Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris...

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany

6858

Alessandro Aldini Roberto Gorrieri (Eds.)

Foundations of Security Analysis and Design VI FOSAD Tutorial Lectures

13

Volume Editors Alessandro Aldini Università degli Studi di Urbino “Carlo Bo” Dipartimento di Scienze di Base e Fondamenti Piazza della Repubblica 13 61029 Urbino, Italy E-mail: [email protected] Roberto Gorrieri Università degli Studi di Bologna Dipartimento di Scienze dell’Informazione Mura Anteo Zamboni 7 40127 Bologna, Italy E-mail: [email protected]

ISSN 0302-9743 e-ISSN 1611-3349 ISBN 978-3-642-23081-3 e-ISBN 978-3-642-23082-0 DOI 10.1007/978-3-642-23082-0 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011934268 CR Subject Classiﬁcation (1998): D.4.6, C.2, K.6.5, K.4, D.3, F.3, E.3 LNCS Sublibrary: SL 4 – Security and Cryptology

© Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, speciﬁcally the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microﬁlms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientiﬁc Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

International School on Foundations of Security Analysis and Design

This book is the sixth in a series of volumes collecting tutorial papers accompanying lectures presented at FOSAD, the International Summer School on Foundations of Security Analysis and Design, which has been held yearly since 2000 at the University Residential Center of Bertinoro, Italy. FOSAD has been one of the foremost educational events established with the goal of disseminating knowledge in the critical area of security in computer systems and networks. Every year, FOSAD oﬀers a good spectrum of current research in foundations of security – ranging from programming languages to analysis of protocols, from cryptographic algorithms to access control policies and trust/identity management – that can be of help for graduate students and young researchers from academia or industry who intend to approach the ﬁeld. The spirit of FOSAD is also characterized by the “open session”, which represents a series of presentations given by selected participants about their ongoing work. The objective of this initiative is to encourage discussions, propose new ideas, comment on open problems, and favor novel scientiﬁc collaborations. The topics covered in this book include privacy and data protection, security APIs, cryptographic veriﬁcation by typing, model-driven security, noninterference analysis, security in governance, risk, and compliance, lattice cryptography, quantitative information ﬂow analysis, and risk analysis. The opening paper presented by Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati gives an overview of the techniques developed for protecting data and ensuring privacy. Riccardo Focardi, Flaminia Luccio, and Graham Steel discuss the subtleties behind the design of secure application program interfaces (security APIs) and show that their analysis through formal techniques has recently proved highly successful both in ﬁnding new ﬂaws and verifying security properties of improved designs. The tutorial paper by C´edric Fournet, Karthikeyan Bhargavan, and Andrew Gordon shows the use of types for verifying authenticity properties of cryptographic protocols. The paper by David Basin, Manuel Clavel, Marina Egea, Miguel Garc´ıa de Dios, Carolina Dania, Gonzalo Ortiz, and Javier Valdazo is a survey of a very promising instance of model-driven security. The authors present an approach and a toolkit supporting the construction of security, data, and graphical user interface (GUI) models together with related Web applications. Roberto Gorrieri and Matteo Vernali extend the notion of intransitive noninterference by Rushby to the frameworks of deterministic labelled transition systems, nondeterministic automata, and the class of Petri nets called elementary net systems. Yudistira Asnar and Fabio Massacci describe a methodology to design systems trading information security with governance, risk, and compliance (GRC) management. Daniele Micciancio gives an introduction to the mathematical theory and appli-

VI

Foundations of Security Analysis and Design

cation of lattice cryptography, which is one of the hottest and fast-moving areas in mathematical cryptography today. M´ ario Alvim, Miguel Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi review two information-theoretic approaches to the quantitative analysis of information ﬂows, namely, the one based on Shannon entropy, and the one based on R´enyi min-entropy. In the last paper, Mass Soldal Lunda, Bjørnar Solhauga, and Ketil Stølen introduce general techniques and guidelines for dealing with risk analysis in systems evolving over time. In particular, the authors propose the CORAS approach to model-driven risk analysis. This year, FOSAD was organized in cooperation with the Network of Excellence on Engineering Secure Future Internet Software Services and Systems (EU FP7 Project NESSoS). Obviously, we would like to thank all the institutions that have promoted and founded FOSAD in the last few years. Finally, we also wish to thank all the staﬀ of the University Residential Center of Bertinoro for the organizational and administrative support. August 2011

Alessandro Aldini Roberto Gorrieri

Table of Contents

Foundations of Security Analysis and Design Protecting Privacy in Data Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati

1

An Introduction to Security API Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . Riccardo Focardi, Flaminia L. Luccio, and Graham Steel

35

Cryptographic Veriﬁcation by Typing for a Sample Protocol Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C´edric Fournet, Karthikeyan Bhargavan, and Andrew D. Gordon Model-Driven Development of Security-Aware GUIs for Data-Centric Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Basin, Manuel Clavel, Marina Egea, Miguel A. Garc´ıa de Dios, Carolina Dania, Gonzalo Ortiz, and Javier Valdazo On Intransitive Non-interference in Some Models of Concurrency . . . . . . . Roberto Gorrieri and Matteo Vernali A Method for Security Governance, Risk, and Compliance (GRC): A Goal-Process Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yudistira Asnar and Fabio Massacci The Geometry of Lattice Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniele Micciancio Quantitative Information Flow and Applications to Diﬀerential Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M´ ario S. Alvim, Miguel E. Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi

66

101

125

152 185

211

Risk Analysis of Changing and Evolving Systems Using CORAS . . . . . . . Mass Soldal Lund, Bjørnar Solhaug, and Ketil Stølen

231

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

275

Protecting Privacy in Data Release Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati Dipartimento di Tecnologie dell’Informazione, Universit` a degli Studi di Milano, Via Bramante 65 - 26013 Crema, Italy [email protected]

Abstract. The evolution of the Information and Communication Technology has radically changed our electronic lives, making information the key driver for today’s society. Every action we perform requires the collection, elaboration, and dissemination of personal information. This situation has clearly brought a tremendous exposure of private and sensitive information to privacy breaches. In this chapter, we describe how the techniques developed for protecting data have evolved in the years. We start by providing an overview of the ﬁrst privacy deﬁnitions (k-anonymity, -diversity, t-closeness, and their extensions) aimed at ensuring proper data protection against identity and attribute disclosures. We then illustrate how changes in the underlying assumptions lead to scenarios characterized by diﬀerent and more complex privacy requirements. In particular, we show the impact on privacy when considering multiple releases of the same data or dynamic data collections, ﬁne-grained privacy deﬁnitions, generic privacy constraints, and the external knowledge that a potential adversary may exploit for inferring sensitive information. We also brieﬂy present the concept of diﬀerential privacy that has recently emerged as an alternative privacy deﬁnition. Keywords: Privacy, microdata protection, data release.

1

Introduction

The advancements in the Information and Communication Technology (ICT) have revolutionized our lives in a way that was unthinkable until few years ago. We live in the Globalization era, where everything we need to do is available within “one mouse click”. Global infrastructure, digital infrastructure, digital society are only few examples of terms used at diﬀerent times for concisely referring to our “computer-based” society. The term that better represents the today’s society is however information society (or information age) since the information has a key role in the daily life activities of everyone. Every time we browse Internet, perform an online transaction, ﬁll in forms to, for example, enter contests or participate in online games, and spend our time online in social networks, information about us is collected, stored, analyzed, and sometimes A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 1–34, 2011. c Springer-Verlag Berlin Heidelberg 2011

2

S. De Capitani di Vimercati et al.

shared with third parties. Furthermore, public and private companies have often the need of publishing aggregate statistical data (macrodata) as well as detailed data (microdata) for research or statistical purposes. The complexity and variety of the today’s information society introduce therefore new risks and pose new research challenges. In fact, the vast amount of personal (user-generated) data collected, stored, and processed, the unclear data ownership, and the lack of control of the users on their own data are creating unprecedented risks of privacy breaches. The problem of properly protecting the privacy of the users is clearly not new and has received (and receives) considerable attention from the research and development communities. In the past, the restricted access to information and its expensive processing represented a form of protection that does not hold anymore. In fact, with the rate at which technology is developing, it is now becoming easier and easier to access huge amount of data by using, for example, portable devices (e.g., PDAs, mobile phones) and ubiquitous network resources. Also, the availability of powerful techniques for analyzing and correlating data coming from diﬀerent information sources makes it simple to infer information that was not intended for disclosure. It is interesting to observe how the problem of guaranteeing privacy protection is changing over the years, in line with the evolution of the ICT. Data were principally released in the form of macrodata, that is, tables (often of two dimensions), where each cell contains aggregate information about users or companies, called respondents. The macrodata protection techniques were principally based on the identiﬁcation and obfuscation of sensitive cells [11]. With the growing importance and use of microdata, the research community dedicated many eﬀorts in designing microdata protection techniques able to preserve the privacy of the respondents while limiting the disclosure risks. Traditionally, the disclosure risks are related to the possibility, for an adversary, to use the microdata for determining conﬁdential information on a speciﬁc individual (attribute disclosure) or for identifying the presence of an individual in the microdata table itself (identity disclosure). To limit the disclosure risks, names, addresses, phone numbers, and other identifying information are removed (or encrypted) from the microdata. For instance, in the microdata table in Figure 1, which contains medical data, the names of the patients as well as their Social Security Numbers are removed, thus obtaining the de-identiﬁed medical data in Figure 2(a). Although a de-identiﬁed microdata table apparently protects the identities of the respondents represented in the table, there is no guarantee of anonymity. The de-identiﬁed microdata may contain other information, called quasi-identiﬁer , such as birth date and ZIP code that in combination can be linked to publicly available information to re-identify individuals. As an example, consider the deidentiﬁed medical data in Figure 2(a) and the voter list for the San Francisco area, publicly released by the local municipality, in Figure 2(b). It is easy to see that the values of attributes DoB, Sex, and ZIP can be exploited for linking the tuples in the microdata with the voter list, thus possibly re-identifying individuals and revealing their illnesses. For instance, in the microdata in Figure 2(a) there is only one female born on 1958/12/11 living in the 94142 area.

Protecting Privacy in Data Release SSN

Name

DoB

Sex

ZIP

123-45-6789 234-56-7890 345-67-8901 456-78-9012 567-89-0123 678-90-1234 789-01-2345 890-12-3456 901-23-4567 012-34-5678 134-56-7890 245-67-8901 356-78-9012 467-89-0123 578-90-1234 689-01-2345 790-12-3456

Diana Smith Nathan Johnson Eric Williams Liz Jones John Brown Luke Davis Barbara Miller Fay Wilson Anthony Moore Matt Taylor Jane Doe Anna Anderson Carol Thomas Gabrielle White Lorna Harris Rob Martin Bob Thompson

1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1958/12/11 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F F M M

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94142 94145 94136 94134 94147 94148

3

Disease H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Pneumonia Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

Fig. 1. An example of microdata table with identifying information

This combination, if unique in the external world as well, uniquely identiﬁes the corresponding tuple in the table as pertaining to Jane Doe, 300 Main St., San Francisco, revealing that she suﬀers from Pneumonia. From a study performed on the data collected for the 2000 US Census, Golle showed that 63% of the US population can be uniquely identiﬁed combining their gender, ZIP code, and full date of birth [21]. This percentage decreases if the gender is combined with the County of residence instead of the ZIP code, and with the month/year of birth (see Figure 3). In the 1990s, several microdata protection techniques were developed [11]. Such techniques can be classiﬁed in two main categories: masking techniques that transform the original data in a way that some statistical analysis on the original and transformed data produce the same or similar results; synthetic data generation techniques that replace the original data with synthetic data that preserve some statistical properties of the original data. Among the microdata protection techniques, k-anonymity [37] is probably one of the most popular, which has inspired the development of algorithms and techniques for both enforcing k-anonymity and for complementing it with other forms of protection (e.g., -diversity [29], and t-closeness [26]). These techniques are based on the assumptions that: quasi-identiﬁers are the only attributes that can be used for inferring the respondents to whom information refers; the same microdata table is published only once; and potential adversaries do not have any external knowledge. Clearly, such assumptions do not hold anymore in the today’s society, where any information can be used to re-identify anonymous data [33]. Two well-known examples of privacy violations, which testiﬁed how ensuring proper privacy protection is becoming a diﬃcult task, are the America OnLine (AOL) and Netﬂix incidents [3,32]. AOL is an Internet services and media company that in 2006 released around 20 millions of search records of 650,000 of its customers. To protect the privacy of its customers, AOL de-identiﬁed such records by substituting personal identiﬁers with numerical identiﬁers. A sample of such data is the following:

4

S. De Capitani di Vimercati et al. SSN Name

DoB

Sex

ZIP

1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1958/12/11 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F F M M

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94142 94145 94136 94134 94147 94148

Disease H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Pneumonia Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

(a) De-identiﬁed medical data Name

Address

City

ZIP

DoB

Sex

... ... ... ... ... ... Jane Doe 300 Main St. San Francisco 94142 58/12/11 female ... ... ... ... ... ...

Education ... secondary ...

(b) Voter list Fig. 2. An example of de-identiﬁed microdata table (a) and of publicly available non de-identiﬁed dataset (b)

ZIP County

year 0.2% 0%

Date of Birth month, year full date 4.2% 63.3% 0.2% 14.8%

Fig. 3. Identiﬁability of the US population in the 2000 US Census data [21]

116874 thompson water seal 2006-05-24 11:31:36 1 http://www.thompsonwaterseal.com 116874 knbt 2006-05-31 07:57:28 116874 knbt.com 2006-05-31 08:09:30 1 http://www.knbt.com 117020 texas penal code 2006-03-03 17:57:38 1 http://www.capitol.state.tx.us 117020 homicide in hook texas 2006-03-08 09:47:35 117020 homicide in bowle county 2006-03-08 09:48:25 6 http://www.tdcj.state.tx.us

that shows records related to two diﬀerent users (116874 and 117020) containing the ID, the term(s) used for the search, the timestamp, whether the user clicked on a result, and the corresponding visited website. With these data, two reporters of the New York Times newspaper were able to identify AOL customer no. 4417749 as Thelma Arnold, a 62 yeas old widow living in Lilburn [3]. In the same year, the on-line movies renting service Netﬂix publicly released 100 millions records, showing the ratings given by 500,000 users to the movies they rent. The records were released within the “Netﬂix Prize” competition that oﬀered $1

Protecting Privacy in Data Release

5

million to anyone who could improve the algorithm used by Netﬂix to suggest movies to its customers. Also in this case, records were de-identiﬁed by replacing personal identiﬁers with numerical identiﬁers. However, some researchers were able to de-anonymize the data by comparing the Netﬂix data against publicly available ratings on the Internet Movie Database (IMDB). For instance, the release of her movie preferences damaged a lesbian mother since she was reidentiﬁed, thus causing the disclosure of her sexual orientation [32]. From the discussion, it is clear that the protection of microdata against improper disclosure is a key issue in today’s information society. The main objective of this chapter is then to provide an overview of how the techniques for protecting microdata releases have evolved in the years, according to the evolution and complexity of the scenarios considered. We will start with a description of the ﬁrst solutions aimed at protecting microdata against identity and attribute disclosures (Section 2). We will then describe recent approaches that removed some of the assumptions characterizing the ﬁrst proposals. In particular, we will describe novel solutions for supporting: i) multiple data releases and the insertion/removal of tuples into/from the released microdata table (Section 3); ii) the deﬁnition of ﬁne-grained privacy preferences by each data respondent (Section 4); iii) generic sensitive associations among released data that need to be kept conﬁdential (Section 5); and iv) external knowledge by malicious data recipients that may cause information leakage (Section 6). We will ﬁnally illustrate a new privacy notion, called diﬀerential privacy, that deﬁnes when a computation is privacy-preserving (Section 7).

2

Privacy in Microdata Publishing

Goal of this section is to present the privacy models speciﬁcally targeted to the protection of microdata from identity and attribute disclosures (i.e., kanonymity [37], -diversity [29], and t-closeness [26]) that have inﬂuenced the work performed by the research community in the data protection area during the last two decades. All these privacy models are based on the following assumptions. First, the attributes composing a microdata table are classiﬁed in four categories, as follows. – Identiﬁers: attributes that uniquely identify a respondent (e.g., Name and SSN). – Quasi-identiﬁers (QI): attributes that, in combination, can be linked with external information to re-identify (all or some of) the respondents to whom information refers, or to reduce the uncertainty over their identities (e.g., DoB, Sex, and ZIP). – Conﬁdential attributes: attributes that represent sensitive information (e.g., Disease). – Non-conﬁdential attributes: attributes that are not considered sensitive by the respondents and whose release is not harmful (e.g., FavoriteColor). A microdata table is then protected by applying microdata protection techniques [11] that transform the values of the quasi-identiﬁer attributes. Second,

6

S. De Capitani di Vimercati et al.

there is a one-to-one correspondence between tuples in the table and data respondents. Third, a microdata table is released only once. Fourth, each table is characterized by a unique quasi-identiﬁer. In the remainder of this section, after a description of k-anonymity, -diversity, and t-closeness, we also brieﬂy discuss their recent enhancements that remove some of the assumptions on which they are based. 2.1

k-Anonymity

k-Anonymity has been proposed for protecting microdata from identity disclosure [37]. It captures the well-known requirement, traditionally applied by statistical agencies, stating that any released data should be indistinguishably related to no less than a certain number of respondents. Basically, it characterizes the protection degree against re-identiﬁcation caused by linking the released dataset with external data sources. Due to the assumption that linking attacks exploit quasi-identiﬁers only, in [37] the general requirement described above has been translated into the following k-anonymity requirement: Each release of data must be such that every combination of values of quasi-identiﬁers can be indistinctly matched to at least k respondents. A microdata table then satisﬁes the k-anonymity requirement if each tuple in the table cannot be related to less than k respondents in the population, and each respondent in the population cannot be related to less than k tuples in the released table. The k-anonymity requirement assumes that the data holder knows any possible external source of information that may be exploited by a malicious recipient for respondents re-identiﬁcation. This assumption is however limiting and highly impractical in most scenarios. k-anonymity then takes a safe approach by requiring that each combination of values of the quasi-identiﬁer attributes appears with at least k occurrences in the released table. The deﬁnition of k-anonymity represents a suﬃcient (not necessary) condition for the k-anonymity requirement. In fact, since each combination of values of the quasi-identiﬁer attributes appears with at least k occurrences, each respondent cannot be associated with less than k tuples in the microdata table. Analogously, each tuple in the table cannot be related to less than k respondents in the population. As a consequence, the risk of re-identiﬁcation is reduced by at least a factor k, independently from the external datasets available to a malicious recipient. Traditional approaches for guaranteeing k-anonymity transform the values of the attributes composing the quasi-identiﬁer and leave the sensitive and nonsensitive attributes unchanged. In particular, k-anonymity relies on generalization and suppression, which have the advantage of preserving the truthfulness of the released information, while producing less precise and less complete tables. Generalization consists in substituting the original values with more general values. For instance, the date of birth can be generalized by removing the day, or the day and the month, of birth. Suppression consists in removing data from the microdata table. The intuition behind the combined use of generalization and suppression is that suppression can reduce the amount of generalization

Protecting Privacy in Data Release

7

necessary to guarantee k-anonymity. In fact, if a limited number of outliers (i.e., quasi-identiﬁer values with less than k occurrences in the table) would force a large amount of generalization to satisfy k-anonymity, these values can be more conveniently removed from the table. For instance, consider the microdata table in Figure 2(a) and assume that the quasi-identiﬁer is attribute DoB only. Since there is only one person born in 1958, attribute DoB should be generalized to the decade of birth to guarantee 4-anonymity. Alternatively, removing the date of birth associated with the eleventh tuple in the table, 4-anonymity can be achieved by generalizing the date of birth to the month of birth, thus reducing the information loss. Both generalization and suppression can be applied at different granularity levels. More precisely, generalization can be applied at the cell or attribute level, while suppression can be applied at the cell, attribute, or tuple level. The combined use of generalization and suppression at diﬀerent granularity levels leads to diﬀerent classes of approaches for guaranteing k-anonymity [10]. However, most of the solutions proposed in the literature adopt attribute generalization and tuple suppression (e.g., Samarati’s algorithm [37], Incognito [23], and k-Optimize [4]). The reason is that cell generalization produces a table where the values in the cells of the same column may be non homogeneous, since they belong to diﬀerent domains (e.g., some tuples report the complete date of birth, while other tuples only report the year of birth). On the other hand, cell generalization has the advantage of causing a reduced information loss if compared to attribute generalization. Therefore, recently also solutions adopting cell generalization have been analyzed (e.g., Mondrian [24]). Consider the microdata table in Figure 2(a) and assume that the quasiidentiﬁer is composed of attributes DoB, Sex, and ZIP. Figure 4 illustrates the 4-anonymous table obtained combining tuple suppression (the eleventh tuple of the table in Figure 2(a) is suppressed) and attribute generalization (DoB has been generalized removing the day of birth, Sex has been generalized to a unique value, denoted *, and ZIP has been generalized removing the last two digits). Since generalization and suppression cause information loss, it is necessary to compute an optimal k-anonymous microdata table that maximizes the utility while preserving the privacy of the respondents. The computation of an optimal k-anonymous table is an NP-hard problem, independently from the granularity level at which generalization and suppression are applied [1,12,31]. As a consequence, both exact and heuristic algorithms have been proposed [12]. The exact algorithms have computational time complexity exponential in the number of quasi-identifying attributes, which is however a small value compared with the number of tuples in the microdata table. 2.2

-Diversity

Although k-anonymity represents an eﬀective solution for protecting respondents’ identities, it has not been designed to protect the released microdata table against attribute disclosure. Given a k-anonymous table it may then be possible to infer (or reduce the uncertainty about) the value of the sensitive attribute associated with a speciﬁc respondent.

8

S. De Capitani di Vimercati et al.

SSN Name

DoB

Sex

ZIP

1950/06 1950/06 1950/06 1950/06

* * * *

941** 941** 941** 941**

H1N1 Gastritis Dyspepsia Pneumonia

Disease

1940/04 1940/04 1940/04 1940/04

* * * *

941** 941** 941** 941**

Peptic Peptic Peptic Peptic

1940/06 1940/06 1940/06 1940/06

* * * *

941** 941** 941** 941**

Broken Leg Short Breath Broken Leg Stomach Cancer

1950/05 1950/05 1950/05 1950/05

* * * *

941** 941** 941** 941**

H1N1 Flu Stomach Cancer Gastritis

Ulcer Ulcer Ulcer Ulcer

Fig. 4. An example of 4-anonymous table

Two well-known attacks that may lead to attribute disclosure in a kanonymous table are the homogeneity attack [29,37] and the external knowledge attack [29]. To illustrate the homogeneity attack, consider a k-anonymous table where all the tuples composing an equivalence class (i.e., all the tuples having the same value for the quasi-identifying attributes) have the same value also for the sensitive attribute. If a data recipient knows that an individual is represented in the microdata table and the value of her quasi-identiﬁer, the recipient can easily identify the equivalence class representing the target respondent. Under the above homogeneity assumption, the recipient can also infer the value of the sensitive attribute of the target respondent. For instance, consider the 4-anonymous table in Figure 4 and suppose that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. Since all the tuples in the equivalence class with quasi-identiﬁer value equal to 1940/04,*,941** have Peptic Ulcer as value for attribute Disease, Alice can infer that her friend suﬀers from Peptic Ulcer. The external knowledge attack occurs when the data recipient may reduce her uncertainty about the sensitive attribute value of a target respondent, exploiting some additional (external) knowledge about the respondent. For instance, consider the 4-anonymous table in Figure 4 and suppose that Alice knows that her neighbor Carol is a female living in 94145 area and born on 1940/06/30. Observing the 4-anonymous table, Alice can only infer that her neighbor suﬀers either from Short Breath or Stomach Cancer, or has a Broken Leg. Suppose that Alice sees Carol running every day: as a consequence, Alice can infer that Carol can neither have broken her leg nor suﬀer from short breath. Hence, Alice discovers that Carol suﬀers from Stomach Cancer. The deﬁnition of -diversity has been proposed to counteract homogeneity and external knowledge attacks [29]. -diversity extends k-anonymity requiring that each equivalence class must be associated with at least well-represented values for the sensitive attribute. In [29], the authors propose diﬀerent deﬁnitions for “well-represented” values. The simplest formalization of -diversity requires

Protecting Privacy in Data Release

SSN Name DoB Sex

ZIP

9

Disease

1940 1940 1940 1940

M M M M

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Short Breath

1950 1950 1950 1950

F F F F

941** 941** 941** 941**

H1N1 Pneumonia H1N1 Flu

1940 1940 1940 1940

F F F F

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Stomach Cancer

1950 1950 1950 1950

M M M M

941** 941** 941** 941**

Gastritis Dyspepsia Stomach Cancer Gastritis

Fig. 5. An example of 4-anonymous and 3-diverse table

that each equivalence class includes at least diﬀerent values for the sensitive attribute. Even with this straightforward deﬁnition, -diversity counteracts the homogeneity attack, and reduces the eﬀectiveness of external knowledge attacks, since the data recipient needs more external knowledge to be able to associate a single sensitive value with a target respondent. For instance, consider the microdata table in Figure 5, which is 4-anonymous and 3-diverse and suppose that Alice knows that her neighbor Carol is a female living in 94145 area and born on 1940/06/30. Observing the 3-diverse table, Alice can infer that her neighbor either suﬀers from Peptic Ulcer , Stomach Cancer , or has a Broken Leg. Since Alice only knows that Carol does not have a Broken Leg, she cannot be certain about her neighbor’s disease. Although any traditional k-anonymity algorithm can be easily adapted to guarantee -diversity [29], in [44] the authors propose a speciﬁc approximation algorithm. This algorithm is based on cell suppression, and aims at minimizing the number of suppressed cells in computing a microdata table that satisﬁes -diversity. 2.3

t-Closeness

Even if -diversity represents a ﬁrst step for counteracting attribute disclosure, this solution may still produce a table that is vulnerable to privacy breaches. In fact, the deﬁnition of -diversity does not take into consideration diﬀerent factors that might be exploited by a malicious data recipient for attribute disclosure, such as: i) the frequency distribution of the values in the sensitive attribute domain; ii) the possible semantic relationships among sensitive attribute

10

S. De Capitani di Vimercati et al.

values; and iii) the diﬀerent sensitivity degree associated with diﬀerent values of the sensitive attribute domain (e.g., HIV is usually considered more sensitive than Flu). Two attacks that may cause attribute disclosure in an -diverse table are the skewness attack and the similarity attack [26]. Skewness attack exploits the possible diﬀerence in the frequency distribution of sensitive attribute values within an equivalence class, with respect to the frequency distribution of sensitive attribute values in the whole population (or in the released microdata table). Indeed, a difference in the distribution may change the probability with which a respondent in the equivalence class is associated with a speciﬁc sensitive value. For instance, consider the 3-diverse table in Figure 5 and suppose that Alice knows that her friend Diana is a female living in 94141 area and born on 1950/06/02. Since two out of the four tuples in the equivalence class with quasi-identiﬁer value equal to 1950,F,941** have value H1N1 for attribute Disease, Alice can infer that her friend has 50% probability of suﬀering from this disease, compared to the 12.5% of the whole released table. Also, although -diversity guarantees that each equivalence class has at least diﬀerent values for the sensitive attribute, it does not impose constraints on their semantics. As a consequence, the sensitive attribute values within an equivalence class may be diﬀerent, but semantically similar, thus causing attribute disclosure. For instance, consider the 3-diverse table in Figure 5 and suppose that Alice knows that her friend Eric is a male living in 94137 area and born on 1950/06/12. Observing the 3-diverse table, Alice can infer that her friend suﬀers from either Gastritis, Dyspepsia, or Stomach Cancer and, therefore, that Eric suﬀers from a stomach-related disease. The deﬁnition of t-closeness has been proposed to counteract skewness and similarity attacks [26]. t-Closeness extends the deﬁnition of k-anonymity, requiring that the frequency distribution of the sensitive attribute values in each equivalence class has to be close (i.e., with distance less than a ﬁxed threshold t) to the frequency distribution of the sensitive attribute values in the released microdata table. In this way, the frequency distribution of sensitive attribute values in each equivalence class is similar to the frequency distribution characterizing the released microdata table. As a consequence, the knowledge of the quasi-identiﬁer value for a target respondent does not change the probability for a malicious recipient of correctly guessing the sensitive value associated with the respondent. The eﬀect of similarity attacks is also mitigated since the presence of semantically similar values in an equivalence class can only be due to the presence, with the same relative frequencies, of these values in the microdata table. To determine if a microdata table satisﬁes the t-closeness property, it is necessary to measure the distance between the frequency distributions of sensitive attribute values characterizing each equivalence class and the whole microdata table. In [26], the authors present diﬀerent techniques to measure the distance between two frequency value distributions, and propose to adopt the Earth Mover Distance (EMD) technique.

Protecting Privacy in Data Release

2.4

11

Extensions

k-anonymity, -diversity, and t-closeness are based on some assumptions that make them not always applicable in speciﬁc scenarios. Such assumptions can be summarized as follows. – Each respondent is represented by only one tuple in the microdata table. In many real-world scenarios diﬀerent tuples in a microdata table may refer to the same respondent. For instance, consider the relation in Figure 2(a): each respondent may be associated with as many tuples in the table as the number of diseases she suﬀers from. k-anonymity is not suited for these scenarios, since an equivalence class composed of k tuples may represent less than k respondents. As a consequence, a k-anonymous table may however violate the k-anonymity requirement. – A unique microdata table is released. Data may be stored in diﬀerent relations, characterized by (functional) dependencies among them. Since different portions of these data may be released at diﬀerent times (e.g., upon request) or to diﬀerent recipients, it is necessary to guarantee that the combined view over all the released pieces of information does not cause privacy violations. Traditional solutions may not be applicable in these scenarios since they assume that the data to be released are stored in a unique microdata table. – The microdata table is characterized by one quasi-identiﬁer . Although traditional solutions assume that the released microdata table is characterized by a unique quasi-identiﬁer, diﬀerent data recipients may posses diﬀerent external data collections to be linked with the released table. The release of a diﬀerent microdata table to each data recipient, anonymized considering the speciﬁc knowledge of the recipient, is however not eﬀective since collusion among recipients would violate respondents’ privacy. On the other hand, a solution that considers the quasi-identiﬁer as the union of the quasiidentiﬁers of any possible data recipient would cause an excessive information loss. – Respondents can be re-identiﬁed through a predeﬁned quasi-identiﬁer . As discussed in Section 1, data respondents can be re-identiﬁed through any piece of information that uniquely (or almost uniquely) pertain to them. As an example, in transactional data (e.g., AOL and Netﬂix datasets) a pattern of items in a transaction (or a set thereof related to the same respondent) may re-identify a respondent. Traditional anonymization solutions are however based on a preliminary identiﬁcation of a quasi-identiﬁer, and are therefore not applicable in this scenario, where the information exploited for reidentiﬁcation cannot be determined a-priori and may not be represented by a set of attributes. In the following, we present a brief overview of some solutions that extend kanonymity, -diversity, and t-closeness by removing (some of) the above assumptions (see Figure 6).

12

S. De Capitani di Vimercati et al. (X,Y )MultiR Butterﬂy km Privacy k-anonymity Anonymity multiple tuples per respondent multiple tables multiple quasi-identiﬁers non-predeﬁned quasi-identiﬁers

× ×

× × × ×

Fig. 6. Extended scenarios (rows) and applicable techniques (columns)

(X,Y )-Privacy [39] addresses the problem of protecting the privacy of the respondents when multiple relations are released and each respondent is possibly associated with a set of tuples in the released tables. In particular, in [39] the authors assume that diﬀerent tables, obtained as the projection of a subset of attributes of a unique microdata table, are sequentially released. To guarantee that a data release, either singularly taken or combined with previous releases, does not compromise the privacy of the respondents, the deﬁnitions of (X,Y )anonymity and (X,Y )-linkability have been introduced, which are both referred with the term (X,Y )-privacy. Given a microdata table and two disjoint sets of attributes X and Y , the microdata table satisﬁes (X,Y )-anonymity if each value of X is linked to at least k diﬀerent values of Y . Note that the deﬁnition of (X,Y )-anonymity is more general than the original k-anonymity requirement. Indeed, the k-anonymity requirement can be satisﬁed by a (X,Y )-anonymous table where X is the set of attributes composing the quasi-identiﬁer and Y is the set of identifying attributes. For instance, consider the microdata table in Figure 2(a) and assume that X = {DoB,Sex,ZIP} and Y = {SSN}. A (X,Y )anonymous table guarantees that each combination of values for attributes DoB, Sex, ZIP is associated with at least k diﬀerent values for attribute SSN. (X,Y )linkability states that no value of Y can be inferred from a value of X with conﬁdence higher than a ﬁxed threshold. To guarantee privacy protection in sequential releases, the join among all the released tables has to satisfy (X,Y )privacy (i.e., it must satisfy both (X,Y )-anonymity and (X,Y )-linkability). MultiR k-anonymity [35] extends the deﬁnition of k-anonymity to multiple relations, while guaranteeing a limited information loss. The considered scenario is characterized by a set S = {T1 , . . . , Tn } of relations that preserves the lossless join property and that includes a person speciﬁc table P T , where each row corresponds to a diﬀerent respondent. The release of a set S of relations satisﬁes multiR k-anonymity if each view over the join JT = P T T1 ... Tn is k-anonymous, meaning that it includes either 0 or at least k occurrences for each combination of values for the attributes in the quasi-identiﬁer. In this scenario, the quasi-identiﬁer is deﬁned as a subset of the attributes of the join relation JT , which can be exploited to re-identify the respondents in the person speciﬁc table

Protecting Privacy in Data Release

13

P T . To protect the privacy of the respondents against attribute (besides identity) disclosure, in [35] the authors propose to extend the deﬁnition of -diversity to the multirelational scenario. A release of a set S of relations satisﬁes multiR ... Tn among all the -diversity if each view over the join JT = P T T1 released relations satisﬁes -diversity with respect to the considered sensitive attribute. Butterfly [36] aims at protecting the privacy of the respondents, while minimizing information loss, when the microdata table to be released is characterized by diﬀerent quasi-identiﬁers. Traditional approaches consider, as a quasi-identiﬁer, the set of all the attributes that may be externally available (although not in combination) to some data recipient. Even if eﬀective, this assumption causes an excessive information loss that can be reduced if the set of attributes available to each recipient is considered separately. Indeed, as shown in [36], a table that is k-anonymous with respect to quasi-identiﬁer QI1 and with respect to QI2 may not be k-anonymous with respect to QI1 ∪ QI2 . In [36] the authors then introduce the k-butterﬂy principle that allows the anonymization of a microdata table so that the k-anonymity requirement is satisﬁed with respect to multiple quasi-identiﬁers while reducing the information loss. More precisely, given a microdata table and two quasi-identiﬁers QI1 and QI2 , a subset of tuples with the same value for the attributes in QI1 ∩ QI2 satisﬁes k-butterﬂy with respect to QI1 and QI2 if the equivalence classes induced by QI1 \ QI2 (QI2 \ QI1 , resp.) include at least k tuples each. A microdata relation satisﬁes k-butterﬂy if all the equivalence classes induced by QI1 ∩ QI2 satisfy k-butterﬂy. It is important to note that a set of tuples that satisﬁes k-butterﬂy with respect to QI1 and QI2 is k-anonymous with respect to both QI1 and QI2 . km -Anonymity [38] aims at protecting the privacy of respondents of transactional data, where a subset of the items composing a transaction may represent a quasi-identiﬁer. For instance, consider a transactional dataset including all the items purchased in a supermarket and assume that Alice has seen her neighbor Bob buying parmesan cheese, strawberries, and soy milk. If the supermarket dataset includes only one transaction with this combination of items, Alice can ﬁnd out the complete list of items purchased by Bob, which may be sensitive (e.g., if it contains a medicine). The k m -anonymity concept solves this problem by requiring that each combination of at most m items must appear in at least k transactions, where m is the maximum number of items per transaction that may be known to a malicious data recipient.

3

Multiple Releases and Data Streams

The proposals described in Section 2 assume that only one instance of the microdata is published. There are however many scenarios where data are subject to frequent changes, due to the insertion, deletion, and update of tuples in the microdata, and need to be published at a regular basis. For instance, a hospital may be forced by law to publish an anonymized version of its data every six months.

14

S. De Capitani di Vimercati et al.

Data may also be continuously generated and may need to be immediately released. This happens, for example, when a credit card company outsources to a third party the transactions generated by its customers whenever they use their credit cards. Such transactions form a microdata stream that has to be continuously monitored by the third party to detect possible misbehaviors. In this section, we ﬁrst describe the privacy issues that arise in the case of multiple data releases and data streams, highlighting why the traditional approaches for protecting microdata are not suitable. We then illustrate some recent approaches that counteract such privacy issues. 3.1

Problem

The multiple releases of a microdata table whose content is updated over time may cause information leakage since a malicious data recipient can correlate the released datasets to gain information about respondents. Also, if a combination of quasi-identiﬁer values appears in one of the released tables only, it may be easier for a malicious recipient to correctly associate it with one of those respondents who have been removed from the subsequent releases (possibly exploiting the information available through external sources). To illustrate, consider the two subsequent releases of the medical data of the patients of a hospital illustrated in Figure 7. Some patients are included in both tables, while others are represented in one of the two tables only (tuples marked with a bullet in Figures 7(a) and 7(b)). Both the ﬁrst and the second release satisfy 4-anonymity and 3-diversity. However, suppose that Alice knows that her friend Fay, who is a female living in the 94130 area, born on 1940/04/20, was in the hospital when both tables have been released. Alice then knows that Fay is represented by a tuple in the tables of Figure 7, and she can identify the two equivalence classes to which Fay belongs in the two tables. In particular, in the ﬁrst release Fay is included in the third class of the table in Figure 7(a), and hence her disease could be either Peptic Ulcer, Broken Leg, or Stomach Cancer. Instead, in the second release Fay is included in the second class of the table in Figure 7(b), and hence her disease could be either Gastritis, Short Breath, Pneumonia, or Peptic Ulcer. By comparing the two sets of possible illnesses, Alice can easily infer that Fay suﬀers from Peptic Ulcer, since this is the unique disease that appears in both sets. When we consider a data stream scenario where only new tuples can be inserted into the released microdata table, it is not necessary to compute diﬀerent releases of the dataset. In fact, it may be more convenient to release only the new tuples. In this case, traditional approaches cannot be adopted, since they need the whole dataset that is instead not available in advance. Furthermore, a data stream is possibly an inﬁnite sequence of tuples, which must be released as soon as they are available since data utility decreases as time passes. As a consequence, it is not possible to assume that the whole dataset can be collected before its publication. For instance, consider a credit card company that releases the stream of data generated by purchases represented as tuples in a microdata table. Such data should be immediately published to check, for example,

Protecting Privacy in Data Release DoB Sex ZIP Disease

15

DoB Sex ZIP Disease

1940 1940 • 1940 1940

M M M M

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Short Breath

1940 1940 • 1940 1940

* * * *

9414* 9414* 9414* 9414*

Peptic Ulcer Peptic Ulcer Measles Stomach Cancer

1950 1950 1950 • 1950

F F F F

941** 941** 941** 941**

H1N1 Pneumonia H1N1 Flu

• 1940 1940 • 1940 1940

* * * *

9413* 9413* 9413* 9413*

Gastritis Short Breath Pneumonia Peptic Ulcer

• 1940 1940 • 1940 1940

F F F F

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Stomach Cancer

1950 1950 • 1950 1950

* * * *

9414* 9414* 9414* 9414*

H1N1 Pneumonia Measles Gastritis

• 1950 1950 • 1950 1950

M M M M

941** 941** 941** 941** (a)

Gastritis Dyspepsia Stomach Cancer Gastritis

• 1950 1950 1950 • 1950

* * * *

9413* 9413* 9413* 9413* (b)

Infract Dyspepsia H1N1 Thrombosis

Fig. 7. An example of two subsequent releases of a table including the medical data of a hospital

possible frauds, while preserving the privacy of the card holders. We note that data streams may include more than one tuple for each respondent (e.g., a card holder may perform multiple purchases). 3.2

Solutions

Recently, the scientiﬁc community has proposed diﬀerent approaches addressing the privacy issues previously discussed. These solutions can be classiﬁed in two categories, depending on whether they consider the re-publication of data or the publication of data streams. In the following, we summarize some of the approaches proposed in both scenarios. Data re-publication (e.g., [39,43]). Most of the solutions proposed to protect the privacy of data respondents in data re-publication scenarios only focus on supporting the insertion of new tuples in the dataset and implicity assume that no tuple is removed from the microdata table. m-Invariance [43] is the ﬁrst technique addressing the problem of data re-publication that takes both insertion and deletion of tuples into account. The possible removal of tuples from the dataset can cause information leakage, due to the critical absence phenomenon. To illustrate, consider the two subsequent releases in Figure 7 and suppose that Alice knows that Gabrielle is represented in both microdata tables and that she suﬀers from either Flu or H1N1 . The value Flu, however, does not appear in the second release. As a consequence, Alice can conclude with certainty that Gabrielle contracted H1N1 . It is important to note that this inference can be drawn independently from how the two released datasets have been generalized to prevent disclosure. To counteract this privacy breach, in [43] the authors introduce the m-invariance property. A sequence T1 , . . . , Tn of released microdata tables satisﬁes m-invariance if the following properties hold: i) each equivalence

16

S. De Capitani di Vimercati et al.

class in Ti , i = 1, . . . , n, includes at least m tuples; ii) no sensitive value appears more than once in each equivalence class in Ti , i = 1, . . . , n; and iii) for each tuple t, the equivalence classes to which t belongs in the sequence [Ti , Tj ], 1 ≤ i ≤ j ≤ n, are characterized by exactly the same set of sensitive values. The rationale of m-invariance is that all the equivalence classes to which a published tuple t belongs must be associated with exactly the same set of (at least) m diﬀerent sensitive values. In this way, the correlation of the tuples in T1 , . . . , Tn does not permit a malicious recipient to associate less than m diﬀerent sensitive values with each respondent in the released datasets. The technique proposed in [43] to achieve m-invariance is incremental, meaning that the n-th release Tn can be determined taking into account only the previous release Tn−1 , and is based on the possible insertion of counterfeits, when it is needed to prevent critical absences. Data streams (e.g., [25,40,45]). One of the most important requirements when releasing data as a stream is timeliness, since these data are usually time critical and need to be published in a timely fashion to be useful for the recipients. The proposed solutions mainly aim at satisfying the k-anonymity requirement. To this purpose, they rely on generalization and on the introduction of a limited delay in data publication. The ﬁrst solution in this direction has been proposed in [45] and is based on the principle that all the tuples in an equivalence class must be published at the same time. Therefore, the data holder locally maintains a set of equivalence classes, which are all initially empty. As a new tuple is generated by the stream, it is inserted into a suitable equivalence class, if such class exits; a new equivalence class suitable for the tuple is generated, otherwise. We note that each equivalence class can include at most one tuple for each respondent. As soon as one of the equivalence classes includes k tuples that, by construction, are related to k diﬀerent respondents, these tuples are generalized to the same quasi-identiﬁer value and published. This technique, although simple, guarantees privacy protection and a timely data publication, at the price of a possibly high information loss. To limit this information loss, in [45] the authors introduce an improvement of their technique that makes a probabilistic estimation of the tuples that will be generated by the data stream, to choose the most convenient generalization strategy. An alternative approach for protecting the privacy of respondents in data streams has recently been proposed in [40], where the authors assume that data are generated and published as “snapshots” (i.e., sets of records available at a given moment of time) of d tuples each. This technique combines generalization and tuple suppression with tuple relocation to guarantee -diversity. Relocation consists in moving a tuple from one snapshot to a more recent one, if this delay in data publishing could be useful to satisfy the -diversity principle. The approach illustrated in [40] guarantees that window queries (i.e., queries that need to be evaluated only on the data released in a speciﬁc time window) evaluated on the released dataset produce the same

Protecting Privacy in Data Release

17

result as if they were evaluated on the original data stream. Another approach for protecting data streams has been illustrated in [25]. This approach is based on noise addition and therefore is not suitable for all those scenarios that require truthfulness of released data.

4

Fine-Grained Privacy Preferences

The privacy-aware publishing techniques illustrated in previous sections guarantee the same amount of privacy to all the respondents represented in the released microdata table. Privacy requirements may however depend on respondents’ preferences, or on the sensitivity of the released values. In the following of this section, we illustrate both the issues that may arise when enforcing the same protection degree to all the respondents, and the advantages of solutions that permit a ﬁne-grained speciﬁcation of privacy preferences. We also describe some recent approaches supporting ﬁne-grained privacy preference speciﬁcations. 4.1

Problem

Privacy is an individual concept [27], since each individual may have her own privacy needs that may be diﬀerent from the requirements of another individual. However, traditional privacy protection techniques provide all the data respondents with the same amount of privacy, without considering their preferences. For instance, the k-anonymity requirement demands that each tuple in a released table cannot be associated with less than k respondents in the population, and viceversa. The anonymity threshold k is ﬁxed by the data holder, without considering the speciﬁc privacy requirements of the respondents. As a consequence, this value may be adequate for some respondents and inadequate for others. For instance, consider the 4-anonymous microdata table in Figure 4 and assume that Gabrielle is a female born on 1950/05/02 and living in the 94136 area, who suﬀers from H1N1. A data recipient knowing her quasi-identiﬁer can infer that Gabrielle suﬀers from either H1N1, Flu, Stomach Cancer, or Gastritis, being these values all equally likely (25% of probability each). Although the microdata table satisﬁes 4-anonymity, Gabrielle may not want people to know, with a probability of 25%, that she suﬀers from H1N1 . On the other hand, Lorna, born on 1950/05/05 and living in the 94134 area and suﬀering from Flu, may agree to release her disease without the need to protect the corresponding tuple in the table. Therefore, the 4-anonymous table in Figure 4 does not protect Gabrielle’s privacy, while it over-protects Lorna’s sensitive value. -diversity, t-closeness, m-invariance, and all the other approaches illustrated in the previous sections suﬀer from this problem, since they are based on a unique protection threshold that is adopted to guarantee the privacy of all the respondents. As a consequence, these techniques cannot be adopted for supporting the following privacy requirements.

18

S. De Capitani di Vimercati et al.

– Value-speciﬁc privacy: diﬀerent values of the sensitive attribute should enjoy a diﬀerent protection degree. For instance, consider the medical microdata table in Figure 2(a). Stomach Cancer is usually considered more sensitive than Flu. As a consequence, tuples representing patients suﬀering from Stomach Cancer should be more carefully protected than tuples of patients suﬀering from Flu. – Respondent-speciﬁc privacy: diﬀerent respondents may have a diﬀerent perception of their privacy, independently from the values of the sensitive attribute. As a consequence, respondents may be willing to specify a diﬀerent threshold for the protection of their data. As an example, consider two patients suﬀering from the same disease. A patient may consider the protection oﬀered by a 3-diverse table adequate, while the other patient may require at least a 5-diverse table. – Sensitive presence: the presence of a respondent in the microdata table may violate her privacy. For instance, consider a microdata table representing patients suﬀering from rare diseases. Even the fact that an individual is represented in the table may violate her privacy. To satisfy the privacy requirements of all the respondents, the privacy parameter used by the techniques previously discussed should be ﬁxed to the most restrictive threshold (e.g., to the highest value of k among the preferences of the respondents), thus causing an excessive information loss. 4.2

Solutions

Recently, diﬀerent solutions supporting ﬁne-grained privacy speciﬁcations have been proposed. In the following, for each of the privacy requirements described above, we illustrate a speciﬁc protection technique addressing it. Value-specific privacy (e.g., [18]). To permit the data holder to specify diﬀerent privacy thresholds for diﬀerent values of the domain of the sensitive attribute, in [18] the authors propose (αi , βi )-closeness. (αi , βi )-closeness is an extension of the t-closeness principle that, instead of adopting a unique threshold value t, associates a range [αi , βi ] with each value si in the domain of the sensitive attribute. An equivalence class satisﬁes (αi , βi )-closeness if, for each sensitive value si , the percentage of tuples in the class with value si for the sensitive attribute is in the range [αi , βi ]. A microdata table satisﬁes (αi , βi )-closeness if each equivalence class in the table satisﬁes (αi , βi )-closeness. (αi , βi )-closeness presents diﬀerent advantages over t-closeness. First, (αi , βi )-closeness is ﬂexible, since it deﬁnes a diﬀerent threshold for each sensitive value. As a consequence, only the values that are considered highly sensitive are associated with a small range, while the frequency of non-sensitive values is possibly not bounded, thus better preserving data utility. Second, (αi , βi )-closeness is easy to check, since it does not require to compute the distance between two frequency distributions, but only to evaluate the number of tuples in the equivalence class associated with each sensitive value. Finally, (αi , βi )-closeness is easy to use, since the deﬁnition

Protecting Privacy in Data Release

19

Any

Stomach-Related

Peptic Ulcer

Stomach Cancer

...

Respiratory

Short Breath

Pneumonia

...

Fig. 8. An example of partial taxonomy tree for attribute Disease

of a range [αi , βi ] for each sensitive value is more intuitive than the choice of the maximum distance t from a known frequency distribution. (αi , βi )-closeness is enforced by applying generalization at the level of attributes [18]. In particular, the authors propose to enforce the (αi , βi )-closeness requirement by extending traditional algorithms used for solving the k-anonymity problem (see Section 2) and, in particular, the Incognito algorithm [23]. Respondent-specific privacy (e.g., [42]). In [42] the authors observed that diﬀerent respondents may have diﬀerent privacy requirements, independently from the value of the sensitive attribute associated with them. As a consequence, the adoption of a unique privacy protection threshold to the whole dataset may result in over-protecting or under-protecting respondents. To overcome this issue, in [42] the authors introduce the concept of personalized anonymity that aims at releasing as much information as possible, while satisfying the speciﬁc privacy requirement deﬁned by each respondent. Personalized anonymity is based on the deﬁnition of a taxonomy tree over the domain of each sensitive attribute in the microdata table. The tree has a leaf for each value in the sensitive attribute domain, and each internal node summarizes the speciﬁc values in its subtree. For instance, Figure 8 illustrates an example of a partial taxonomy tree for attribute Disease in the microdata table in Figure 2(a). Each respondent speciﬁes her privacy preference by choosing a guarding node along the path from the root of the taxonomy tree to the leaf node representing the respondent’s sensitive attribute value. The technique proposed in [42] for enforcing the privacy preferences of the respondents relies on the application of generalization, performed in two steps. In the ﬁrst step, the attributes composing the quasi-identiﬁer are generalized following traditional generalization schemes [37]. In the second step, each equivalence class generated in the ﬁrst step is further modiﬁed to respect the privacy level required by each respondent. In particular, for each tuple in an equivalence class, the value of the sensitive attribute is generalized (based on the taxonomy tree deﬁned for the attribute) if needed to respect the guarding node chosen by the tuple respondent. A table to be released satisﬁes personalized anonymity if, for each respondent, a malicious recipient cannot infer with probability higher than p that the respondent is associated with a speciﬁc value in the subtree rooted at the guarding node chosen by the respondent herself. The value of p

20

S. De Capitani di Vimercati et al.

represents the maximum conﬁdence with which a recipient is allowed to infer sensitive attribute values and is set by the data holder. For instance, consider the taxonomy tree in Figure 8 and suppose that Carol , who suﬀers from Stomach Cancer , chooses Stomach-Related disease as her guarding node. Carol ’s privacy is violated if a data recipient can infer with probability higher than p that Carol suﬀers from a disease among Peptic Ulcer , Stomach Cancer , and the other values children of the Stomach-Related node in the taxonomy tree. Another respondent, Matt , who suﬀers from Short Breath, does not consider a privacy violation the release of his disease. As a consequence, he sets his guarding node as ∅ (i.e., a special guarding node denoting that a respondent believes that the release of her sensitive attribute value does not violate her privacy). Sensitive presence (e.g., [34]). In [34], the authors propose δ-presence as a metric to evaluate the risk that a data recipient can identify the presence of an individual in the released table. The released dataset includes a subset of the tuples of a larger data collection D. The released microdata table T ∗ , obtained by generalizing (at the level of attributes) T ⊆ D, satisﬁes δ-presence if δmin ≤ P (t ∈ T |T ∗ ) ≤ δmax , for all t ∈ D, where P (t ∈ T |T ∗ ) is the probability that a data recipient correctly guesses that tuple t belongs to the released dataset, observing the released microdata table T ∗ . If the released dataset satisﬁes δpresence, a malicious recipient cannot determine the inclusion of a respondent in T ∗ with probability lower than δmin or higher than δmax . We note that, by tuning the values of δmin and δmax , it is possible to ﬁnd a good trade-oﬀ between data utility and privacy of the released generalized microdata table T ∗ . In fact, a small [δmin ,δmax ] range favors privacy, while a large [δmin ,δmax ] range favors data utility.

5

Group-Based Approaches for Protecting Sensitive Associations

The approaches described in the previous sections typically apply generalization and suppression for guaranteeing k-anonymity, -diversity, t-closeness, or other (more enhanced) privacy requirements. Generalization and suppression, however, cause information loss that could be reduced by adopting diﬀerent protection techniques. In the following of this section, we highlight the disadvantages caused by generalization and illustrate an alternative approach that overcomes these shortcomings, while protecting sensitive data against disclosure. 5.1

Problem

The adoption of well-known generalization and suppression techniques results in tables that are less complete and less detailed than the original microdata tables. In fact, the released table is composed of a set of equivalence classes, including all the tuples that have been generalized to the same value for the quasi-identiﬁer. The values of the quasi-identiﬁer in the released table are then less precise than

Protecting Privacy in Data Release

21

the values in the original data collection, thus destroying the correlation among the values of the quasi-identifying attributes and the sensitive attribute. The generalization-based approaches are therefore not suitable for those publishing scenarios where the precision of aggregate queries is of paramount importance and the exact distribution of the values of the quasi-identiﬁer must be released. For instance, if the correlation existing among Sex, ZIP, and Disease is important for the analysis of the impact of an infectious disease, the release of the 4-anonymous table in Figure 4 is of limited utility for the ﬁnal recipient, since both Sex and ZIP have been generalized to a unique value. 5.2

Solutions

An alternative technique to generalization that permits the release of the exact distribution of the quasi-identiﬁer values, while guaranteeing to preserve the privacy of the respondents, is fragmentation. Basically, fragmentation consists in splitting the original microdata table in vertical fragments, such that the attributes composing the quasi-identiﬁer and the sensitive attribute are not represented in the same fragment. In the following, we illustrate two diﬀerent solutions adopting a group-based approach to protect the privacy of the respondents. Anatomy [41] is a group-based proposal addressing the issue of guaranteing diversity in microdata release without resorting to generalization. Anatomy ﬁrst partitions the tuples in the microdata table in groups that satisfy the -diversity principle (i.e., each group includes at least well-represented values for the sensitive attribute). Each group is then associated with a unique group identiﬁer and the microdata table is split into two fragments, F1 and F2 , including the attributes composing the quasi-identiﬁer and the sensitive attribute, respectively. For each tuple, both F1 and F2 report the identiﬁer of the group to which it belongs. For simplicity, each group in the fragment storing the sensitive attribute has a tuple for each sensitive value appearing in the group, and reports the frequency with which the value is represented in the group. For instance, consider the microdata table in Figure 2(a) and assume that the data holder is interested in releasing a 3-diverse table. Figure 9 illustrates the two fragments F1 and F2 obtained by partitioning the tuples in the table in Figure 2(a) in groups that satisfy 3-diversity. Although a malicious recipient may know the quasi-identiﬁer value of a target respondent, she can only infer that the respondent belongs to one group (say, g1 ) in F1 , and that the sensitive value of the target respondent is one of the values in the group in F2 that is in relation with g1 . To illustrate, assume that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. Alice can easily infer that her friend is represented by the ninth tuple of the table in Figure 9(a). However, since the tuples in the third group in Figure 9(a) are in relation with the tuples in the third group in Figure 9(b), Alice can only infer that Barbara suﬀers from either Peptic Ulcer , Broken Leg, or Stomach Cancer . Note that the privacy guarantee oﬀered by Anatomy is exactly the same oﬀered by traditional generalization-based approaches. In fact, a malicious data recipient cannot associate less than diﬀerent

22

S. De Capitani di Vimercati et al.

DoB

Sex ZIP GroupID

GroupID Disease

Count

1940/04/01 1940/04/02 1940/06/07 1940/06/05

M M M M

94143 94142 94130 94131

1 1 1 1

1 1 1

Peptic Ulcer Broken Leg Short Breath

2 1 1

1950/06/02 1950/06/05 1950/05/02 1950/05/05

F F F F

94141 94144 94136 94134

2 2 2 2

2 2 2

H1N1 Pneumonia Flu

2 1 1

1940/04/10 1940/04/20 1940/06/25 1940/06/30

F F F F

94139 94130 94142 94145

3 3 3 3

3 3 3

Peptic Ulcer Broken Leg Stomach Cancer

2 1 1

1950/06/20 1950/06/12 1950/05/10 1950/05/30

M M M M

94132 94137 94147 94148

4 4 4 4

4 4 4

Gastritis Dyspepsia Stomach Cancer

2 1 1

(a) F1

(b) F2

Fig. 9. An example of two fragments satisfying 3-diversity obtained adopting the Anatomy approach

sensitive values with each respondent in the released table. On the other hand, by releasing the exact distribution of the values of the attributes composing the quasi-identiﬁer, the evaluation of aggregate queries can be more precise [41]. Loose associations [15] represent a more ﬂexible solution to guarantee privacy in data publication without adopting generalization. Loose associations have been proposed to protect generic sensitive associations among the attributes in a data collection. For instance, consider the microdata table in Figure 1 and suppose that attribute Treatment is also represented in the table. A possible set of sensitive associations deﬁned among attributes {SSN,Name,DoB, Sex,ZIP,Disease,Treatment} could include: i) both the association between the values of attributes SSN and Disease, and the association between the values of attributes Name and Disease; ii) the association between the values of quasiidentifying attributes DoB, Sex, ZIP and the values of sensitive attribute Disease; iii) the association between the values of attributes Disease and Treatment. Given a set of sensitive associations deﬁned among the attributes included in a microdata table, they are broken by publishing a set of diﬀerent fragments. It is easy to see that the problem of protecting the association of a sensitive attribute with the respondents’ quasi-identiﬁer can be modeled through the deﬁnition of a sensitive association among the sensitive attribute and quasi-identifying attributes. Like Anatomy, the original microdata table can then be split in diﬀerent fragments in such a way that the sensitive attribute is not stored together with all the attributes composing the quasi-identiﬁer. It is in fact suﬃcient to store a subset of the quasi-identifying attributes in a fragment F1 , and all the other quasi-identifying attributes in another fragment F2 , together with the sensitive attribute. For instance, consider the microdata table in Figure 2(a). A fragmentation that would protect against identity and attribute disclosures could be

Protecting Privacy in Data Release 1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/06/07 1940/04/10 1940/06/30 1940/04/02 1940/06/05 1940/06/25 1940/04/20 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F M M

94141 94132 94137 94144 94143 94130 94139 94145 94142 94131 94142 94130 94136 94134 94147 94148

•ZZZZZZZZ r• ZZZZZZZZZrZrZrZrZ• • rZZZZZ• •ZZZZZZrZrZ r ZZZZZ r •UUUr • UUUU rr UUUU rr• •r U •ZZZZZZZZ rrr U• rZZZZZ• •UUUU rr rUUU U r r • UUUU • rrr U• • •ZZZZZZZZ ZZZmmm• Z • ZZZZZZmZmmm ZZ• m ZZZZZ• •ZZZZmm mmmm ZZZZZZZZZ • • d• d •ZZZZZZd d d d Z d Z d Z ddddd ZZZZ• • • •

23

Peptic Ulcer H1N1 Gastritis Dyspepsia Peptic Ulcer Pneumonia Broken Leg Stomach Cancer Peptic Ulcer H1N1 Short Breath Broken Leg Peptic Ulcer Stomach Cancer Flu Gastritis

Fig. 10. An example of loose association deﬁned on the table in Figure 2(a)

composed of the following two fragments: F1 (DoB,Sex,ZIP) and F2 (Disease). Note that a fragmentation is not unique: F1 (DoB,Sex) and F2 (ZIP,Disease) is another solution that still protects the association between the sensitive attribute and the quasi-identiﬁer. To provide the data recipient with some information on the associations between the quasi-identiﬁer and the sensitive attribute values existing in the original relation, provided a given privacy degree of the association is respected, in [15] the authors propose to publish a loose association between the tuples composing F1 and F2 . The tuples in F1 and in F2 are independently partitioned in groups of size at least k1 and k2 , respectively. Each group in F1 and in F2 is then associated with a diﬀerent group identiﬁer. For each tuple, both F1 and F2 report the identiﬁer of the group to which the tuple belongs. The group-level relationships between the tuples in F1 and in F2 are represented by an additional table A that includes, for each tuple t in the original microdata table, a tuple modeling the relationship between the group where t appears in F1 and the group where t appears in F2 . For instance, Figure 10 represents two fragments F1 and F2 for the microdata table in Figure 2(a). Both the fragments have been partitioned into groups of 2 tuples each and the lines between the tuples in F1 and F2 represent their relationships in the original microdata table. Figure 11 illustrates the three relations, F1 , A, and F2 that are released instead of the original microdata. It is easy to see that, even if a malicious recipient knows the quasi-identiﬁer of a respondent, she can only identify the tuple related to the target respondent in F1 , but not the corresponding Disease in F2 . For instance, assume that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. By looking at the released tables, Alice discovers that her friend is represented by the seventh tuple in F1 , which belongs to group dsz4. However, since group dsz4 is associated in A with two diﬀerent groups in F2 (i.e., d4 and d5) Alice cannot identify the illness Barbara suﬀers from, since it could be either Peptic Ulcer , Broken Leg, Stomach Cancer , or H1N1.

24

S. De Capitani di Vimercati et al. F1 DoB 1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

A

Sex ZIP F M M F M M F F M M F F F F M M

(a)

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94145 94136 94134 94147 94148

G dsz1 dsz1 dsz2 dsz2 dsz3 dsz5 dsz4 dsz6 dsz3 dsz5 dsz6 dsz4 dsz7 dsz7 dsz8 dsz8

F2

G1 G2

Disase

G

dsz1 dsz1 dsz2 dsz2 dsz3 dsz3 dsz4 dsz4 dsz5 dsz5 dsz6 dsz6 dsz7 dsz7 dsz8 dsz8

H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

d1 d2 d2 d3 d1 d3 d5 d7 d4 d6 d6 d4 d5 d8 d7 d8

d1 d2 d2 d3 d1 d4 d5 d4 d3 d6 d7 d6 d5 d8 d7 d8

(b)

(c)

Fig. 11. An example of 4-loose association

The partitioning of the tuples in the two fragments should be carefully designed to guarantee an adequate protection degree. In fact, a loose association enjoys a degree k of protection if every tuple in A indistinguishably corresponds to at least k distinct associations among tuples in the two fragments (i.e., it could have been generated starting from k diﬀerent tuples in the microdata table). The release of F1 , F2 , and A satisﬁes k-looseness, with k ≤ k1 · k2 , if for each group g1 in F1 (group g2 in F2 , respectively), the union of the tuples in all the groups with which g1 (g2 , respectively) is associated in A is a set of at least k diﬀerent tuples. Figure 11 represents an example of a 4-loose association. This implies that it is not possible, for a malicious data recipient, to associate with each quasi-identiﬁer value in F1 less than 4 diﬀerent diseases in F2 .

6

Microdata Publishing with Adversarial External Knowledge

Another source of complication for the protection of microdata is the external (or background) knowledge that an adversary may exploit for inferring information about the respondents represented in the microdata. The research community has recently dedicated many eﬀorts for counteracting this problem whose diﬃculty lies in the fact that the data holder is unaware of the type of knowledge an adversary may have. In this section, we illustrate the privacy problems that may arise due to the adversarial external knowledge, and we present recent protection techniques speciﬁcally designed to consider such knowledge. 6.1

Problem

When publishing a microdata table it is necessary to take into consideration the fact that a malicious recipient may exploit, besides the released table, also

Protecting Privacy in Data Release

25

additional information to infer the sensitive attribute value associated with a target respondent. This knowledge can be obtained by similar data collections released by other organizations or competitors, by social networking sites, or by personal knowledge. For instance, consider the microdata table in Figure 5 and suppose that Alice knows that her neighbor Gabrielle is a female born on 1950/05/02 and living in the 94136 area. If Alice does not have any additional knowledge, by looking at the released microdata table she can only infer that her friend suﬀers from either H1N1 , Pneumonia, or Flu. However, Alice can improve this inference by exploiting external sources of information. For instance, Alice is a close friend of Liz (who appears in the same equivalence class as Gabrielle in the table in Figure 5) and therefore knows that she suﬀers from Pneumonia. As a consequence, Alice can infer that Gabrielle either suﬀers from Flu or H1N1 . However, Gabrielle’s sister Hellen has been recently hospitalized for having contracted H1N1 , and she posted it on her Facebook proﬁle. Since H1N1 is a contagious disease, Alice can infer with high probability that also Gabrielle contracted H1N1 . The problem of protecting released data against malicious recipients exploiting external knowledge has been acknowledged since the ﬁrst introduction of proposals addressing privacy issues in microdata release (see Section 2). However, traditional approaches only consider a few speciﬁc types of external knowledge. For instance, k-anonymity assumes that data recipients only know publicly available datasets associating the identity of respondents with their quasi-identiﬁer. -diversity considers also the fact that a recipient may have additional (personal) knowledge about a subset of the respondents, which permits her to discard a subset of the sensitive values in the equivalence class of a target respondent. Taking external knowledge into consideration when releasing a microdata collection requires the deﬁnition of an adequate modeling of the knowledge that a malicious recipient may possess. This task is complicated by the fact that it is not realistic to assume that data holders have complete knowledge of all the data available to recipients. Furthermore, information is collected and publicly released every day and, consequently, the external information that could be exploited for re-identifying respondents is continuously changing. The external knowledge modeling should therefore be ﬂexible enough to possibly capture any kind of information that might be available to the data recipient. 6.2

Solutions

We summarize some of the most important results modeling the external knowledge of an adversary. (c,k)-Safety [30] introduces a formal modeling of external knowledge assuming a worst case scenario, where the malicious recipient knows the set of respondents represented by a tuple in the published microdata table, and the values of both quasi-identifying and non-sensitive attributes associated with each respondent.

26

S. De Capitani di Vimercati et al.

Besides this identiﬁcation information, a malicious recipient may also possess additional information modeled with the concept of basic unit of knowledge. A basic unit of knowledge is deﬁned as an implication formula (∧i Ai ) → (∨j Bj ), where Ai and Bj are atoms of the form tp [S] = s that represent the fact that respondent p is associated with value s for sensitive attribute S. For instance, suppose that Alice knows that Bob and Carol , who are married, are both sick. Since Flu is highly contagious, Alice knows that if Bob has Flu also Carol suﬀers from the same disease. This knowledge can be modeled through the following basic unit: tBob [Disease]=Flu→tCarol[Disease]=Flu. To avoid unrealistic scenarios where the malicious recipient has unbounded external knowledge, in [30] the authors assume that the knowledge available to a data recipient is composed of at most k basic units of knowledge. Therefore, the overall external knowledge of a recipient is represented as the conjunction ϕ of all her basic units of knowledge. Given the released microdata table T (either obtained using a generalization-based or a group-based approach), the maximum disclosure risk to which the released table is exposed can be computed as the maximum probability that a data recipient with knowledge ϕ can infer that respondent p is associated with value s for sensitive attribute S. More formally, the maximum disclosure of table T is deﬁned as max P (tp [S] = s|T ∧ ϕ), computed on the tuples tp of table T that are consistent with external knowledge ϕ, and that take value s in the domain of S. A released table T satisﬁes (c, k)-safety if the maximum disclosure risk of T is lower than a threshold 0 ≤ c ≤ 1 ﬁxed by the data holder, assuming that malicious recipients have at most k basic units of external knowledge. Given a microdata table, the goal is therefore to determine a corresponding (c, k)-safety microdata table that minimizes the loss of information due to the adoption of generalization-based or group-based approaches. In [30] the authors note that traditional algorithms proposed to guarantee kanonymity can be easily adapted to guarantee (c, k)-safety, since the maximum disclosure is monotonic with respect to generalization. The authors also note that Anatomy [41] can be easily adapted to guarantee (c, k)-safety, since the disclosure decreases merging the groups of tuples in the fragments. Privacy skyline [6] introduces a more intuitive and usable approach than (c, k)safety to measure the external knowledge available to a data recipient. In fact, the deﬁnition of an adequate value of k modeling the adversarial knowledge is not intuitive. Also, (c, k)-safety is based on the assumption that each data respondent is associated with a unique sensitive value and that all the values in the sensitive attribute domain are equally sensitive. Privacy skyline tries to overcome these limitations. The basic idea is that external knowledge can be seen as composed of several categories, which need to be quantiﬁed adopting diﬀerent measures. As a consequence, the disclosure risk cannot be a numeric value, but it is decomposed according to the external knowledge components that are considered relevant for the speciﬁc scenario. In [6], the authors classify the external knowledge in the following three categories:

Protecting Privacy in Data Release

27

– the knowledge about a target individual (e.g., Alice knows that Bob does not suﬀer from Cancer ); – the knowledge about individuals other than the target individual (e.g., Alice knows that Carol , who is also a respondent of the considered microdata table, suﬀers from H1N1 ); – the knowledge about same-value families that model the fact that a group (or family) of respondents have the same value for the sensitive attribute (e.g., Alice knows that David and Hellen are married and, therefore, if any of them suﬀers from Flu, it is highly probable that also the other person suﬀers from Flu). It is important to note that the composition of a same-value family is value speciﬁc, meaning that the relationship among users may be diﬀerent depending on the speciﬁc sensitive value s considered. For instance, the same-value family for an infectious disease may be composed of colleagues and people leaving in the same house, while the same-value family for HIV may only include a married couple. The external knowledge available to a malicious recipient is expressed as a triple (,k,m) that quantify, for each of the categories illustrated above, the amount of knowledge held by the recipient for each sensitive value s. A triple (,k,m) indicates that the recipient knows: sensitive values that the target respondent does not have; the sensitive value associated with k individuals diﬀerent from the target respondent; and m individuals in the same-value family of the target respondent. For instance, suppose that, with respect to respondent Ron and sensitive value H1N1, Alice knows that Ron does not suﬀer from Ovarian Cancer and Measles ( = 2). Also, suppose that Alice already knows the diseases of other three respondents represented in the table where Ron is represented (k = 3). Finally, suppose that Alice knows that other ﬁve people work in Ron’s oﬃce (m = 6). As a consequence, the triple expressing such external knowledge of Alice is (2,3,6). The release of a microdata table T is safe if the maximum probability that a data recipient can infer that an arbitrary respondent p is associated with sensitive value s in the original dataset is lower than a threshold c ﬁxed by the data holder, assuming the recipient’s external knowledge to be bounded by (, k, m). For instance, assume that the external knowledge of a malicious recipient for an arbitrary target respondent p and sensitive value H1N1 is represented by the triple (2,5,1) and that c=0.5. A released table is considered safe if the malicious recipient cannot determine with probability higher than 0.5 that p suﬀers from H1N1 , provided the malicious user knows: at most 2 sensitive attribute values that p does not have; the sensitive value of at most 5 respondents other than p; and at most 1 member in the same-value family as p. To provide the data holder with a more ﬂexible privacy measure, in [6] the authors propose to bound the external knowledge not through a unique triple (,k,m), but through a set of triples (representing incomparable points in the three-dimensional space ,k,m) that form a skyline. The release of a microdata table T , protected applying generalization or group-based techniques, is safe if the probability that the malicious recipient violates the privacy of the respondents is lower than

28

S. De Capitani di Vimercati et al.

4

(1,4)

≤1 (3,3)

3 m 2 (5,1)

1

1

2

3 k

4

5

Fig. 12. An example of privacy skyline obtained ﬁxing three points {(1,1,4), (1,3,3),(1,5,1)}

threshold c, assuming that the external knowledge of the recipient is bounded by the skyline (e.g., the grey area in Figure 12, depicted in only two dimensions for simplicity, represents the area in which the external knowledge of the recipient is bounded). In [6] the authors extend their deﬁnition of safe release to take into account the fact that each respondent may be associated with more than one sensitive attribute value (e.g., a respondent that suﬀers from more than one disease). -Privacy [28] is aimed at deﬁning a realistic adversarial model. Indeed, traditional approaches either assume that the adversary has a very limited knowledge, or an inﬁnite knowledge. Both these assumptions are however unrealistic. To overcome this issue, in [28] the authors propose to measure the privacy degree oﬀered to a respondent p as the diﬀerence in the recipient’s belief about the respondent’s sensitive attribute value when p belongs to the released table and when p does not belong to the same. This measure is based on the observation that looking at the released dataset, a data recipient can learn information about the sensitive value of a respondent p, even if the tuple representing p is not released [22]. Therefore, it is necessary to consider this unavoidable gain of information when evaluating the quality of a released table. In particular, -privacy states that the release of a table T obtained through generalization exposes the in privacy of a respondent if the ratio ppout is greater than a threshold ﬁxed by the data holder, where pin (pout , respectively) is the probability that a recipient infers the sensitive value associated with an arbitrary respondent p when p is not represented in T (belongs to T , respectively). To compute pin and pout , the authors consider two diﬀerent kinds of external knowledge: i) full information on a subset of the tuples in T ; and ii) information on the distribution of the sensitive attribute values in the dataset. We note that, although both (c, k)-safety and privacy skyline consider the ﬁrst kind of external knowledge, they do not make any assumption on the second kind of knowledge modeled by -privacy.

Protecting Privacy in Data Release

7

29

Diﬀerential Privacy

The proposals described so far typically measure the disclosure risk associated with the release of a microdata table as the increase of the probability that an adversary may correctly guess the identity or the values of sensitive attributes of a respondent represented in the table. Such approaches do not consider that the microdata table can also be exploited for inferring information of respondents that are not represented in the table. In the remainder of this section, we ﬁrst illustrate in more details such a privacy issue, and then brieﬂy describe diﬀerential privacy, a recent privacy notion that is becoming popular in the data protection community. 7.1

Problem

One of the ﬁrst deﬁnitions of privacy in data publishing scenarios states that: anything that can be learned about a respondent from the statistical database should be learnable without access to the database [13]. Although this deﬁnition has been thought for statistical databases, it is also well suited for the microdata publishing scenario. Unfortunately, this deﬁnition of ideal privacy cannot be achieved by any privacy-aware microdata publication technique that is aimed at preserving data utility. In fact, as proved in [16], only an empty dataset can guarantee absolute disclosure prevention. This is also due to the fact that the release of a dataset may violate the privacy of any respondent, independently of whether the respondent is represented in the dataset. For instance, suppose that the released dataset permits to compute the average annual income of people living in city A for each ethnic group, and suppose that this information is not publicly available (and therefore a malicious recipient can only gain this information by looking at the released table). Assume also that Alice knows that Bob’s annual income is 1,000$ more than the average annual income of Asian people living in city A. Although this piece of information alone does not permit Alice to gain any information about Bob’s annual income, if combined with the released dataset, it allows Alice to infer Bob’s annual income. It is important to note that the disclosure of Bob’s annual income does not depend on his representation in the released dataset. The solutions proposed in the literature for protecting microdata tables implicitly assume that the privacy of individuals not included in the dataset is not at risk. As a consequence, they cannot be adopted to prevent the attack described above. 7.2

Solution

Diﬀerential privacy [16] is a novel privacy notion whose goal is to guarantee that the release of a microdata table does not disclose sensitive information about any individual, represented or not by a tuple in the table. In particular, a data release is considered safe if the inclusion in the dataset of tuple tp , related to respondent p, does not change the probability that a malicious recipient can

30

S. De Capitani di Vimercati et al.

correctly identify the sensitive attribute value associated with p. More formally, given two datasets T and T diﬀering only for one tuple tp , an arbitrary randomized function K operating on the dataset satisﬁes -diﬀerential privacy if and only if P (K(T ) ∈ S) ≤ exp() · P (K(T ) ∈ S), where S is a subset of the possible outputs of function K and is a public privacy parameter. Intuitively, -diﬀerential privacy holds if the removal (insertion, respectively) of one tuple tp from (into, respectively) the dataset does not signiﬁcatively aﬀect the result of the evaluation of function K. As an example, consider an insurance company that consults a medical dataset to decide whether an individual p is eligible for an insurance contract. If diﬀerential privacy is satisﬁed, the presence or absence of tuple tp representing p in the dataset does not signiﬁcantly aﬀect the ﬁnal decision by the insurance company. It is also important to note that the external knowledge that an adversary may possess cannot be exploited for breaching the privacy of individuals. In fact, the knowledge that the recipient gains looking at the released dataset is bounded by a multiplicative factor exp(), for any individual either represented or not in the released microdata table. Diﬀerential privacy is applicable to both the non-interactive publishing scenario (i.e., public release of a dataset) and the interactive publishing scenario (i.e., evaluation of queries over a private dataset). The techniques proposed in the literature to guarantee -diﬀerential privacy are based on the addition of noise, and therefore do not preserve data truthfulness. To achieve diﬀerential privacy, by deﬁnition, it is necessary to hide the presence (or absence) of the tuple associated with an individual. Therefore, considering simple count queries, the query result returned to the requesting recipient may diﬀer by at most one (0 if the tuple is not removed; 1 otherwise) from the result computed on the original dataset. To compute the query result, the solution proposed in [17] consists in adding random noise to the query result evaluated on the original dataset. The distribution considered for the random noise is the Laplace distribution Lap(Δ(f )/) with probability density function P (x|b) = exp(−|x|/b)/2b, where b = Δ(f )/ and Δ(f ) is the maximum diﬀerence between the query result evaluated over T and over T (which is equal to 1 for count queries, since T and T diﬀer for one tuple). The addition of independently generated noise, with distribution Lap(Δ(f )/), to the query result guarantees -diﬀerential privacy [17]. This strategy can also be adopted in a non-interactive scenario, where the data holder releases the frequency matrix representing the dataset and each cell in the matrix is the result of a count query. The frequency matrix has a dimension for each attribute in the table and the entries in each dimension are labeled with the values in the attribute domain. Each cell in the matrix reports the number of tuples in the table with value, for each attribute, equal to the label of the corresponding entry in the frequency matrix.

8

Conclusions

The public and semi-public release of large microdata collections is becoming increasingly popular, thanks to the high availability of computational power at

Protecting Privacy in Data Release

31

low prices, which makes data analysis an easy task for most data recipients. Although microdata collections represent a valuable resource for data recipients, the release of ﬁne-grained information related to single individuals may put the privacy of respondents at risk. In this chapter, we ﬁrst illustrated the traditional approaches designed for preventing identity and attribute disclosure in microdata publishing. We then discussed the privacy risks that may arise when changing the underlying assumptions, and described some techniques recently proposed in the literature to overcome these privacy risks. Acknowledgments. This work was supported in part by the EU within the 7FP project “PrimeLife” under grant agreement 216483, by the Italian Ministry of Research within the PRIN 2008 project “PEPPER” (2008SY2PH4), and by the Universit` a degli Studi di Milano within the “UNIMI per il Futuro - 5 per Mille” project “PREVIOUS”.

References 1. Aggarwal, G., Feder, T., Kenthapadi, K., Motwani, R., Panigrahy, R., Thomas, D., Zhu, A.: Approximation algorithms for k-anonymity. Journal of Privacy Technology (November 2005) 2. Azzini, A., Marrara, S., Sassi, R., Scotti, F.: A fuzzy approach to multimodal biometric continuous authentication. Fuzzy Optimization and Decision Making 7(3), 215–302 (2008) 3. Barbaro, M., Zeller, T.: A face is exposed for AOL searcher no. 4417749. New York Times, August 9 (2006) 4. Bayardo, R.J., Agrawal, R.: Data privacy through optimal k-anonymization. In: Proc. of the 21st IEEE International Conference on Data Engineering (ICDE 2005), Tokyo, Japan (April 2005) 5. Bezzi, M., De Capitani di Vimercati, S., Livraga, G., Samarati, P.: Protecting privacy of sensitive value distributions in data release. In: Proc. of the 6th Workshop on Security and Trust Management (STM 2010), Athens, Greece (September 2010) 6. Chen, B.C., LeFevre, K., Ramakrishnan, R.: Privacy skyline: Privacy with multidimensional adversarial knowledge. In: Proc. of the 33rd International Conference on Very Large Data Bases (VLDB 2007), Vienna, Austria (September 2007) 7. Cimato, S., Gamassi, M., Piuri, V., Sassi, R., Scotti, F.: Privacy-aware biometrics: Design and implementation of a multimodal veriﬁcation system. In: Proc. of the 24th Annual Computer Security Applications Conference (ACSAC 2008), Anaheim, CA, USA (December 2008) 8. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Fragmentation design for eﬃcient query execution over sensitive distributed databases. In: Proc. of the 29th International Conference on Distributed Computing Systems (ICDCS 2009), Montreal, Canada (June 2009) 9. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Combining fragmentation and encryption to protect privacy in data storage. ACM Transactions on Information and System Security (TISSEC) 13(3), 22:1–22:33 (2010)

32

S. De Capitani di Vimercati et al.

10. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: k-Anonymity. In: Yu, T., Jajodia, S. (eds.) Secure Data Management in Decentralized Systems. Springer, Heidelberg (2007) 11. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: Microdata protection. In: Yu, T., Jajodia, S. (eds.) Secure Data Management in Decentralized Systems. Springer, Heidelberg (2007) 12. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: Theory of privacy and anonymity. In: Atallah, M., Blanton, M. (eds.) Algorithms and Theory of Computation Handbook, 2nd edn. CRC Press, Boca Raton (2009) 13. Dalenius, T.: Towards a methodology for statistical disclosure control. Statistik Tidskrift 15, 429–444 (1977) 14. De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Encryption policies for regulating access to outsourced data. ACM Transactions on Database Systems (TODS) 35(2), 12:1–12:46 (2010) 15. De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Fragments and loose associations: Respecting privacy in data publishing. Proc. of the VLDB Endowment 3(1), 1370–1381 (2010) 16. Dwork, C.: Diﬀerential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006) 17. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating Noise to Sensitivity in Private Data Analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006) 18. Frikken, K.B., Zhang, Y.: Yet another privacy metric for publishing micro-data. In: Proc. of the 7th Workshop on Privacy in Electronic Society (WPES 2008), Alexandria, VA, USA (October 2008) 19. Gamassi, M., Lazzaroni, M., Misino, M., Piuri, V., Sana, D., Scotti, F.: Accuracy and performance of biometric systems. In: Proc. of the 2004 IEEE Instrumentation & Measurement Technology Conference (IMTC 2004), Como, Italy (May 2004) 20. Gamassi, M., Piuri, V., Sana, D., Scotti, F.: Robust ﬁngerprint detection for access control. In: Proc. of the 2nd RoboCare Workshop (RoboCare 2005), Rome, Italy (May 2005) 21. Golle, P.: Revisiting the uniqueness of simple demographics in the US population. In: Proc. of the 5th Workshop on Privacy in the Electronic Society (WPES 2006), Alexandria, VA, USA (October 2006) 22. Kifer, D.: Attacks on privacy and deFinetti’s theorem. In: Proc. of the 2009 ACM SIGMOD International Conference on Management of Data (SIGMOD 2009), Providence, RI, USA (June 2009) 23. LeFevre, K., DeWitt, D.J., Ramakrishnan, R.: Incognito: Eﬃcient full-domain kanonymity. In: Proc. of the 2005 ACM SIGMOD International Conference on Management of Data (SIGMOD 2005), Baltimore, MD, USA (June 2005) 24. LeFevre, K., DeWitt, D.J., Ramakrishnan, R.: Mondrian multidimensional kanonymity. In: Proc. of the 22nd IEEE International Conference on Data Engineering (ICDE 2006), Atlanta, GA, USA (April 2006) 25. Li, F., Sun, J., Papadimitriou, S., Mihaila, G., Stanoi, I.: Hiding in the crowd: Privacy preservation on evolving streams through correlation tracking. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 26. Li, N., Li, T., Venkatasubramanian, S.: t-closeness: Privacy beyond k-anonymity and -diversity. In: Proc.of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007)

Protecting Privacy in Data Release

33

27. Li, T., Li, N.: On the tradeoﬀ between privacy and utility in data publishing. In: Proc. of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2009), Paris, France (June-July 2009) 28. Machanavajjhala, A., Gehrke, J., G¨ otz, M.: Data publishing against realistic adversaries. Proc. of the VLDB Endowment 2(1), 790–801 (2009) 29. Machanavajjhala, A., Kifer, D., Gehrke, J., Venkitasubramaniam, M.: -diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD) 1(1), 3:1–3:52 (2007) 30. Martin, D.J., Kifer, D., Machanavajjhala, A., Gehrke, J., Halpern, J.Y.: Worstcase background knowledge for privacy-preserving data publishing. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 31. Meyerson, A., Williams, R.: On the complexity of optimal k-anonymity. In: Proc. of the 23rd ACM-SIGMOD-SIGACT-SIGART Symposium on the Principles of Database Systems (PODS 2004), Paris, France (June 2004) 32. Narayanan, A., Shmatikov, V.: Robust de-anonymization of large sparse datasets. In: Proc. of the 2008 IEEE Symposium on Security and Privacy (SP 2008), Berkeley/Oakland, CA, USA (May 2008) 33. Narayanan, A., Shmatikov, V.: Myths and fallacies of personally identiﬁable information. Communications of the ACM (CACM) 53, 24–26 (2010) 34. Nergiz, M.E., Atzori, M., Clifton, C.: Hiding the presence of individuals from shared databases. In: Proc. of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD 2007), Beijing, China (June 2007) 35. Nergiz, M., Clifton, C., Nergiz, A.: Multirelational k-anonymity. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 36. Pei, J., Tao, Y., Li, J., Xiao, X.: Privacy preserving publishing on multiple quasiidentiﬁers. In: Proc. of the 25th IEEE International Conference on Data Engineering (ICDE 2009), Shanghai, China (March-April 2009) 37. Samarati, P.: Protecting respondents’ identities in microdata release. IEEE Transactions on Knowledge and Data Engineering (TKDE) 13(6), 1010–1027 (2001) 38. Terrovitis, M., Mamoulis, N., Kalnis, P.: Privacy-preserving anonymization of setvalued data. Proc. of the VLDB Endowment 1, 115–125 (2008) 39. Wang, K., Fung, B.C.M.: Anonymizing sequential releases. In: Proc. of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2006), Philadelphia, PA, USA (August 2006) 40. Wang, K., Xu, Y., Wong, R., Fu, A.: Anonymizing temporal data. In: Proc. of the 2010 IEEE International Conference on Data Mining (ICDM 2010), Sydney, Australia (December 2010) 41. Xiao, X., Tao, Y.: Anatomy: Simple and eﬀective privacy preservation. In: Proc. of the 32nd International Conference on Very Large Data Bases (VLDB 2006), Seoul, Korea (September 2006) 42. Xiao, X., Tao, Y.: Personalized privacy preservation. In: Proc. of the 2006 ACM SIGMOD International Conference on Management of Data (SIGMOD 2006), Chicago, IL, USA (June 2006) 43. Xiao, X., Tao, Y.: m-invariance: Towards privacy preserving re-publication of dynamic datasets. In: Proc. of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD 2007), Beijing, China (June 2007)

34

S. De Capitani di Vimercati et al.

44. Xiao, X., Yi, K., Tao, Y.: The hardness and approximation algorithms for diversity. In: Proc. of the 13th International Conference on Extending Database Technology (EDBT 2010), Lausanne, Switzerland (March 2010) 45. Zhou, B., Han, Y., Pei, J., Jiang, B., Tao, Y., Jia, Y.: Continuous privacy preserving publishing of data streams. In: Proc. of the 12th International Conference on Extending Database Technology (EDBT 2009), Saint Petersburg, Russia (March 2009)

An Introduction to Security API Analysis Riccardo Focardi1 , Flaminia L. Luccio1 , and Graham Steel2 1 2

1

DAIS, Universit` a Ca’ Foscari Venezia, Italy {focardi,luccio}@dsi.unive.it LSV, ENS Cachan & CNRS & INRIA, France [email protected]

Introduction

A security API is an Application Program Interface that allows untrusted code to access sensitive resources in a secure way. Examples of security APIs include the interface between the tamper-resistant chip on a smartcard (trusted) and the card reader (untrusted), the interface between a cryptographic Hardware Security Module, or HSM (trusted) and the client machine (untrusted), and the Google maps API (an interface between a server, trusted by Google, and the rest of the Internet). The crucial aspect of a security API is that it is designed to enforce a policy, i.e. no matter what sequence of commands in the interface are called, and no matter what the parameters are, certain security properties should continue to hold. This means that if the less trusted code turns out to be malicious (or just faulty), the carefully designed API should prevent compromise of critical data. Designing such an interface is extremely tricky even for experts. A number of security ﬂaws have been found in APIs in use in deployed systems in the last decade. In this tutorial paper, we introduce the subject of security API analysis using formal techniques. This approach has recently proved highly successful both in ﬁnding new ﬂaws and verifying security properties of improved designs. We will introduce the main techniques, many of which have been adapted from language-based security and security protocol veriﬁcation, by means of two case studies: cryptographic key management, and Personal Identiﬁcation Number (PIN) processing in the cash machine network. We will give plenty of examples of API attacks, and highlight the areas where more research is needed.

2

Background

The ﬁrst security vulnerability that may properly be called an ‘API attack’, and thus highlighted the security API as a critical design point, was discovered by Longley and Rigby in the early 1990s [41]. Their article showed how the logic programming language Prolog could be used to analyse a key management interface of a cryptographic device. Although the device was not identiﬁed at

Work partially supported by the RAS Project “TESLA: Techniques for Enforcing Security in Languages and Applications”.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 35–65, 2011. c Springer-Verlag Berlin Heidelberg 2011

36

R. Focardi, F.L. Luccio, and G. Steel

the time, it later became known that it was an HSM manufactured by Eracom and used in the cash machine network. In 2000, Anderson published an attack on key loading procedures on another similar module manufactured by Visa [5], and the term ‘security API’ was coined by Bond and Anderson [12,13] in two subsequent papers giving more attacks. Clayton and Bond showed how one of their more computationally intensive attacks could be implemented against a real IBM device using programmable FPGA hardware [21]. Independently from the Cambridge group, an MSc thesis by Clulow gave more examples of attacks, mostly speciﬁc to the PIN translation and veriﬁcation commands oﬀered by the API of Prism HSMs [22]. Clulow also published attacks on the industry standard for cryptographic key management APIs, RSA PKCS#11 [23]. Up until this point all the attacks had been discovered by manual analysis or by ad-hoc semi-formal techniques speciﬁc to the particular API under consideration. A ﬁrst eﬀort to apply more general formal tools, speciﬁcally the automatic ﬁrst-order theorem prover Otter, was not especially successful, and the results remain unpublished (though they are available in a technical report [52]). The researchers were unable to discover any new attacks, and because the modelling lacked formal groundwork, when no attacks were found they were unable to conclude anything about the security of the device. One member of the team later remarked that “It ended up being more about how to use the tools than about analysing the device” [35]. Meanwhile, the formal analysis of protocols for e.g. cryptographic key exchange and authentication had become a mature ﬁeld. A particularly successful approach had centred around the so-called Dolev-Yao (DY) abstract model, where bitstrings are modelled as terms in an abstract algebra, and cryptographic functions are functions on these terms [27]. Together with suitable abstractions, lazy evaluation rules and other heuristics, this model has proved highly amenable to automated analysis, by model checking or theorem proving techniques [8,11,28,42]. Modern automated tools can check secrecy and authentication properties of (abstract models of) widely-used protocols such as Kerberos and TLS in a few seconds. The idea of applying protocol analysis techniques to the analysis of security APIs seemed very attractive. However, initial experiments showed that existing tools were not suitable for the problem [15,38] for a number of reasons: In particular many of the attacks pertinent to security APIs are outside the scope of the normal DY model. For example, they might involve an attacker learning a secret value, such as a cash machine PIN, by observing error messages returned by the API (a so-called error oracle attack). Furthermore, the functionality of security APIs typically depends on global mutable state which may loop, a feature which invalidates many abstractions and optimisations made by protocol analysis tools, particularly when freshly generated nonces and keys are considered. There are also problems of scale - a protocol might describe an exchange of 5 messages between two participants, while an API will typically oﬀer dozens of commands.

An Introduction to Security API Analysis

37

In the sections that follow, we will show how formal analysis techniques have been adapted to security API problems. The aim of this paper is to serve as an introduction to the ﬁeld. As such, the technical material is introduced by means of two case studies. The ﬁrst is based around the RSA standard for cryptographic interfaces PKCS#11, while the second concerns commands for PIN processing using HSMs in the cash machine network. We follow these two case studies with a discussion of open problems.

3

Key Management with RSA PKCS#11

Cryptographic key management, i.e. the secure creation, storage, backup, use and destruction of keys has long been identiﬁed as a major challenge in applied cryptography. Indeed, Schneier calls it “the hardest part of cryptography and often the Achilles’ heel of an otherwise secure system.” [48]. In real-world applications, key management often involves the use of HSMs or cryptographic tokens, since these are considered easier to secure than commodity hardware, and indeed are mandated by standards in certain sectors [37]. There is also a growing trend towards enterprise-wide schemes based around key management servers oﬀering cryptographic services over open networks [18]. All these solutions aim to enforce security by dividing the system into trusted parts (HSM, server) and untrusted parts (host computer, the rest of the network). The trusted part makes cryptographic functions available via an API. Hence we arrive at our ﬁrst security API problem: how to design an interface for a key management device so that we can create, delete, import and export keys from the device, as well as permitting their use for encryption, decryption, signature and veriﬁcation, all so that if the device comes into contact with a malicious application we can be sure the keys stay secure. This is far from trivial, as well will see from our case study of the most widely used standard for such interfaces, RSA PKCS#11. 3.1

The PKCS#11 Standard

RSA Public Key Cryptography Standards (PKCS) aim to standardise various aspects of cryptography to promote interoperability and security. PKCS#1, for example, deﬁnes the RSA asymmetric encryption and signing algorithm. PKCS#11 describes the ‘Cryptoki’ API for cryptographic hardware. Version 1.0 was published in 1995. The latest oﬃcial version is v2.20 (2004) which runs to just under 400 pages [47]. Adoption of the standard is almost ubiquitous in commercial cryptographic tokens and smartcards, even if other interfaces are frequently oﬀered in addition. In a PKCS#11-based API, applications initiate a session with the cryptographic token, by supplying a PIN. Note that if malicious code is running on the host machine, then the user PIN may easily be intercepted, e.g. by a keylogger or by a tampered device driver, allowing an attacker to create his own sessions with the device, a point conceded in the security discussion in the standard [47, p. 31]. PKCS#11 is intended to protect its sensitive cryptographic keys even when connected to a compromised host.

38

R. Focardi, F.L. Luccio, and G. Steel

Once a session is initiated, the application may access the objects stored on the token, such as keys and certiﬁcates. However, access to the objects is controlled. Objects are referenced in the API via handles, which can be thought of as pointers to or names for the objects. In general, the value of the handle, e.g. for a secret key, does not reveal any information about the actual value of the key. Objects have attributes, which may be bitstrings e.g. the value of a key, or Boolean ﬂags signalling properties of the object, e.g. whether the key may be used for encryption, or for encrypting other keys. New objects can be created by calling a key generation command, or by ‘unwrapping’ an encrypted key packet. In both cases a new handle is returned. When a function in the token’s API is called with a reference to a particular object, the token ﬁrst checks that the attributes of the object allow it to be used for that function. For example, if the encrypt function is called with the handle for a particular key, that key must have its encrypt attribute set. To protect a key from being revealed, the attribute sensitive must be set to true. This means that requests to view the object’s key value via the API will result in an error message. Once the attribute sensitive has been set to true, it cannot be reset to false. This gives us the principal security property stated in the standard: attacks, even if they involve compromising the host machine, cannot “compromise keys marked ‘sensitive’, since a key that is sensitive will always remain sensitive”, [47, p. 31]. Such a key may be exported outside the device if it is encrypted by another key, but only if its extractable attribute is set to true. An object with an extractable attribute set to false may not be read by the API, and additionally, once set to false, the extractable attribute cannot be set to true. Protection of the keys essentially relies on the sensitive and extractable attributes. 3.2

The Wrap-Decrypt Attack

Clulow ﬁrst published attacks on PKCS#11 based APIs in 2003 [23], where he gave many examples of ways in which keys with the sensitive attribute set to true could be read in clear outside the device. The most straightforward of these is the ‘key separation’ attack, where the attributes of a key are set in such a way as to give a key conﬂicting roles. Clulow gives the example of a key with the attributes set for decryption of ciphertexts, and for ‘wrapping’, i.e. encryption of other keys for secure transport. To determine the value of a sensitive key, the attacker simply wraps it and then decrypts it, as shown in Fig. 1. The attack is described with a notation for PKCS#11 based APIs brieﬂy deﬁned in the following and more formally in the next section: h(n1 , k1 ) is a predicate stating that there is a handle encoded by n1 for a key k1 stored on the device. The symmetric encryption of k1 under key k2 is represented by {|k1}| k2 . Note also that according to the wrapping formats deﬁned in PKCS#11, the device cannot tell whether an arbitrary bitstring is a cryptographic key or some other piece of plaintext. Thus when it executes the decrypt command, it has no way of telling that the packet it is decrypting contains a key.

An Introduction to Security API Analysis

39

Initial knowledge: The intruder knows h(n1 , k1 ) and h(n2 , k2 ). The name n2 has the attributes wrap and decrypt set whereas n1 has the attribute sensitive and extractable set. Trace: Wrap: h(n2 , k2 ), h(n1 , k1 ) → {|k1}| k2 SDecrypt: h(n2 , k2 ), {|k1}| k2 → k1

Fig. 1. Wrap/Decrypt attack

It might appear easy to prevent such an attack, but as we shall see, it is in fact rather diﬃcult within the conﬁnes of PKCS#11. Before treating this in detail, we will introduce our language for formal modelling of the API. 3.3

Formal Model

Our model follows the approach used by Delaune, Kremer and Steel (DKS) [25]. The device is assumed to be connected to a host under the complete control of an intruder, representing a malicious piece of software. The intruder can call the commands of the API in any order he likes using any values that he knows. We abstract away details of the cryptographic algorithms in use, following the classical approach of Dolev and Yao [27]. Bitstrings are modelled as terms in an abstract algebra. The rules of the API and the abilities of an attacker are written as deduction rules in the algebra. Basic Notions. We assume a given signature Σ, i.e. a ﬁnite set of function symbols, with an arity function ar : Σ → N, a (possibly inﬁnite) set of names N and a (possibly inﬁnite) set of variables X . Names represent keys, data values, nonces, etc. and function symbols model cryptographic primitives, e.g. {|x|}y representing symmetric encryption of plaintext x under key y, and {x}y representing public key encryption of x under y. Function symbols of arity 0 are called constants. This includes the Boolean constants true () and false (⊥). The set of plain terms PT is deﬁned by the following grammar: t, ti := x | n | f (t1 , . . . , tn )

x∈X n∈N f ∈ Σ and ar (f ) = n

We also consider a ﬁnite set F of predicate symbols, disjoint from Σ, from which we derive a set of facts. The set of facts is deﬁned as F T = {p(t, b) | p ∈ F , t ∈ PT , b ∈ {, ⊥}} In this way, we can explicitly express the Boolean value b of an attribute p on a term t by writing p(t, b). For example, to state that the key referred to by n has the wrap attribute set we write wrap(n, ).

40

R. Focardi, F.L. Luccio, and G. Steel

The description of a system is given as a ﬁnite set of rules of the form new n ˜

T ; L −−−→ T ; L where T, T ⊆ PT are sets of plain terms L, L ⊆ F are sets of facts and n ˜⊆N is a set of names. The intuitive meaning of such a rule is the following. The rule can be ﬁred if all terms in T are in the intruder knowledge and if all the facts in L hold in the current state. The eﬀect of the rule is that terms in T are added to the intruder knowledge and the valuation of the attributes is updated to satisfy L . The new n ˜ means that all the names in n ˜ need to be replaced by fresh names in T and L . This allows us to model nonce or key generation: if the rule is executed several times, the eﬀects are diﬀerent as diﬀerent names will be used each time. Example 1. The following rule models wrapping: h(x1 , y1 ), h(x2 , y2 ); wrap(x1 ,), extract(x2 ,) → {|y2 }| y1 Intuitively, h(x1 , y1 ) is a handle x1 for key y1 while term {|y2 }| y1 represents a key y2 wrapped with y1 . Since the attribute wrap for key y1 is set, noted as wrap(x1 ,), and key y2 is extractable, written extract(x2 ,), then we can wrap y2 with y1 , creating {|y2 }| y1 . The semantics of the model is deﬁned in a standard way in terms of a transition system. Each state in the model consists of a set of terms in the intruder’s knowledge, and a set of global state predicates. The intruder’s knowledge increases monotonically with each transition (he can only learn more things), but the global state is non-monotonic (attributes may be set on and then oﬀ). For a formal semantics, we refer to the literature [25]. We now present in Fig. 2 a subset of PKCS#11 commands suﬃcient for some basic symmetric key management commands (the asymmetric key has also been treated in the literature [26]). This will suﬃce to demonstrate the modelling technique and some attacks. 3.4

Using the Formal Model

We ﬁrst add some rules to the model for the intruder that allow him to encrypt and decrypt using his own keys (see Fig. 3). The intruder is assumed not to be able to crack the encryption algorithm by brute-force search or similar means, thus he can only read an encrypted message if he has the correct key. We analyse security as reachability, in the model, of attack states, i.e. states where the intruder knows the value of a key stored on the device with the sensitive attribute set to true, or the extractable attribute set to false. We give the intruder some initial knowledge, typically just some key ki that is not loaded on the device, and then use a tool such as a model checker to search the model for a chain of commands and intruder steps that leads to an attack state. Unfortunately we immediately encounter both theoretical and practical problems. First, we know that the reachability problem in general for languages like

An Introduction to Security API Analysis

41

Fig. 2. PKCS#11 Symmetric Key Fragment

x, y → {|x|}y {|x|}y , y → x Fig. 3. Intruder rules for symmetric key cryptography

this is undecidable: even with ﬁxed message size there is a reduction to the Post correspondence problem [29, table 12, page 298]. Even so we might hope to ﬁnd some attacks in practice. But in fact the model checker is quickly swamped by the combinatorial possibilities, many of which at ﬁrst sight seem unlikely to lead to an attack. For example, the intruder can take an already encrypted term {|k1}| k2 , and use it as input to the encrypt command along with some handle h(n, k3 ) to obtain {|{|k1}| k2 }| k3 . Unfortunately one cannot in general just delete these terms from the intruder knowledge: they may be a necessary step for some ﬁendish attack. Fortunately we can address both the the theoretical and practical problems at once, by means of a well-modedness result [26]. There it is shown that each function symbol such as h(., .), {|.|}. can be given a unique interpretation in terms of modes. For example, h(., .) has mode Nonce × Key → Handle. We assign to each constant symbol a mode. A term is well-moded if all the function symbols in it are applied to symbols such that the modes are respected. Furthermore, any reachability query that can be expressed with well-moded terms is satisﬁable if and only if it it is reachable by a sequence of steps where the intruder learns only well-moded terms. This allows us to prune the search space dramatically, since any branch that results in the creation of an ill-moded term can be ignored. If

42

R. Focardi, F.L. Luccio, and G. Steel

the function symbols can be moded acyclically, we can also show decidability provided we bound fresh handles and keys. Consult the paper for details and proofs [26]. 3.5

A Suite of Attacks

Equipped with a suitable formal model and a model checker, we can attempt to ﬁnd secure conﬁgurations of the standard. Clulow’s ﬁrst suggestion for preventing the attack in Fig. 1 is to prevent attribute changing operations from allowing a stored key to have both wrap and decrypt set. Note that in order to do this, it is not suﬃcient merely to check that decrypt is unset before setting wrap, and to check wrap is unset before setting decrypt. One must also add wrap and decrypt to a list of ‘sticky’ attributes which once set, may not be unset, or the attack is not prevented, [51]. Eﬀectively this means the unset rules will be omitted from the model for these attributes. Having applied these measures, we discover the attack given in Fig. 4. The intruder imports his own key k3 by ﬁrst encrypting it under k2 , and then unwrapping it. He can then export the sensitive key k1 under k3 to discover its value. Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ) and the key k3 ; n1 has the attributes sensitive and extract set whereas n2 has the attributes unwrap and encrypt set. Trace: Encrypt: h(n2 , k2 ), k3 → {|k3}| k2 new n3 Unwrap: h(n2 , k2 ), {|k3}| k2−−−→h(n 3 , k3 ) Set wrap: h(n3 , k3 ) → wrap(n3 , ) Wrap: h(n3 , k3 ), h(n1 , k1 ) → {|k1}| k3 Intruder: {|k1}| k3 , k3 → k1

Fig. 4. Attack using encrypt and unwrap

To prevent the attack shown in Fig. 4, we add encrypt and unwrap to the list of conﬂicting attribute pairs. Another new attack is discovered (see Fig. 5) of a type discussed by Clulow, [23, Section 2.3]. Here the key k2 is ﬁrst wrapped under k2 itself, and then unwrapped, gaining a new handle h(n4 , k2 ). The intruder then wraps k1 under k2, and sets the decrypt attribute on handle h(n4 , k2 ), allowing him to obtain k1. One can attempt to prevent the attack in Fig. 5 by adding wrap and unwrap to our list of conﬂicting attribute pairs. Now in addition to the initial knowledge from the ﬁrst three experiments, we give the intruder an unknown key k3 encrypted under k2. Again he is able to aﬀect an attack similar to the one above, this time by unwrapping {|k3}| k2 twice (see Fig. 6).

An Introduction to Security API Analysis

43

Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ); n1 has the attributes sensitive, extract and whereas n2 has the attribute extract set. Trace: Set wrap: h(n2 , k2 ) → Wrap: h(n2 , k2 ), h(n2 , k2 ) → Set unwrap: h(n2 , k2 ) → new n4 Unwrap: h(n2 , k2 ), {|k2}| k2 −−−→ Wrap: h(n2 , k2 ), h(n1 , k1 ) → Set decrypt: h(n4 , k2 ) → Decrypt: h(n4 , k2 ), {|k1}| k2 →

wrap(n2 , ) {|k2}| k2 unwrap(n2 , ) h(n4 , k2 ) {|k1}| k2 decrypt(n4 , ) k1

Fig. 5. Re-import attack 1

Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ); n1 has the attributes sensitive, extract and whereas n2 has the attribute extract set. The intruder also knows {|k3}| k2 . Trace: Set unwrap: h(n2 , k2 ) → new n3 Unwrap: h(n2 , k2 ), {|k3}| k2 −−−→ new n4 Unwrap: h(n2 , k2 ), {|k3}| k2 −−−→ Set wrap: h(n3 , k3 ) → Wrap: h(n3 , k3 ), h(n1 , k1 ) → Set decrypt: h(n4 , k3 ) → Decrypt: h(n4 , k3 ), {|k1}| k3 →

unwrap(n2 , ) h(n3 , k3 ) h(n4 , k3 ) wrap(n3 , ) {|k1}| k3 decrypt(n4 , ) k1

Fig. 6. Re-import attack 2

This sample of the attacks found show how diﬃcult PKCS#11 is to conﬁgure in a safe way, and indeed there are several more attacks documented in the literature [17,23,26,33] and perhaps more to discover. Another line of research has consisted of trying to propose secure subsets of the API together with a suitable security proof. An obstacle here is the fresh generation of keys and handles: if there are no attacks in the model after generating n fresh keys, how do we know there are no attacks after generating n+1? To address this problem, Fr¨oschle and Steel proposed abstractions for handles and keys that allow security proofs for unbounded fresh data [33]. In particular, they showed the security of a symmetric key management subset based around the proprietary extensions to PKCS#11 made by Eracom, where keyed hashes are used to bind attributes to keys under wrapping, ensuring that they are re-imported with the same attributes.

44

R. Focardi, F.L. Luccio, and G. Steel

Fig. 7. Tookan system diagram

3.6

Finding Attacks on Real Devices

Attacks on the standard are interesting in themselves, but in reality every device implements a diﬀerent subset of PKCS#11 with diﬀerent restrictions on the use of each command. How can one know whether a particular device is vulnerable? To address this, Bortolozzo, Centenaro, Focardi and Steel developed the Tookan1 tool [17]. Tookan functions as shown in Fig. 7. First, Tookan extracts the capabilities of the token following a reverse engineering process (1). The results of this task are written in a meta-language for PKCS#11 models. Tookan uses this information to generate a model the language described above (2), which is encoded for input to the SATMC model checker [9]. If SATMC ﬁnds an attack, the attack trace (3) is sent to Tookan for testing directly on the token (4). Changes to the model. There are several diﬀerences between Tookan’s model and the model of section 3.3. One is that Tookan takes into account key templates. In section 3.3, the key generation commands create a key with all attributes unset (see Fig. 2). Attributes are then enabled one by one using the SetAttribute command. In our experiments with real devices, we discovered that some tokens do not allow attributes of a key to be changed. Instead, they use a key template specifying settings for the attributes which are given to freshly generated keys. Templates are used for the import of encrypted keys (unwrapping), key creation using CreateObject and key generation. The template to be used in a speciﬁc command instance is speciﬁed as a parameter, and must come from a set of valid templates, which we label G, C and U for the valid templates for key generation, creation and unwrapping respectively. Tookan can construct the set of templates in two ways: the ﬁrst, by exhaustively testing the commands using templates for all possible combinations of attribute settings, which may be very time consuming, but is necessary if we aim to verify the security of a token. The second method is to construct the set of templates that should be allowed based on the reverse-engineered attribute policy (see next paragraph). This is an approximate process, but can be useful for quickly ﬁnding attacks. Indeed, in our experiments, we found that these models reﬂected well the operation of the token, i.e. the attacks found by the model checker all executed on the tokens without any ‘template invalid’ errors.

1

Tool for cryptoki analysis.

An Introduction to Security API Analysis

45

Attribute Policies. Most tokens tested attempt to impose some restrictions on the combinations of attributes that can be set on a key and how these may be changed. There are four kinds of restriction that Tookan can infer from its reverse engineering process: Sticky on These are attributes that once set, may not be unset. The PKCS #11 standard lists some of these [47, Table 15]: sensitive for secret keys, for example. The UnsetAttribute rule is only included for attributes which are not sticky on. To test if a device treats an attribute as sticky on, Tookan attempts to create a key with the attribute on, and then calls SetAttribute to change the attribute to oﬀ. Sticky off These are attributes that once unset may not be set. In the standard, extractable is listed as such an attribute. The SetAttribute rule is only included for attributes which are not sticky oﬀ. To test if a device treats an attribute as sticky on, Tookan attempts to create a key with the attribute oﬀ, and then calls SetAttribute to change the attribute to on. Conflicts Many tokens (appear to) disallow certain pairs of attributes to be set, either in the template or when changing attributes on a live key. For example, some tokens do not allow sensitive and extractable to be set on the same key. The SetAttribute rule is adjusted to prevent conﬂicting attributes from being set on an object or on the template. When calculating the template sets C, G, U (see above), we forbid templates which have both the conﬂicting attributes set. To test if a device treats an attribute pair as a conﬂict, Tookan attempts to generate a key with the the pair of attributes set, then if no error is reported, it calls GetAttribute to check that the token really has created a key with the desired attributes set. Tied Some tokens automatically set the value of some attributes based on the value of others. For example, many tokens set the value of always sensitive based on the value of the attribute sensitive. The SetAttribute and UnsetAttribute rules are adjusted to account for tied attributes. The template sets C, G, U are also adjusted accordingly. To test if a device treats an attribute pair as tied, Tookan attempts to generate a key with some attribute a on and all other attributes oﬀ. It then uses GetAttribute to examine the key as it was actually created, and tests to see if any other attributes were turned on. Limitations of Reverse Engineering. Tookan’s reverse engineering process is not complete: it may result in a model that is too restricted to ﬁnd some attacks possible on the token, and it may suggest false attacks which cannot be executed on the token. This is because in theory, no matter what the results of our ﬁnite test sequence, the token may be running any software at all, perhaps even behaving randomly. However, if a token implements its attribute policy in the manner in which we can describe it, i.e. as a combination of sticky on, sticky oﬀ, conﬂict and tied attributes, then our process is complete in the sense that the model built will reﬂect exactly what the token can do (modulo the usual Dolev-Yao abstractions for cryptography).

46

R. Focardi, F.L. Luccio, and G. Steel Table 1. Summary of results on devices

Company

Device Model

Supported Functionality Attacks found sym asym cobj chan w ws a1 a2 a3 a4 a5 mc

Aladdin Athena Bull Eutron Feitian USB Feitian Feitian Gemalto MXI Security RSA SafeNet Sata ACS Card Athena Gemalto Gemalto Gemalto Siemens

eToken PRO ASEKey Trustway RCI Crypto Id. ITSEC StorePass2000 ePass2000 ePass3003Auto SEG Stealth MXP Bio SecurID 800 iKey 2032 DKey ACOS5 ASE Smartcard Cyberﬂex V2 Classic TPC IS V1 Classic TPC IS V2 CardOS V4.3 B

Eracom IBM

HSM simulator opencryptoki 2.3.1

Soft

a1

a1

a3 a3 a3

a3 a3

a2 a3 a4

a1 a1

In our testing, the model performed very well: the Tookan consistently found true attacks on ﬂawed tokens, and we were unable to ﬁnd ‘by hand’ any attacks on tokens which the model checker deemed secure. This suggests that real devices do indeed implement their attribute policies in a manner similar to our model. 3.7

Results

Table 1 summarises the results obtained by Tookan on a number of devices as well as two software simulators. Supported functionality and attacks are summarized in Table 2 and described below. Implemented functionality. Columns ‘sym’ and ‘asym’ respectively indicate whether or not symmetric and asymmetric key cryptography are supported. Column ‘cobj’ refers to the possibility of inserting external, unencrypted, keys on the device via C CreateObject PKCS#11 function. This is allowed by almost all of the analysed tokens, although it wasn’t included in the original model of the standard used by Delaune, Kremer and Steel [26]. The next column, ‘chan’, refers to the possibility of changing key attributes through C SetAttributeValue. The following two columns, ‘w’ and ‘ws’, respectively indicate whether the token permits wrapping of nonsensitive and sensitive keys.

An Introduction to Security API Analysis

47

Table 2. Key for table 1 Acronym Description

Supported functionality

Attacks

sym asym cobj chan w ws a1 a2 a3 a4 a5 mc

symmetric-key cryptography asymmetric-key cryptography inserting new keys via C CreateObject changing key attributes wrapping keys wrapping sensitive keys wrap/decrypt attack based on symmetric keys wrap/decrypt attack based on asymmetric keys sensitive keys are directly readable unextractable keys are directly readable (forbidden by the standard) sensitive/unextractable keys can be changed into nonsensitive/extractable ﬁrst attack found by Tookan

Attacks. Attack a1 is a wrap/decrypt attack as discussed in section 3.2. The attacker exploits a key k2 with attributes wrap and decrypt and uses it to attack a sensitive key k1 . Using our notation from section 3.3: Wrap: h(n2 , k2 ), h(n1 , k1 ) →{|k1}| k2 Decrypt: h(n2 , k2 ), {|k1}| k2 →k1 As we have discussed above, the possibility of inserting new keys in the token (column ‘cobj’) might simplify further the attack. It is suﬃcient to add a known wrapping key: new n

2 CreateObject: k2 −−−−→h(n 2 , k2 ) Wrap: h(n2 , k2 ), h(n1 , k1) →{|k1}| k2

The attacker can then decrypt {|k1 }| k2 since he knows key k2 . SATMC discovered this variant of the attack on several vulnerable tokens. Despite its apparent simplicity, this attack had not appeared before in the PKCS#11 security literature [23,25]. Attack a2 is a variant of the previous ones in which the wrapping key is a public key pub(z) and the decryption key is the corresponding private key priv(z): Wrap: h(n2 , pub(z)), h(n1 , k1 ) →{k1 }pub(z) ADecrypt: h(n2 , priv(z)), {k1 }pub(z) →k1 In this case too, the possibility of importing key pairs simpliﬁes even more the attacker’s task by allowing him to import a public wrapping key while knowing the corresponding private key. Once the wrap of the sensitive key has been performed, the attacker can decrypt the obtained ciphertext using the private key.

48

R. Focardi, F.L. Luccio, and G. Steel

Attack a3 is a clear ﬂaw in the PKCS#11 implementation. It is explicitly required that the value of sensitive keys should never be communicated outside the token. In practice, when the token is asked for the value of a sensitive key, it should return some “value is sensitive” error code. Instead, we found that some of the analysed devices just return the plain key value, ignoring this basic policy. Attack a4 is similar to a3: PKCS#11 requires that keys declared to be unextractable should not be readable, even if they are nonsensitive. If they are in fact readable, this is another violation of PKCS#11 security policy. Finally, attack a5 refers to the possibility of changing sensitive and unextractable keys respectively into nonsensitive and extractable ones. Only the Sata and Gemalto SafeSite Classic V2 tokens allow this operation. However, notice that this attack is not adding any new ﬂaw for such devices, given that attacks a3 and a4 are already possible and sensitive or unextractable keys are already accessible. Model-checking results. Column ‘mc’ reports which of the attacks was automatically rediscovered via model-checking. SATMC terminates once it has found an attack, hence we report the attack that was found ﬁrst. Run-times for ﬁnding the attacks vary from a couple of seconds to just over 3 minutes. 3.8

Finding Secure Configurations

As we observed in the last section, none of the tokens we tested are able to import and export sensitive keys in a secure fashion. In particular, all the analysed tokens are either insecure or have been drastically restricted in their functionality, e.g. by completely disabling wrap and unwrap. In this section, we present CryptokiX, a software implementation of a Cryptoki token which can be ﬁXed by selectively enabling diﬀerent patches. The starting point is openCryptoki [45], an open-source PKCS#11 implementation for Linux including a software token for testing. As shown in Table 1, the analysis of openCryptoki software token has revealed that it is subject to all the non-trivial attacks. This is in a sense expected, as it implements the standard ‘as is’, i.e., with no security patches. CryptokiX extends openCryptoki with: Conflicting attributes. We have seen, for example, that it is insecure to allow the same key to be used for wrapping and decrypting. In CryptokiX it is possible to specify a set of conﬂicting attributes. Sticky attributes. We know that some attributes should always be sticky, such as sensitive. This is also useful when combined with the ‘conﬂicting attributes’ patch above: if wrap and decrypt are conﬂicting, we certainly want to avoid that the wrap attribute can be unset so as to allow the decrypt attribute to be set. Wrapping formats. It has been shown that specifying a non-conﬂicting attribute policy is not suﬃcient for security [23,25]. A wrapping format should also be used to correctly bind key attributes to the key. This prevents attacks where the key is unwrapped twice with conﬂicting attributes.

An Introduction to Security API Analysis

49

Secure templates. We limit the set of admissible attribute combinations for keys in order to avoid that they ever assume conﬂicting roles at creation time. This is conﬁgurable at the level of the speciﬁc PKCS#11 operation. For example, we can deﬁne diﬀerent secure templates for diﬀerent operations such as key generation and unwrapping. A way to combine the ﬁrst three patches with a wrapping format that binds attributes to keys in order to create a secure token has already been demonstrated [33] and discussed in section 3.5. One can also use the fourth patch to produce a secure conﬁguration that does not require any new cryptographic mechanisms to be added to the standard, making it quite simple and cheap to incorporate into existing devices. We consider here a set of templates with attributes sensitive and extractable always set. Other attributes wrap, unwrap, encrypt and decrypt are set as follows: Key generation. We allow three possible templates: 1. wrap and unwrap, for exporting/importing other keys; 2. encrypt and decrypt, for cryptographic operations; 3. neither of the four attributes set, i.e. the default template if none of the above is speciﬁed. Key creation/import. We allow two possible templates for any key created with CreateObject or imported with Unwrap: 1. unwrap,encrypt set and wrap,decrypt unset; 2. none of the four attributes set. The templates for key generation are rather intuitive and correspond to a clear separation of key roles, which seems a sound basis for a secure conﬁguration. The rationale behind the single template for key creation/import, however, is less obvious and might appear rather restrictive. The idea is to allow wrapping and unwrapping of keys while ‘halving’ the functionality of created/unwrapped keys: these latter keys can only be used to unwrap other keys or to encrypt data, wrapping and decrypting under such keys are forbidden. This, in a sense, oﬀers an asymmetric usage of imported keys: to achieve full-duplex encrypted communication two devices will each have to wrap and send a freshly generated key to the other device. Once the keys are unwrapped and imported in the other devices they can be used to encrypt outgoing data in the two directions. Notice that imported keys can never be used to wrap sensitive keys. Note also that we require that all attributes are sticky on and oﬀ, and that we assume for bootstrapping that any two devices that may at some point wish to communicate have a shared long term symmetric key installed on them at personalisation time. This need only be used once in each direction. Our solution works well for pairwise communication, where the overhead is just one extra key, but would be more cumbersome for group key sharing applications. The developed solution has been implemented and analysed by extracting a model using Tookan. A model for SATMC was constructed using the abstractions

50

R. Focardi, F.L. Luccio, and G. Steel

described above (see end of section 3.5). Given the resulting model, SATMC terminates with no attacks in a couple of seconds, allowing us to conclude the patch is safe in our abstract model for unbounded numbers of fresh keys and handles. Note that although no sensitive keys can be extracted by an intruder, there is of course no integrity check on the wrapped keys that are imported. Indeed, without having an encryption mode with an integrity check this would seem to be impossible. This means that one cannot be sure that a key imported on to the device really corresponds to a key held securely on the intended recipient’s device. This limitation would have to be taken into account when evaluating the suitability of this conﬁguration for an application. CryptokiX is available online2 . 3.9

Summary

We have seen how RSA PKCS#11 describes an API for key management where the usage policy for each key is described by a set of attributes. We have seen how this interface, if not conﬁgured carefully, can be vulnerable to a variety of attacks, and we have seen that these vulnerabilities aﬀect not just theoretical models but real devices. We have also seen how to use formal modelling techniques to systematically ﬁnd attacks and then verify the security of models. However there are still many open questions about designs of a secure API and cryptographic soundness of models, for example, which we will discuss in section 5.

4

PIN Processing

We now consider our second case study, which addresses the problem of protecting a user’s PIN when withdrawing some money at an Automated Teller Machine (ATM). International bank networks are structured in such a way that an access to an ATM implies that the user’s PIN is sent from the ATM to the issuing bank for the veriﬁcation. While travelling, the PIN is decrypted and reencrypted by special tamper-resistant HSMs which are placed on each network switch, as illustrated in Fig. 8. Indeed, international standards mandate the use of these devices to keep PINs secure [37]. The ﬁrst PIN encryption is performed by the ATM keypad which is an HSM itself, using a symmetric key k1 shared with the neighbour acquiring bank. While travelling from node to node, the encrypted PIN is decrypted and re-encrypted by the HSM located in the switch with another key shared with the destination node. This is done using a so called translation API. The ﬁnal veriﬁcation and acceptance/refusal of the PIN is done by the issuing bank. This check is performed via a veriﬁcation API. At ﬁrst sight this setup seems highly secure. However, several API-level attacks have been discovered on these HSMs in recent years [10,16,22]. These attacks work by assuming that the attacker is an insider who has gained access to the HSM at some bank switch, a threat considered in-scope by the standard 2

http://secgroup.ext.dsi.unive.it/cryptokix

An Introduction to Security API Analysis

51

Fig. 8. Bank network

[37]. The attacker performs some unexpected sequence of API calls from which he is able to deduce the value of a PIN. API-level attacks on PINs have recently attracted attention from the media [1,3]. This has increased the interest in developing formal methods for analysing PIN recovery attacks. We will brieﬂy survey the literature here. Examining the PIN veriﬁcation API, Bond and Zielinski [16] and Clulow [22] discovered the so called decimalization attack that we will discuss in section 4.1 at more or less the same time. The original version of the attack requires an average of 16.5 API calls to obtain a 4-digit PIN [16]. Later variations proposed by Steel [49] and Focardi and Luccio [31] reduced that value to 16.145 and 14.47 respectively. The decimalisation table attack is similar to the Mastermind game, and in section 4.2, we will illustrate how to reuse known techniques to solve Mastermind in order to ﬁnd eﬃcient attacks on bank PINs. Bond and Clulow have also demonstrated an attack that works even if the decimalisation table is ﬁxed [14], although it requires many more calls to the API (some 10s of thousands). Attacks on the PIN translation API have been presented by Clulow [22] and subsequently by Berkman and Ostrovsky [10]. These attacks often exploit details of the formats used to pad a PIN into a 64 bit block for encryption. Berkman and Ostrovsky’s attacks also exploit commands for changing a customer PIN, though they generally require a known plaintext-ciphertext pair (i.e. a clear PIN and an encryption of the same PIN) to be introduced into the system. The ﬁrst formal analysis work on PIN recovery attacks is due to Steel [49], who showed hos a combination of logic programming and probabilistic model checking could be used to ﬁnd new attacks and analyse possible patches to the APIs. Recently Centenaro and the authors of this survey presented a

52

R. Focardi, F.L. Luccio, and G. Steel Table 3. The veriﬁcation API

PIN V(EPB , len, oﬀset, vdata, dectab ) { x1 := encpdk (vdata); x2 := left(len, x1 ); x3 := decimalize(dectab, x2 ); x4 := sum mod10(x3 , oﬀset ); x5 := deck (EPB ); x6 := fcheck(x5 ); if (x6 =⊥) then return( format wrong ); if (x4 = x6 ) then return(PIN correct ); else return(PIN wrong )}

language-based setting for analysing PIN processing API via a type-system [20]. We have formally modelled existing attacks, proposed some ﬁxes and proved them correct via type-checking. These ﬁxes typically require to reduce and modify the HSM functionality by, e.g., deciding on a single format of the transmitted PIN or adding MACs for the integrity of user data. This will be the subject of section 4.3. As upgrading the bank network HSMs worldwide is complex and very expensive in [30] the authors have proposed a method for the secure upgrade of HSMs in wide networked systems. This method incrementally upgrades the network so to obtain upgraded, secure subnets, while preserving the compatibility towards the legacy system. Finally, in [32] we have also have proposed a low-impact, easily implementable ﬁx that involves very little change to the existing ATM network infrastructure. Attacks are still possible, but they are 50000 times slower. 4.1

API-Level Attacks on PIN Verification

In this section we show in detail a real API-level attack on cash machine PINs. We focus our attention on the PIN veriﬁcation API (PIN V for short) and we specify its code in Table 3. It takes as input the encrypted PIN block EPB , i.e. the encrypted PIN arriving from the ATM, the PIN length len and oﬀset oﬀset, the validation data vdata and the decimalization table dectab. The API returns the result of the veriﬁcation or an error code. PIN V behaves as follows: – The user PIN of length len is computed by ﬁrst encrypting validation data vdata with the PIN derivation key pdk (x1 ) and obtaining a 16 hexadecimal digit string. Then, the ﬁrst len hexadecimal digits are chosen (x2 ), and decimalized through dectab (x3 ), obtaining the “natural” PIN assigned by the issuing bank to the user. decimalize is a function that associates to each possible hexadecimal digit (of its second input) a decimal one as speciﬁed by its ﬁrst parameter (dectab). Finally, if the user wants to choose her own PIN, an oﬀset is calculated by digit-wise subtracting (modulo 10) the natural PIN from the user-selected one (x4 ).

An Introduction to Security API Analysis

53

– To recover the trial PIN EPB is ﬁrst decrypted with key k (x5 ), then the PIN is extracted by the formatted decrypted message (x6 ). This last operation depends on the speciﬁc PIN format adopted by the bank. In some cases, for example, the PIN is padded with random digits so to make its encryption immune from codebook attacks. In this case, extracting the PIN involves removing this random padding. – If the format in incorrect (⊥ represents failure) then an error message is returned. Otherwise, the equality between the user PIN and the trial PIN is veriﬁed. Example 2. For simplicity we consider the ‘default’ decimalisation table: 0 1 2 3 4 5 6 7 8 9 A B C D E F 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 We will write it as dectab = 0123456789012345. Let also EPB = {|5997, r|}k , len=4, oﬀset =4732 and x1 = encpdk (vdata) = BC6595FDE32BA101 then: x2 x3 x4 x5 x6 x6 x4

= left(4, BC6595F DE32BA101) = BC65 = decimalize(dectab, BC65) = 1265 = sum mod10(1265, 4732) = 5997 = deck ({|5997, r|}k ) = (5997, r) = fcheck(5997, r) = 5997 =⊥ = x6 return(PIN correct )

Since x6 is diﬀerent from ⊥ and x4 = x6 , the API returns PIN correct . The decimalization attack. We now illustrate a real attack on PIN V ﬁrst reported in [16,22], called the decimalization table attack. It consists of deducing the PIN digits by modifying the way numbers are decimalized and by observing whether this aﬀects the result of PIN V. The position of the guessed PIN digits is reconstructed by manipulating the oﬀset, which is a public parameter. By combining all this information the attacker is able to reconstruct the whole PIN. More speciﬁcally, the attack works by iterating the following two steps, until the whole PIN is recovered: 1. To discover whether or not a decimal digit d is present in the intermediate value contained in x3 the intruder picks digit d, changes the dectab function so that values previously mapped to d now map to d + 1 mod 10, and then checks whether the system still returns PIN correct . If this is the case digit d does not appear in x3 . 2. It the API call returns PIN wrong the attacker discovers that d is one of the digits of x3 . To locate the position of the digit the intruder also decreases the oﬀset digits by 1, position by position, until the API returns again that the PIN is correct.

54

R. Focardi, F.L. Luccio, and G. Steel

We illustrate the attack through a simple example. Example 3. Consider again Example 2. The attacker, unaware of the value of the PIN, ﬁrst tries to discover whether or not 0 appears in x3 , so it changes the dectab as dectab = 1123456789112345 i.e., he replaces the two 0’s by 1’s. Invoking the API with dectab he obtains decimalize(dectab , BC65) = decimalize(dectab, BC65) = 1265 that is x3 remains unchanged and so the attacker deduces 0 does not appear in the PIN. The attacker proceeds by replacing the 1’s of dectab by 2’s: dectab = 0223456789022345 which gives x3 x4 x5 x6 x6 x4

= decimalize(dectab , BC65) = sum mod10(2265, 4732) = deck ({|5997, r|}k ) = fcheck(5997, r) =⊥ = x6

= 2265 = 1265 = 6997 = 5997 = (5997, r) = 5997 return(PIN wrong )

The intruder now knows that digit 1 occurs in x3 . To discover its position and multiplicity, he now varies the oﬀset so as to “compensate” for the modiﬁcation of the dectab. In particular, he tries to decrement each oﬀset digit by 1. For example, testing the position of one occurrence of one digit amounts to trying the following oﬀset variations: 3732, 4632, 4722, 4731 Notice that, in this speciﬁc case, oﬀset value 3732 makes the API return again PIN correct . The attacker now knows that the ﬁrst digit of x3 is 1. Given that the oﬀset is public, he also calculates the ﬁrst digit of the user PIN as 1 + 4 mod 10 = 5. He then proceeds using the same technique to discover the other PIN digits. 4.2

PIN Attacks as a Mastermind Game

As observed in [15,31], the above attack resembles the Mastermind game, invented in 1970 by the Israeli postmaster and telecommunication expert Mordecai Meirowitz [46, page 442]. The game is shown in Fig. 9 which is taken from [2]. It is played as a board game between two players, a codebreaker and a codemaker [50]. The codemaker chooses a linear sequence of coloured pegs and conceals them behind a screen. Duplicates of coloured pegs are allowed. The codebreaker has to guess, in diﬀerent trials, both the colour and the position of the pegs. During each trial he learns something and based on this he decides the next guess: in

An Introduction to Security API Analysis

55

Fig. 9. Guessing a code of a Mastermind game is similar to guessing a PIN

particular, a response consisting of a red marker represents a correct guess of the colour and the position of a peg (but the marker does not indicate which one is correct), a response consisting of a white marker represents only the correct guess of a colour but at the wrong position. The analogy between the game and the attack is as follows: each API call represents a trial of the codebreaker and the API return value is the corresponding answer of the codemaker. Modifying the dectab corresponds to disclosing the presence of certain digits in the PIN, like the white marker in the Mastermind Game. On the other hand, manipulating the dectab and the oﬀset together is similar to asking the codemaker to conﬁrm a guess of both the value and the position of one PIN digit, like the red marker of the game. An extended Mastermind game. To make the above analogy more precise, it is in fact necessary to extend the Mastermind game so as to allow the codebreaker to pose a guess of sets of coloured pegs, instead of just single pegs. Intuitively, the sets represent alternative guesses, i.e., it is suﬃcient that one of the pegs in the set is correct to get a red or a white marker. E.g., given the secret (1, 5, 3, 1) and the (extended) guess ({1}, {3, 4, 5}, {1, 2}, {0}), then the result is two red markers for the ﬁrst two positions of the guess and one white marker for the third position of the guess. The correspondence between the values of the secret and the values in the guess are shown as underlined values in ({1}, {3, 4, 5}, {1, 2}, {0}). In [31] it is proved that the decimalization attack is equivalent to playing an extended Mastermind game, only focusing on answers containing 4 red markers. Intuitively, given a modiﬁed dectab and oﬀset, we construct the sets for the extended guess by picking all digits whose modiﬁcation in the oﬀset are correctly compensated by the variation in the dectab. The ﬁrst set corresponds to the

56

R. Focardi, F.L. Luccio, and G. Steel

ﬁrst oﬀset digit, the second set to the second digit and so on. For example, consider an original oﬀset (1, 5, 4, 2) modiﬁed as (0, 3, 1, 0), giving a variation of (−1, −2, −3, −2). Suppose now that the modiﬁed dectab turns the mapping of digit 4 into 5, with a variation of +1, then digit 4 will be in the ﬁrst set of the extended guess, since the oﬀset has the correct (opposite) variation -1 in the ﬁst position. As an example, consider the following default dectab and a modiﬁed one, named dectab : dectab 0 1 2 3 4 5 6 7 8 9 dectab 0 2 4 3 5 8 6 9 1 9 variation +1 +2 +1 +3 +2 +3

0 0

1 2 3 4 5 2 4 3 5 8 +1 +2 +1 +3

Since the variation on the ﬁrst oﬀset digit is −1 we collect, in the ﬁrst set of the guess, all digits with variation +1 in the dectab, i.e., {1, 4}. For the second and fourth oﬀset digits, both with variation −2, we obtain set {2, 7}. Finally, for the third oﬀset digit, with variation −3, we have set {5, 8}. The resulting extended guess is ({1, 4}, {2, 7}, {5, 8}, {2, 7}). Let us now see how the answer to this guess in the Mastermind game relates to the result of an API call with the above modiﬁed oﬀset and dectab. Recall that we start from a dectab and oﬀset that give PIN correct . Thus, we will still have PIN correct only if the variations in the dectab are correctly compensated for by the ones in the oﬀset. For example, if digit 1 originally occurred in the ﬁrst place after decimalization (x3 ), with dectab it will now be a 2. Summing 1 with the ﬁrst digit of the original oﬀset we had 1 + 1 = 2. With the modiﬁed oﬀset we now have 2 + 0 = 2. In fact the variation +1 in the dectab is correctly compensated for by the oﬀset. Notice that 1 occurs in the ﬁrst set of the guess. Observe also that 1 does not occur in the other sets of the guess. Thus, if 1 originally occurred in the second place after decimalization it would still be turned into 2 by dectab but the change in the oﬀset this time would not compensate for this variation. In fact, with the original dectab and oﬀset we would get 1 + 5 = 6 (where 5 is the second value of the oﬀset) while with the modiﬁed ones we would get 2 + 3 = 5 (where 3 is the second value of the new oﬀset). Thus, the modiﬁcation of +1 in the dectab this time would not correspond to the variation −2 in the second oﬀset digit. By iterating this reasoning on all digits, we obtain that the API call gives PIN correct if and only if the extended guess in the Mastermind game gives 4 red markers, meaning that the four secret digits, correct after decimalization and before summing the oﬀset, are in the four sets of the extended guess. This correspondence has been formally proved [31]. Improving attack strategies by playing Mastermind. Now that we have shown a strict analogy between PIN cracking and an extension of the Mastermind game, we can ﬁnd eﬃcient attacks for recovering PINs by simply devising minimal strategies for winning the Mastermind game. The solution proposed by Knuth [39] is to build a balanced search tree by minimizing the maximum number of solutions surviving after a guess. A guess, in our case, gives a Boolean

An Introduction to Security API Analysis

57

Table 4. An algorithm for ﬁnding PIN cracking strategies do_guess ( S ) : if | S | != 1: # not a s i n g l e s o l u t i o n yet min = | S |

# s t a r t i n g value , we want to d e c r e a s e it

# d e p e n d i n g on the round and the n u m b e r of s u r v i v i n g solutions , # g e n e r a t e s the g u e s s e s ( h e u r i s t i c s here ...) guesses = g e n e r a t e _ g u e s s e s( S ) for g in guesses : # for each guess g # i n t e r s e c t S and s u r v i v i n g s o l u t i o n s for g M_SOLS = S & s u r v i v i n g( g ) # count m a t c h i n g and non - m a t c h i n g and take the max n_sol = max (| m_sols | ,| S | -| m_sols |) # if we got a minimum , we store it if mas < min : min = n_sol MIN_SOLS = M_SOLS if min < | S |: # we are d e c r e a s i n g the s u r v i v i n g s o l u t i o n set size : # let us p e r f o r m the guess # case1 : guess was right , e x p l o r e s o l u t i o n s in M I N _ S O L S do_guess ( MIN_SOLS ) # case2 : guess was wrong , e x p l o r e s o l u t i o n s in S - M I N _ S O L S do_guess ( S - MIN_SOLS ) else : FAIL

# we are l o o p i n g

answer which partitions the set of possible solutions into two disjoint sets: matching or non-matching ones. The guess that is more balanced, i.e., that minimizes the maximum of the two sets, is picked and the procedure is recursively applied until it reaches a unique solution. This method does not guarantee an optimal solution as an unbalanced guess might produce a more balanced subtree later on, and vice versa, but it has been shown to work well in practice [31,39]. In Table 4, we report an algorithm inspired by the above idea, adapted to our setting. The recursive function do guess is initially invoked with the whole S set of possible secrets. It then ﬁnds a guess that minimizes the maximum number of surviving solutions on the two possible answers and it descends recursively into the two sets in which the actual set is partitioned. A critical issue is how the set of candidate guesses are picked (function generate guesses(S)). In our setting, considering all extended guesses consisting of all possible combinations of digit subsets for each position is computationally intractable. In [31] it is observed that the ‘shape’ of the guesses used in the strategies found is quite regular. This makes it possible to devise a few heuristics that incrementally extend the set of

58

R. Focardi, F.L. Luccio, and G. Steel

Fig. 10. Noninterference

guesses when needed. As a result, the existing bounds on the average number of API calls required to recover a PIN has been reduced to 14.47 and a new bound of 19.3 has been given for 5-digits PINs. Both cases are close to the optimum, i.e., the depth of a perfectly balanced search tree. 4.3

Devising Robust PIN Management APIs

In the previous sections, we described one of the known attacks on PIN management APIs and its similarity to the Mastermind game. We now focus on remedies and we ﬁrst try to understand what is the source of the attacks, i.e., what security property is missing. This will allow us to propose formal models and tools supporting a more ‘robust’ development of PIN processing APIs. Preventing information flow. Since the PIN recovery attacks involve the leakage of a secret value, a ﬁrst attempt to understand the problem might be in terms of classical noninterference [34]. In this setting, data items are labelled with security levels taken from a lattice. Noninterference requires that data from a higher level, e.g. the secret level, should never aﬀect the value of data with a lower level, e.g. the public level. The idea is depicted in Fig. 10. The system is (highly) simpliﬁed as a box containing two smaller boxes, with a switch for the input and a display for the output. The top box (high level input/output) contains the secret information while the bottom box (low level input/output) is public and possibly under the control of the attacker. Noninterference holds if a variation on the high input does not produce any change in the low level output. The noninterference property is very strong and ensures that no secret information will ever be leaked, but this is too much for our setting. In fact, PIN veriﬁcation needs to leak 1 bit of information about the correctness of the PIN. To see how the property is broken, it is enough to consider an API call with the correct encrypted PIN block (giving PIN correct ), and another API call with

An Introduction to Security API Analysis

59

Fig. 11. Robust declassiﬁcation

a wrong encrypted PIN block (giving PIN wrong ). This for example happens if the user types the wrong PIN at the ATM. By changing the secret data, i.e., the encrypted PIN, we have two diﬀerent outcomes of the API, thus two diﬀerent low level outputs. Clearly a leak of information about the correctness of the PIN is inevitable, but we would somehow like to limit this to prevent attacks. Admitting controlled information leakage. One approach to relaxing noninterference while still controlling information release is called robustness, and was proposed by Myers, Sabelfeld and Zdancewic (MSZ) [43]. This property admits some form of declassiﬁcation (or downgrading) of conﬁdential data, but requires that an attacker cannot inﬂuence the secret information which is declassiﬁed. In our case study of section 4.1, PIN V returns the correctness of the typed PIN which is a one-bit leakage of information about a secret datum. Robustness requires that an attacker cannot abuse such a declassiﬁcation in order to gain more information than intended. The idea is illustrated in Fig. 11: at the top of the ﬁgure, moving from left to right, we see that a change in the high, i.e., secret input is observable at the low, i.e., public level, meaning that noninterference does not hold and some information is leaked. However, moving from the top to the bottom, we notice that a change in the low level input does not aﬀect the leaked information. This amounts to stating that what is leaked is never aﬀected by untrusted, low level users. If we consider the decimalization attack we notice that the attacker, by manipulating public data such as the dectab and the oﬀset, can aﬀect what is declassiﬁed, i.e., the outcome of the API call. By forbidding this, the attack should be prevented.

60

R. Focardi, F.L. Luccio, and G. Steel

PIN V M(EPB , len, oﬀset, vdata, dectab, MAC ) { if (macak (EPB , len, oﬀset, vdata, dectab) == MAC ) then PIN V(EPB , len, oﬀset, vdata, dectab ) ; else return(“integrity violation ); }

Fig. 12. A ﬁxed PIN veriﬁcation API

A robust PIN verification API. In [20] we have formalized the above property in the setting of PIN management APIs, and we have given a patched version of PIN V that prevents attacks by checking the integrity of the public data. The idea is to have a Message Authentication Code (MAC) of such data, i.e., an unforgeable cryptographic signature that once checked by the HSM, guarantees the integrity of data. If the attacker tries to tamper with the decimalization table or oﬀset, the MAC check fails and the HSM aborts the call with an error message. Fig. 12 reports the code of the ﬁxed PIN veriﬁcation API, called PIN V M, which recomputes and checks the MAC before calling the original PIN V.3 In the paper we have also devised a type-system to statically prove the robustness of the new API. Even if the proposed ﬁx has been proved to be secure, the infrastructure changes needed to add full MAC calculation to ATMs and switches are seen as prohibitive by the banks [6]. We have proposed a way to implement a weaker version of our scheme whilst minimising changes to the existing infrastructure [32]. In the paper, we observe that cards used in the cash machine network already have an integrity checking value: the card veriﬁcation code (CVC), or card veriﬁcation value (CVV), is a 5 (decimal) digit number included in the magnetic stripe of most cards in use and contains a MAC of the customer’s PAN, the expiry date of the card and some other data. The current purpose of the CVV is to make card cloning more diﬃcult, since the CVV is not printed in the card and does not appear on printed POS receipts.4 Our idea is to use the CVV calculation method twice, in the manner of a hashed MAC or HMAC function. We calculate the CVV of a new set of data, containing the decimalisation table and oﬀset. Then, we insert the result of the original CVV calculation to produce a ﬁnal 5-digit MAC. The scheme is easily implementable but we loose some security, since our MACs now have an entropy of only 5 decimal digits (216 < 105 < 217 ).

3

4

For technical reasons, the speciﬁcation in [20] has that the Personal Account Number is in the list of parameters. Here we consider it as part of the validation data vdata. The CVV/CVC should not be confused with the CVC2 or CVV2, which is printed on the back of the card and is designed to counteract “customer not present” fraud when paying over the Internet or by phone.

An Introduction to Security API Analysis

4.4

61

Summary

We have seen how bank networks use special HSMs to secure ATM transactions. These sophisticated devices perform cryptographic operations such as PIN encryption/decryption and PIN veriﬁcation via a security API. We have discussed attacks on this API and we have shown in detail the so called decimalization table attack. We have seen how these attacks progressively leak partial information about PIN digits similarly to what happens in the Mastermind game. We have shown that algorithms for the solution of Mastermind can be adapted to search for eﬃcient attacking strategies on bank PINs. Finally, we have illustrated how the security of PIN management APIs can be modelled in terms of an information ﬂow property called robustness and we have described a ﬁxed API that can be proved to be robust. Intuitively, a robust API can leak partial information on a secret, e.g., whether the PIN typed at the ATM is correct or not, but this leakage should never be under the control of the attacker, as occurs in the decimalization attack.

5

Conclusions

There are many open research problems in security APIs, both theoretical and practical. In the domain of key management, we have seen that the current standard for APIs, PKCS#11, is very diﬃcult to conﬁgure securely. This is evidenced by a number of attacks on models of the standard as well as attacks on real PKCS#11 security tokens: in our sample of 18 devices, we found 6 tokens that trivially gave up their sensitive keys in complete disregard of the standard, 3 that were vulnerable to a variety of key separation attacks, and a further smartcard that allowed unextractable keys to be read in breach of the standard. The remainder provide no functionality for secure transport of sensitive keys. We have seen that it is possible to propose secure conﬁgurations, [17,33], but security proofs here are only in the symbolic model of cryptography: there is more work to be done to reconcile these proofs to the cryptographic details of real implementations, though work in this direction is underway [40]. Given the current situation it is perhaps not surprising that there have been articles proposing completely new designs for key management APIs [19,24]. Indeed at least two new industrial standards which address key management are currently at the draft stage: IEEE 1619.3 [36] (for secure storage) and OASIS Key Management Interoperability Protocol (KMIP) [44]. It remains to be seen how these latter designs will address security concerns. We have described attacks on bank HSMs for the ATM network. We have discussed possible ﬁxes for the decimalization attack but the extent to which these ﬁxes can be implemented in real devices is still unclear. There is a great deal of existing infrastructure in the ATM network that would be costly to replace. Since the attacks involve obtaining PINs in order to withdraw money using closed magnetic stripe cards, one possibility would be to upgrade the whole network to the new chip-based cards [4]. This seems unlikely to happen in the short term, and even if a decision were taken to do so it might take years to cover

62

R. Focardi, F.L. Luccio, and G. Steel

the whole world-wide network. In the meanwhile the network will fall-back to the magnetic stripe protocol whenever the chip is not supported by the ATM, and well-organised attackers will continue to ‘cash out’ their stolen PINs in countries where magnetic stripe ATMs are still the norm. We have seen that the problem of PIN processing API attacks, in a sense, an excessive parametrization of some of the functionalities. Since these parameters are public, and attacker can manipulate them in order to aﬀect the information that is declassiﬁed by the PIN check. Some of these parameters could be ‘lockeddown’: the dectab, for example, could be ﬁxed on the HSMs of a particular bank, assuming that one bank only uses one dectab for all users. As a more general solution, we have proposed adding a MAC of (a subset of) the function parameters, but this of course requires to change HSMs to support this new operation. A cheap low-impact ﬁx could be obtained using CVVs but this is not completely satisfactory from a security point of view, as it only mitigates the attacks. In our opinion, the direction that bank industry will take is still unclear. The ﬁeld of security API analysis is highly active with an annual international workshop (ASA - Analysis of Security APIs) and a chapter of Anderson’s essential textbook [7] devoted to the theme. Recent articles at ASA and elsewhere have proposed security API techniques for solving problems in anything from social networks to browser Javascript engines to utility meters. Thanks to the ASA forum, the formal methods and security community has come closer to industrial users of security APIs, leading to research results that are applicable in the short term, and hopefully, in the longer term, inﬂuencing the way secure devices are developed, programmed and used. It is, in fact, a very exciting time to be researching security APIs.

References 1. Hackers crack cash machine PIN codes to steal millions. The Times online, http:// www.timesonline.co.uk/tol/money/consumer_affairs/article4259009.ece 2. Mastermind, http://commons.wikimedia.org/wiki/File:Mastermind.jpg 3. PIN Crackers Nab Holy Grail of Bank Card Security. Wired Magazine Blog ’Threat Level’, http://blog.wired.com/27bstroke6/2009/04/pins.html 4. The EMV Standard, http://www.emvco.com/ 5. Anderson, R.: The correctness of crypto transaction sets. In: 8th International Workshop on Security Protocols (April 2000), http://www.cl.cam.ac.uk/ftp/ users/rja14/protocols00.pdf 6. Anderson, R.: What we can learn from API security (transcript of discussion). In: Security Protocols, pp. 288–300. Springer, Heidelberg (2003) 7. Anderson, R.: Security Engineering, 2nd edn. Wiley, Chichester (2007) 8. Armando, A., Basin, D.A., Boichut, Y., Chevalier, Y., Compagna, L., Cu´ellar, J., Hankes Drielsma, P., H´eam, P., Kouchnarenko, O., Mantovani, J., M¨ odersheim, S., von Oheimb, D., Rusinowitch, M., Santiago, J., Turuani, M., Vigan` o, L., Vigneron, L.: The AVISPA tool for the automated validation of internet security protocols and applications. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 281–285. Springer, Heidelberg (2005)

An Introduction to Security API Analysis

63

9. Armando, A., Compagna, L.: SAT-based model-checking for security protocols analysis. Int. J. Inf. Sec. 7(1), 3–32 (2008), Software available at http://www. ai-lab.it/satmc, Currently developed under the AVANTSSAR project, http:// www.avantssar.eu 10. Berkman, O., Ostrovsky, O.M.: The unbearable lightness of PIN cracking. In: Dietrich, S., Dhamija, R. (eds.) FC 2007 and USEC 2007. LNCS, vol. 4886, pp. 224–238. Springer, Heidelberg (2007) 11. Blanchet, B.: From secrecy to authenticity in security protocols. In: Hermenegildo, M.V., Puebla, G. (eds.) SAS 2002. LNCS, vol. 2477, pp. 342–359. Springer, Heidelberg (2002) 12. Bond, M.: Attacks on cryptoprocessor transaction sets. In: Ko¸c, C ¸ .K., Naccache, D., Paar, C. (eds.) CHES 2001. LNCS, vol. 2162, pp. 220–234. Springer, Heidelberg (2001) 13. Bond, M., Anderson, R.: API level attacks on embedded systems. IEEE Computer Magazine 34(10), 67–75 (2001) 14. Bond, M., Clulow, J.: Encrypted? randomised? compromised (when cryptographically secured data is not secure). In: Cryptographic Algorithms and their Uses, pp. 140–151 (2004) 15. Bond, M., Clulow, J.: Extending security protocol analysis: New challenges. Electronic Notes in Theoretical Computer Science 125(1), 13–24 (2005) 16. Bond, M., Zielinski, P.: Decimalization table attacks for pin cracking. Technical Report UCAM-CL-TR-560, University of Cambridge, Computer Laboratory (2003), http://www.cl.cam.ac.uk/TechReports/UCAM-CL-TR-560.pdf 17. Bortolozzo, M., Centenaro, M., Focardi, R., Steel, G.: Attacking and ﬁxing PKCS#11 security tokens. In: Proceedings of the 17th ACM Conference on Computer and Communications Security (CCS 2010), pp. 260–269. ACM Press, Chicago (2010) 18. Cachin, C., Camenisch, J.: Encrypting keys securely. IEEE Security & Privacy 8(4), 66–69 (2010) 19. Cachin, C., Chandran, N.: A secure cryptographic token interface. In: Computer Security Foundations (CSF-22), pp. 141–153. IEEE Computer Society Press, Long Island (2009) 20. Centenaro, M., Focardi, R., Luccio, F.L., Steel, G.: Type-based analysis of PIN processing APIs. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 53–68. Springer, Heidelberg (2009) 21. Clayton, R., Bond, M.: Experience using a low-cost FPGA design to crack DES keys. In: Kaliski Jr., B.S., Ko¸c, C ¸ .K., Paar, C. (eds.) CHES 2002. LNCS, vol. 2523, pp. 579–592. Springer, Heidelberg (2003) 22. Clulow, J.: The design and analysis of cryptographic APIs for security devices. Master’s thesis, University of Natal, Durban (2003) 23. Clulow, J.: On the security of PKCS#11. In: Walter, C.D., Ko¸c, C ¸ .K., Paar, C. (eds.) CHES 2003. LNCS, vol. 2779, pp. 411–425. Springer, Heidelberg (2003) 24. Cortier, V., Steel, G.: A generic security API for symmetric key management on cryptographic devices. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 605–620. Springer, Heidelberg (2009) 25. Delaune, S., Kremer, S., Steel, G.: Formal analysis of PKCS#11. In: Proceedings of the 21st IEEE Computer Security Foundations Symposium (CSF 2008), pp. 331–344. IEEE Computer Society Press, Pittsburgh (2008) 26. Delaune, S., Kremer, S., Steel, G.: Formal analysis of PKCS#11 and proprietary extensions. Journal of Computer Security 18(6), 1211–1245 (2010)

64

R. Focardi, F.L. Luccio, and G. Steel

27. Dolev, D., Yao, A.: On the security of public key protocols. IEEE Transactions in Information Theory 2(29), 198–208 (1983) 28. Durante, A., Focardi, R., Gorrieri, R.: A compiler for analyzing cryptographic protocols using noninterference. ACM Transactions on Software Engineering and Methodology 9(4), 488–528 (2000) 29. Durgin, N.A., Lincoln, P., Mitchell, J.C.: Multiset rewriting and the complexity of bounded security protocols. Journal of Computer Security 12(2), 247–311 (2004) 30. Focardi, R., Luccio, F.L.: Secure upgrade of hardware security modules in bank networks. In: Armando, A., Lowe, G. (eds.) ARSPA-WITS 2010. LNCS, vol. 6186, pp. 95–110. Springer, Heidelberg (2010) 31. Focardi, R., Luccio, F.L.: Guessing bank pins by winning a mastermind game. Theory of Computing Systems (to appear, 2011) 32. Focardi, R., Luccio, F.L., Steel, G.: Blunting diﬀerential attacks on PIN processing APIs. In: Jøsang, A., Maseng, T., Knapskog, S.J. (eds.) NordSec 2009. LNCS, vol. 5838, pp. 88–103. Springer, Heidelberg (2009) 33. Fr¨ oschle, S., Steel, G.: Analysing PKCS#11 key management APIs with unbounded fresh data. In: Degano, P., Vigan` o, L. (eds.) ARSPA-WITS 2009. LNCS, vol. 5511, pp. 92–106. Springer, Heidelberg (2009) 34. Goguen, J.A., Meseguer, J.: Security policies and security models. In: IEEE Symposium on Security and Privacy, pp. 11–20 (1982) 35. Herzog, J.: Applying protocol analysis to security device interfaces. IEEE Security & Privacy Magazine 4(4), 84–87 (2006) 36. IEEE 1619.3 Technical Committee. IEEE storage standard 1619.3 (key management) (draft), https://siswg.net/ 37. International Organization for Standardization. ISO 9564-1: Banking personal identiﬁcation number (PIN) management and security, 30 pages 38. Keighren, G.: Model checking security APIs. Master’s thesis, University of Edinburgh (2007) 39. Knuth, D.: The Computer as a Master Mind. Journal of Recreational Mathematics 9, 1–6 (1976) 40. Kremer, S., Steel, G., Warinschi, B.: Security for key management interfaces. In: Proceedings of the 24th IEEE Computer Security Foundations Symposium (CSF 2011), Cernay-la-Ville, France, pp. 266–280. IEEE Computer Society Press, Los Alamitos (2011) 41. Longley, D., Rigby, S.: An automatic search for security ﬂaws in key management schemes. Computers and Security 11(1), 75–89 (1992) 42. Lowe, G.: Breaking and ﬁxing the Needham Schroeder public-key protocol using FDR. In: Margaria, T., Steﬀen, B. (eds.) TACAS 1996. LNCS, vol. 1055, pp. 147– 166. Springer, Heidelberg (1996) 43. Myers, A.C., Sabelfeld, A., Zdancewic, S.: Enforcing robust declassiﬁcation and qualiﬁed robustness. Journal of Computer Security 14(2), 157–196 (2006) 44. OASIS Key Management Interoperability Protocol (KMIP) Technical Committee. KMIP – key management interoperability protocol (February 2009), http://xml. coverpages.org/KMIP/ 45. openCryptoki, http://sourceforge.net/projects/opencryptoki/ 46. Pickover, C.A.: The Math Book: From Pythagoras to the 57th Dimension, 250 Milestones in the History of Mathematics. Sterling (2009) 47. RSA Security Inc., v2.20. PKCS #11: Cryptographic Token Interface Standard (June 2004) 48. Schneier, B.: Applied Cryptography, 2nd edn. John Wiley and Sons, Chichester (1996)

An Introduction to Security API Analysis

65

49. Steel, G.: Formal Analysis of PIN Block Attacks. Theoretical Computer Science 367(1-2), 257–270 (2006) 50. Stuckman, J., Zhang, G.: Mastermind is NP-Complete. INFOCOMP Journal of Computer Science 5, 25–28 (2006) 51. Tsalapati, E.: Analysis of PKCS#11 using AVISPA tools. Master’s thesis, University of Edinburgh (2007) 52. Youn, P., Adida, B., Bond, M., Clulow, J., Herzog, J., Lin, A., Rivest, R., Anderson, R.: Robbing the bank with a theorem prover. Technical Report UCAM-CL-TR-644, University of Cambridge (August 2005)

Cryptographic Verification by Typing for a Sample Protocol Implementation C´edric Fournet1, Karthikeyan Bhargavan2, and Andrew D. Gordon1,3 1 2 3

Microsoft Research INRIA Rocquencourt University of Edinburgh

Abstract. Type systems are effective tools for verifying the security of cryptographic protocols and implementations. They provide automation, modularity and scalability, and have been applied to large protocols. In this tutorial, we illustrate the use of types for verifying authenticity properties, first using a symbolic model of cryptography, then relying on a concrete computational assumption. (1) We introduce refinement types (that is, types carrying formulas to record invariants) for programs written in F# and verified by F7, an SMT-based type checker. (2) We describe a sample authenticated RPC protocol, we implement it in F#, and we specify its security against active adversaries. (3) We develop a sample symbolic library, we present its main cryptographic invariants, and we show that our RPC implementation is perfectly secure when linked to this symbolic library. (4) We implement the same library using concrete cryptographic primitives, we make a standard computational assumption, and we show that our RPC implementation is also secure with overwhelming probability when linked to this concrete library.

1 Verifying Protocol Implementations Cryptographic Protocols Go Wrong. Security flaws still regularly appear in widelyused protocol implementations, sometimes years after their deployment, despite the best efforts of skilled designers, developers, testers, and reviewers. We may organize these flaws into three categories: logical, cryptographic, and implementation flaws. – As a classic example of a logical flaw, consider the vulnerability in the publickey protocol of Needham and Schroeder (1978), discovered by Lowe (1996) in his seminal paper on model-checking security protocols. This man-in-the-middle attack is the staple example for protocol verification; it is well known in the formal methods research community, and many tools can discover it. Still, for instance, Cervesato et al. (2008) recently discovered that the IETF issued a public-key variant of Kerberos, shipped by multiple vendors, with essentially the same design flaw. – As an example of a cryptographic flaw, consider the padding-oracle attacks on unauthenticated ciphertexts, discovered by Bleichenbacher (1998) and Vaudenay A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 66–100, 2011. c Springer-Verlag Berlin Heidelberg 2011

Cryptographic Verification by Typing for a Sample Protocol Implementation

67

(2002). This is a well-known, practical class of side-channel attacks. Still, it involves subtle interferences between cryptographic algorithms and error handling, and remains difficult to prevent without inspecting many implementation details. For instance, Albrecht et al. (2009) recently showed that SSH implementations were still vulnerable to a similar attack. – Less interestingly, perhaps, implementation flaws due to coding errors are still commonplace. For instance, several security updates patched flaws in the processing of X.509 certificates in TLS clients, leading to secure connections with the wrong server. In contrast with functional testing, security testing does not provide much coverage against active attacks. The difficulty lies in the details of a large amount of code that might compromise the security of a protocol, for example in the parsing of untrusted data or the handling of runtime errors. We would like to reliably design, develop, and deploy protocols and similar programs that rely on cryptography. To this end, we build formal methods and automated tools to relate protocol implementations and cryptographic models. Before presenting one such method, based on refinement types, we briefly discuss cryptographic models and their practical application to production code. Symbolic and Computational Models for Cryptography. In theory, the main security properties of protocols are now well understood. Two main verification approaches have been successfully applied: (1) Symbolic, “black box” models of cryptography, first proposed by Needham and Schroeder (1978) and formalized by Dolev and Yao (1983), focus on the logical structure of the protocol, and assume that their cryptographic primitives are perfect. Cryptographic values range over abstract terms, in an algebra subject to a few rewrite rules. Adversaries range over arbitrary programs that control the network and can intercept, rewrite, and inject messages. These models naturally benefit from formalisms developed for programming languages; their security can be verified using automated tools that scale up to large systems (Abadi and Blanchet 2005). (2) Computational models of cryptography focus on algorithms and the complexity of attacks. Cryptographic values range over concrete bitstrings of bounded length. Adversaries range over probabilistic, polynomial machines. These models are more precise, but also more difficult to apply to large systems. Automated verification is more challenging, although formal tools are starting to appear. Reconciling these two approaches is an active subject of research. Under suitable restrictions and computational assumptions, one can prove that some symbolic models are computationally sound for all protocols, then conduct program verification using off-the shelf symbolic tools. On the other hand, when considering the details of concrete protocols, the existence of a sound symbolic model is far from granted, and it is often more effective to adapt verification tools to deal directly with the cryptographic patterns and assumptions of the protocol being considered. Specifications, Models, and Implementations. In practice, production code and cryptographic models evolve independently, and interact only through informal specifications.

68

C. Fournet, K. Bhargavan, and A.D. Gordon

Specifications and standards are large text documents: their main purpose is to define message formats and promote interoperability between implementations, rather than stating security goals and detailing their local enforcement. Models are short, abstract descriptions handwritten by specialists, using either formal specification languages or pseudo-code. They state what must be verified, but necessarily ignore the details of full-fledged implementations. Implementations, typically coded in C or C++, deal with low-level details. Sadly, their main security concerns still is to avoid buffer overruns and maintain memory safety. Cryptographic issues are manually reviewed from time to time, but this kind of code review can hardly be applied each time the code is modified. We propose to bridge the gap between models and implementations from both directions: we build tools that automatically verify cryptographic properties on source code, rather than simplified models; and we develop models as executable code (in this tutorial, written in F#); this code may serve as a reference implementation, aiming at simplicity rather than performance. It may additionally be used for testing other legacy implementations. Contents. This tutorial illustrates the verification of authentication properties on a sample RPC protocol coded in F#. The protocol relies on cryptographic keyed hashes, also known as MACs. It is a variant of protocols formally studied by Bhargavan et al. (2010) and Fournet et al. (2011). It is small enough to explain its code and its verification in details. We show how to verify it by typechecking against F7 interfaces, first using a symbolic model of MACs, then using a more concrete, computational assumption, namely resistance to existential forgery chosen-message attacks. The rest of the paper is organized as follows. Section 2 reviews F7, our main verification tool. Section 3 describes our sample protocol, presents its F# implementation, and discusses its security. Section 4 develops its modular, logical specification. Section 5 shows how to verify our code against active adversaries within a sample symbolic model of cryptography. Section 6 shows how to verify the same code against active, probabilistic polynomial adversaries, within a simple computational model of cryptography. Finally, Section 7 offers points for further reading. This tutorial omits most formal definitions. Another, complementary tutorial by Gordon and Fournet (2010) presents the refinement type system at the core of F7, and outlines its applications, but does not deal with cryptography. A website http://research.microsoft.com/f7 hosts additional materials, including related research papers, the latest release of the F7 tool, and the code presented in this tutorial.

2 F7: Automated Program Verification Using Refinement Types We now describe a syntax for F# programs annotated with logical specifications and refinement types and show how such programs can be verified using the F7 typechecker. This section is largely independent of the example developed in the rest of the paper. For a more detailed tutorial on the formal language and type system underlying F7, see Gordon and Fournet (2010). Here, we focus on the syntax and command-lines that are commonly used when verifying cryptographic protocols.

Cryptographic Verification by Typing for a Sample Protocol Implementation

69

2.1 Security Programming in F# We first give the core syntax of the fragment of F# we use for writing programs. This fragment is a standard, ML-like call-by-value functional language that includes recursive functions and recursive algebraic datatypes, plus two constructs for writing specifications. We can define data structures such as lists, options, trees, and XML documents in this language. Notably, however, our fragment does not support F# classes, objects, or looping constructs. Still, the language described here has proved adequate to program several thousand lines of security applications and large cryptographic protocols (Bhargavan et al. 2009a, 2010). We believe that for the security programmer, writing in this restricted fragment is a reasonable trade-off in exchange for automated verification support using F7. A program is a sequence of modules whose syntax can be defined as follows: Syntax of F# Modules (patterns, expressions, definitions) P, Q ::= (), true, false, i, “s” x (P1 , . . . , Pn ) hP e ::= (), true, false, i, “s” x (e1 , . . . , en ) fun x → e he e1 e2 let P = e1 in e2 match e with Σi (Pi → ei ) if e then e1 else e2 ) assume (p M) assert (p M) δ ::= let x = e let rec f = fun x → e type α = T type α = Σi (hi : Ti ) open M S ::= module M δ1 . . . δn

Patterns constants: unit, booleans, integers, strings variable tuple constructor Expressions constants: unit, booleans, integers, strings variable tuple function (scope of x is e) constructor application function application sequential composition (scope of variables in P is e2 ) pattern matching (scope of variables in Pi is ei ) conditional assume that p M is true require that p M must be true Definitions value definition recursive function definition (scope of f and x is e) type abbreviation recursive algebraic datatype import another module Module

We first state some syntactic conventions. Our phrases of syntax may contain two kinds of identifier. Type constructors, written α , refer to datatypes and type abbreviations, such as key or bytes. Program variables, written x, y, z refer to values and functions; by convention, we use f to refer to function names, h to refer to constructors, p to refer to predicate names, and l to refer to logical function names (explained later). Each variable has a scope and we identify phrases of syntax up to consistent renaming of bound variables. We write ψ {φ /ι } for the capture-avoiding substitution of the phrase φ for each free occurrence of identifier ι in the phrase ψ .

70

C. Fournet, K. Bhargavan, and A.D. Gordon

Each module (M) is written in a separate file (m.fs) and consists of a sequence of definitions. A definition (δ ) may introduce a new global variable (M.x, M.f) that represents a value or a (possibly recursive) function, or it may define a new type constructor symbol (M.α ) that represents a type abbreviation or a (possibly recursive) algebraic datatype with a set of term constructors (M.hi that take arguments of type Ti ). The types that appear in type definitions and abbreviations are in standard F# type syntax (not the extended F7 syntax defined later in this section). Values and type constructors in another module (N) can be referred to either by using their fully-qualified names (N.x, N.f, N.α ) or by opening the module (open N). As an illustration of the syntax, we display the module Msg from the file msg.fs Serializing messages as byte arrays: (msg.fs) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

module Msg type msg = | Request of string | Response of string let tag0 = Lib.utf8 "0" let tag1 = Lib.utf8 "1" let msg to bytes = fun m → match m with Request s → Lib.concat tag0 (Lib.utf8 s) | Response s → Lib.concat tag1 (Lib.utf8 s) let bytes to msg = fun b → let (tag,sb) = Lib.split 2 b in let s = Lib.iutf8 sb in if tag = tag0 then Request s else if tag = tag1 then Response s else failwith "incorrect tag"

The module Msg defines an algebraic datatype msg, two values tag0 and tag1 and two functions msg to bytes and bytes to msg. The type msg has two constructors Request and Response; each takes a string as argument and constructs a value of type msg. The right-hand-side of each value and function definition is an expression (e). The expressions in Msg use auxiliary functions from the module Lib; utf8 and iutf8 convert between strings and byte arrays; concat and split concatenate and split byte arrays. In addition, they use two predefined functions: = (structural equality) and failwith (to raise an error and terminate the program). An expression (e) may be a constant (2, “0”), a local or global variable (m, Msg.tag0), a tuple, a lambda expression representing a function (e.g. the definition of msg to bytes on line 10), a constructor application (Request s), a function application (Lib.utf8 s), a let-expression for sequential composition (line 17) , or a conditional expression written using either pattern matching (line 11) or if-then-else (line 18).

Cryptographic Verification by Typing for a Sample Protocol Implementation

71

Patterns (P) are used for ML-style pattern matching in match and let expressions. A pattern can be used to check the shape of an expression: whether it is a constant, a tuple (e.g. (tag,sb) on line 16), or uses a given constructor (Request s on line 12). If an expression matches a pattern, then its components are bound to the corresponding variables in the pattern. In addition, we introduce two new expression constructs (not used in Msg) to enable logical specifications within expressions. The expression assume(p M) states that at this point in the program, the logical predicate p can be assumed to hold at value M. Conversely, the expression assert(p M) states that at this point in the program, the logical predicate p is expected to hold at value M. During typechecking, these constructs are interpreted as logical annotations that serve as assumptions and verification goals (explained later in this section.) However, they have no effect at run-time—both constructs are implemented as noops in F#. Each predicate symbol p that appears in an expression must be previously declared as a datatype constructor; hence, each assume or assert is a well-formed F# expression. The programs in the rest of this tutorial conform to the syntax described here. In addition, they may use predefined values, functions, and types from the F# standard library (“Pervasives”). These include the function = that tests for the (structural) equality of two values, and the type list that has constructors for nil ([]) and cons (::). 2.2 Security Specifications in F7 We define a language for writing logical specifications for F# modules as F7 interfaces. The F7 interface language extends F# interfaces, hence every F# interface is an F7 interface, and every F7 interface may be erased to a normal F# interface. The full syntax of interfaces is given on the next page. An F7 interface for a module M is written in a separate file (m.fs7) and consists of a sequence of declarations. A declaration (σ ) may declare the F7 type of a variable or function (M.x, M.f), and this type is then expected to be the type of the corresponding value or function definition in the F# module M. A type abbreviation declaration introduces a new type constructor symbol (M.α ) as an abbreviation for an F7 type (T ). An algebraic type declaration introduces a new type constructor (M.α and a set of term constructors (hi that takes an argument of F7 type Ti ). An abstract type declaration introduces a new type constructor (M.α ) but leaves its implementation completely hidden. More generally, each type constructor may be parametrized by type arguments (e.g. ’t list) or by values (e.g. (’t; p:string) key), but we elide this detail here, and instead we treat parametrized types simply as schemas that are eliminated by inlining. Each value, function, or type declaration may have a visibility qualifier, public (default) or private: public declarations are considered part of the exported interface of the module and mark functions and types that are visible to all other F# modules (including the adversary), whereas private declarations are typically used for auxiliary functions and values that are needed for typechecking but are not exported by the module. We illustrate the syntax by displaying the interface for the Lib module used by Msg. The interface file lib.fs7 declares an abstract type bytes that represents byte arrays. It then introduces two logical functions Utf8 and Length, declares F7 types for the functions utf8, iutf8, length, concat, and split, and introduces three assumes.

72

C. Fournet, K. Bhargavan, and A.D. Gordon

Syntax of F7 Interfaces (values, formulas, types, declarations) M, N ::= (), true, false, i, “s” x (M1 , . . . , Mn ) hM lM C ::= true | false M1 = M2 | M1 = M2 pM not C | C ∧C | C ∨C C ⇒C |C ⇔C !x.C ?x.C T ::= unit, bool, int, string T ref α x : T {C} x : T1 → T2 x : T1 × T2 π ::= public | private σ ::= π val x : T π type α = T π type α = Σi (hi of Ti ) type α function val l : T1 → T2 predicate val p : T → bool assume C assert C I ::= module M σ1 . . . σn

Values constants: unit, booleans, integers, strings variable tuple constructor logical function application First-order Logic Formulas constants comparison predicate application boolean operators implication, if-and-only-if universal, first-order quantification (for all) existential, first-order quantification (exists) Types base types reference type constructor x of type T such that C (scope of x is C) dependent function type (scope of x is T2 ) dependent pair type (scope of x is T2 ) Visibility qualifier Declarations value or function declaration type abbreviation recursive algebraic datatype abstract type logical function logical predicate assume formula assert formula Interface

A logical function declaration introduces a new function symbol (M.l) that may only be used within formulas. The function symbol is initially uninterpreted, but we may add assumptions about its behaviour using an assume declaration. For example, our only assumption about Utf8 is stated in the assume on line 14, namely, that Utf8 is an injective function. When a logical function has more than one argument, they must be written in uncurried form (l(M1 , . . . , Mn )). The Lib interface uses a predefined uninterpreted logical function (|) with two arguments that represents byte array concatenation. A logical predicate declaration (not used in Lib) introduces a new predicate symbol (M.p) that may be used in formulas in interfaces and assume or assert expressions. An assume declaration states that a formula (C) is assumed to be true. Such declarations are used to define logical predicates and functions, and to state verification assumptions. For example, the assumes on lines 15 and 16 states our assumptions on

Cryptographic Verification by Typing for a Sample Protocol Implementation

73

Byte array conversions: (lib.fs7) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

module Lib type bytes function val Utf8: string → bytes function val Length: bytes → int val utf8: s:string → b:bytes {b = Utf8(s)} val iutf8: b:bytes → s:string {b = Utf8(s)} val length: b:bytes → l:int {Length(b) = l} val concat: b1:bytes → b2:bytes → b:bytes{b = b1|b2} val split: i:int{i > 0} → b:bytes → b1:bytes ∗ b2:bytes{b = b1|b2 ∧ Length(b1) = i} assume !x:string, y:string. (Utf8(x)=Utf8(y) ⇒ x = y) assume !x:bytes, y:bytes. Length(x | y) = Length(x) + Length(y) assume !b1,b2,c1,c2. (b1|b2 = c1|c2 ∧ Length(b1)=Length(c1)) ⇒ (b1=c1 ∧ b2=c2)

the relationship between Length and byte array concatenation (|); in particular, concatenating n bytes to an array increases its length by n, and if two equal byte arrays are split at the same index, the results are pointwise equal. An assert declaration requires that a formula (C) must be entailed by the formulas previously assumed. Such declarations are used to formulate (and verify) lemmas that serve as hints for subsequent steps. F7 types (T ) subsume and extend F# types. Like in F#, an F7 type may be a base type, a mutable reference containing a value of type T (T ref), or a type constructor (M.α ) representing a type abbreviation or algebraic type. A refinement type, written x : T {C}, represents a value M of type T such that C{M/x} holds. For example, the type b:bytes {b = Utf8(s)} (line 8) represents all byte arrays M that are equal to Utf8(s). A dependent pair type, written x : T1 × T2 , represents a pair of values (M, N) such that M has type T1 and N has type T2 {x/M}. In particular, we can use dependent pair types to write dependent tuples of the form x1 : T1 × . . . × xn : Tn . For example, the type b1: bytes ∗ b2:bytes{b = b1|b2 ∧ Length(b1)= i} (line 12) represents a pair of byte arrays (M,N) where the refinement type of the second component refers to the first. A dependent function type, written x : T1 → T2 , represents a function that takes an argument M of type T1 and returns a result of type T2 {M/x}. A function type is often written in the form x : T1 {C1 } → y : T2 {C2 }; here, the formula C1 is called the precondition and C2 is called the post-condition. For example, the type of split (line 12) is a function type that takes two arguments, an integer i and byte array b, and returns a pair of byte arrays (b1,b2); its precondition is that i must be greater than 0; its post-condition is that b is the concatenation of b1 and b2 and that the length of b1 is i. Formulas (C) appear in refinement types and in assume and assert declarations. Formulas are written in an untyped first-order logic with uninterpreted function and

74

C. Fournet, K. Bhargavan, and A.D. Gordon

predicate symbols. They may use standard operators for conjunction, disjunction, implication, and universal and existential quantification. For example, the assume on line 16 uses equality, conjunction, implication, and universal quantification. Each predicate or logical function has a fixed arity and must be declared before it can be used. For example, the predefined predicates for equality and inequality have arity 2. The behaviour of predicates can be defined by assuming formulas of the form ∀x.p x ⇔ C. Formulas speak about values (M). A value may be a constant such as a boolean, an integer, or a string (interpreted as a 0-ary function symbol). It may also be a constructed term (hM), where the constructor h is treated as a 1-ary function symbol that is oneto-one and invertible. It may be an n-tuple ((M1 , . . . , Mn )), which is interpreted as the application of an n-ary tuple constructor (Tuplen (M1 , . . . , Mn )). Finally, a value may be the application of a logical function symbol (l M), such as Length(b). 2.3 Modular Verification by Typechecking We consider programs of the form: L1 · · · Lm P1 · · · Pn · O where the modules L1 , . . . , Lm are trusted libraries that are assumed to behave correctly, the modules P1 , . . . , Pn are program modules that we wish to verify, and the module O represents an arbitrary adversary that does not contain any assumes or asserts. The adversary O is given access to the library and program modules through their exported (public) interfaces and hence, it may read any public variable and call any public function (any number of times). The adversary does not have to be well-typed in F7, but it must obey the F# types of the exported interface. Our verification goal is semantic safety: in every run of the program, every asserted formula logically follows from previously assumeed formulas. In particular, if the assert expressions in a program represents security goals, then these security goals should be satisfied in all runs of the program for all adversaries O. Our proof method is typechecking: we verify that each F# module in the program satisfies its typed F7 interface. Each judgment of the F7 type system is given relative to an environment, E, which is a sequence μ1 , . . . , μn , where each μi may be a value or type declaration (σ ), or subtype assumption α fsc lib.fs mac.fs format.fs rpc.fs test.fs -o rpc.exe

and the messages exchanged over TCP are: > ./rpc.exe Connecting to localhost:8080 Sending {BgAyICsgMj9mhJa7iDAcW3Rrk...} (28 bytes) Listening at ::1:8080 Received Request 2 + 2? Sending {AQA0NccjcuL/WOaYS0GGtOtPm...} (23 bytes) Received Response 4

4 Logical Refinements for the RPC Protocol We now develop more precise typed interfaces that embed logical refinements to support our verification effort. The interfaces presented is this section are used both for symbolic and computational verification. 4.1 Refined Interface for Formatting the Authenticated Digests We introduce two auxiliary predicates for the payload formats: Requested and Responded are the (typechecked) postconditions of the functions request and response. Hence, the Format module used by the protocol implementation exports the following interface. Interface to Format Modules (format.fs7): module Format open Lib predicate val Requested: bytes ∗ string → bool predicate val Responded: bytes ∗ string ∗ string → bool val request: s:string → b:bytes {Requested(b,s)} val response: s:string → t: string → b:bytes {Responded(b,s,t)} assert!v,v’,s,s’,t’. (Requested(v,s) ∧ Responded(v’,s’,t’)) ⇒ (v = v’) assert!v,v’,s,s’. (Requested(v,s) ∧ Requested(v’,s’) ∧ v = v’) ⇒ (s = s’) assert!v,s,s’,t,t’. (Responded(v,s,t) ∧ Responded(v,s’,t’)) ⇒ (s = s’ ∧ t = t’)

Cryptographic Verification by Typing for a Sample Protocol Implementation

83

Typechecking involves the automatic verification that our formatting functions are injective and have disjoint ranges, as informally explained in informal assumption (3). Verification is triggered by asserting the formulas marked with assert, so that Z3 proves them. For verifying the rest of the protocol, we can hide the definition of Requested and Responded and rely only on these three derived logical properties, which confirms that the security of our protocol depends only on these properties, rather than a specific digest format. 4.2 Refined, Ideal Interface for Cryptographic MACs To capture the intended properties of MACs, we rely on another, refined ideal interface for the Mac module, as follows: type bytespub = bytes type key type text = bytes type mac = b:bytes{Length(b) = macsize} predicate val Msg: key ∗ text → bool val GEN: unit → key val MAC: k:key → t:text{Msg(k,t)} → mac val VERIFY: k:key → t:text → m:mac → v:bool{v=true ⇒ Msg(k,t)} val LEAK: k:key{!t. Msg(k,t)} → b:bytes {Length(b) = keysize}

This refined interface is designed for protocol verification, and is similar to those used for typing symbolic models of cryptography (Bhargavan et al. 2010). It declares the type of keys as abstract, thus preventing accidental key leakage in data sent on a public network or passed to MAC. To support authentication properties, the interface introduces a logical predicate on keys and texts, Msg(k,t), to indicate that t is an authentic message MACed with key k. (In F7 syntax, predicates are declared as constructors.) This predicate occurs in the types of MAC and VERIFY, as a precondition for MACing and as a post-condition of successful verification. The interpretation of Msg(k,t) is protocol-dependent: as illustrated next for the RPC protocol, each protocol defines Msg according to the properties it wishes to authenticate using MACs, possibly giving a different interpretations to each key. Since our interface must be sound for any logical definition of Msg, this entails that, until the key is leaked, calls to VERIFY succeeds at most for texts previously passed as arguments to MAC, thus excluding the possibility of forging a MAC for any other text. The precondition of LEAK accounts for (well-typed) dynamic key compromise: the protocol may call LEAK to access the actual key bytes at any point, while other parts of the protocol still rely on the postcondition of VERIFY. Thus, the precondition !t, Msg(k,t) demands that, before leaking the key, Msg(k,t) holds for all texts. (In F7 syntax, !t is logical universal quantification over t.) The function LEAK can be used to model the potential corruption of principals by leaking their keys to the adversary; see Bhargavan et al. (2009b, 2010) for protocols verified by typing despite partial key compromise. Taking advantage of refinements, the interface also enforces the consistency of sizes for MACs and key bytes. (In F7, Length is a logical library function.) Thus, for instance,

84

C. Fournet, K. Bhargavan, and A.D. Gordon

typechecking would catch parsing errors leading to a call to VERIFY with a truncated MACs, as its argument would not match the refined type mac. Intuitively, the ideal interface is too good to be true: a simple information-theoretic argument shows that any implementation such that (1) MACs are shorter than texts and (2) MAC and VERIFY communicate only through MACs must be unsafe. (In particular, we do not have C ; I.) In the next two sections, we will show how to formally meet the properties exported by the ideal interface, first symbolically, by implementing MACs using abstracts terms instead of concrete byte arrays, then computationally, by implementing an intermediate error-correcting ideal functionality for MACs. 4.3 Instantiating the MAC Library to the RPC Protocol We are now ready to define the predicate Msg for the RPC protocol, by relating the format of the bytes that are cryptographically authenticated to their interpretation in terms of authentic requests and responses. Usage of Keys: (rpc.fs7) definition !a,b,k. KeyAB(k,a,b) ⇒ ( !m. Msg(k,m) ⇔ Corrupt(a) ∨ Corrupt(b) ∨ (?s. Requested(m,s) ∧ Request(a,b,s)) ∨ (?s,t. Responded(m,s,t) ∧ Response(a,b,s,t)))

For each key generated for the protocol, the definition gives four valid interpretations: one of the two parties is corrupted, or this is a correctly-formatted request, or this is a correctly-formatted response. This definition is not in itself trusted. It is included in the F7 interface for RPC and used to typecheck its code. If the definition was too strong, then we would get a typing error in the code that calls MAC and needs to prove Msg(k,t) as a precondition. If the definition was too weak, then we would get a typing error at the assertions, as the postcondition of VERIFY would not suffice to prove the asserted formula. Exercise 3. Explain how to model the re-use of keys in both directions, so that a key k shared between a and b may be used also for authenticating requests from b to a and responses from a to b? What is the resulting definition of predicate Msg(k,m)? Is the resulting sample protocol well-typed? Is it secure? Otherwise, describe how to fix it. Exercise 4. Explain how to model sessions that consist of a sequence of requests and responses? What is the resulting definition of predicate Msg(k,m)? 4.4 Towards Security Theorems for Our Sample Protocol Before presenting our verified cryptographic libraries, we outline the general form of our security theorems for the RPC protocol, given in the next two sections. We consider the runtime safety of programs of the form Lib · MAC · Format · RPC · Adv

Cryptographic Verification by Typing for a Sample Protocol Implementation

85

where – – – –

Lib is a standard library; MAC is our cryptographic library for MACs; RPC is our protocol code; and Adv is an active adversary, ranging over programs well-typed against the adversary interface. These programs cannot directly assume or assert logical properties, although they can cause RPC and MAC to do so by calling them through the adversary interface. Without loss of generality, we assume that Adv returns (): unit.

Our protocol is deemed secure when all runs of programs of the form above are safe, that is, all their assertions follow from their assumptions. Informally, this excludes any authentication and correlation attack against our protocol using the capabilities granted by the adversary interface. To prove that Lib · MAC · Format · RPC · Adv is safe, we will mostly rely on typing: we prove that Lib · MAC · Format · RPC · Adv : unit

(1)

then conclude by type safety. More precisely, we use F7 to perform the following tasks: (1) (2) (3) (4) (5)

typecheck Lib ; ILib (possibly assuming properties of primitive functions); typecheck ILib MAC ; IMAC ; typecheck ILib Format ; IFormat ; typecheck ILib , IMAC , IFormat RPC ; IRPC ; check the subtyping relation ILib , IMAC , IRPC f7 mac-symb.fs7 formats-symb.fs7 formats-symb.fs This verifies, in particular, that (1) our implementations of request and response comply with their specifications Request and Response, and (2) these specifications logical imply the asserted properties exported by our interface for verifying RPC. To get the details, we may insert the command-line option -scripts formats, which causes F7 to keep an SMP trace of all its calls to Z3, its underlying prover. We then observe that typechecking formats−symb.fs involves calling Z3 on 9 logical proof obligations: 2 for checking implementations against specifications (1); 4 for proving that terms are indeed logical Bytes; and 3 for proving the asserted formulas (2). 5.2 Refinement Types for the RPC Protocol Using F7, we check that our protocol code (with the Net and Crypto library interfaces, and the assumed formulas above) is a well-typed implementation of the interface below. Typed Interface for the RPC Protocol: (rpc.fs7) val keygen: a:principal → b:principal → k:key {KeyAB(k,a,b)} val corrupt: a:principal → b:principal {Corrupt(a) ∨ Corrupt(b)} → k:key {KeyAB(k,a,b)} → b:bytes //omitted symbolically: {Length(b) = keysize} val client: a:principal → b:principal → k:key {KeyAB(k,a,b)} → r:req{ Request(a,b,r) } → unit val server: a:principal → b:principal → k:key {KeyAB(k,a,b)} → ( r:req → s:rsp {Request(a,b,r) ⇒ Response(a,b,r,s)}) → unit

5.3 Modelling the Symbolic Adversary As outlined in Section 3.6, we model an active adversary as an arbitrary program welltyped against an adversary interface, Iadv . In the symbolic model, we define the interface as follows. Recall that type bytespub = t:bytes {Pub(t)}. The function GENPUB generates a new key k, assumes that Msg(k,t) holds for all t, which establishes Pub(k), and returns k. The function princ corrupt assumes the event Corrupt(a) for its argument a, and returns all the keys associated with a; we have Pub(k) for each k associated with a, because, by definition of KeyAB in Section 4.3, the predicate Msg(k,t) holds for all t.

90

C. Fournet, K. Bhargavan, and A.D. Gordon

Adversary Interface: Iadv val send: bytespub → unit val recv: (bytespub → unit) → unit val utf8: string → bytespub val iutf8: bytespub → string val tag: int → bytespub → bytespub val itag: bytespub → int ∗ bytespub val pair: bytespub → bytespub → bytespub val ipair: bytespub → bytespub ∗ bytespub val GENPUB: unit → bytespub val MAC: bytespub → bytespub → bytespub val VERIFY: bytespub → bytespub → bytespub → bool val keygen: principal → principal → unit val client: principal → principal → string val server: principal → principal → (string → string) → unit val princ corrupt: principal → bytespub list

Let a symbolic adversary Adv be a program that contains no assume or assert and that is well-typed against the adversary interface, that is, Iadv Adv : unit. 5.4 Symbolic Soundness for the RPC Protocol We are now ready to formally state our target security theorem for this protocol. We say that an expression is semantically safe when every executed assertion logically follows from previously-executed assumptions. For the formal details, see Bhargavan et al. (2010). The proof of the following is by typechecking with F7, as sketched in Section 4.4. Theorem 2 (Symbolic RPC Safety). For any symbolic adversary Adv, the module Lib · MAC · Format · RPC · Adv is semantically safe.

6 Computational Soundness We now explain how to re-use our refinement-based specification to verify the RPC protocol implementation using a more concrete model of cryptography. Formally, we need to adapt our language semantics to reason about probabilistic and polynomial-time expressions—we refer the interested reader to Fournet et al. (2011) for a detailed presentation. We also need to develop another, computationally-sound implementation of our Mac module given a standard cryptographic assumption, as explained below. On the other hand, computational soundness makes little difference as regards automated protocol verification: we still use the F7 typechecker, and we only need to slightly adapt our F# code and F7 interfaces for the RPC protocol.

Cryptographic Verification by Typing for a Sample Protocol Implementation

91

In contrast with symbolic models, our reasoning relies on the concrete representation of values of type bytes, as they are used for instance in the system implementation of Mac in Section 3.4. We let type bytes = byte array, where byte is the primitive F# type for 8-bit integers. More precisely, we often keep track of the length of byte arrays, so that we can reason about message formats and call cryptographic primitives with correct arguments. Hence, for instance, we let type mac = b:bytes {Length(b)= macsize} for concrete MAC values, so that we consistently use MACs with exactly macsize bytes. Another difference is that our computational libraries need not share a single definition for all cryptographic terms. In Section 5, the symbolic definition of bytes covers MACs, but also UTF8 encodings and concatenation of bytes; more generally, the symbolic libraries of Bhargavan et al. (2010) jointly define all message and cryptographic processing, using large algebraic data types. In contrast, our computational variants of the modules Mac and Format are independent, and more generally each cryptographic primitive can be specified and implemented in a separate module (Fournet et al. 2011). On the other hand, computationally-sound interfaces are usually more restrictive than symbolic ones. We first present our concrete implementation for authenticated message digests, as a first example of byte-level verification, then we discuss computational soundness for MACs using an intermediate, ideal implementation, and we finally give a sample computational security theorem for the RPC code. 6.1 Verifying Binary Message Formats We write (and verify) another pair of formatting functions, this time using concrete bytes and inserting 2-byte lengths for variable-length fields instead of relying on abstract pairs—these two functions are used for compiling and running the protocol. The implementation is: F# Concrete Implementation of Formatting: (format.fs) module Format open Lib let tag0 = [| 0uy |] let tag1 = [| 1uy |] let request s = concat tag0 (utf8 s) let response s t = let sb = utf8 s concat tag1 (concat (int2bytes (length sb)) (concat sb (utf8 t)))

We typecheck this implementation against the interface listed in Section 4.1 supplemented with the declarations below: val tag0: b:bytes { Length(b)=1 } val tag1: b:bytes { Length(b)=1 ∧ b = tag0 } definition !v,s. Requested(v,s) ⇔ v = tag0 | Utf8(s) definition !v,s,t. Responded(v,s,t) ⇔ v = tag1 | Int2Bytes(Length(Utf8(s ))) | Utf8(s) | Utf8(t)

92

C. Fournet, K. Bhargavan, and A.D. Gordon

The refinements on tags explicitly state that they are distinct one-byte arrays; the logical definitions give a complete specification of our digest formats. The automated verification relies on precise type annotations in our library to keep track of the lengths of concatenated byte arrays. For instance, the second asserted property of the interface logically follows by inlining the definition of Requested on both sides of v = v’, and then applying our axiom on equality of concatenations when the length of the first byte arrays are the same. 6.2 A Concrete Refined Interface for MACs We first give more precise types to the concrete MAC implementation of Section 3.4. These types reflect basic assumptions on the underlying cryptographic algorithms, which are outside the scope of this code verification effort; they are used in the proof but they are not sufficient to ensure message authentication. We use the interface type mac = b:bytes{Length(b) = macsize} predicate val GENerated: key → bool predicate val MACed: key ∗ text ∗ mac → bool val GEN: unit → k:key {GENerated(k)} val MAC: k:key → t:text → m:mac {MACed(k,t,m)} val VERIFY: k:key → t:text → m:mac → v:bool{ GENerated(k) ∧ MACed(k,t,m) ⇒ v = true } val LEAK: k:key → b:bytes{Length(b) = keysize} assume !k,t,m0,m1. GENerated(k) ∧ MACed(k,t,m0) ∧ MACed(k,t,m1) ⇒ m0 = m1 In this interface, written I C in the following, two auxiliary predicates GENerated and MACed keep track of the results returned by GEN and MAC. Relying on these predicates, the post-condition of VERIFY states a functional correctness property: verification of a MAC computed using matching keys and texts always succeeds; and the final assumption states that MAC is a deterministic function. These properties are easily met by any implementation that just recomputes the MAC for verification, as our concrete implementation does: to match IC , it suffices to add events at the end of GEN and MAC to record their result, and to rely on the functionality of MAC to establish the post-condition of VERIFY. 6.3 Resistance to Existential Chosen-Message Forgery Attacks (CMA) We now describe our computational notion of security for MACs, expressed as a game between the cryptographic functionality and a probabilistic polynomial-time adversary. We adopt a standard notion, named resistance against existential Chosen-Message forgery Attacks (CMA) and first proposed by Goldwasser et al. (1988) for signatures; we follow the presentation of Bellare et al. (1996; 2005). As usual with concrete cryptography, the MAC key is a finite bitstring, so we cannot entirely exclude that a lucky adversary may just guess the key, or some valid MACs, using for instance random sampling. Instead, our notion of security is relative to a security parameter, which informally controls the length of the key and other cryptographic

Cryptographic Verification by Typing for a Sample Protocol Implementation

93

values, and we bound the probability that an adversary succesfully forges a MAC: we say that a probabilistic, polynomial-time program is asymptotically safe when the probability that any logical assertion fails at runtime is negligible, that is, is a function of the security parameter that converges to zero faster than the inverse of any polynomial. In the game that defines CMA, an adversary attempts to forge a valid MAC; to this end, it can adaptively call two oracles for computing and verifying MACs on a single, hidden key; the adversary wins if it can produce a text and a MAC such that verification succeeds but the text has not been passed to the MAC oracle. We set up this game using an F# module, written CMA and defined as follows: let k = GEN() let log = ref [] let mac t = log := t::!log; MAC k t let verify t m = let v = VERIFY k t m in assert(not v || List.mem t !log); v This code generates a key, implements the two oracles as functions using that key, and maintains a log of all texts passed as arguments to the mac function. We intend to run this code with an adversary given access to mac and verify, but not k or log. The verification oracle also tests whether any verified pair wins the game: the assert claims that this does not happen: either v is false, or t is in the log. (In the asserted formula, || is logical disjunction.) Definition 1. A Mac implementation C is CMA-secure when, for all p.p.t. expressions A with no assume or assert such that mac: text → mac, verify: text → mac → bool A: unit, the expression C · CMA · A is asymptotically safe. We recall the two main security theorems of Fournet et al. (2011) for MACs, which establish asymptotic safety for programs that are well-typed against the ideal interface of a CMA-secure MAC, detailed in Section 4.2 and written IMAC below. Crucially, although keys are just bytes, their type in IMAC is abstract, so that protocols cannot directly generate or access their values. Theorem 3 (Asymptotic Safety for MAC). Let C be a p.p.t. implementation of the refined concrete interface: C ; I C . Let A be a p.p.t. expression such that I A : unit. If C is CMA-secure, then C · A is asymptotically safe. We will apply this theorem to show that our sample RPC code is secure against all active probabilistic polynomial-time adversaries, but first we explain how it is established by programming and typechecking. 6.4 An Ideal Functionality for MACs Our main tool is an intermediate implementation of Mac, written F and defined below, such that C · F is well-typed against the ideal interface IMAC . (We call F an ideal functionality for MACs, and call C · F an ideal MAC implementation; in terms of universal composability, C emulates F, with C as a simulator.) A second security theorem states that the concrete and ideal implementations are indistinguishable (≈ε ), that is,

94

C. Fournet, K. Bhargavan, and A.D. Gordon

the probability that a probabilistic polynomial adversary guesses which of the two implementations it is running with minus 12 (the probability of guessing at random) is negligible. Theorem 4 (Ideal Functionality for MAC). Let C be a p.p.t. CMA-secure implementation of the concrete-refined interface IC . Let A be p.p.t. and such that I A : bool. We have C · A ≈ε C · F· A. The proof relies on a standard code-based cryptographic argument to relate the code of F to the similar, but simpler code that defines CMA security (see Fournet et al. 2011). Intuitively, the ideal implementation is thus both perfectly secure and indistinguishable from the concrete one. This is achieved by intercepting all calls to the concrete implementation and correcting the result of concrete verification from true to false when the message has not been MACed and the key has not been leaked—our CMA assumption implies that such corrections occur with negligible probability. Our ideal functionality re-defines the type for keys, now represented as integer indexes, and re-defines the functions GEN, MAC, VERIFY, and LEAK using those provided by any implementation of the refined concrete interface I C . It maintains a global, private association table ks that maps each key index to some internal state, consisting of a concrete MAC key (kv), a list of previously-MACed texts (log), and a boolean flag (leaked) indicating whether the key has been leaked or not. In the code below, qualified names such as Mac.GEN refer to entries from the concrete interface I C , while names such as GEN refer to their re-definition. Ideal Functionality for MACs: (macf.fs) let ks = ref [] (* state for all keys generated so far *) let GEN () = let k = list length !ks let kv = Mac.GEN() in ks := (k,(kv,empty log k,false flag k))::!ks; k let MAC k t = let (kv,log,leaked) = assoc k !ks in log := t:: !log; Mac.MAC kv t let VERIFY k t m = let (kv,log,leaked) = assoc k !ks in Mac.VERIFY kv t m && (mem k t !log || !leaked) let LEAK k = let (kv,log,leaked) = assoc k !ks in leaked := true; Mac.LEAK kv In this code, the auxiliary functions list length, assoc, and mem are the standard ML functions List.length, List.assoc, and List.mem with more precise refinement types, and the functions empty log and false flag allocate mutable references that initially contain an empty list and false, respectively. The four main functions are as follows:

Cryptographic Verification by Typing for a Sample Protocol Implementation

95

– GEN allocates a key with index k (the number of keys allocated so far), generates a concrete key, and records its initial state, with an empty log and a flag set to false. When called with a key, the three other functions first perform a table lookup to retrieve its state (kv,log,leaked). – MAC computes the MAC using the concrete kv and adds the MACed text to the log. – VERIFY performs the concrete MAC verification, then it corrects its result from true to false when (i) the concrete verification succeeds; but (ii) the text has not been MACed, as recorded in log; and (iii) the key has not been leaked, as recorded by leaked. – LEAK returns the concrete key bytes, and sets the leaked flag to true. The state entries in the table ks are typed using the following auxiliary declarations: type (;k:key) msg = t:text{ Msg(k,t) } type (;k:key) leak = b:bool{ b=true ⇒ !t. Msg(k,t) } type (;k:key) state = Mac.key ∗ (;k) msg list ref ∗ (;k) leak ref val ks: (k:key ∗ (;k) state) list ref These types enforce runtime invariants: all texts t kept in the log for key k are such that Msg(k,t), and we have !t, Msg(k,t) whenever the flag is set to true. (In F7 syntax, (;k:key) msg declares a type whose refinements depend on the value k.) We automatically typecheck I C F ; I ; intuitively, this holds because the result of the verification function is always consistent with the log and the flag, whose refinement types record enough facts Msg(k,t) to establish the post-condition of VERIFY. We have also supposed that C ; IC , so the module C · F obtained by composing the concrete MAC implementation C with F is a well-typed implementation of the ideal interface IMAC , as F corrects any unsafe verification of C, but it is unrealistic, as F relies on global shared state. We conclude this development with an informal proof of Theorem 3 (Asymptotic Safety for MAC). For a given A such that IMAC A, we obtain C · F · A by composing our typing judgments, so C · F · A is perfectly safe, by type safety. Moreover, by Theorem 4 (Ideal Functionality for MAC). C · F · A and C · A are indistinguishable, so C · A is also safe except with a negligible probability. 6.5 Computational Soundness for the RPC Protocol We give on the next page a variant of our code of the RPC protocol. This code differs from that presented in Section 3 only in its use of the network: in the computational model of computation, we cannot rely on parallel threads to send and receive messages; instead, we let the adversary schedule the two stages of the client and the server. On the other hand, the code for generating and verifying messages is unchanged. For the second stage of the client, client2, the type interface is as follows val client2: a:principal → b:principal → k:key {KeyAB(k,a,b)} → r:req{ Request(a,b,r) } → msg → (s:rsp{ Response(a,b,r,s) ∨ Corrupt(a) ∨ Corrupt(b) }) option

96

C. Fournet, K. Bhargavan, and A.D. Gordon

F# Implementation for the Authenticated RPC Protocol: (rpc.fs) module RPC open Pi open Lib open Mac open Format type principal = string type req = string type rsp = string type msg = bytes type fact = KeyAB of key ∗ principal ∗ principal let keygen (a:principal) (b:principal) = let k = GEN() in assume(KeyAB(k,a,b)); k let corrupt (a:principal) (b:principal) k = LEAK k let client (a:principal) (b:principal) k s = let msg1 = concat (utf8 s) (MAC k (request s)) send msg1 let client2 (a:principal) (b:principal) k s msg2 = let (v,m’) = split macsize msg2 in let t = iutf8 v in if VERIFY k (response s t) m’ then Some(t) else None let server (a:principal) (b:principal) k service = recv (fun msg1 → let (v,m) = split macsize msg1 in let s = iutf8 v in if VERIFY k (request s) m then let t = service s let msg2 = concat (utf8 t) (MAC k (response s t)) send msg2)

Cryptographic Verification by Typing for a Sample Protocol Implementation

97

Let Net range over modules implementing the network and the scheduler, informally controlled by the adversary, with the following interface for the protocol:

INet = send: bytes → unit, recv: (bytes → unit)→ unit

In addition, Net may export an arbitrary interface INET for the adversary. Let CRPC be the sample protocol code above, after inlining its other (trusted) library functions, with the

following interface for the adversary: IRPC = client: string → unit, server: unit → unit We arrive at the following computational safety theorem:

Theorem 5 (Computational RPC Safety). Let C be a CMA-secure p.p.t. module with no assert such that C ; I C . Let Net be a module with no assert such that Net ;

INET , Net ; INET , and Net ·CRPC is p.p.t. Let A be a polynomial expression with no

assert such that INET , IRPC A : unit. The expression C · Net ·CRPC · A is asymptotically safe. (Technically, we state our p.p.t. assumption on Net composed with CRPC because Net

has a second-order interface.) In case INET consists of first-order functions on bytes, and for each concrete definition of Net, we obtain as a special case a computational safety theorem for an adversary A with traditional oracle access to the network and the protocol. Typability of A is then irrelevant, since A exchanges only raw bytes with the rest of the system and may implement any polynomial computation on those bitstrings. Although we now rely on computational assumptions, our protocol code still typechecks essentially for the same reasons as in Section 5. Compared with our symbolic theorem, the computational theorem involves a more complex semantics, but a simpler adversary interface: the computational adversary need not be given access to the cryptographic library or formatting functions such as concat, since it may instead include its own concrete implementation of MAC and formatting algorithms.

7 Further Reading The companion tutorial (Gordon and Fournet 2010) focuses on the type theory underlying F7, and includes pointers for further reading on operational semantics, type systems, process calculi, and refinement types. That tutorial also discusses some of the initial applications of F7, including synthesis of security protocols for multiparty sessions (Bhargavan et al. 2009a), security of multi-tier web programming (Baltopoulos and Gordon 2009), auditability by typing (Guts et al. 2009), and the use of the refined state monad to verify imperative F# code (Borgstr¨om et al. 2011), a line of work resulting in the use of F7 to verify that SQL transactions maintain database integrity (Baltopoulos et al. 2011). Recent applications of F7 to cryptographic software include verifications for F# reference implementations of the XML-based CardSpace protocol (Bhargavan et al. 2008a, 2010) and of the DKM key management library (Acar et al. 2011). The present tutorial introduces F7 as a tool for verifying cryptographic software in both the symbolic and computational models. We have illustrated an approach to computational verifications based on the work of Fournet et al. (2011); Backes et al.

98

C. Fournet, K. Bhargavan, and A.D. Gordon

(2010) develop an alternative approach, in which they obtain computationally sound verifications of F7 programs by appeal to the general CoSP framework. F7 does not support union or intersection types; Backes et al. (2011) extend the type theory of F7 with these features and show their utility for dealing with public-key cryptography and zero-knowledge proofs. In other recent work, Swamy et al. (2011) present the design and implementation of F*, which integrates the refinement types of F7 into the Fine language (Swamy et al. 2010), and hence supports cryptographic software. In earlier work on protocols in F#, Bhargavan et al. (2008b) develop FS2PV, which analyzes F# software by compilation to ProVerif (Abadi and Blanchet 2005). ThepioneersofcryptographicsoftwareverificationareGoubault-Larrecq and Parrennes (2005), whose tool Csur analyzes secrecy properties of C code via a custom abstract interpretation. Another verification tool for C is ASPIER (Chaki and Datta 2009), which relies on various abstractions and on software model-checking techniques. Dupressoir et al. (2011) recast the F7 approach to symbolic verification of protocol code in the setting of the general-purpose C verifier VCC (Cohen et al. 2009) so as to verify protocol code in C; their work is the first to prove memory safety and security properties simultaneously for C. (F7 need only prove security properties, as the underlying F# type system proves memory safety directly.) Other approaches to C cryptographic protocols include: automatic generation of protocol monitors, to detect if production protocols deviate from their specification, from abstract protocol descriptions Pironti and J¨urjens (2010), and the use of symbolic execution (Corin and Manzano 2011; Aizatulin et al. 2011) to extract potentially verifiable symbolic representations of cryptographic computations in C. In summary, research on verifying cryptographic implementations is an active area. As this tutorial illustrates, F7 shows the viability of refinement types for verifying security properties of F# code in both the symbolic and computational models. It remains a research challenge to scale up verification to large, existing codebases, and to languages more commonly used for cryptographic software, including C, Java, C#, and even JavaScript.

References Abadi, M., Blanchet, B.: Analyzing security protocols with secrecy types and logic programs. JACM 52(1), 102–146 (2005) Abadi, M., Gordon, A.D.: A calculus for cryptographic protocols: The spi calculus. Information and Computation 148, 1–70 (1999) Acar, T., Fournet, C., Shumow, D.: DKM: Design and verification of a crypto-agile distributed key manager (2011), http://research.microsoft.com/en-us/ um/people/fournet/dkm/ Aizatulin, M., Gordon, A.D., J¨urjens, J.: Extracting and verifying cryptographic models from C protocol code by symbolic execution (in submission, 2011) Albrecht, M., Paterson, K., Watson, G.: Plaintext recovery attacks against SSH. In: 2009 30th IEEE Symposium on Security and Privacy, pp. 16–26 (May 2009) Backes, M., Maffei, M., Unruh, D.: Computationally sound verification of source code. In: ACM CCS, pp. 387–398 (2010) Backes, M., Hrit¸cu, C., Maffei, M.: Union and intersection types for secure protocol implementations. In: TOSCA (2011)

Cryptographic Verification by Typing for a Sample Protocol Implementation

99

Baltopoulos, I., Gordon, A.D.: Secure compilation of a multi-tier web language. In: ACM SIGPLAN Workshop on Types in Language Design and Implementation (TLDI 2009), pp. 27–38 (2009) Baltopoulos, I., Borgstr¨om, J., Gordon, A.D.: Maintaining database integrity with refinement types. In: ECOOP 2011 (2011) Bellare, M., Rogaway, P.: Introduction to modern cryptography. In: UCSD CSE 207 Course Notes (2005) Bellare, M., Canetti, R., Krawczyk, H.: Keying hash functions for message authentication. In: Koblitz, N. (ed.) CRYPTO 1996. LNCS, vol. 1109, pp. 1–15. Springer, Heidelberg (1996) Bhargavan, K., Fournet, C., Gordon, A.D., Swamy, N.: Verified implementations of the Information Card federated identity-management protocol. In: Proceedings of the ACM Symposium on Information, Computer and Communications Security (ASIACCS 2008), pp. 123–135. ACM Press, New York (2008) Bhargavan, K., Fournet, C., Gordon, A.D., Tse, S.: Verified interoperable implementations of security protocols. ACM TOPLAS 31, 5:1–5:61 (2008) Bhargavan, K., Corin, R., Deni´elou, P.-M., Fournet, C., Leifer, J.J.: Cryptographic protocol synthesis and verification for multiparty sessions. In: CSF, pp. 124–140. IEEE Computer Society, Los Alamitos (2009) Bhargavan, K., Corin, R., Deni´elou, P.-M., Fournet, C., Leifer, J.J.: Cryptographic protocol synthesis and verification for multiparty sessions. In: 22nd IEEE Computer Security Foundations Symposium (CSF 2009), pp. 124–140. IEEE Computer Society, Los Alamitos (2009) Bhargavan, K., Fournet, C., Gordon, A.D.: Modular verification of security protocol code by typing. In: ACM Symposium on Principles of Programming Languages (POPL 2010), pp. 445–456 (2010) Bleichenbacher, D.: Chosen ciphertext attacks against protocols based on the RSA encryption standard PKCS #1. In: Krawczyk, H. (ed.) CRYPTO 1998. LNCS, vol. 1462, pp. 1–12. Springer, Heidelberg (1998) Borgstr¨om, J., Gordon, A.D., Pucella, R.: Roles, stacks, histories: A triple for Hoare. Journal of Functional Programming 21(2), 159–207 (2011) Cervesato, I., Jaggard, A.D., Scedrov, A., Tsay, J.-K., Walstad, C.: Breaking and fixing public-key Kerberos. Information and Computation 206(2-4), 402–424 (2008) Chaki, S., Datta, A.: ASPIER: An automated framework for verifying security protocol implementations. In: IEEE Computer Security Foundations Symposium, pp. 172– 185 (2009) Cohen, E., Dahlweid, M., Hillebrand, M.A., Leinenbach, D., Moskal, M., Santen, T., Schulte, W., Tobies, S.: VCC: A practical system for verifying concurrent C. In: Berghofer, S., Nipkow, T., Urban, C., Wenzel, M. (eds.) TPHOLs 2009. LNCS, vol. 5674, pp. 23–42. Springer, Heidelberg (2009) Corin, R., Manzano, F.A.: Efficient symbolic execution for analysing cryptographic pro´ Wieringa, R., Zannone, N. (eds.) ESSoS tocol implementations. In: Erlingsson, U., 2011. LNCS, vol. 6542, pp. 58–72. Springer, Heidelberg (2011) Dolev, D., Yao, A.: On the security of public key protocols. IEEE Transactions on Information Theory IT-29(2), 198–208 (1983)

100

C. Fournet, K. Bhargavan, and A.D. Gordon

Dupressoir, F., Gordon, A., J¨urjens, J., Naumann, D.: Guiding a general-purpose C verifier to prove cryptographic protocols. In: 24th IEEE Computer Security Foundations Symposium (to appear, 2011) Eastlake, D., Jones, P.: US Secure Hash Algorithm 1 (SHA1). RFC 3174 (2001) Fournet, C., Kohlweiss, M., Strub, P.-Y.: Modular code-based cryptographic verification. Technical report, sample code, and formal proofs (2011), http:// research.microsoft.com/˜fournet/comp-f7/ Goldwasser, S., Micali, S., Rivest, R.: A digital signature scheme secure against adaptive chosen-message attack. SIAM Journal on Computing 17(2), 281–308 (1988) Gordon, A.D., Fournet, C.: Principles and applications of refinement types. In: Esparza, J., Spanfelner, B., Grumberg, O. (eds.) Logics and Languages for Reliability and Security. IOS Press, Amsterdam (2010); Available as Microsoft Research Technical Report MSR–TR–2009–147 Goubault-Larrecq, J., Parrennes, F.: Cryptographic protocol analysis on real C code. In: Cousot, R. (ed.) VMCAI 2005. LNCS, vol. 3385, pp. 363–379. Springer, Heidelberg (2005) Guts, N., Fournet, C., Zappa Nardelli, F.: Reliable evidence: Auditability by typing. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 168–183. Springer, Heidelberg (2009) Krawczyk, H., Bellare, M., Canetti, R.: HMAC: Keyed-hashing for message authentication. RFC 2104 (1997) Lowe, G.: Breaking and fixing the Needham-Schroeder public-key protocol using FDR. In: Margaria, T., Steffen, B. (eds.) TACAS 1996. LNCS, vol. 1055, pp. 147–166. Springer, Heidelberg (1996) Milner, R.: Communicating and Mobile Systems: the π -Calculus. Cambridge University Press, Cambridge (1999) Needham, R., Schroeder, M.: Using encryption for authentication in large networks of computers. Commun. ACM 21(12), 993–999 (1978) Pironti, A., J¨urjens, J.: Formally-based black-box monitoring of security protocols. In: Massacci, F., Wallach, D., Zannone, N. (eds.) ESSoS 2010. LNCS, vol. 5965, pp. 79–95. Springer, Heidelberg (2010) Swamy, N., Chen, J., Chugh, R.: Enforcing stateful authorization and information flow policies in fine. In: Gordon, A.D. (ed.) ESOP 2010. LNCS, vol. 6012, pp. 529–549. Springer, Heidelberg (2010) Swamy, N., Chen, J., Fournet, C., Strub, P.-Y., Bhargavan, K., Yang, J.: Secure distributed programming with value-dependent types. In: International Conference on Functional Programming, ICFP 2011 (to appear, 2011) Vaudenay, S.: Security flaws induced by CBC padding - applications to SSL, IPSEC, WTLS... In: Knudsen, L.R. (ed.) EUROCRYPT 2002. LNCS, vol. 2332, pp. 534– 545. Springer, Heidelberg (2002)

Model-Driven Development of Security-Aware GUIs for Data-Centric Applications David Basin1 , Manuel Clavel2,3 , Marina Egea2, Miguel A. Garc´ıa de Dios2 , Carolina Dania2 , Gonzalo Ortiz2 , and Javier Valdazo2 1

ETH Z¨ urich, Switzerland [email protected] 2 IMDEA Software Institute, Madrid, Spain {manuel.clavel,marina.egea,miguelangel.garcia}@imdea.org, {carolina.dania,gonzalo.ortiz,javier.valdazo}@imdea.org 3 Universidad Complutense, Madrid, Spain [email protected]

Abstract. In this tutorial we survey a very promising instance of modeldriven security: the full generation of security-aware graphical user interfaces (GUIs) from models for data-centric applications with access control policies. We describe the modeling concepts and languages employed and how model transformation can be used to automatically lift security policies from data models to GUI models. We work through a case study where we generate a security-aware GUI for a chatroom application. We also present a toolkit that supports the construction of security, data, and GUI models and generates complete, deployable, web applications from these models.

1

Introduction

Model building is at the heart of system design. This is true in many engineering disciplines and is increasingly the case in software engineering. Model-driven engineering (MDE) [7] is a software development methodology that focuses on creating models of diﬀerent system views from which system artifacts such as code and conﬁguration data are automatically generated. Proponents of model-driven engineering have in the past been guilty of making overambitious claims: positioning it as the Holy Grail of software engineering where modeling completely replaces programming. This vision is, of course, unrealizable in its entirety for simple complexity-theoretic reasons. If the modeling languages are suﬃciently expressive then basic problems such as the consistency of the diﬀerent models or views of a system become undecidable. However, there are specialized domains where MDE can truly deliver its full potential: in our opinion, security-aware GUIs for data-centric applications is one of them. Data-centric applications are applications that manage information, typically stored in a database. In many cases, users access this information through graphical user interfaces (GUIs). Informally, a GUI consists of widgets (e.g., windows, text-ﬁelds, lists, and combo-boxes), which are visual elements that display and A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 101–124, 2011. Springer-Verlag Berlin Heidelberg 2011

102

D. Basin et al.

store information and support events (like “clicking-on” or “typing-in”). A GUI deﬁnes the layout for the widgets, as well as the actions that the widgets’ events trigger either on the application’s database (e.g., to create, delete, or update information) or upon other widgets (e.g., to open or close a window). There is an important, but little explored, link between visualization and security: When the application data is protected by an access control policy, the application GUI should be aware of and respect this policy. For example, the GUI should not display options to users for actions (e.g., to read or update information) that they are not authorized to execute on application data. This, of course, prevents the users from getting (often cryptic) security warnings or error messages directly from the database management system. It also prevents user frustration, for example from ﬁlling out a long electronic form only to have the server reject it because the user lacks a permission to execute some associated action on the application data. However, manual encoding the application’s security policy within the GUI code is cumbersome and error prone. Moreover, the resulting code is diﬃcult to maintain, since any changes in the security policy will require manual changes to the GUI code. In this tutorial we spell out our model-driven engineering approach for developing security-aware GUIs for data-centric applications. The backbone of this approach, illustrated in Figure 1, is a model transformation that automatically lifts the access control policy modeled at the level of the data to the level of the GUI [2]. More precisely, given a security model (specifying the access control policy on the application data) and a GUI model (specifying the actions triggered by the events supported by the GUI’s widgets), our model transformation generates a GUI model that is security-aware. The key idea underlying this transformation is that the link between visualization and security is ultimately deﬁned in terms of data actions, since data actions are both controlled by the security policy and triggered by the events supported by the GUI. Thus, under our approach, the process of modeling and generating security-aware GUI has the following parts: 1. 2. 3. 4.

Software engineers specify the application-data model. Security engineers specify the security-design model. GUI designers specify the application GUI model. A model transformation automatically generates a security-aware GUI model from the security model and the GUI model. 5. A code generator automatically produces a security-aware GUI from the security-aware model. The other key components of this approach are the languages that we propose for modeling the data (ComponentUML), the access control policy (SecureUML), and the GUI (ActionGUI). These languages are deﬁned by their corresponding metamodels and support the rigorous modeling of a large class of data models, security models, and GUI models. For data models, the main modeling elements are entities, along with their attributes and associations; for security models,

Model-Driven Development of Security-Aware GUIs

Software Engineer

Security Engineer

103

Data Model

Security Model

GUI Model

GUI Engineer

Model Transformation

Security-aware GUI Model

Code Generation

Security-aware GUI Application

Fig. 1. Model-driven development of security-aware GUIs

these elements are roles, permissions (possibly constrained at runtime to satisfy given properties), and the actions associated to these permissions. For GUI models, these elements are widgets, the (possibly conditional) events associated to these widgets, and the (possibly conditional) actions associated to these events. The constraint language OCL [10] is used in all of these models. For security models, OCL is used to formalize the constraints on the permissions. For GUI models, it is used to formalize the conditions on the actions, as well as to specify the information to be displayed in widgets, updated in the database, or passed from one widget to another. To support a full model-driven engineering development process, we have built a toolkit, named the ActionGUI Toolkit. This features specialized model editors for data, security, and GUI models, and implements the aforementioned model transformation to automatically generate security-aware GUI models. Moreover, our toolkit includes a code generator that, given a security-aware GUI model, automatically produces a complete web application, ready to be deployed in web containers such as Tomcat or GlassFish. A key component of this code generator is our translator from OCL to an SQL-based query language [5], which handles the OCL expressions appearing in the security-aware GUI models. More information about our ActionGUI Toolkit can be found at the URL http:// www.bm1software.com/actiongui.html.

104

D. Basin et al.

Overall, we see the full generation of security-aware GUIs from models for data-centric applications as a very promising application for model-driven engineering. By working with models and using code-generators to produce the ﬁnal products, GUI designers can focus on the GUI’s layout and behavior, instead of wrestling with the diﬀerent, often complex, technologies that are used to implement them. Moreover, by using model transformations, the problem of establishing the link between visualization and security is successfully addressed. To appreciate this last point, consider the standard alternative: the default, “ad-hoc” solution of directly hard-coding the security policy within the GUI. This is clearly disadvantageous. First, the GUI designer is often unaware of the application data security policy. Second, even if the designer is aware of it, manual hard-coding the application data security policy within the GUI code is cumbersome and error-prone. Finally, any changes in the security policy will require manual changes to the GUI code that implements this policy, which again is cumbersome and error-prone. Organization We explain in this tutorial our approach for developing security-aware GUIs for data-centric applications and present a toolkit, named ActionGUI, supporting this approach. We begin in Sections 2–4 by introducing our modeling languages for data models (ComponentUML), security models (SecureUML), and GUI models (ActionGUI). We also introduce our running example: a basic chatroom application. In Section 5, we discuss the problem of lifting the security requirements from data models to GUI models, and we present our solution: a model transformation that automatically transforms a GUI model into a security-aware GUI model with respect to the security requirements imposed on the underlying data model. We conclude this tutorial with a discussion on current and future work. All of the models we present here (and many more) are available at the ActionGUI home page. The interested reader can also evaluate there the code generated by the ActionGUI Toolkit from these models.

2

Data Models: ComponentUML

In this and the next two sections, we introduce the modeling languages that we use for the model-driven development of security-aware GUIs for data centric applications. These languages are: ComponentUML, for modeling data; SecureUML, for modeling the access control policy; and ActionGUI, for modeling the application’s GUI. To illustrate the main modeling concepts and relationships provided by these languages, we work through a running example: a simple chat application named ChitChat, which supports multiple chat rooms where users can converse with each other in diﬀerent chat rooms. We begin

Model-Driven Development of Security-Aware GUIs

105

then with ComponentUML, which is the language that we use for modeling the application data. ComponentUML also gives us a context to introduce the constraint language OCL [10], which we intensively use in our approach when modeling both access control policies and the application’s GUIs. Data models provide a data-oriented view of a system. Typically they are used to specify how data is structured, the format of data items, and their logical organization, i.e., how data items are grouped and related. ComponentUML essentially provides a subset of UML class models where entities (classes) can be related by associations and may have attributes and methods. Attributes and association-ends have types (either primitive types or entity types). As expected, the type associated to an attribute is the type of the attribute’s values, and the type associated to an association-end is the type of the objects which may be linked at this end of the association. The ComponentUML metamodel is shown in Figure 2.

Fig. 2. The ComponentUML metamodel

The ChitChat data model. In our ChitChat application, each user has a nickname, a password, an e-mail address, a mood message, and a status. The user may participate and be invited to participate in any number of chat rooms. Each chat room is created by a user, has a name, a starting and ending date, and it manages the messages sent by its participants. The table shown in Figure 3 speciﬁes the data model of the ChitChat application, using the concepts and relationships provided by ComponentUML. Each row in this table corresponds to an entity, and shows its attributes and association-ends, with their corresponding types. Notice that we also indicate, for each association-end, the association-end which corresponds to the other end of the association. The reader can ﬁnd on the ActionGUI home page the graphical representation of the ChitChat data model using the concrete syntax for ComponentUML that is supported by the ActionGUI Toolkit.

106

D. Basin et al. Entity

ChatUser

ChatRoom

body ChatMessage

AssocEnd msgSent participates owns invitedTo

Type ChatMessage ChatRoom ChatRoom ChatRoom

Other end from participants ownedBy invitees

messages participants ownedBy invitees String from chat

ChatMessage ChatUser ChatUser ChatUser ChatUser ChatRoom

chat participates owns invitedTo msgSent messages

Attribute Type nickname String password String email String moodMsg String status String name String start Date end Date

Fig. 3. The ChitChat data specification

OCL: constraints and queries The Object Constraint Language (OCL) [10] is a speciﬁcation language for expressing constraints and queries using a textual notation. As part of the UML standard, it was originally intended for modeling properties that could not be easily or naturally captured using graphical notation (e.g., class invariants in a UML class diagram). In fact, OCL expressions are always written in the context of a model, and they are evaluated on an instance of this model. This evaluation returns a value but does not change anything; OCL is a side-eﬀect free language. We summarize here the main elements of the OCL language which are used in this tutorial. OCL is a strongly type language. Expressions either have a primitive type (namely, Boolean, Integer, Real, and String), a class type, or a collection type, whose base type is either a primitive type or a class type. OCL provides the standard operators on primitive types and on collections. For example, the operator includes checks whether an object is part of a collection, and the operator isEmpty checks whether a collection is empty. More interestingly, OCL provides a dot-operator to access the values of the objects’ attributes and association-ends. For example, let u be an object of the class ChatUser. Then, the expression u.nickname refers to the value of the attribute nickname for the ChatUser u, and the expression u.participates refers to the objects linked to the ChatUser u through the association-end participates. Furthermore, OCL provides the operator allInstances to access to all the objects of a class. For example, the expression ChatRoom.allInstances() refers to all the objects of the class ChatRoom. Finally, OCL provides operators to iterate on collections. These are forAll, exists, select, reject, and collect. For example, ChatUser.allInstances()−>select(u|u.status=’on−line’) refers to the collection of objects of the class ChatUsers whose attribute status has the value “on-line”. ChitChat’s entity invariants. To illustrate the syntax (and the semantics) of the OCL language, we formalize here some entity (class) invariants for ChitChat’s

Model-Driven Development of Security-Aware GUIs

107

data model. For example, the following OCL expression formalizes that users’ nicknames must be unique: ChatUser.allInstances()−>forall(u1,u2| u1 u2 implies u1.nickname u2.nickname).

Similarly, we can formalize that the status of a ChitChat user is either “oﬀline” or “on-line” using the following OCL expression: ChatUser.allInstances() −>forall(u|u.status=’oﬀ−line’ or u.status=’on−line’).

Finally, we can formalize that each message has exactly one sender: ChatMessage.allInstances()−>forAll(m|m.from−>size()= 1).

3

Security Models: SecureUML+ComponentUML

SecureUML [3] is a modeling language for formalizing access control requirements that is based on RBAC [6]. In RBAC, permissions specify which roles are allowed to perform given operations. These roles typically represent job functions within an organization. Users are granted permissions by being assigned to the appropriate roles based on their competencies and responsibilities in the organization. RBAC additionally allows one to organize the roles in a hierarchy where roles can inherit permissions along the hierarchy. In this way, the security policy can be described in terms of the hierarchical structure of an organization. However, it is not possible in RBAC to specify policies that depend on dynamic properties of the system state, for example, to allow an operation only during weekdays. SecureUML extends RBAC with authorization constraints to overcome this limitation. SecureUML provides a language for specifying access control policies for actions on protected resources. However, it leaves open what the protected resources are and which actions they oﬀer to actors. These are speciﬁed in a so-called “dialect”. Figure 4 shows the SecureUML metamodel. Essentially, it provides a language for modeling roles (with their hierarchies), permissions, actions, resources, and authorization constraints, along with their assignments, i.e., which permissions are assigned to a role, which actions are allowed by a permission, which resource is aﬀected by the actions allowed by a permission, which constraints need to be satisﬁed for granting a permission, and, ﬁnally, which resource is aﬀected by an action. In our approach, we use a speciﬁc dialect of SecureUML, named SecureUML+ComponentUML, for modeling the access control policy on data models. The SecureUML+ComponentUML metamodel provides the connection between SecureUML and ComponentUML. Essentially, in this dialect of SecureUML, the protected resources are the entities, along with their attributes, methods, and association-ends (but not the associations as such), and the actions that they offer to the actors are those shown in Figure 5. Essentially, there are two classes of actions: atomic and composite. The atomic actions are intended to map directly

108

D. Basin et al.

Fig. 4. The SecureUML metamodel Resource Entity Attribute Method AssociationEnd

Actions create, read, update, delete, full access read, update, full access execute read, create, delete, full access

Fig. 5. The SecureUML+ComponentUML actions on protected resources

onto actual operations on the database. These actions are: create and delete for entities; read and update for attributes; create and delete for association-ends; and execute for methods. The underlined actions are the composite actions, which hierarchically group lower-level actions. Composite actions allow modelers to conveniently specify permissions for sets of actions. For example, the full access action for an attribute groups together the read and update actions for this attribute. Finally, in SecureUML+ComponentUML, authorization constraints are speciﬁed using OCL, extended by four keywords, self, caller, value, and target. These keywords have the following meanings: – self refers to the root resource upon which the action will be performed, if the permission is granted. The root resource of an attribute, an association-end, or a method is the entity to which it belongs. – caller refers to the actor that will perform the action, if the permission is granted. – value refers to the value that will be used to update an attribute, if the permission is granted. – target refers to the object that will be linked at the end of an association, if the permission is granted. The ChitChat access control policy. For the sake of our running example, consider the following (partial) access control policy for the ChitChat application:

Model-Driven Development of Security-Aware GUIs

109

– Only administrators can create or delete users; – Administrators can read any user’s nickname, email, mood message, and status. – Any user can read and update its own nickname, password, email, mood message, and status. – Any user can read other users’ nicknames, mood messages, and status. – Users can join a chat room by invitation only, but they can leave at any time. The table shown in Figure 6 speciﬁes this (partial) access control policy, using the concepts and relationships provided by SecureUML+ComponentUML. Each row in this table corresponds to a role, and shows its permissions. Moreover, for each permission it shows the actions allowed by the permission, the resource aﬀected by each of these actions, and the constraint that must be satisﬁed for the permission to be granted. For example, the permission DisjointChat authorizes any user to leave a chat room at any anytime. More precisely, it allows any user caller to delete a participates-link between a user self and a chat room target (meaning that the user self leaves the chat room target), but only if the user caller is indeed the user self (that is, the user caller is the one leaving the chat room target), and also the chat room target indeed belongs to the collection of chat rooms linked to the user caller through the association-end participates (that is, the caller is actually participating in the chat room target). The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s access control policy using the concrete syntax for SecureUML+ComponentUML that is supported by the ActionGUI Toolkit. Role

Permission Action create, delete read Admin AnyUser read read read read, update read, update User SelfUser read, update read, update read, update read OtherUser read read JointChat create DisjointChat delete

Resource ChatUser nickname email moodMsg status nickname password email moodMsg status nickname moodMsg status participates

Authorization Constraint

true

caller=self

true

self=caller and caller.invitedTo−>includes(target) participates self=caller and caller.participates−>includes(target)

Fig. 6. The ChitChat access control policy (partial)

110

4

D. Basin et al.

GUI Models: ActionGUI

ActionGUI is a modeling language for formalizing the GUIs of a rich class of data-centric applications. The ActionGUI metamodel is shown in Figure 7. In a nutshell, ActionGUI provides a language to model widgets (e.g., windows, textﬁelds, buttons, lists, and tables), events (e.g., clicking-on, typing-in), and actions, which can be on data (e.g., to update a property of an element in the database) or on other widgets, (e.g., to open a window), as well as the associations that link the widgets with the events that they support and the events with the actions that they trigger. In addition, ActionGUI provides support to formally model the following features: – Widgets can be displayed in containers, which are also widgets (e.g., a window can contain other widgets). – Widgets may own variables, which are in charge of storing information for later use. – Events may be only supported upon the satisfaction of speciﬁc conditions, whose truth value can depend on the information stored in the widgets’ variables or in the database. – Actions may be only triggered upon the satisfaction of speciﬁc conditions, whose truth value can depend on the information stored in the widgets’ variables or in the database. – Actions may take their arguments (values that instantiate their parameters) from the information stored in the widgets’ variables or in the database. The ActionGUI metamodel’s invariants specify: (i) for each type of widget, the “default” variables that widgets of this type always own; (ii) for each type of widget, the type of events that widgets of this type may support; and (iii) for each type of action, the arguments that actions of this type require, as well as the arguments (if any) that these actions may additionally take. In particular, the invariants of ActionGUI’s metamodel formalize, among others, the following constraints about the diﬀerent types of widgets: – Windows. They can contain any type of widget, except windows. Windows are not contained in any widget. – Text-field. They can be typed-in. By default, each text-ﬁeld owns a variable text of type string, which stores the last string typed-in by the user. The value of the variable text is permanently displayed in the text-ﬁeld. – Button. They can be clicked-on. – List. They contain exactly one text-ﬁeld. By default, each list owns a variable rows of type collection. A list displays as many rows as elements are in the collection stored by the variable rows, each row containing exactly one instance of the text-ﬁeld contained by the list. By default, each instance of this text-ﬁeld owns a variable row whose value is the element associated to this row from those stored by the variable rows. Finally, by default, each list owns a variable selected that holds the element associated to the last row selected by the user.

Model-Driven Development of Security-Aware GUIs

111

Fig. 7. The ActionGUI metamodel

– Combo-box. They are similar to lists, except that rows are displayed in a drop-down box. – Table. They are similar to lists, except that they can contain any number of text-ﬁelds, buttons, lists, or even tables. Also, the invariants of ActionGUI’s metamodel formalize, among others, the following invariants about the diﬀerent types of actions: – Create. It creates a data item in the database. It takes two arguments: the type of the new data item (type) and the variable that will store this element for later reference (variable). – Delete (entities). It deletes a data item in the database. It takes as argument the element to be deleted (object). – Read. It reads the value of a data item’s attribute in the database. It takes three arguments: the data item whose property is to be read (object); the property to be read (attribute); and the variable that will store, for later reference, the value read (variable). – Update. It modiﬁes the value of a data item’s attribute in the database. It takes three arguments: the data item whose attribute is to be modiﬁed (object); the attribute to be modiﬁed (attribute); and the new value (value). – Create (association-ends). It creates a new link in the database between two data items. It takes three arguments: the source data item (sourceObject); the target data item (targetObject); and the association-end (associationEnd) through which the target data item will be linked to the source data item.

112

D. Basin et al.

– Delete (association-ends). It deletes a link in the database between two data items. It takes three arguments: the source data item (sourceObject); the target data item (targetObject); and the association-end (associationEnd) from where the target data item will be removed. – Open. It opens a window. It takes as argument the window to be opened (target); additionally, for any of this window’s variables, it can take as argument a value to be assigned to this variable when opening the window. – Back. It goes back to the window from which a window was open. – Set. It assigns a new value to a widget’s variable. It takes two arguments: the variable (target) and the value to be assigned to this variable (value). Finally, actions’ conditions and arguments are speciﬁed in ActionGUI models using OCL, extended with the widget’s variables (always enclosed in square brackets). As expected, when evaluating an OCL expression that contains a widget’s variable, the value of the corresponding subexpression is the value currently stored in the variable. In case of ambiguity, a widget’s variable is denoted by its name, preﬁxed by the name of its widget (followed by a dot). Also, in case of ambiguity, the name of a widget is preﬁxed by the name of its container (followed by a dot). Notice that, within the same containers, widgets have unique names. Moreover, a widget’s variable can only be used within the window that contains its widget, either directly or indirectly. The ChitChat login window. To continue with our running example, consider the following interface for allowing a registered user to login into the ChitChat application: a window (loginWi) containing: – a writable text-ﬁeld (nicknameEn), for the user to type its nickname in; – a writable text-ﬁeld (passwordEn), for the user to type its password in; and – a clickable button (loginBu), for the user to login, using as its nickname and password the strings that it typed in the text-ﬁelds nicknameEn and passwordEn, respectively. Upon successful authentication, the user will be directed to the application’s main menu window (menuWi) as the logged-in user (caller). The table shown in Figure 8 speciﬁes this login window, using the concepts and relationships provided by ActionGUI. Each row in this table correspond to a widget, where the containment relationship is denoted by displaying the widgets using tree-like notation. For each widget, we show the variables owned by the widget and the events that it supports. Moreover, for each event, we show the actions triggered by this event, as well as its arguments, indicating the values for each of the actions’ parameters. However, we neither show in this table the “default” widget’s variables nor the events supported by the widgets when they do not trigger any action. Notice also that there are two elements that we have intentionally left undeﬁned in this table: the value to be assigned to the menuWi’s variable caller when opening the window menuWi, and the condition for this action. We now describe how both elements can be deﬁned using ActionGUI’s extension of OCL.

Model-Driven Development of Security-Aware GUIs

113

Widgets Variables Events Actions Arguments loginWi |nicknameEn |passwordEn |loginBu click-on open() window = menuWi | menuWi.caller = | authenticated user () Upon successful authentication. Fig. 8. The ChitChat login window

– The authenticated user should be the registered user in the database whose nickname and password coincide with the values of the text-variables owned, respectively, by text-ﬁelds nicknameEn and passwordEn. Using our extended OCL, we can deﬁne the authenticated user as follows: ChatUser.allInstances()−>any(u| u.nickname= [nicknameEn.text] and u.password=[passwordEn.text])

Notice that, as one of the invariants of the ChitChat data model, we speciﬁed that nicknames shall be unique. Thus, although the any-iterator will return any registered user satisfying the body of the any-iterator, there will be at most one such registered users. – The condition for opening the window menuWi should be the existence in the database of a registered user whose nickname and password coincide with the values of the text-variables owned, respectively, by text-ﬁelds nicknameEn and passwordEn. We can deﬁne this condition as follows: ChatUser.allInstances()−>exists(u| u.nickname= [nicknameEn.text] and u.password=[passwordEn.text])

The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s login window using the concrete syntax for ActionGUI that is supported by the ActionGUI Toolkit. The ChitChat menu window. Consider now the following interface for allowing a logged-in user to choose an option from ChitChat’s main menu: a window (menuWi), owning a variable caller which stores the logged-in user, and containing: – a selectable list (usersLi) with as many rows as registered users are online, each of these rows containing an unwritable text-ﬁeld (nicknameLa) showing the nickname of the registered user associated to this row; – a clickable button (editProﬁleBu ) for the caller to access the interface for editing the proﬁle (i.e., name, password, email, mood message, and status) of the user selected in the list usersLi;

114

D. Basin et al.

– a clickable button (createChatBu) for the caller to access the interface for creating a new chat room; and – a clickable button (closeChatBu) for the caller to close the window. The table shown in Figure 9 speciﬁes this menu window, using the concepts and relationships provided by ActionGUI. Notice that the collection of data items to be displayed in the list usersLi, namely, the online users, is not formally deﬁned in this table. Using ActionGUI’s extension of OCL, we can deﬁne this collection as follows: ChatUser.allInstances()−>select(u|u.status= ’on−line’)

The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s menu window using the concrete syntax for ActionGUI that is supported by the ActionGUI Toolkit. Widgets Variables Events Actions menuWi caller | usersLi on-create set | | | nicknameLa on-create read || || | editProﬁleBu click-on open | | | createChatBu click-on open | closeChatBu click-on close

Arguments target = rows value = on-line users object = [UsersLi.row] attribute = nickname variable = text window = editProﬁleWi editProﬁleWi.selectedUser = [UsersLi.selected] window = createChatWi

Fig. 9. The ChitChat menu window

5

Security-Aware ActionGUI Models

In this section, we propose our solution to what we believe is the key challenge when modeling security-aware GUIs for data-centric applications: Which method should the GUI designer use for establishing the link between visualization and security? In other words, how should the GUI designer model their GUIs so as to make them aware of and respect the access control policy that protects the application data? As mentioned before, establishing this link is crucial when modeling security-aware GUIs for data-centric applications. Basically, a GUI should not display options to users for actions (e.g., to read or update information) that they are not authorized to execute on application data. This prevents the users from getting security warnings or cryptic error messages directly from the database management system. It also prevents user frustration from ﬁlling out a long electronic form only to have the server reject it because the user lacks a permission to execute some associated action on the application data.

Model-Driven Development of Security-Aware GUIs

115

As we will motivate in this section, manually modeling (and even worse, manually encoding) the application’s security policy within the GUI model is cumbersome, error prone, and scales poorly to large applications. Moreover, the resulting models are diﬃcult to maintain, since any changes in the security policy will require manual changes to the GUI models. Our solution to this problem uses a standard technique in model-driven development: model transformation. In particular, we introduce a model transformation that automatically transforms a GUI model into a security-aware GUI model with respect to the access control policy imposed on the underlying data model. One additional and substantial advantage of our solution is that, by keeping the security models and the GUI model apart, the security engineers and the GUI designers can independently model what they know best and maintain their models independently. The ChitChat edit-profile window. To motivate the problem faced by a GUI designer when modeling a security-aware GUI, let us continue with our running example. Consider the interface for allowing a logged-in user (caller) to edit the proﬁle of a previously chosen user (selectedUser), which may of course be the caller itself. More speciﬁcally, this interface shall consist of a window such that: 1. The current values of the selectedUser’s proﬁle are displayed when opening the window. 2. The caller can type in the new values (if any) for the selectedUser’s proﬁle. 3. The selectedUser’s proﬁle is updated with the new values typed in by the caller when he or she clicks on a designated button. Recall that a registered user’s proﬁle is composed of the following attributes: nickname, password, mood message, email, and status. Recall also that the access control policy for reading and updating users’ proﬁles, as speciﬁed in Figure 6, is the following: 4. A user is always allowed to read and update its own nickname, password, mood message, email, and status. 5. A user is allowed to read another user’s nickname, mood message, and status, but not the user’s password or email. 6. An administrator is always allowed to read a user’s nickname, mood message, status, and email, but not the user’s password. Now, if the GUI designer only takes into consideration the functional requirements (1–3), the ChitChat edit-proﬁle window can be modeled as shown in Figure 10. Namely, a window editProﬁleWi, which owns the variable selectedUser and caller, and contains: – A writable text-ﬁeld nicknameEn, for the caller to type in the new value (if any) with which to update the selectedUser’s nickname. Notice that when the text-ﬁeld nicknameEn is created, its “default” variable text will be assigned the current value of the selectedUser’s nickname, and therefore this value will be the string initially displayed in the text-ﬁeld nicknameEn, as requested.

116

D. Basin et al. Widgets Variables editProﬁleWi caller, selectedUser | nicknameEn | | | passwordEn | | | moodMsgEn | | | emailEn | | | statusEn | | | updateBu | |

Events Actions Arguments

on-create read

on-create read

on-create read

on-create read

on-create read

click-on

update

update | | update | | update | | update | |

object = [selectedUser] attribute = nickname variable = text object = [selectedUser] attribute = password variable = text object = [selectedUser] attribute = moodMsg variable = text object = [selectedUser] attribute = email variable = text object = [selectedUser] attribute = status variable = text object = [selectedUser] attribute = nickname value = [nicknameEn.text] object = [selectedUser] attribute = password value = [passwordEn.text] object = [selectedUser] attribute = moodMsg value = [moodMsgEn.text] object = [selectedUser] attribute = email value = [emailEn.text] object = [selectedUser] attribute = status value = [statusEn.text]

Fig. 10. The ChitChat edit-profile window (although security-unaware)

– Analogous writable text-ﬁelds for each of the other elements in a registered user’s proﬁle: password, mood message, email, and status. – A clickable button updateBu for the caller to trigger the sequence of actions that will update, as requested, the selectedUser’s nickname, password, mood message, and status, with the new values (if any) typed by the caller in the corresponding text-ﬁelds. Obviously, the edit-proﬁle window modeled in Figure 10 does not satisfy the security requirements (4–6). Any caller can read and update any value contained in the proﬁle of any selectedUser! So how can the GUI designer model the editproﬁle window to make it aware of and respect the security requirements (4–6)?

Model-Driven Development of Security-Aware GUIs

117

There are essentially two solutions available. Unfortunately both of them, if applied manually, are cumbersome and error prone and therefore impractical for large applications with complex security policies. To understand the challenge that faces a GUI designer when modeling security-aware GUIs, let us ﬁrst highlight the knowledge that he or she must acquire to accomplish this task. We decompose this into two steps. Step 1: For each action triggered by an event, and for each role considered by the access control policy, the GUI designer must determine (i) the conditions under which the given action can be securely executed by a user with the given role. This depends, of course, on the underlying access control policy, and, more speciﬁcally, on the authorization constraints assigned to the permissions which grant access to execute the given action for the given role. Also, recall that permissions are inherited along the role hierarchy. Notice however that, to obtain (i), the GUI designer can not simply compose, using disjunctions, the authorization constraints assigned to the aforementioned permissions. In particular, the variable caller, if it appears in any of these authorization constraints, must be replaced by the (widget) variable that stores the current application’s user. Furthermore, the variables self, value, and target, if they appear in any of the aforementioned authorization constraints, must also be replaced by the appropriate expressions, based on the arguments taken by the given action. For example, in the case of a readaction, the variable self should be replaced by the value of the parameter object for this action. Step 2: For each event supported by a widget, and for each role considered by the access control policy, the GUI designer must determine (ii) the conditions under which all the actions triggered by the given event can be securely executed by a user with the given role. In this case, for each given role, the GUI designer can simply compose, using conjunctions, the results to determine (i) for every action triggered by the given event. Let us now illustrate, using our running example, the two solutions that are currently available to the GUI modeler for turning a security-unaware GUI model into a security-aware one. To simplify the discussion, we assume from now on that the user currently logged-in is always stored in a (widget) variable caller, that this variable is owned by every window, and that this variable’s value is automatically passed from one window to another window when opening the latter from the former. Similarly, we assume the role of the user currently logged-in (if he or she has several roles, then the “active” role) is always stored in a (widget) variable role, that this variable is owned by every window, and that this variable’s value is automatically passed from one window to another window when opening the latter from the former. Solution A. The ﬁrst solution for the GUI designer consists in modeling as many diﬀerent edit-proﬁle windows as possible security scenarios. In our case, the

118

D. Basin et al.

GUI designer must model three diﬀerent edit-proﬁle windows (editMyProﬁleWi, editOthersProﬁle, and editUsersProﬁle), one for each of the following scenarios: 1. When the caller has the role ‘User’ and coincides with the selectedUser. 2. When the caller has the role ‘User’ but does not coincide with the selectedUser. 3. When the caller has the role ‘Admin’. In particular, the window editMyProﬁleWi associated to the security scenario (1) will contain exactly the same widgets as the window editProﬁleWi in Figure 10. In contrast, the window editOthersProﬁleWi associated to the security scenario (2), will only contain the text-ﬁelds nicknameEn, moodMsgEn , and statusEn (and not the button updateBu), since a user with the role ‘User’ can only read (but not update) other user’s nickname, mood message, and status. Similarly, the window editProﬁlesWi will only contain the text-ﬁelds nicknameEn, moodMsgEn , emailEn, and statusEn (and not the button updateBu), since a user with the role ‘Admin’ can read (but not update) any user’s proﬁle, except its password. Furthermore, for this solution to work, every event intended to give access to the interface for editing users’ proﬁles must also be aware of the security scenarios (1)–(3), in order to open the appropriate edit-proﬁle window. In particular, the GUI designer must associate the sequence of conditional actions shown in Figure 11 to each of the aforementioned events. Notice that for each open-action, the imposed condition formalizes the conditions under which all the actions triggered by all the events supported by the widgets which are contained in the window being opened can be securely executed by a user. That is, the GUI designer must gather all the knowledge corresponding to (ii), in Step 2 above, for every event supported by any widget contained in the window which the action under consideration is about to open. Actions Arguments if [role] = ’User’ and [caller] = [selected user] then open window = editMyProﬁleWi editMyProﬁleWi.selectedUser = [selected user] if [role] = ’User’ and [caller] [selected user] then open window = editOthersProﬁleWi editOthersProﬁleWi.selectedUser = [selectedUser] if [role] = ’Admin’ then open window = editUsersProﬁleWi editUsersProﬁleWi.selectedUser = [selectedUser] Fig. 11. Solution A: Conditions for opening the edit-profile windows

Solution B. Another solution for the GUI designer consists of specifying, for each of event supported by a widget, the conditions under which all the actions triggered by the given event can be securely executed by a user. In our case, the GUI design must convert the edit-proﬁle window model shown in Figure 10 into the one shown in Figure 12.

Model-Driven Development of Security-Aware GUIs

119

Widgets Events Actions Arguments nicknameEn on-create if true then read object = [selectedUser] attribute = nickname variable = text passwordEn on-create if [role] = ’User’ and [caller] = [selectedUser] then read object = [selectedUser] attribute = password variable = text moodMsgEn on-create if true then read object = [selectedUser] attribute = moodMsg variable = text emailEn on-create if (([role] = ’User’ and [caller] = [selectedUser]) or [role] = ’Admin’) then read object = [selectedUser] attribute = email variable = text statusEn on-create if true then read object = [selectedUser] attribute = status variable = text updateBu click-on if [role] = ’User’ and [caller] = [selectedUser] then update object = [selectedUser] attribute = nickname value = [nicknameEn.text] update object = [selectedUser] attribute = password value = [passwordEn.text] update object = [selectedUser] attribute = moodMsg value = [moodMsgEn.text] update object = [selectedUser] attribute = email value = [emailEn.text] update object = [selectedUser] attribute = status value = [statusEn.text] Fig. 12. Solution B: The ChitChat edit-profile window (now security-aware)

This solution is certainly simpler and less intrusive than Solution A with respect to the original, security-unaware GUI model (since, for example, no new windows need to be added to the original design). However, in order to implement this solution, the GUI designer must gather all the knowledge corresponding to (ii), in Step 2 above, for every event supported by every widget in the model.

120

D. Basin et al.

A model-transformation approach. Clearly, manually modeling the application’s security policy within the GUI model is problematic. It is cumbersome, error prone, and scales poorly to large applications. Moreover, it requires the GUI designer to have complete knowledge of the access control policy on the application data. Finally, the resulting GUI models are diﬃcult to maintain. To address the problem of establishing the link between visualization and security, we employ a standard technique from model-driven development: model transformation. A model transformation takes as input a model (or several models) conforming to given metamodel (respectively, several metamodels) and produces as output a model conforming to a given metamodel. More speciﬁcally, the ActionGUI Toolkit implements a model transformation that takes as input an ActionGUI model and a SecureUML+ComponentUML model, and automatically produces as output an ActionGUI model. This output model is identical to the input ActionGUI model except that it is now security-aware with respect to the access control policy speciﬁed in the input SecureUML+ComponentUML model. In particular, our model transformation follows the ideas behind Solution B: it speciﬁes for each event supported by a widget, the conditions under which all the actions triggered by this event can be securely executed by a user. As expected, at the core of this model transformation, we have implemented a function (on the input models) that automatically gathers all the knowledge corresponding to (i) and (ii), in Step 1 and Step 2 above, for each action triggered by an event and for each event supported by a widget, respectively. Continuing with our running example, let us consider how models can be made security-aware and maintained over time. First, how can the GUI designer model the edit-proﬁle window to make it aware of and respect the security requirements (4–6)? Using our ActionGUI Toolkit, the designer simply calls the aforementioned model transformation on ChitChat’s access control policy shown in Figure 6 and the (security-unaware) ChitChat edit-proﬁle window model shown in Figure 10. Our model transformation then automatically generates (in practically no time) the security-aware ChitChat edit-proﬁle window model shown in Figure 12. Finally, what must the GUI designer do, with respect to the edit-proﬁle window model, if ChitChat’s security policy happens to change? For example, suppose that any user is allowed to read the email of other users when the former participates in a chat room where the latter is also participating. The designer must simply call again the model transformation, this time taking as inputs the modiﬁed model of ChitChat’s access control policy and, as before, ChitChat’s (security-unaware) edit-proﬁle window model. shown in Figure 10.

6

Related Work and Conclusions

The ever-growing development and use of information and communication technology is a constant source of security and reliability problems. Clearly we need better ways of developing software systems and approaching software engineering as a well-founded discipline. In many engineering disciplines, model building is at the heart of system design. This is increasingly the case in software engineering

Model-Driven Development of Security-Aware GUIs

121

since model-driven software engineering was ﬁrst proposed over a decade ago. In our opinion, the late adoption of this methodology is due to the diﬃculty in deﬁning eﬀective domain-speciﬁc modeling languages and also the eﬀort required for developers to learn modeling languages and the art of model building. Deﬁning a good domain-speciﬁc modeling language requires ﬁnding the right abstractions and degree of precision to capture relevant aspects of the structure and the logic of a software system. In addition, for a modeling language to be really usable and useful for software developers, appropriate tools must be provided to build models, analyze them, and keep them synchronized with end products. We wish to emphasize in this regard the need to focus on concrete domains like access controls, GUIs, data models, and the like. In our experience, only by limiting the domain is it possible to build suﬃciently precise modeling languages that support the automatic generation of fully functional applications. Automatic code generation brings with it important advantages. Once a code generator is implemented for a platform (a one-oﬀ cost), modelers can use it like a compiler for a very high-level language, dramatically increasing their productivity. But even when the model-driven development process is not completely automatic, there are experience reports (see, for instance, [9,11]), in which the productivity of a developer in industry is said to increase by a factor of 2 or 3. There is an additional argument for using model-driven development to develop security-critical systems, i.e., for model-driven security [1]. Namely, security is often built redundantly into systems. For example, in a web-application, access control may be enforced at all tiers: at the web server, in the back-end databases, and even in the GUI. There are good reasons for this. Redundant security controls is an example of defense in depth and is also necessary to prevent data access in unanticipated ways, for example, directly from the database thereby circumventing the web application server. Note that access control on the client is also important, but more from the usability rather than the security perspective. Namely, although client-side access control may be easy to circumvent, it enhances usability by presenting honest users an appropriate view of their options: unauthorized options can be suppressed and users can be prevented from entering states where they are unauthorized to perform any action, e.g., where their actions will result in security exceptions thrown by the application server or database. The above raises the following question: must one specify security policies separately for each of these tiers? The answer is “no” for many applications. Security can often be understood in terms of the criticality of data and an access control policy on data can be speciﬁed at the level of component (class) models, as discussed in Section 3. Afterwards, an access control policy modeled at the level of components may be lifted to other tiers. When the tiers are also modeled, this lifting can be accomplished using model transformation techniques and in a precise and meaningful way as we have illustrated in Section 5. In general, model transformations support problem decomposition during development where design aspects can be separated into diﬀerent models which are later composed. As a methodology for designing security-aware GUIs, this approach supports

122

D. Basin et al.

the consistent propagation of a security policy from component models to GUI models and, via code generation, to GUI implementations. This decomposition also means that security engineers and GUI designers can independently model what they know best and maintain their models independently. Related Work There are also other tools like WebRatio [12], Olivanova [4], and Lightswitch [8] that support development methodologies for building data-centric applications that are similar to the model-driven methodology presented in this work. In these tools, application development starts by building a data model that reﬂects the data structure required for the database. The development process continues by applying diﬀerent UI generation patterns to the data model. These patterns enable data retrieval, data editing, data creation, and database search. A detailed comparison of the expressiveness provided by the languages supported by these tools with the languages supported by the ActionGUI Toolkit is interesting; however it falls out of the scope of this paper. Nevertheless, we note that, in contrast to what these tools can provide, the ActionGUI Toolkit oﬀers developers the full ﬂexibility to create designs without burdening them with the restrictions imposed by the obligatory use of a ﬁxed number of given patterns. The above tools also impose a major restriction at the level of data management, i.e., at the level of data access and visualization: The information that can be referenced and therefore that can be accessed and visualized within one screen can only come from one table of the database or, at most, from two tables that are reachable from each other within one navigation step. The three tools, WebRatio, Olivanova, and Lightswitch, support the deﬁnition and generation of RBAC policies at diﬀerent granularity levels. Lightswitch supports granting or denying permissions (whose actual behavior must be manually programmed) to execute create, read, update, or delete actions on entities for users in diﬀerent roles. WebRatio and Olivanova also support granting or denying permissions to execute a similar set of actions on entity’s properties, individually, for users in diﬀerent roles. In WebRatio and Olivanova, the role of the authorization constraints could be played by preconditions restraining the invocation of actions through a concrete UI. Note, however, that none of these tools implement an algorithm capable of lifting to the UIs the security policy that governs the access to data. Future Work The toolkit we presented supports the construction of security, data, and GUI models and generates complete, deployable, security-aware web applications from these models. However, there is still much work ahead to turn this toolkit into a full, robust commercial application. In particular, we plan to add to the language (and to its code-generator) other actions which do not act upon the database or the GUI elements. Examples are sending an email to a contact selected in a list or printing a table.

Model-Driven Development of Security-Aware GUIs

123

Our work here, as well as our past work in model-driven security, has focused primarily on access control. However, many systems have security requirements that go beyond access control, for example, obligations on how data must or must not be used once access is granted. We are currently working on handling usage control policies in the context of model-driven security. The challenge here is to deﬁne modeling languages that are expressive enough to capture these policies, support their formal analysis, and provide a basis for generating infrastructures to enforce or, at least, monitor these policies. There are many challenging questions concerning model analysis. Here, our goal is to be able to analyze the consistency of diﬀerent system views. For example, suppose that access control is implemented at multiple tiers (or levels) of a system, e.g., at the middle tier implementing a controller for a web-based application and at the back-end persistence tier. If the policies for both of these tiers are formally modeled, we would like to answer questions like “will the controller ever enter a state in which the persistence tier throws a security exception?” Note that with advances in model transformations, perhaps such questions will some day not even need to be asked, as we can uniformly map a security policy across models of all tiers. Ultimately we see model-driven security playing an important role in the construction and certiﬁcation of critical systems. For example, certiﬁcation under the Common Criteria requires models for the higher Evaluation Assurance Levels. Model-driven security provides many of the ingredients needed: models with a well-deﬁned semantics, which can be rigorously analyzed and have a clear link to code. As the acceptance of model-driven development techniques spread, and as they become better integrated with well-established formal methods that support a detailed behavioral analysis, such applications should become a reality. Acknowledgements. This work is partially supported by the EU FP7-ICT Project “NESSoS: Network of Excellence on Engineering Secure Future Internet Software Services and Systems” (256980) by the Spanish Ministry of Science and Innovation Project “DESAFIOS-10” (TIN2009-14599-C03-01), and by Comunidad de Madrid Program “PROMETIDOS-CM” (S2009TIC-1465).

References 1. Basin, D., Clavel, M., Egea, M.: A decade of model driven security. In: Proceedings of the 16th ACM Symposium on Access Control Models and Technologies (SACMAT 2011). ACM Press, New York (2011) (invited paper, in press) 2. Basin, D., Clavel, M., Egea, M., Schl¨ apfer, M.: Automatic generation of smart, security-aware GUI models. In: Massacci, F., Wallach, D., Zannone, N. (eds.) ESSoS 2010. LNCS, vol. 5965, pp. 201–217. Springer, Heidelberg (2010) 3. Basin, D., Doser, J., Lodderstedt, T.: Model driven security: From UML models to access control infrastructures. ACM Transactions on Software Engineering and Methodology 15(1), 39–91 (2006) 4. Care Technologies. Olivanova – the programming machine (2011), http://www.care-t.com

124

D. Basin et al.

5. Egea, M., Dania, C., Clavel, M.: MySQL4OCL: A stored procedure-based MySQL code generator for OCL. Electronic Communications of the EASST 36 (2010) 6. Ferraiolo, D.F., Sandhu, R.S., Gavrila, S., Kuhn, D.R., Chandramouli, R.: Proposed NIST standard for role-based access control. ACM Transactions on Information and System Security 4(3), 224–274 (2001) 7. Kleppe, A., Bast, W., Warmer, J.B., Watson, A.: MDA Explained: The Model Driven Architecture–Practice and Promise. Addison-Wesley, Reading (2003) 8. Microsoft. Visual studio lightswitch (2010), http://www.microsoft.com/ visualstudio/en-us/lightswitch 9. Mohan, R., Kulkarni, V.: Model driven development of graphical user interfaces for enterprise business applications - experience, lessons learnt and a way forward. In: Sch¨ urr, A., Selic, B. (eds.) MODELS 2009. LNCS, vol. 5795, pp. 307–321. Springer, Heidelberg (2009) 10. Object Management Group. Object Constraint Language specification Version 2.2 (February 2010), OMG document, http://www.omg.org/spec/OCL/2.2 11. Schramm, A., Preußner, A., Heinrich, M., Vogel, L.: Rapid UI development for enterprise applications: Combining manual and model-driven techniques. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) MODELS 2010. LNCS, vol. 6394, pp. 271– 285. Springer, Heidelberg (2010) 12. Web Models Company. Web ratio – you think, you get (2010), http://www. webratio.com

On Intransitive Non-interference in Some Models of Concurrency Roberto Gorrieri and Matteo Vernali Dipartimento di Scienze dell’Informazione, Universit` a di Bologna, Mura A. Zamboni, 7, 40127 Bologna, Italy [email protected]

Abstract. Intransitive non-interference (INI for short) is a behavioural property extensively studied by Rushby over deterministic automata with outputs associated to transitions (Mealy machines) in order to discuss the security of systems where declassiﬁcation of secret information is allowed. In this paper, we ﬁrst propose a natural transposition of Rushby’s deﬁnition on deterministic labelled transition systems, we call INI as well, and then an alternative, yet more easily checkable, formulation of INI, called NI with downgraders (NID for short). We show how NID can be naturally extended to the case of nondeterministic automata by using a variation of it based on bisimulation equivalence (BNID). The most novel contribution of this paper is the extension of this theory on the class of Petri nets called elementary net systems: we propose a semistatic technique, called PBNID and based on the inspection of the net structure, that is shown to be equivalent to BNID.

1

Introduction

Non-interference has been deﬁned in the literature as an extensional property based on some observational semantics: the high part (usually considered the secret part) of a system does not interfere with the low part (i.e., the public one) if whatever is done at the high level produces no visible eﬀect on the low part of the system. The original notion of non-interference in [14] was deﬁned, using trace semantics, for deterministic automata with outputs. Generalized notions of noninterference were then designed to include (nondeterministic) labeled transition systems and ﬁner notions of observational semantics such as bisimulation (see, e.g., [24,7,25,9]). The security properties in this class are based on the dynamics of systems; they are deﬁned by means of one (or more) equivalence check(s); hence, non-interference checking is as diﬃcult as equivalence checking, a wellstudied hard problem in automata theory and concurrency theory. When it is necessary to declassify information (e.g., when a secret plan has to be made public for realization), the two-level approach (H/secret – L/public) is usually extended with one intermediate level of downgrading (D), so that when an action in that class is performed, the previously performed high actions become observable to low users. This security policy is known under the name of A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 125–151, 2011. c Springer-Verlag Berlin Heidelberg 2011

126

R. Gorrieri and M. Vernali

intransitive noninterference (INI for short) because the information ﬂow relation is considered not transitive: even if information ﬂows from level H to D and from D to L are allowed, direct ﬂows from H to L are forbidden. Rusby [23] deﬁnes INI only for deterministic ﬁnite automata with output associated to transitions (Mealy machines), based on earlier work of Haigh and Young [13], in turn inﬂuenced by [15]. The basic intuition is that a machine in this class is INI if for any trace σ, the states reached after σ and ipurge(σ) (where all the action in H not justiﬁed by subsequent actions in D are removed) oﬀer the same output. Here we redeﬁne it on deterministic ﬁnite labeled transition systems, by interpreting output equivalence as low language equivalence of the reached states. This extensional deﬁnition is, however, rather cumbersome as it requires an equivalence check for any possible trace, hence inﬁnitely many checks, despite the fact that ﬁnite automata have only ﬁnitely many states and transitions. Therefore, we propose a diﬀerent property, called NID (non-interference with downgraders) that essentially requires that for any high transition the source state and the target one are equivalent for low observers. NID is reminescent of the so-called unwinding condition in [23], even if ours better reveals the role of downgraders. We prove that NID is an equivalent, yet more easily checkable, characterization of INI. When extending the approach to nondeterministic systems, we observe that NID is inadequate in some cases, as trace semantics is too weak to discriminate situations of possible danger. Nonetheless, by considering bisimulation semantics in place of trace semantics, we get BNID which seems to be fully satisfactory. Intuitively, the many deﬁnitions of (transitive or intransitive) noninterference that have been proposed in the literature try to capture the essence of information ﬂow as an extensional property. On the contrary, one may think that there are clear physical reasons for the occurrence of an information ﬂow, that can be better understood if one exploits a computational model where causality of actions and conﬂict among actions can be modelled directly. Indeed, this is not the case of labeled transitions systems, a typical example of an interleaving model, where parallelism is not primitive. For this reason, in [2,3,4] Busi and Gorrieri have shown that these extensional noninterference properties can be naturally deﬁned also on Petri Nets, in particular on Elementary Nets [6], a well-known model of computation where causality and conﬂict are primitive concepts. More interestingly, they address the problem of deﬁning statically non-interference for Elementary Nets, by looking at the structure of the net systems under investigation: • in order to better understand the relationship between a ﬂow of information and the causality (or conﬂict) relation between the activities originating such a ﬂow, hence grounding more ﬁrmly the intuition about what is an interference, and • in order to ﬁnd more eﬃciently checkable noninterference properties that are suﬃcient (sometimes also necessary) conditions for those that have already received some support in the literature.

On Intransitive Non-interference in Some Models of Concurrency

127

Structural noninterference was ﬁrst proposed in [3,4] on the basis of the absence of particular places in the net. Two special classes of places are of interest: causal places, i.e., places for which there are an incoming high transition and an outgoing low transition; and, conﬂict places, i.e. places for which there are both low and high outgoing transitions. Intuitively, causal places represent potential source of interference because the occurrence of the high transition is a prerequisite for the execution of the low transition. Similarly, conﬂict places represent potential source of interference because if the low event is not executable, then we can derive that a certain high transition has occurred. The absence of causal and conﬂict places is clearly a static property that can be easily checked (linear in the size of the net) by a simple inspection of the (ﬁnite) net structure. Interestingly enough, we show that absence of such places is a suﬃcient condition to ensure BNID. In order to characterize more precisely BNID, the notion of causal place and conﬂict place is slightly reﬁned, yielding the so-called active causal place and active conﬂict place. These new deﬁnitions are based also on a limited exploration of the state-space of the net (i.e. of its marking graph), hence, the absence of such places is not a purely structural property, rather a hybrid, semi-static property. When active causal and active conﬂict places are absent, we get a property, called Positive Place–Based Non–Interference with Downgraders (PBNID for short), which turns out to be equivalent to BNID. 1.1

Contribution of This Paper

The ﬁrst, minor contribution of the paper is a reformulation of Rushby’s deﬁnition [23] of (basic) noninterference (NI for short) in the setting of deterministic labeled transition systems (LTSs for short). Apparently, this reformulation, called NI with abuse of notation, is new because it is the only one requiring low equivalence of the reached states. Then, an alternative characterization of NI called SNDC, is given in terms of a ﬁnite number (equal to the number of high transitions in the LTS) of equivalence checks. One further contribution is its extension to nondetermistic systems, yielding the bisimulation-based SNDC, called SBNDC for short. Both SNDC and SBNDC are not original [7,9,5], but new is the result that proves that NI is equivalent to SNDC for deterministic LTSs. The same game is played for intransitive noninterference. First, we provide a reformulation of Rushby’s deﬁnition [23] in the setting of deterministic LTSs, which is new. Then, the alternative characterization of INI, called NID, is given in terms of a ﬁnite number (equal to the number of high transitions in the LTS) of equivalence checks. Then, one further contribution is its extension to nondetermistic systems, yielding the bisimulation-based NID, called BNID for short. Also in this case, NID and BNID were deﬁned already in [5] under the name of DSN DC and DSBN DC, but new is the result of equivalence between INI and NID.

128

R. Gorrieri and M. Vernali

The main result of the paper is the extension of the structural approach to noninterference of [2,3,4] to the richer setting of intransitive noninterference. We prove that an elementary net is BNID iﬀ it is PBNID. Hence, we essentially prove that there is an illegal information ﬂow directly from H to L iﬀ there is a direct causality (or conﬂict) relation between a high transition and a low one. From a complexity point of view, BNID over LTSs has complexity O(n3 ) (see [5] for a discussion on the complexity of DSBN DC) where n is the number of states. By following the same arguments reported in [10], we can conclude that BNID on elementary nets has complexity O(pn23p ), where p is the number of places and n the number of transitions; hence, since the marking graph has O(2p ) states, BNID is cubic in the number of the states also for elementary net systems. The complexity of checking for the absence of potential causal/conﬂict places is O(f + p), where f is the number of arcs in the net. The complexity of PBNID is O(pn22p ) in the worst case, hence PBNID is quadratic in the number of states. These algorithms, for the case without downgrading actions (i.e., classic noninterference), have been already implemented in a software tool, called the Petri Net Security Checker (PNSC for short), which provides functionalities for creating, editing and executing Petri nets, as well as automatically detecting places that are potentially/actively causal/conﬂict [10]. 1.2

Related Work

Deterministic Systems In [20] Pinsky studies intransitive noninterference (INI) over deterministic Mealy machine; his deﬁnition is very close to Rushby’s original one; his main contribution is an algorithm for detecting this property, whose complexity is not examined, but seems exponential. This work has been extended in [21] to cover the case of nondeterministic machines. In [11,12] an algorithmic approach is proposed to solve the problem of veriﬁcation of the property of INI, using tools and concepts of the theory of supervisory control of discrete event systems (DES). The algorithm is exponential in the number of states of the deterministic automaton. In [17] Ron van der Meyden discusses some alternative variations of INI in the area of deterministic Moore machines. He suggests that Rushby’s deﬁnition of INI is somehow inadequate in some practical cases and he proposes some stronger variations that better ﬁt with Rushby’s unwinding conditions. Nondeterministic Systems In [22] Roscoe and Goldsmith propose a rather discriminating deﬁnition of INI for nondeterministic systems which is based on strong assumptions about determinacy of low-level view. Their view is somehow related to the debate about what nondeterminism can be actually observed.

On Intransitive Non-interference in Some Models of Concurrency

129

Mullins in [19] study the problem of intransitive non-interference for a variant of CCS [18], hence on (nondeterministic) LTSs. His proposed property, called Admissible Interference (AI for short), is based on trace semantics: E is AI if for all reachable state E , (E \ D)/H (where downgrading actions are forbidden and high level actions are invisible) is (weak) trace equivalent to E \ H ∪ D (where both downgrading actions and high level actions are forbidden). It is not diﬃcult to prove that AI is strictly weaker than BNID. For instance, the system E = h.l1 .0 + l1 .0 + l2 .0 is AI, while it is not even NID and indeed, E is not secure because a low level user that cannot perform l2 is sure that h has been performed. Lafrance and Mullins in [16] propose a (weak) bisimulation-based version of AI, called BN AI (reminiscent of Focardi and Gorrieri’s SBSN N I), which turns out to be strictly weaker than BNID. For instance, system F = h.l1 .0 + τ.l1 .0 + l2 .0, which is a slight variant of E above, is BN AI, but it is not even NID and indeed, F is not secure for the same reason as above. In their thorough study [5] Bossi et al. consider various deﬁnitions of unwinding-based INI properties for a CCS-like language, with no precise indication of which of them is the right one. Among these properties, DSBN DC is actually the same deﬁnition as our BNID. No attempt is made to compare their many variants with Rushby’s original deﬁnition. They prove general composability properties (w.r.t. some operators of CCS) and provide conditions under which horizontal and vertical reﬁnements are preserved. Petri Nets To the best of our knowledge, this is the ﬁrst paper approaching INI over Petri nets. It builds over the results of [3,4] for the case of (basic) two-level noninterference, where SBNDC (the property obtained from BNID in absence of downgrading actions) is proved to be equivalent to PBNI+(the property obtained from PBNID when downgrading actions are absent). The paper is organised as follows. In Section 2 we recall the basic background deﬁnitions about Labeled Transition Systems and Elementary Net systems. In Section 3 we study basic noninterference, NI for short, starting from the deﬁnition proposed by Rushby for deterministic automata with outputs. We reformulate it over deterministic LTS and then propose our alternative, unwinding-like deﬁnition (SNDC) and prove the equivalence of the two. We discuss the extension to nondeterministic LTSs, yielding SBNDC. In Section 4 we present Rushby’s deﬁnition of Intransitive Noninterference, INI for short, then we reformulate it over deterministic LTSs and provide an alternative characterization in terms of a local property, called NID. We then consider LTSs in general, and the property BNID, based on bisimulation. In Section 5 we deﬁne BNID over elementary net systems. We then formulate the semi-static property PBNID by looking at the presence of (active) causal/conﬂict places, and provide the proof that BNID is the same as PBNID. In Section 6 we draw some conclusions.

130

2

R. Gorrieri and M. Vernali

Background

2.1

Labeled Transition Systems

Here we recall some basic deﬁnitions over LTSs. Definition 1. A labeled transition system is a triple T S = (St, E, →) where • St is the set of states • E is the set of events • →⊆ St × E × St is the transition relation. e

e

In the following we use s → s to denote (s, e, s ) ∈→. Given a transition s → s , s is called the source, s the target and e the label of the transition. A rooted transition system is a pair (T S, s0 ) where T S = (St, E, →) is a transition system and s0 ∈ St is the initial state. A transition system T S = (St, E, →) is ﬁnitestate if both St and E are ﬁnite sets. Definition 2. A labeled transition system T S = (St, E, →) is deterministic iﬀ e e the following holds: ∀s ∈ St, ∀e ∈ E if s → s1 and s → s2 then s1 = s2 . e

This means that ∀s ∈ St, ∀e ∈ E there is at most one s such that s → s , but such an s may also not exist. Definition 3. Given a labeled rooted transition system T S = (St, E, →, s0 ), a e1 en s2 . . . sn → sn+1 . We say path from s1 to sn+1 is a sequence of transitions s1 → that s is reachable from s if there exists a path from s to s . T S is reduced when the set of states reachable from s0 is exactly St, i.e., all the states are reachable from the initial state. In the following, we will usually consider only reduced labeled rooted transition systems. Definition 4. Let T S = (St, E, →, s0 ) be a rooted transition system. A trace of TS is a (possibly empty) sequence of events e1 . . . en such that there exists a path e1 en . . . sn → sn+1 with s1 = s0 . The set of traces of TS is denoted by T r(s0 ). s1 → Let T S = (St, E, →) be a transition system and let s1 , s2 ∈ St. We say that s1 and s2 are trace equivalent (denoted with s1 ∼ s2 ) iﬀ T r(s1 ) = T r(s2 ). Definition 5. A bisimulation between T S1 and T S2 is a relation R ⊆ (St1 × St2 ) such that if (s1 , s2 ) ∈ R then for all e ∈ (E1 ∪ E2 ) e

e

• ∀s1 such that s1 → s1 , then ∃s2 such that s2 → s2 and (s1 , s2 ) ∈ R e e • ∀s2 such that s2 → s2 , then ∃s1 such that s1 → s1 and (s1 , s2 ) ∈ R. If T S1 = T S2 we say that R is a bisimulation on T S1 . It is well-known that trace semantics and bisimulation semantics do coincide over deterministic LTSs, while in general bisimulation semantics is more discriminating (see, e.g., [18]).

On Intransitive Non-interference in Some Models of Concurrency

2.2

131

Elementary Net Systems

Here we introduce basic deﬁnitions about the class of Petri Nets we use. Some familiarity with Petri net terminology is assumed. More details can be found in [6,3]. Definition 6. An elementary net is a tuple N = (S, T, F ), where • S and T are the (ﬁnite) sets of places and transitions, such that S ∩ T = ∅ • F ⊆ (S × T ) ∪ (T × S) is the ﬂow relation, usually represented as a set of directed arcs connecting places and transitions. A ﬁnite subset of S is called a marking. Given a marking m and a place s, if s ∈ m then we say that the place s contains a token, otherwise we say that s is empty. Let x ∈ S ∪ T . The preset of x is the set • x = {y | F (y, x)}. The postset of x is the set x• = {y | F (x, y)}. The preset and postset functions are generalized in the obvious way to sets of elements: if X ⊆ S ∪ T then • X = x∈X • x and X • = x∈X x• . A transition t is enabled at marking m if • t ⊆ m and t• ∩ m = ∅. The ﬁring (i.e., execution) of a transition t enabled at m produces the marking m = (m \ • t) ∪ t• . This is usually written as m[tm . With the notation m[t we mean that there exists m such that m[tm . An elementary net system is a pair (N, m0 ), where N is an elementary net and m0 is a marking of N , called initial marking. With abuse of notation, we use (S, T, F, m0 ) to denote the net system ((S, T, F ), m0 ). The set of markings reachable from m, denoted by [m, is deﬁned as the least set of markings such that • m ∈ [m • if m ∈ [m and there exists a transition t such that m [tm then m ∈ [m. The set of ﬁring sequences is deﬁned inductively as follows: • m0 is a ﬁring sequence; • if m0 [t1 m1 . . . [tn mn is a ﬁring sequence and mn [tn+1 mn+1 then also m0 [t1 m1 . . . [tn mn [tn+1 mn+1 is a ﬁring sequence. Given a ﬁring sequence m0 [t1 m1 . . . [tn mn , we call t1 . . . tn a transition sequence. We use σ to range over transition sequences. The marking graph of a net system N is the transition system M G(N ) = ([m0 , T, {(m, t, m ) | m ∈ [m0 ∧ t ∈ T ∧ m[tm }). A net is transition simple if the following condition holds for all x, y ∈ T : if x = • y and x• = y • then x = y. A marking m contains a contact if there exists a transition t ∈ T such that • t ⊆ m and t• ∩ m = ∅. A net system is contact–free if no marking in [m0 contains a contact. A net system is reduced if each transition can occur at least one time: for all t ∈ T there exists m ∈ [m0 such that m[t. In the following we consider contact-free elementary net systems that are transition simple and reduced. •

132

3

R. Gorrieri and M. Vernali

Basic Noninterference on LTSs

3.1

Rushby’s Definition

The deﬁnition of Rushby [23], that elaborates over the original one of Goguen and Meseguer [14], is given over ﬁnite-state deterministic automata with outputs associated to transitions: these are essentially Mealy machines, where every action is allowed from each state. Definition 7. A system M is a tuple (S, A, O, step, output) where: • • • • •

S is a ﬁnite set of states, with a distinguished initial state s0 ; A is a ﬁnite set of actions (or inputs); O is a ﬁnite set of outputs; step : S × A → S is the transition function, and output : S × A → O is the function that returns the output associated to the transition.

Function step can be extended to sequences of actions by means of function run : S × A∗ → S as follows: run(s, ) = s run(s, aα) = run(step(s, a), α) where denotes the empty sequence and α a sequence of actions. Rushby’s deﬁnition is given in general for a set of security domains onto which a particular security policy is described. For simplicity’s sake, we restrict our attention to the simple case of two levels only, usually called high – H (for classiﬁed or secret information) and low – L (for public information). Hence, we reformulate his deﬁnition in this simpliﬁed setting.1 Some auxiliary deﬁnitions are necessary. Given a set D = {L, H} of security domains we consider a function dom : A → D which associates a security domain to each action. A security policy (or interference relation) is a reﬂexive relation →i ⊆ D × D, which, in our simpliﬁed setting, is given by the relation {(L, L), (L, H)(H, H)}, stating that information can ﬂow from low to high. With i we denote the complementary relation of noninterference, that in our case is just the relation {(H, L)}, meaning that information ﬂows from high to low are forbidden. A security policy is transitive if its interference relation→i is so, which is true in our case. There is an information ﬂow from domain u to domain v when the actions performed by domain u make the system, as observed by domain v, diﬀerent from the case when such actions are not performed. 1

This simpliﬁcation is not against generality: given a n-level system, this can be veriﬁed by means of a set of two level checks, for any partitioning of the n levels in two groups.

On Intransitive Non-interference in Some Models of Concurrency

133

Definition 8. Given v ∈ D and α ∈ A∗ , we deﬁne purge(α, v) as the subsequence of α obtained by removing all the actions associated with the domains u such that u i v: purge(, v) = a purge(α, v) if dom(a) →i v purge(aα, v) = purge(α, v) otherwise. In our specialized setting, as the forbidden information ﬂow is only the one from H to L, purge(α, v) returns α when v = H, while it simply removes the high actions from α when v = L. Definition 9. A system M is NI (i.e., non-interferent) iﬀ the following holds: ∀α ∈ A∗ ∀a ∈ A output(run(s0 , α), a) = output(run(s0 , purge(α, dom(a))), a). This essentially amounts to say that, whenever a low action a is to be performed, the state s1 , reached by performing α, and the state s2 , reached by performing purge(α, L), oﬀer the same output when performing a. Hence, NI holds if low outputs (i.e., the outputs performed in correspondence of low inputs) do not depend on high inputs. 3.2

How to Define NI on Deterministic LTSs?

Our aim is to analyse systems that can perform two kinds of actions: high level actions, representing the interaction of the system with high level users, and low level actions, representing the interaction with low level users. We want to verify if the interplay between the high user and the high part of the system can aﬀect the view of the system as observed by a low user. We assume that the low user knows the structure of the system, and we check if, in spite of this, he is not able to infer the behavior of the high user by observing the low view of the execution of the system. Hence, we consider LTSs whose set of events E is partitioned into two subsets: the set EH of high level events and the set EL of low level events. To emphasize this partition we use the following notation: with (St, EL , EH , →) we denote the LTS (St, E, →) where E = EL ∪ EH and EL ∩ EH = ∅. We would like to redeﬁne property NI over deterministic labeled transition systems in a way to preserve as much as possible the original intuition of the deﬁnition given by Rushby. We have to cope with some issues. First of all, the label of a transition in an LTS is either an input (high or low) or an output (high or low). Hence, the rigid synchrony of inputs and outputs is invalid in this more general model. Moreover, the equivalence on outputs required by the deﬁnition of NI above makes little sense on deterministic LTSs, where usually equivalence is expressed in terms of language equality (i.e., equality of the set of traces). We ﬁrst deﬁne the variant purge function in this context, that we call hide. Definition 10. Given a deterministic LTS (St, EL , EH , →), function hide (or low view) of a sequence of events α ∈ E ∗ is deﬁned as follows:

134

R. Gorrieri and M. Vernali

• hide() = a hide(α) if a ∈ EL • hide(aα) = hide(α) otherwise Also function run should be adapted as follows, because deterministic LTSs do not ensure that in any state a transition is present for any event: • run(s, ) = s a run(s , α) if s → s • run(s, aα) = undeﬁned otherwise So the deﬁnition of NI in this setting should be something like this: ∀α ∈ E ∗ run(s0 , α) ∼L run(s0 , hide(α)). where ∼L is some notion of low-equivalence we have not yet identiﬁed and the equivalence is meant to hold whenever the left hand side of the equation is deﬁned, i.e. when trace α is executable. Now let us try to identify what is a sensible candidate for ∼L . The idea is to have that the two reached states are indistinguishable for a low observer, at least until some high action takes place. Hence, we ﬁrst deﬁne the initial low view Λ(α) of a trace α ∈ E ∗ as follows: • Λ() = a Λ(α) if a ∈ EL • Λ(aα) = otherwise. Λ is similar to function hide, however it diﬀers because Λ truncates α to its ﬁrst high level action. Definition 11. Given a deterministic LTS, we say that two states s1 and s2 are initial low-view equivalent, denoted with s1 ∼L s2 , iﬀ Λ(T r(s1 )) = Λ(T r(s2 )). It is immediate to observe that ∼L is an equivalence relation. Moreover, it is decidable for ﬁnite-state LTS’s (actually PSPACE-complete), as it can be reduced to the trace equivalence problem. One may wonder why we need function Λ in the deﬁnition above and do not use instead function hide. The following example explains this point. Example 1. The deterministic LTS in Figure 1 is clearly insecure because if a low user observes action l2 then (s)he is sure that the high action h has been performed an odd number of times. Indeed, this system is not NI, but it would satisfy the variant deﬁnition where function hide is used in place of Λ. So, we are now ready to deﬁne our notion of noninterference, we call NI with abuse of notation, as from now on this deﬁnition surpasses the previous one. Definition 12. Given a deterministic labeled rooted transition system T S = (St, EL , EH , →, s0 ), we say that T S satisﬁes NI if the following holds: ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then so is run(s0 , hide(α)) and run(s0 , α) ∼L run(s0 , hide(α))

On Intransitive Non-interference in Some Models of Concurrency h s1

s0

h

l1

135

s2 l2

s3 Fig. 1. An LTS not satisfying N I

3.3

Unwinding – SNDC

The deﬁnition of NI is rather cumbersome and diﬃcult to check, because of the universal quantiﬁcation over all possible traces in E ∗ . As a matter of fact, a direct, brute force algorithm checking NI would fail miserably because the number of equivalence checks is inﬁnite in principle, even for ﬁnite-state LTS’s. However, in [15,23] are reported some local conditions ensuring NI that allows for a better algorithmic veriﬁcation of NI. These are called unwinding conditions in the jargon of information ﬂow security. In this section we want to propose one local property which is necessary and suﬃcient to prove NI. This property is called SNDC [7,9,5]. Definition 13. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SNDC h

iﬀ ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ∼L s . Hence, checking SNDC is decidable for ﬁnite-state LTS’s, as we have to make as many equivalence checks as are the high transitions in the LTS, which are ﬁnitely many, and each equivalence check s ∼L s is decidable. We can then state the main theorem for this part of the paper, whose proof is postponed to Appendix A. Theorem 1. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SNDC iﬀ it satisﬁes NI. 3.4

Extending the Approach to Nondeterminism

The deﬁnition of SNDC is rather satisfactory for deterministic LTSs, but when we move to general (i.e., possibly nondeterministic) LTSs, it is somehow inadequate, as the following example shows. Example 2. The system in Figure 2 is SNDC because the states s0 and s1 are low-view equivalent. However, such a system is insecure, because a low level user willing to perform trace ll may be unable to do so and in such a case (s)he is sure that h has been performed. This example clariﬁes that we need a more discriminating notion of initial lowview equivalence. A natural way out could be to reﬁne the deﬁnition by using the ﬁner bisimulation equivalence in place of trace equivalence.

136

R. Gorrieri and M. Vernali s1

s0 l l

h

l

l

l

Fig. 2. An LTS that satisﬁes SNDC but that is not secure.

Definition 14. Let T S = (St, EL , EH , →) be and LTS. An initial low–view bisimulation on T S is a relation R ⊆ St × St such that if (s1 , s2 ) ∈ R then for all a ∈ EL : a

a

• if s1 → s1 then there exists s2 such that s2 → s2 , with (s1 , s2 ) ∈ R a a • if s2 → s2 then there exists s1 such that s1 → s1 , with (s1 , s2 ) ∈ R We say that two states s1 and s2 are initial low-view bisimulation equivalent, denoted with s1 ≈L s2 , if there exists an initial low-view bisimulation R with (s1 , s2 ) ∈ R. It is immediate to observe that ≈L is an equivalence relation. Moreover, for ﬁnite-state LTS’s, ≈L is decidable in polynomial time, because so is strong bisimulation. We are now ready to deﬁne the improved information ﬂow property. Definition 15. A reduced LTS (St, EL , EH , →, s0 ) satisﬁes SBNDC if and only h if ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ≈L s . It is easy to observe that the system in Example 2 is not SBNDC. Moreover, SBNDC is decidable for ﬁnite-state LTS’s because ﬁnite is the number of decidable equivalence checks s ≈L s . Finally, SBNDC is a conservative extension of SNDC as the following proposition states. Theorem 2. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SBNDC iﬀ it satisﬁes SNDC. By our experience, we conjecture that SBNDC is the right property in the setting of nondeterministic LTSs.

4

Intransitive Noninterference on LTSs

Basic (or transitive) noninterference is, from a practical point of view, too draconian: one often wants to make public some data that were secret previously. This operation is known as declassiﬁcation and can be modeled naturally with an intransitive version of noninterference. Typically, we have three levels: H for secret actions, L for public actions and D for downgrading actions; we admit ﬂows from H to L only if mediated by an action in D. Hence, in this three-level approach, the interference relation is {(L, L), (L, H)(H, H), (D, D), (L, D), (D, L), (H, D), (D, H)} (where only (H, L) is forbidden), which is clearly not transitive.

On Intransitive Non-interference in Some Models of Concurrency

4.1

137

Rushby’s Definition

In Section 3.1 we have presented Rushby’s model, based on Mealy machines, for NI. Now we report his extension to intransitive non-interference, called INI, as described in [23]. We ﬁrst need to identify which actions in a trace α are not to be deleted with the intransitive version of the purge function. Definition 16. Function sources : A∗ × D → P(D) is deﬁned as follows: sources(, u) = {u} ⎧ ⎨ sources(α, u) ∪ {dom(a)} if ∃v : v ∈ sources(α, u) ∧dom(a) →i v sources(aα, u) = ⎩ sources(α, u) otherwise. Essentially, v ∈ sources(α, u) means either that v = u or that there exists a subsequence of α composed of actions in the domains w1 , w2 , . . . , wn such that w1 →i w2 →i · · · →i wn , v = w1 and u = wn . Hence, when considering if a, performed before α, is allowed to inﬂuence domain u, we ask if there is any v ∈ sources(α, u) such that dom(a) →i v. Definition 17. Function ipurge : A∗ × D → A∗ (that is intransitive-purge) is deﬁned as follows: ipurge(, u) = a ipurge(α, u) if dom(a) ∈ sources(aα, u) ipurge(aα, u) = ipurge(α, u) otherwise. Essentially, ipurge(α, u) is the subsequence of α where all the actions that cannot interfere with u are removed. It is interesting to observe that, in the simpliﬁed case of three levels only we consider in this paper, ipurge(α, H) = α, ipurge(α, D) = α, while ipurge(α, L) returns the subsequence of α where all the high level actions occurring before a low level action are removed, unless a downgrading action does occur in between the two. Definition 18. A system M is INI (i.e., intransitive non-interferent) iﬀ the following holds: ∀α ∈ A∗ ∀a ∈ A output(run(s0 , α), a) = output(run(s0 , ipurge(α, dom(a))), a). This essentially amounts to say that, whenever a low action a is to be performed, the state s1 , reached by performing α, and the state s2 , reached by performing ipurge(α, L), oﬀer the same output when performing a. Hence, INI holds if low outputs (i.e., the outputs performed in correspondence of low inputs) do not depend on non-downgraded high inputs. 4.2

Defining INI for Deterministic LTSs

As we have three levels of actions, the deﬁnition of LTS should reﬂect this partitioning, hence T S = (St, EL , ED , EH , →). We start by deﬁning an intransitive variant of function hide.

138

R. Gorrieri and M. Vernali

Definition 19. Given a deterministic LTS (St, EL , ED , EH , →), the function ihide (or intransitive low view) of a sequence of events α ∈ E ∗ is deﬁned as follows: • ihide() = ⎧ ⎨ ihide(α)a if a ∈ EL • ihide(αa) = ihide(α) if a ∈ EH ⎩ αa if a ∈ ED Function Λ extends naturally to the case of the presence of downgrading actions: it truncates traces at the point the ﬁrst high action or downgrading action is met. Even if the deﬁnition of Λ works as expected also in the three level setting, we prefer to change its name to Δ to emphasize that we are working on this richer scenario. Hence, also the deﬁnition of initial low-view equivalence is syntactically slightly changed: s1 ∼D s2 iﬀ Δ(T r(s1 )) = Δ(T r(s2 )). It is immediate to observe that ∼D is an equivalence relation. Moreover, it is decidable for ﬁnite-state LTS’s. Definition 20. Given a deterministic labelled rooted transition system T S = (St, EL , ED , EH , →, s0 ), we say that T S satisﬁes INI iﬀ the following holds: ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then so is run(s0 , ihide(α)) and then run(s0 , α) ∼D run(s0 , ihide(α)). The deﬁnition above is indeed a conservative extension of NI because, if we consider a deterministic labelled rooted transition system with empty set of downgrading actions, we get that INI and NI do coincide. This is due to the fact that ihide(α) = hide(α) if α ∈ (EL ∪ EH )∗ . Theorem 3. Given a deterministic labelled rooted transition system T S = (St, EL , ∅, EH , →, s0 ), it holds that T S satisﬁes NI iﬀ T S satisﬁes INI. 4.3

Unwinding – NID

For basic (i.e., two-level) noninterference, we have shown that SNDC is a local property characterizing NI. Here we play the same game, by providing an obvious generalization of SNDC, we call NID (NonInterference with Downgraders) and then by showing that NID and INI are the same property (proof postponed to Appendix B). Definition 21. A reduced deterministic LTS (St, EL , ED , EH , →) satisﬁes NID h

iﬀ ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ∼D s . Theorem 4. A reduced deterministic LTS (St, EL , ED , EH , →, s0 ) satisﬁes NID iﬀ it satisﬁes INI. NID is decidable for ﬁnite-state LTS’s: we have to perform as many equivalence checks as are the transitions labeled with an high level action; moreover, each equivalence check of the form s ∼D s is decidable, as it can be reduced to the trace equivalence problem.

On Intransitive Non-interference in Some Models of Concurrency

4.4

139

Extending the Approach to Nondeterminism

The deﬁnition of NID seems rather satisfactory for deterministic LTSs, but when we move to nondeterministic systems, it may be inadequate, as the system in Figure 2 shows: the states s0 and s1 are low-view equivalent, but a low level user willing to perform trace ll may be unable to do so and in such a case (s)he is sure that h has been performed. A similar example is reported in Figure 3. s1

s0 h d l

h s3

s2 d

h l

l

l

l

l

s4

Fig. 3. An insecure nondeterministic system satisfying N ID

As discussed in Section 3.4, we should replace the trace based notion of initial low-view equivalence with a ﬁner one based on bisimulation equivalence. The deﬁnition of low-view observational equivalence of Deﬁnition 14 is unchanged for this richer scenario, but to emphasize the diﬀerence (presence also of the intermediate level D), we denote it with ≈D instead of ≈L . We are now ready to deﬁne the improved information ﬂow property, we call BNID (Bisimulation-based NID). Definition 22. A reduced LTS (St, EL , ED , EH , →) satisﬁes BNID if and only h

if ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ≈D s . It is easy to observe that the system in Figure 3 is not BNID. Moreover, BNID is decidable for ﬁnite-state LTS’s because ﬁnite is the number of decidable equivalence checks s ≈D s . Finally, based on our experience, we conjecture that BNID is the right property in the setting of nondeterministic LTSs.

5

Noninterference on Elementary Nets

We ﬁrst brieﬂy survey the work in [3,4], presenting the adaptation of the property SBNDC [7,9] in the case of elementary net systems, and then the idea of structural noninterference. Then, Section 5.3 reports the original extension to the case of intransitive noninterference. We remind that we consider elementary net systems that are contact-free, reduced and transition simple.

140

5.1

R. Gorrieri and M. Vernali

A Dynamic Non-interference Property: SBNDC

We consider nets whose set of transitions is partitioned into two subsets: the set H of high level transitions and the set L of low level transitions. To emphasize this partition we use the following notation. Let L and H be two disjoint sets: with (S, L, H, F, m0 ) we denote the net system (S, L ∪ H, F, m0 ). Among the many non-interference properties deﬁned by Focardi and Gorrieri in [7,8,9], here we consider SBNDC (Strong Bisimulation Non-Deducibility on Composition). To properly deﬁne it over Petri nets, we need the auxiliary deﬁnition of initial low-view bisimulation over elementary net systems. Definition 23. Let N = (S, L, H, F, m0 ) be a net system. An initial low–view bisimulation is a relation R ⊆ Pf in (S) × Pf in (S) such that if (m1 , m2 ) ∈ R then for all t ∈ L: • if m1 [tm1 then there exists m2 such that m2 [tm2 , and (m1 , m2 ) ∈ R; • if m2 [tm2 then there exists m1 such that m1 [tm1 , and (m1 , m2 ) ∈ R. We say that m1 is initial low–view bisimilar to m2 , denoted by m1 ≈L m2 , if there exists an initial low–view bisimulation R such that (m1 , m2 ) ∈ R. Now we are ready to deﬁne SBNDC. Definition 24. Let N = (S, L, H, F, m0 ) be a net system. N is SBNDC iﬀ for all markings m ∈ [m0 and for all h ∈ H the following holds: m[hm implies m ≈L m . The intuition behind SBNDC is that, whenever a high transition h is performed, the markings before h and after h are observationally indistinguishable for a low observer. Note that SBNDC is clearly decidable for ﬁnite elementary net systems because the number of reachable markings is ﬁnite, as well as the set H of high transitions; hence, the number of equivalence checks is ﬁnite; moreover, each equivalence check m ≈L m is decidable, because so is bisimulation equivalence over ﬁnite elementary net systems. Example 3. As a simple case study, consider the net in Figure 4, which represents a mutually exclusive access to a shared resource (represented by the token in s) by a high-user (left part of the net) and a low-user (right part of the net). Even if it might appear, at ﬁrst sight, that the system is secure (and indeed, it is BSNNI (Bisimulation Strong Nondeterministic Non-Interference) [7,9]), actually it is not SBNDC: consider the reachable marking m = {p1,2 , s, p2,2 }; it is easy to observe that m[h2 m = {p1,3 , s, p2,2 } and m[l2 , but l2 is not ﬁreable from m . Indeed, the system is not secure, because an unsuccesful attempt of a low level user to perform l2 just after having performed l1 will give him the information that the high user has performed h2 but not yet h3 .

On Intransitive Non-interference in Some Models of Concurrency p1,2

s

p2,2

h2

h1

l2

p1,3

p2,3

h3 p1,1

141

l1

l3 s

p2,1

Fig. 4. The net system for a mutually exclusive access to a shared resource

5.2

Structural Non-interference

Consider a net system N = (S, L, H, F, m0 ). Consider a low level transition l of the net: if l can ﬁre, then we know that the places in the preset of l are marked before the ﬁring of l; moreover, we know that such places become unmarked after the ﬁring of l. If there exists a high level transition h that produces a token in a place s in the preset of l (see the system N1 in Figure 5), then the low level user can infer that h has occurred if he can perform the low level transition l. We note that there exists a causal dependency between the transitions h and l, because the ﬁring of h produces a token that is consumed by l. In this case we will say that s is a potential causal place. Consider now the situation illustrated in the system N2 of Figure 5: in this case, place s is in the preset of both l and h, i.e., l and h are competing for the use of the resource represented by the token in s. Aware of the existence of such a place, a low user knows that the high-level action h has been performed, if he is not able to perform the low-level action l. Place s represents a conﬂict

h s

s

l

N1

h

l

N2

Fig. 5. Examples of net systems containing causal and conﬂict places

142

R. Gorrieri and M. Vernali

between transitions l and h, because the ﬁring of h prevents l from ﬁring. In this case we will call s a potential conﬂict place. In order to avoid the deﬁnition of a security notion that is too strong, and that rules out systems that do not reveal information on the high-level actions that have been performed, we need to reﬁne the concepts illustrated above. In particular the potential causal place is an active causal place if there is an execution where the token produced by the high level transition is eventually consumed by the low level transition. Similarly, a potential conﬂict place is active if the token that could be consumed immediately by a high level transition can be later on also consumed by a low level transition. The formal deﬁnitions follow. Definition 25. Let N = (S, L, H, F, m0 ) be an elementary net system. Let s be a place of N such that s• ∩ L = ∅ (i.e., a token in s can be consumed by a low transition). The place s ∈ S is a potentially causal place if • s ∩ H = ∅ (i.e., a token in s can be produced by a high transition). A potentially causal place s is an active causal place if the following condition holds: there exist l ∈ s• ∩ L, h ∈ • s ∩ H, m ∈ [m0 and a transition sequence σ such that m[hσl and s ∈ t• for all t ∈ σ. The place s ∈ S is a potentially conﬂict place if s• ∩ H = ∅ (i.e., the token in s can be consumed also by a high transition). A potentially conﬂict place is an active conﬂict place if the following condition holds: there exist l ∈ s• ∩ L, h ∈ s• ∩ H, m ∈ [m0 and a transition sequence σ such that m[h, m[σl and s ∈ t• for all t ∈ σ. Definition 26. Let N = (S, L, H, F, m0 ) be an elementary net system. We say that N is PBNI+ (positive Place Based Non-Interference) if, for all s ∈ S, s is neither an active causal place nor an active conﬂict place. The following non-trivial result, proved in [4], states that the behavioural noninterference property SBNDC is equivalent to the semi-static, structural property PBNI+. Theorem 5. Let N = (S, L, H, F, m0 ) be an elementary net system. Then N is PBNI+ iﬀ N is SBNDC. An obvious consequence is that if N has no potentially causal and potentially conﬂict places, then N is SBNDC. Hence, a simple strategy to check if N is SBNDC is to ﬁrst identify potential causal/conﬂict places, a procedure that has complexity O(f + p) in the size of the net (p is the number of places and f of arcs). If no place of these sorts is found, then N is PBNI+, hence SBNDC. Otherwise, any such a candidate place should be better studied to check if it is actually an active causal/conﬂict place, a procedure that requires a limited exploration of the marking graph. The complexity of PBNI+ in the worst case is O(pn22p ) (see [10] for details). Observe that the net in Figure 4 of our running example is not PBNI+ because place s is an active conﬂict (and also active causal) place.

On Intransitive Non-interference in Some Models of Concurrency

5.3

143

Extending the Approach for Intransitive Noninterference

Not too surprisingly, the theory presented in the two previous subsections can be adapted to the more general scenario when also downgrading actions are present. First of all, observe that the notion of initial low view bisimulation equivalence ≈L of Deﬁnition 23 works as expected also for the three level scenario; however, to better reﬂect the fact that also actions in D are possible, we rename it with ≈D . Hence, the deﬁnition of BNID over elementary net systems is essentially the same as SBNDC. Definition 27. Let N = (S, L, D, H, F, m0 ) be a net system. N is BNID iﬀ for all markings m ∈ [m0 and for all h ∈ H the following holds: m[hm implies m ≈D m . Then, we adapt the structural noninterference approach to the three level scenario. Definition 28. Let N = (S, L, D, H, F, m0 ) be an elementary net system. A place s ∈ S is a potentially causal place if • s ∩ H = ∅ and s• ∩ L = ∅. Place s ∈ S is an active causal place if it is potentially causal and there exist h ∈ • s∩H, l ∈ s• ∩ L, m ∈ [m0 and a transition sequence σ ∈ (H ∪ L)∗ such that m[hσl and for all t ∈ σ, s ∈ / t• . Observe that the only diﬀerence w.r.t. Deﬁnition 25 is that the transition sequence σ is constrained not to contain downgrading actions. The idea is that if a downgrading action d is performed inevitably in between h and l, then the ﬂow is mediated by d and so the information ﬂow becomes legal. Definition 29. Let N = (S, L, D, H, F, m0 ) be an elementary net system. A place s ∈ S is a potentially conﬂict place if s• ∩ H = ∅ and s• ∩ L = ∅. Place s ∈ S is an active conﬂict place if it is potentially conﬂict and there exist h ∈ s• ∩ H, l ∈ s• ∩ L, m ∈ [m0 and a transition sequence σ ∈ (H ∪ L)∗ such that m[h, m[σl and for all t ∈ σ, s ∈ / t• . Also for active conﬂict places, the only diﬀerence is the constraint on σ about downgrading actions. Indeed, if a downgrading action d is to be performed before l, then h and l are not really conﬂicting, because that conﬂict is made public by action d. Definition 30. Let N = (S, L, D, H, F, m0 ) be an elementary net system. We say that N is PBNID ( Positive Place Based Non Interference with Downgraders) if for all s ∈ S, s is neither a active causal place, nor an active conﬂict place. The main result is that the dynamic property BNID and the semi-static property PBNID are actually equivalent on elementary net systems. The proof of the following theorem, based on the analogous proof in [4] of Theorem 5, is reported in Appendix C.

144

R. Gorrieri and M. Vernali

Theorem 6. Let N = (S, L, D, H, F, m0 ) be an elementary net system. N is BNID if and only if N is PBNID. The complexity of checking BNID is the same as checking SBNDC and, as reported in [10], it is O(pn23p ) where p is the number of places and n the number of transitions. Similarly, the complexity of checking PBNID is the same as checking PBNI+ and, as reported in [10], varies from a minimum of O(f + p) (where f is the number of arcs) to a maximum of O(pn22p ). Example 4. Consider again the insecure net in Figure 4. A possible way to make the net secure is to include additional downgrading transitions as reported in Figure 6. d1 p1,2

s

p2,2

h2

h1

l2

p1,3

p2,3 l3

h3 p1,1

l1

s

p2,1 d2

Fig. 6. Secure mutually exclusive access to a shared resource

6

Conclusion

In this paper we presented a study about the noninterference properties in the basic case of two levels of conﬁdentiality (H and L) and in the richer case of three levels, where the level D of downgrading actions is considered additionally. In both settings, we start by ﬁrst recalling the deﬁnitions by Rushby in [23] for deterministic ﬁnite state automata with outputs (Mealy machines); we try to provide similar deﬁnitions for the more popular model of deterministic labeled transition system, following as much as possible the original intuition, yielding our deﬁnition of NI and INI. Then, in both scenarios, we investigate local information ﬂow properties, called SNDC and NID, respectively, that are necessary and suﬃcient conditions to ensure NI and INI, respectively. In both cases, we discuss the need of bisimulation-based deﬁnition of information ﬂow security properties as soon as one considers nondeterministic LTSs. Finally, we extend the work in [3,4] over the model of elementary net systems to the case of intransitive noninterference.

On Intransitive Non-interference in Some Models of Concurrency

145

Future work will be devoted to study systematic techniques for adapting an insecure system to make it secure. This is a rather challenging problem because there is a large range of possible modiﬁcations of a system that can be considered; e.g., enlarging the behaviour of the low part of the system, inserting suitable declassiﬁers whenever possible (as we have done in Example 4) or even cutting some possible high level behaviour. Moreover, we plan to study more selective forms on intransitive noninterference where the occurrence of a downgrading action does not reveal all the previously performed high level actions, as it is now prescribed in Rushby’s deﬁnition (cf. Deﬁnition 19). Furthermore, it is interesting to see if the approach presented here can be extended to the richer setting of P/T Petri nets. Some results are reported in a recent paper [1] (actually written after this), where it is shown that SBNDC and BNID are decidable for ﬁnite P/T nets (hence for a class of inﬁnite-state systems), even if it is left open the problem of understanding if (transitive as well as intransitive) noninterference can be equivalently characterized as a relation of causality or conﬂict among particular high actions and low ones. Acknowledgements. The authors would like to thank the anonymous referees for helpful comments.

References 1. Best, E., Darondeau, P., Gorrieri, R.: On the Decidability of Non Interference over Unbounded Petri Nets. EPTCS 51, 16–33 (2010); Procs. SecCo 2010 2. Busi, N., Gorrieri, R.: A Survey on Non-Interference with Petri Nets. In: Desel, J., Reisig, W., Rozenberg, G. (eds.) Lectures on Concurrency and Petri Nets. LNCS, vol. 3098, pp. 328–344. Springer, Heidelberg (2004) 3. Busi, N., Gorrieri, R.: Positive Non-Interference in Elementary and Trace Nets. In: Cortadella, J., Reisig, W. (eds.) ICATPN 2004. LNCS, vol. 3099, pp. 1–16. Springer, Heidelberg (2004) 4. Busi, N., Gorrieri, R.: Structural Non-Interference in Elementary and Trace Nets. Mathematical Structures in Computer Science 19(6), 1065–1090 (2009) 5. Bossi, A., Piazza, C., Rossi, S.: Modelling Downgrading in Information Flow Security. In: Procs. 17th IEEE Computer Security Foundations Workshop (CSFW 2004), pp. 187–201. IEEE Press, Los Alamitos (2004) 6. Engelfriet, J., Rozenberg, G.: Elementary Net Systems. In: Reisig, W., Rozenberg, G. (eds.) APN 1998. LNCS, vol. 1491, pp. 12–121. Springer, Heidelberg (1998) 7. Focardi, R., Gorrieri, R.: A Classiﬁcation of Security Properties. Journal of Computer Security 3(1), 5–33 (1995) 8. Focardi, R., Gorrieri, R.: The Compositional Security Checker: A Tool for the Veriﬁcation of Information Flow Security Properties. IEEE Transactions on Software Engineering 23(9), 550–571 (1997) 9. Focardi, R., Gorrieri, R.: Classiﬁcation of Security Properties (Part I: Information Flow). In: Focardi, R., Gorrieri, R. (eds.) FOSAD 2000. LNCS, vol. 2171, pp. 331–396. Springer, Heidelberg (2001) 10. Frau, S., Gorrieri, R., Ferigato, C.: Structural Noninterference at Work: the Petri Net Security Checker. In: Degano, P., Guttman, J., Martinelli, F. (eds.) FAST 2008. LNCS, vol. 5491, pp. 210–225. Springer, Heidelberg (2009)

146

R. Gorrieri and M. Vernali

11. Hadj-Alouane, N.B., Lafrance, S., Lin, F., Mullins, J., Yeddes, M.M.: Characterizing Intransitive Noninterference for 3-Domain Security Policies With Observability. IEEE Transactions on Automatic Control 50(6), 920–925 (2005) 12. Hadj-Alouane, N.B., Lafrance, S., Lin, F., Mullins, J., Yeddes, M.M.: On the veriﬁcation of intransitive noninterference in multilevel security. IEEE Transactions on Systems, Man, and Cybernetics, Part B 35(5), 948–958 (2005) 13. Haigh, J.T., Young, W.D.: Extending the noninterference version of MLS for SAT. IEEE Trans. on Software Engineering SE-13(2), 141–150 (1987) 14. Goguen, J.A., Meseguer, J.: Security Policy and Security Models. In: Proc. of Symposium on Security and Privacy (SSP 1982), pp. 11–20. IEEE CS Press, Los Alamitos (1982) 15. Goguen, J.A., Meseguer, J.: Unwinding and Inference Control. In: Proc. of Symposium on Security and Privacy (SSP 1984), pp. 75–86. IEEE CS Press, Los Alamitos (1984) 16. Lafrance, S., Mullins, J.: Bisimulation-based Nondeterministic Admissible Interference and ita Applications to the Analysis of Cryptographic Protocols. Information and Software Technology 45(11), 779–790 (2003); Preliminary version in Electronic Notes in Theoretical Computer Science 61, 1–24 (2002) 17. van der Meyden, R.: What, Indeed, Is Intransitive Noninterference? In: Biskup, J., L´ opez, J. (eds.) ESORICS 2007. LNCS, vol. 4734, pp. 235–250. Springer, Heidelberg (2007) 18. Milner, R.: Communication and Concurrency. Prentice-Hall, Englewood Cliﬀs (1989) 19. Mullins, J.: Nondeterministic Admissible Interference. Journal of Universal Computer Science 11, 1054–1070 (2000) 20. Pinsky, S.: Absorbing Covers and Intransitive Noninterference. In: Proc. of IEEE Symposium on Security and Privacy (SSP 1995), pp. 102–113. IEEE CS press, Los Alamitos (1995) 21. Pinsky, S., Zieglar, E.: Noninterference Equations for Nondeterministic Systems. In: Proc. of IEEE Computer Security Foundations Workshop (CSFW 2001), pp. 3–14. IEEE CS press, Los Alamitos (2001) 22. Roscoe, A.W., Goldsmith, M.-H.: What is Intransitive Noninterference? In: Proc. of 12th Computer Security Foundations Workshop (CSFW), pp. 228–238. IEEE CS Press, Los Alamitos (1999) 23. Rushby, J.: Noninterference, Transitivity, and Channel-control Security Policies. Technical Report CSL-92-02, SRI International (1992) 24. Ryan, P.Y.A.: Mathematical Models of Computer Security. In: Focardi, R., Gorrieri, R. (eds.) FOSAD 2000. LNCS, vol. 2171, pp. 1–62. Springer, Heidelberg (2001) 25. Ryan, P.Y.A., Schneider, S.: Process Algebra and Noninterference. In: Proc. of 12th Computer Security Foundations Workshop (CSFW), pp. 214–227. IEEE CS Press, Los Alamitos (1999)

On Intransitive Non-interference in Some Models of Concurrency

147

Appendix A

Proofs of Section 3

Proposition 1. If a deterministic reduced LTS satisﬁes NI, then it satisﬁes SNDC. Proof. By hypothesis, we have that for all α ∈ E ∗ such that run(s0 , α) is deﬁned, run(s0 , α) ∼L run(s0 , hide(α)). Let s = run(s0 , α ) and t = run(s0 , hide(α )); b hence, s ∼L t. Take α = α b. If b ∈ EH then s → s and run(s0 , hide(α b)) = t, hence s ∼L t. Hence, by transitivity of ∼L , we conclude that s ∼L s , i.e., SNDC holds. In order to prove the reverse implication (i.e., SNDC implies NI ), it is useful to prove ﬁrst the following lemma. Lemma 1. If a deterministic reduced LTS satisﬁes SNDC then, for all s, t ∈ St a a and for all a ∈ EL , s ∼L t and s → s imply t → t and s ∼L t . Proof. As s ∼L t, for all α = aβ we have that α ∈ Λ(T r(s)) iﬀ α ∈ Λ(T r(t)). If a aβ ∈ Λ(T r(s)), there must exist a state s such that s → s with β ∈ Λ(T r(s )). a Similarly, If aβ ∈ Λ(T r(t)) there must exist a state t such that t → t with β ∈ Λ(T r(t )). Hence for all β we have that β ∈ Λ(T r(s )) iﬀ β ∈ Λ(T r(t )), i.e., s ∼ L t . Proposition 2. If a deterministic reduced LTS satisﬁes SNDC, then it satisﬁes NI. Proof. We prove by induction on the length of traces α that s ∼L t ⇒ run(s, α) ∼L run(t, hide(α))

(1)

for all α such that run(s, α) is deﬁned. When in the equation (1) above we set s = t = s0 , we get the thesis. The base case when α = is trivial. For the inductive case, assume that the thesis holds for all α of length n for which run(s, α) is deﬁned, and consider trace aα. If run(s, a) is undeﬁned, then trace aα is to be ignored; otherwise let s = run(s, a). We have that run(s, aα) = run(s , α). On the other hand for run(t, hide(aα)) we have to distinguish two cases. (1. a ∈ EL ) In this case, the deﬁnition of hide justiﬁes that run(t, hide(aα)) = run(t, a hide(α)). By hypothesis that s ∼L t and since s = run(s, a), there must exist a t = run(t, a). Hence, run(t, a hide(α)) = run(t , hide(α)). As s ∼L t and SNDC holds, we get by Lemma 1 that s ∼L t . By applying the inductive hypothesis, we get run(s , α) ∼L run(t , hide(α)), and so the implication in (1) holds for aα. (2. a ∈ EH ) In this case, the deﬁnition of hide justiﬁes that run(t, hide(aα)) = run(t, hide(α)). On the other hand, as a ∈ EH and SNDC holds, we get s ∼L s , hence (since by hypothesis s ∼L t) also s ∼L t. By applying the inductive hypothesis, we get run(s , α) ∼L run(t, hide(α)), and so the implication in (1) holds for aα.

148

R. Gorrieri and M. Vernali

Corollary 1. A deterministic reduced LTS is NI if and only if it is SNDC. Proof. It follows directly from Proposition 1 and Proposition 2.

B

Proofs of Section 4

Proposition 3. If a deterministic reduced LTS satisﬁes INI, then it satisﬁes NID. Proof. By hypothesis, we have that for all α ∈ E ∗ such that run(s0 , α) is deﬁned, run(s0 , α) ∼D run(s0 , ihide(α)). Let s = run(s0 , α ) and t = run(s0 , hide(α )); b

hence we have that s ∼D t. Take α = α b. If b ∈ EH then s → s and run(s0 , ihide(α b)) = t, hence s ∼D t. Hence, by transitivity of ∼D , we conclude that s ∼D s , i.e., NID holds. In order to prove the reverse implication (i.e., NID implies INI), it is useful to prove ﬁrst the following lemma. Lemma 2. If a deterministic reduced LTS satisﬁes NID then, for all s, t ∈ St a a and for all a ∈ EL , s ∼D t and s → s imply t → t and s ∼D t . Proof. As s ∼D t, for all α = aβ we have that α ∈ Δ(T r(s)) iﬀ α ∈ Δ(T r(t)). If a aβ ∈ Δ(T r(s)), there must exist a state s such that s → s with β ∈ Δ(T r(s )). a Similarly, If aβ ∈ Δ(T r(t)) there must exist a state t such that t → t with β ∈ Δ(T r(t )). Hence for all β we have that β ∈ Δ(T r(s )) iﬀ β ∈ Δ(T r(t )), i.e., s ∼L t . Proposition 4. If a deterministic reduced LTS satisﬁes NID, then it satisﬁes INI. Proof. We prove, by induction on the length of α, that ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then run(s0 , α) ∼D run(s0 , ihide(α))

(2)

The base case when α = is trivial. For the inductive case, assume that the thesis holds for all α of length n for which run(s0 , α) = s is deﬁned, and consider trace αa. If run(s, a) is undeﬁned, then trace αa is to be ignored; otherwise let s = run(s, a). We have that run(s0 , αa) = run(s, a) = s . On the other hand for run(s0 , ihide(αa)) we have to distinguish three cases. (1. a ∈ EL ) In this case, the deﬁnition of ihide justiﬁes that run(s0 , ihide(αa)) = run(s0 , ihide(α)a) = run(t, a). The inductive hypothesis ensures that s ∼D t; hence, by Lemma 2 we have that s ∼D t , where t = run(t, a), and so the implication in (2) holds for aα. (2. a ∈ EH ) In this case, the deﬁnition of ihide justiﬁes that run(s0 , ihide(αa)) = run(s0 , ihide(α)) = t. On the other hand, run(s0 , α) = s and run(s, a) = s . By inductive hypothesis, s ∼D t. On the other hand, as a ∈ EH and NID holds, we get s ∼D s , hence also s ∼D t, and so the implication in (2) holds for aα. (3. a ∈ ED ) In this case, by deﬁnition of ihide, we get run(s0 , ihide(αa)) = run(s0 , αa). Since ∼D is reﬂexive, the thesis follows.

On Intransitive Non-interference in Some Models of Concurrency

149

Corollary 2. A deterministic reduced LTS is INI if and only if it is NID. Proof. It follows directly from Proposition 3 and Proposition 4.

C

Proofs of Section 5

Theorem 7. Let N = (S, L, D, H, F, m0 ) be an elementary net system. If N satisﬁes P BN ID, then N satisﬁes BN ID. Proof. Let N be PBNID. We will show that N is BNID. Take m ∈ [m0 such that m[hm for h ∈ H. We have to prove that there exists an initial low-view bisimulation R on N such that (m, m ) ∈ R. Let R = {(m1 , m2 ) | ∀l ∈ L ∀s ∈ • l : m1 (s) = m2 (s) ⇒ (∀σ∀i ∈ {1, 2} : mi [σl ⇒ ∃l1 ∈ σ : s ∈ l1 • )} be the candidate relation. 1. We show that R is an initial low-view bisimulation on N . Let (m1 , m2 ) ∈ R. Suppose m1 [lm1 . We show that also m2 [l. Suppose that there exists s ∈ • l such that m2 (s) = 0, hence m1 (s) = m2 (s). As (m1 , m2 ) ∈ R and m1 [l, by deﬁnition of R (with σ = ), there must exists t ∈ , reaching a contradiction. Hence ∀s ∈ • l m2 (s) ≥ 1, and so there exists m2 such that m2 [lm2 (by contact-freeness). Now we show that (m1 , m2 ) ∈ R. Suppose that (m1 , m2 ) ∈ / R. Then there exist l , s ∈ • l such that m1 (s ) = m2 (s ) and there exist σ and i such that mi [σl and s ∈ l1 • for no l1 ∈ σ. As mi [lmi for i = 1, 2, m1 (s ) = m2 (s ) and there exists j ∈ {1, 2} such that mj [lσl and s ∈ l1 • for no l1 ∈ σ. Seeing that m1 (s ) = m2 (s ) necessarily s ∈ / l • , hence s ∈ l1 • for no l1 ∈ lσ. Thus we obtain (m1 , m2 ) ∈ / R, reaching a contradiction. Hence, we have that (m1 , m2 ) ∈ R. The symmetric case can be proved in the same way, hence we obtain that R is an initial low-view bisimulation on N . 2. We show that (m, m ) ∈ R. If there does not exist s and l ∈ s• such that m(s) = m (s), we are done: (m, m ) ∈ R. Hence, suppose that there exist s and l ∈ s• such that m(s) = m (s). We show that ∀σ : m[σl ⇒ ∃t ∈ σ : s ∈ t• and ∀σ : m [σl ⇒ ∃t ∈ σ : s ∈ t• . As m[hm , from m(s) = m (s) we deduce that one of the following holds: • s ∈ h• . Hence s is a potentially causal place. Take a sequence σ such m[σl. We show that there exists t ∈ σ such that s ∈ t• . Two subcases can happen: - σ = hσ . As PBNID holds, s is not an active causal place. Hence, for all m ¯ ∈ [m0 and for all σ ¯ : if m[h¯ ¯ σ l then there exists t ∈ σ ¯ such that s ∈ t• . As m[σl and σ = hσ , then there exists t ∈ σ ¯ such that s ∈ t• . - σ = or σ = t σ with t = h. As s ∈ h• we obtain m(s) = 0. As m[σl and s ∈ • l, there must exist a transition t ∈ σ that produces one token in s, i.e., such that s ∈ t• . In particular, σ = .

150

R. Gorrieri and M. Vernali

Consider now a sequence σ such that m [σl. We show that there exists t ∈ σ such that s ∈ t• . As m[hm , we have that m[hσl, hence, because PBNID holds, there exists t ∈ σ such that s ∈ t• . • s ∈ • h. Hence, s is a potentially conﬂict place. Take a sequence σ such that m[σl. We show that there exists t ∈ σ such that s ∈ t• . As PBNID holds s cannot be an active conﬂict place. Hence, for all m ¯ ∈ [m0 and for all σ ¯ : if m[h ¯ and m[¯ ¯ σ l then there exists t ∈ σ ¯ such that s ∈ t• . As m[hm and m[σl, there exists t ∈ σ such that s ∈ t• . Take now a sequence σ such that m [σl. As s ∈ • h, we have m (s) = 0. As s ∈ • l from m [σl we obtain that there must exists a transition t ∈ σ producing one token in s, i.e., s ∈ t• . Theorem 8. Let N = (S, L, D, H, F, m0 ) be an elementary net system. If N is BN ID then N is P BN ID. Proof. Suppose that N is BN ID. We show that no place in N can be an active causal place or an active conﬂict place. • Suppose that s is an active causal place. Then, there exist h ∈ • s, l ∈ s• , m ∈ [m0 and σ ∈ (H ∪ L)∗ such that m[hσl and ∀t ∈ σ, s ∈ / t• . Among the markings and the transition sequences that satisfy the conditions above, take m and σ such that σ contains the minimun number of transitions in H. Two cases can happen: 1. All transitions in σ belong to L. We have that m[hm . By BNID there exists an initial low-view bisimulation on N containing the pair (m, m ). As m [σl, also m[σl. But from h ∈ • s and m[h we deduce that s ∈ / m; we also know that ∀t ∈ σ, s ∈ / t• ; hence, after the ﬁring of σ place s is still empty, contradicting the fact that m[σl. 2. There exists a high level transition in σ. Let h be the last high transition in σ. Hence, there exist σ1 , σ2 , such that σ = σ1 h σ2 and all transitions in σ2 belong to L. Thus, there exist m1 , m2 such that m[hσ1 m1 [h m2 [σ2 l. From m1 [h m2 , by BNID there exists an initial low-view bisimulation on N containing the pair (m1 , m2 ). From m2 [σ2 l, we obtain that also m1 [σ2 l, thus obtaining the ﬁring sequence m[hσ1 σ2 l, contradicting the fact that the chosen transition sequence was one with the least number of high transitions. • Suppose that s is an active conﬂict place. There there exist h ∈ s• , l ∈ s• , m ∈ [m0 and σ ∈ (H ∪ L)∗ such that m[h, m[σl and ∀t ∈ σ, s ∈ / t• . Among the markings and the transition sequences that satisfy the conditions above, take m and σ such that σ contains the minimun number of transitions in H.

On Intransitive Non-interference in Some Models of Concurrency

151

Two cases can happen: 1. All transitions in σ belong to L. We have that m[hm . By BNID there exists an initial low-view bisimulation on N containing the pair (m, m ). As m[σl, then m [σl. But from h ∈ s• and m[h we deduce that s ∈ / m , • we also know that ∀t ∈ σ, s ∈ / t ; hence, after the ﬁring of σ place s is still empty, contradicting the fact that m [σl. 2. There exists a high level transition in σ. Let h be the last high transition in σ. Hence, there exist σ1 , σ2 , such that σ = σ1 h σ2 and all transitions in σ2 belong to L. Thus, there exist m1 , m2 such that m[σ1 m1 [h m2 [σ2 l. From m1 [h m2 , by BNID there exists an initial low-view bisimulation on N containing the pair (m1 , m2 ). From m2 [σ2 l, we obtain that also m1 [σ2 l, thus obtaining the ﬁring sequence m[σ1 σ2 l, contradicting the fact that the chosen transition sequence was one with the least number of high transitions.

A Method for Security Governance, Risk, and Compliance (GRC): A Goal-Process Approach Yudistira Asnar and Fabio Massacci Department of Information Engineering and Computer Science, University of Trento, Italy [email protected]

Abstract. The Governance, Risk, and Compliance (GRC) management process for Information Security is a necessity for any software systems where important information is collected, processed, and used. To this extent, many standards for security managements at operational level exists (e.g., ITIL, ISO27K family etc). What is often missing is a process to govern security at organizational level. In this tutorial, we present a method to analyze and design security controls that capture the organizational setting of the system and where business goals and processes are the main citizen. The SI*-GRC method is a comprehensive method that is composed of i) a modeling framework based on a requirement engineering framework, with some extensions related to security & GRC concerns, such as: trust, permission, risk, and treatment, 2) a analysis process defining systematical steps in analyzing and design security controls, 3) analytical techniques to verify that certain security properties are satisfied and the risk level is acceptable, and at last 4) a CASE tool, namely the SI* Tool to support analysts in using the method. To illustrate this method, we use a running example on e-Health adapted from a real-life process in an hospital partner.

1 Introduction The last decade has seen two parallel and conflicting demands on businesses services: business processes are increasing in complexity and unpredictability, while demands for accountability, regulatory compliance and security are becoming mandatory. Thus, a structured approach to Governance, Risk and Compliance (GRC) of Information Security has become a high priority goal [1]: – Governance is the set of policies, laws, culture, and institutions that define how an organization should be managed (as opposed to the way it is actually managed); – Risk Management is coordinate activities that direct and control an organization forecasting and managing events/risks that might have a negative impact on the business; – Compliance is the act of adhering to regulations as well as corporate policies and procedures. GRC products support the achievement of compliance by automatic monitoring of controls across different applications, offers audit facilities, the prioritization of corrective actions according risk based profiles [2]. Traditional audit methods involve the review A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 152–184, 2011. c Springer-Verlag Berlin Heidelberg 2011

A Method for Security Governance, Risk, and Compliance (GRC)

153

of transactions at a given time (i.e., sampling). However, this approach does not give sufficient assurance over the level of security and compliance of business processes over an entire period, and more advanced GRC products (e.g. SAP and Oracle Financials) offer Continuous Controls Monitoring techniques. Tools such as Audit Command Language [3] are becoming increasingly popular as cheap and flexible options for the implementation of continuous controls. GRC solutions for technical systems are well understood (even if not thoroughly deployed) and a number of textbooks clearly identify the desired security properties. For example in [4] we find confidentiality, availability, and integrity of computer-related assets, and it also includes possession/ownership, utility, authenticity, non-repudiation, and privacy of information. In addition to those qualities, compliance with specific regulations might require some additional qualities, such as accountability (SoX, Basel II), assurance (SAS 70, CMMI, CommonCriteria), etc. However, it is becoming increasingly evident that managing security and GRC of a system is not only a matter of technical solutions. Most of the state of practices (e.g., ISACA RiskIT[5], COBIT [6]) define risks as the obstruction originated from the availability of resources (e.g., documents, servers, production machines). However, the availability of resources is not sufficient to guarantee the security or GRC of a system, because sometime a disruption is originated from the process or even the objective level of the system. In other words, security violations might be originated from any layer of the organization (i.e., strategic, process, infrastructure, information). The focus of this work is to consider as a “system”, a more complex construction and namely a socio-technical system [7] where both social and technical aspects, and its relations are considered as an integrated whole. The method presented in this tutorial distills a number of research articles and empirical studies of the University of Trento and a number of colleagues from industry and academia (See the acknowledgments). It analyzes security and GRC aspects of a socio-technical system and focuses on the organization’s goals and processes of the system. The method as a whole [8] assumes that a security initiative is a long-life program that an organization needs to carry on (i.e., not a project-based initiative) based on the Deming management cycle of plan-docheck-act [9]. In other words, one needs to consider how to monitor and continuously improve the implemented controls in the system. Here we put the emphasis on the Plan phase where analysts model the target system and analyze the security GRC issues and design security mechanisms to manage the excess risk. Table1 shows the basic checklist for the method. In the following sections, we demonstrate how the SI*-GRC method is used to analyze and design security and GRC concerns of the information system used in a rich scenario from e-Health (§2). We then we provide an overview of the method describing who the participating roles are, what the steps are, and what is contained in the models of the method (§3). We explain in details how the method support the Security and GRC of the system during the Plan phase. We start by describing how to capture a target of analysis the “SI*-GRC” way (§4), then how to analyze security and GRC concerns (§5), and finally how to define the required controls (§6). We discuss briefly the technique for in-vivo monitoring and update of the controls (§7) and conclude.

154

Y. Asnar and F. Massacci Table 1. SI*-GRC Method Checklist – The Role Identification and Organizational Set-up specifies the roles and responsibilities for the SI*-GRC team, the business process owners and the IT management. – The Specification of the Target of Analysis captures at high level the socio-technical system with the key relevant information. The analyst should quickly identify the following aspects: • Actor Models capture relevant actors (i.e., social and technical ones) their organizational structure. • Goal Models capture the objectives of the organizations by spelling out the intentions and capability of identified actors and the functional interdependency between actors and goals. • Process and Services Maps identify the processes which achieve the organization’s goals. In this model, one can see the relation between how a goal is being carried out by a business process, and how the business process is supported by series of business services and/or resources. An organization typically has formalized its business process in some formalization (e.g., BPMN, Flow-chart, UML Activity Diagram). This formalization should be re-used as much as possible. – The Security and Risk Analysis captures which aspects of the system are not protected • Business Continuity Critical Points Identification specifies the business processes and actors whose commitment is essential to achieve the high-level goals of the organization. • Unauthorized Processing Identification identifies existing permissions (if known) and how they are delegated to other actors. Provide as output potential unlawful processing and over-entitlement scenarios. • Trusted Computing Base Identification identifies the trust relationships between actors in possibly making references to specific goals. Provided as output the boundaries of the TCB that can be sources of potential failures of reliability and misuses of permission in terms of scenarios for failures of the organizational goals. • Unwanted Scenario Identification supports the analysts in identifying threat, events and gives guidance how to structure them at resource, process or strategic level. • Risk Assessment estimates the risk level of identified risks and specifies which risks will be treated or accepted – Control Analysis identifies the control mechanisms put in place to address the risks. • Control Goal Model specifies and elaborate the control goals in order to cover most (if not all) risks, with appropriate and precise measures. • The Control Processes and Services Map identifies the control processes that achieve the control goals.

2 The Running Example Scenario The scenarios used in this article are adopted from the drug reimbursement process in a major hospital in Italy [10]. This process is only applied for a specific set of drugs (called File F drugs) to be delivered to outpatients, patients in “day hospital” regimen, and patients during the last day of hospitalization. This list includes innovative and expensive drugs, caring chronic diseases with a high social impact (tumors, multiple sclerosis, HIV, etc.).

A Method for Security Governance, Risk, and Compliance (GRC)

155

The economic aspect of this business process is particularly relevant as the regional wide expenditure for these drugs is raising year by year (in +12% in 2006, +10% in 2008), bringing to an augmented attention of the Health Authority toward the existence of an effective system of controls over the dispensations of these drugs1. As a consequence, this business process is subject to significant security and compliance requirements: all regulations2 of the File F Regulatory Framework have to be respected by the hospital in order to obtain the reimbursement from the regional health authority. Moreover, since the drug reimbursement process includes the processing of personal and sensitive data (the recipients of the drugs), it must also satisfy the regulations on privacy from the Legislative Decree no. 196 of 30.06.2003 called ”personal data protection code”, in addition to well-established security requirements (e.g., nonrepudiation of a drug dispensation, availability of the prescription data, integrity of the drug dispensation process) In nutshell, the Drug Reimbursement process is a mechanism that allows the hospitals to refund the costs of the dispensed File F drugs. The process is composed of three macro-phases: Drugs Prescription, Drugs Dispensation, Generation & Sending File F reports to the Health Authority. In the Drugs Prescription phase, a prescription is done by a doctor to a patient using an IT system. The doctor selects File F drugs to administer to the patient and of their posology and quantity. The selection of the drugs can be done by copying the past prescriptions (in case of a chronic diseases) or also choosing them from a complete list of File F drugs. Finally, a prescription sheet is produced containing the patient’s name and other personal data (e.g. the Fiscal Code), and also sensitive data (e.g., the patient disease, the prescribed drugs and their quantities/posologies). In the Drug Dispensation phase, dispensers (doctors or nurses) dispense File F drugs following a prescription given to a patient. At this phase the dispensers need to indicate which drugs, from the prescription, are being dispensed, since ones might dispense a part of the prescription for some reasons (e.g., the drug is currently unavailable). In some occasion, some drugs appear to be unavailable in the computer system though in reality they are available in the hospital ward drug stock. In this case, the dispensers must dispense the drugs because it is critical for the patient’s safety. Finally, a dispensation sheet is printed and needs to be signed by the dispensers and the patients. This sheet contains similar information as the prescription except the patient disease. The Report Generation phase is constituted two parts: 1) the generation of the File F reports to be sent to the Health Authority; and 2) the sending of the File F reports to the Health Authority. In the first part, the Accounting Office retrieves all the data about the dispensed drugs in a month, checks the data, produces the File F reports, and sends them to the Hospital Medical Director. In the second part, an operator from the 1

2

The Regional Directive n. 5743 - 31.10.2007 provides indications to optimize and improve the process design about prescription/dispensation/accounting of File F drugs, and the Regional Directive VIII/1375 - 14.12.2005 stresses the priority to implement actions towards the verification of the appropriateness of the use of File F drugs. To have an idea, without mentioning privacy requirements, the File F mechanism was instituted by the regional circular 17/SAN 3.4.1997, and successively has been emended by the Circular No.5/SAN 30.1.2004, by the Circular No.45/SAN 23.12.2004, by the Note 30.11.2007 H1.2007.0050480, by the Note 27.3.2008 H1.2008.0012810 and by the Note 04.12.2008 H1.2008.0044229.

156

Y. Asnar and F. Massacci

Medical Director verifies the File F reports and sends them to the Health Authority supervised by the head of department. All the operations are assisted by the IT system and the File F reports have to be sent to the Health Authority following a special guideline, namely the “debito informativo” IT channel, that permits to send and receive information through common e-mail clients with additional security mechanisms that guarantee the confidentiality of the exchanged data (encryption of the message and of the attachments, digital signature, etc.). It is another security &compliance requirement posed in the information system. For technical details, we assume the drug dispensation process is supported by an information system implemented in terms of services, in SOA [11] sense. Still, we believe this method is applicable for other technical architecture (e.g., object-oriented, web-based)

3 Overview of the Method This section illustrate the SI*-GRC method at high-level, starting from the organizational context to be prepared before using the method, the conceptual modeling used in the SI*-GRC, namely the SI* modeling framework, and finally illustrate the SI*-GRC process that will be detailed in the following sections. 3.1 Roles and Organizational Set-Up In order to manage security and GRC, the preliminary organizational steps are essential to ensure that security & compliance is achieved effectively across the organization and in a timely manner. More importantly, they aim to obtain management buy-in, to clarify the ownership of a process & the responsibility of controls, to align the organization’s strategic objectives with regulatory requirements, and above all to maintain security across system evolution. Often such aspect is usually neglected in research papers on information system engineering (e.g., [11,12]) which directly focuses on the functional design of the system. The preparation steps are the following ones: 1. The Identification or set-up a GRC council ensures the periodic review of the organization’s security policies and procedures; the level of compliance with these policies and procedures; the level of exposure of key information assets during implementation; the ongoing improvement of existing control implementations (i.e., control processes and indicators); and the availability of sufficient resources for SI*-GRC. In other words, these actors are essential staff members during the Check and Act phase. 2. The Definition of roles and responsibilities for the SI*-GRC team. Actors that make up the analysis-design Council: (i) management (e.g., members of the Board of Directors, the Chief Executive Officer (CEO), and Shareholders); (ii) business process owners (key personnel that are responsible for the daily operations of the concerned business process); and (iii) IT management (those responsible for the governance, daily operations and maintenance of the IT infrastructure).

A Method for Security Governance, Risk, and Compliance (GRC)

157

The followings are actors involved in the design of SI*-GRC controls: Business Analysts are those responsible for the analysis of the organization’s business goals & processes; Risk Analysts are responsible to assess the level of risk of some threats to the business, and decide whether they are acceptable or not; Security Analysts are responsible for the design and implementation of actions aimed at providing a reasonable level of assurance concerning compliance with security policies. Security analysts generally make part of IT management; SI*-GRC Analysts are proficient with the method that can coordinate the analysis and design process of control processes and indicators. Moreover, the SI*-GRC analysts are responsible for ensuring that the SI*-GRC controls and indicators are implemented according to the designs and provides an appropriate/improved level of assurance concerning the achievement of strategic objectives.3 Note the performance of these actors is critical for the success of security & GRC initiatives. 3. The creation of a timeline for the SI*-GRC deployment. indicating the key goals and milestones of the SI*-GRC application, as well as those responsible for each milestone. The timeline should list all tasks to be carried out during the SI*-GRC application including decision points. 3.2 Process Overview The SI*-GRC method includes a description of systematic steps to support the PlanDo-Check-Act (PDCA) of the Deming Cycle [9] to ensure the continuous monitoring and improvement of the secure system as depicted in Figure 1. The Plan phase aims at capturing and analyzing the problems in the target system and at the end results in design of necessary controls. The Do phase concerns on the implementation up to the execution of such controls. The Check phase aims at reviewing the implemented controls and the existing organization context. Finally, analysts need to re-act upon the review results to improve the security and GRC of the system at the Act phase. These reactions must be planned before they are implemented in the system. The general steps of the Plan phase are depicted in Figure 2, where includes 1. Target System Definition where an information system is formalized in the context of an organization; 2. Security and Risk Analysis where and it captures and analyzes security and GRC aspects (i.e., including trust and permission) and then assess the possible risks; 3. Control Analysis where some necessary controls are formalized including their quantitative indicators indicating their effectiveness and performance. The Target System Modeling step covers three modeling activities. Firstly, the Actor Modeling captures actors (i.e., human or technical) involve in the system including their structure in the organization. Analysts then capture and analyze the strategic interests in 3

In this work, we assume an improvement at the information security, governance, compliance, and risk management will lead to the better assurance of the business.

158

Y. Asnar and F. Massacci

• ! * ! • *+ &!* • *+&!

• '! • !*! • )&!!

SI-GRC Method

• ') "! • +) " ( ! • !& (!

• '( /"' ! & ! • '(&!& $ &!*/- .

Fig. 1. The Plan-Do-Check-Act Cycle of the SI*-GRC Method

the Goal Modeling. Moreover, analysts need to capture strategic rationales and dependencies of an actor [13] and each capability. To achieve those interests, an organization defines systematic steps (i.e., a business process). In our method, we use BPMN [14]) to capture the business process though we believe other process models (e.g., YAWL [15]) will also be applicable. The Processes and Services Mapping indicates which process satisfies which goal. Moreover, in the business process one must indicate which resources (i.e., data object in BPMN or underlying business services [11]) involve in the execution of business process. At Security Analysis step, it composed of four activities: 1. Unwanted Scenario Identification which identifies potential events 2. the Unauthorized Processing Identification models actors’ entitlements and the delegation permission to another actor, the 3. Trusted Computing Base Identification where capture the trust relationship between actors involved in the system, 4. the Risk Assessment needs to measure the level of risk for the high-level goals of the target system after considering its trust and permission models, and finally decides which risks that are unacceptable therefore require some controls to mitigate them. The Control Analysis aims at analyzing further the control mechanisms. It starts from the Control Goal Establishment describing the objectives of the controls. The Control Goal Analysis specifies and elaborate the established control goal so that they cover most (if not all) risks, accurate, and have a clear-cut definition of their fulfillment. At last, the analysts need Control Process and Services Map which details the means to achieve a control goal. This map is very similar to the one used for business process and

A Method for Security Governance, Risk, and Compliance (GRC)

• • •

• •

• • •

159

• •

•

Fig. 2. The SI*-GRC Process Overview for the Plan Phase

business goals excepts that it aims at protecting the business process from the excess risks by achieving the control goals. The final outcomes of the SI*-GRC for the plan phase are a list of detailed control mechanisms composed of control goals describing the state-of-affairs to be achieved/protected, a process description of how those goals are implemented in term of control processes, and a series of indicators indicating their effectiveness and performance, and the organization’s trust and permission model. Example 1. Based on the scenario given in Section 2. – Control Goal - File F reports must respect the privacy preference defined by a patient; – Control Process - Anonymize the name of patient that requests to, after File F reports are generated; – Indicators • The number of privacy litigation coming from patients – effectiveness; • The number of File F records not being anonymized where the patient requests to be anonymized – performance; • How often the anonymization process is executed – performance; • How many records have been anonymized – performance. – Trust Model - Accounting Officers trust that dispensers will be honest in inputting dispensation data; – Permission Model - Doctors have permissions to access a patient’s medical info. More details will follow in the incoming sections (i.e., Section 4-6); and for other phases (i.e., Do, Check, and Act, we only illustrate the usages of the method in-vivo of the information system (Section 7). 3.3 SI* Modeling Framework The SI*-GRC method uses the SI* modeling framework [16] for most of its modeling except the modeling of business process where uses BPMN. The SI* framework is a

160

Y. Asnar and F. Massacci

contribution

delegatum/ trustum

decomposition

Event

contribution

Delegation/ Trust

has

Impact

delegatee/ trustee

delegator/trustor impacting

Business Object

Actor

alleviation

Goal

Process mean-end

Resource mean-end decomposition

decomposition

Control Goal

Control Process

Fig. 3. The SI* Conceptual Modeling

modeling framework extending the i* framework [13] to support security requirement analysis. In Figure 3, the conceptual model of this modeling framework is based on basic concepts representing requirement analysis Actor is an autonomous entity that has its own intentions (human and software), capabilities, and entitlements. This concept is then realized as; Goal is a state-of-affair that an actor intends to achieve; Process is a means to fulfill a goal, to furnish a resource; Resource is an artifact that is consumed/produced by a process; Event is an uncertain circumstance that affects a goal satisfaction (in/directly); Trust captures a believe of the capability/honesty of one actor (trustor) to another actor (trustee) in fulfilling/using a business object (trustum); Delegation depicts a transfer of the responsibility/right from one actor (delegator) to another actor (delegatee) in fulfilling/using a business object (delegatum). The Appendix A lists all constructs (i.e., concepts and relation) in the SI* Framework including the graphical diagrams and DLV predicates. Details relations between constructs will be explained and illustrated along the analysis and design process. To avoid confusion, we use an italic text to indicate a SI*-GRC basic concepts and a sans-serif text for the one related to the scenario.

4 Target System Definition The first step of the SI*-GRC process is modeling the target system where one needs to analysts its security and GRC concerns. In some literature in the organization behavior [17], management information system [18], and enterprise architecture [19], one can organize an information system in an enterprise into three levels of abstraction as depicted in Figure 4. Essentially, an information system always serves the objectives of the stakeholders (i.e., including the organization’s shareholders). These objectives are implemented by series of business processes that are inter-related one to another. To

Objective

A Method for Security Governance, Risk, and Compliance (GRC)

Stakeholder Goals

161

Company Vision-Mission

Business Objective BO 1

BO 2

Patient

Go for Medical ExaminationGo for Patient Medical Personal Examination Information Go for

Receive Drugs

Process

Patient

+

+

Medical Personal Examination Information Go for Medical Personal Examination Information

Write Prescription

Doctor

+

Receive Drugs Dispensation Receive Sheet Drugs + Dispensation Receive Sheet Drugs

Patient

Doctor

+

Dispensation Sheet

Write

Personal Information Prescription Drugs Write Prescription Prescription + Drugs Write

DoctorPrescription +

Dispensation Sheet

Doctor Paramedics

Prescription Dispense + Prescription Drugs Drugs

Paramedics Receive

Prescription

+

Receive Prescription

Dispense Prescription Drugs Drugs

Paramedics

+

Paramedics Receive

Prescription Receive Prescription

Dispense Drugs +

Dispense Drugs

Resource

+

Phsical & Logical Resource

Software

Application Documents Server

Fig. 4. Three Layer Model of an Information System in an Enterprise

execute such processes, participating actor might need some resources either physical or logical ones. To capture such settings, analysts need to define the Target System in terms of the structure of actors in the organization, their interests and capabilities, and interdependencies between them. Finally, we need to map which process is associated to which stakeholders’ goal. The overall process of this phase is described in Figure 5. Note that in the SI*-GRC method analysts might choose any approach for a Process Modeling in the organization, such as Event-Driven [20], Workflow (e.g., BPMN [14], YAWL [15]), UML-based [21], Declarative BPs [22], or Case-base [23]. 4.1 Actor Modeling First task towards modeling the target system is the identification of the key social actors whose intentions need to be fulfilled by the system or are mostly involved in the system. Depending on the level of analysis one might consider and analyze strategic interests of social actors[24] or technical actors (e.g., email service, data management agent) and analyze the social actors’ goals that are delegated to the technical actors. For a comprehensive analysis we need to consider most actors involve in all phases of the system existence (if relevant) and in particular

162

Y. Asnar and F. Massacci

Actor Modeling

Process Modeling

Goal Modeling

Process Mapping

Fig. 5. The Process Model of the Target System Definition Step

– subject actors, whose information will be stored/processed by the information system - e.g., Patient; – usage actors will use/consume the information produced/managed by the information system - e.g.,: Pharmacy, the Health Authority, the Auditor, the Accreditation Office (e.g., Joint Commission); – system actors who are part of the information system to use and maintain it - e.g., the IT departments, the Risk & Compliance Office, business analysts, and all actors in Figure 6; Analysts need to be not confusing between role or agent. A role is an abstract characterization of intentional entity’s behaviors, and an agent is an intentional entity with a concrete manifestation. In other words, one can distinguish an agent as an object level entity that has one-to-one correspondence with an entity in the system world; while role as the class actor needs to be distinguished into role. Example 2. In our scenario, the Hospital “H”, the healthcare authority, Dr. Dave, Alice, and the Head of Hospital are considered as agent; while doctor, nurse, dispenser, and accounting officer as role. Given a series of actors, analysts need to structure them using the following relations: 1. play indicates an agent that plays a particular role; 2. is part of indicates an actor that is part/member/composed of another actor; 3. is a captures a specialization relation between two actors (i.e., role-to-role or agentto-agent); 4. supervise captures a supervision relation between two actors (i.e., role-role or agentagent). Example 3. In Figure 6, we model the Hospital “H” that is composed of a series of operational unit (e.g., Cardiology, Gynecology, etc.), an Accounting Office and a Medical Director; leads by a head of hospital Mr. Henri. Dr Dave and Dr Dubois play a role of doctor, while Alice plays a role as nurse and dispenser. All these agents are member of (is part of ) the Cardiology Department. Since Doctor and Nurse are (is a) dispenser, all these can act as a dispenser in the Cardiology Department. These actors are essential to set up the scope of the following modeling activities. Note in Example 3, we only consider actors that are part of the drug reimbursement system.

A Method for Security Governance, Risk, and Compliance (GRC)

163

Play

Doctor Dr. Dave

Hospital H

Head of Operational Unit

Is Part Of

Is Part Of Is Part Of

Dispenser

Operational Unit

Accounting Office

Henri

Is Part Of

Is Part Of

Play

Nurse

Head of Med. Director

Is Part Of

Is A

Play

Medical Director

Is Part Of Is Part Of

Alice

Is A

Is Part Of

Is A

Dr. Dubois

Head of Hospital

Is Part Of

Play Play

Is Part Of

Accounting Officer

Medical Director Officer

Play

Is A

Play

Is Part Of

Harry

Is Part Of

Is Part Of

Cardiology Dept.

Andrea

Mary

Fig. 6. The Actor Model in the Hospital “H”

4.2 Goal Modeling For a given actor model, analysts need to identify the strategic interests (goals or objectives) that the actors intend to achieve and model using a Request relation depicted in Figure 7. These cover responsibilities, motivations, and intentions of actors in the organization and business objectives of the organization. Example 4. The patient needs to get medical service and the hospital interest to obtain the reimbursement of dispensed drugs. In particular to the context of security and GRC, one considers also the high level norms imposed by the organization information policy or by the standardization body or by the regulator. Example 5. The hospital “H” management intends to keep the all medical records in two modalities (e.g., paper and digital) for the next 3 years; The Joint Commission (a standardization bodies) requires an hospital to initiate and maintain all clinical records of every patient assessed or treated (MCI.19) - completeness, sufficiency, integrity, and accountability; while MCI.10 & MCI.11 require information privacy & security are maintained; – The health authority, in Regional Circular No.5/SAN 30-1-2004, obliges an hospital to follow a strict guideline to send a drug reimbursement report in to a defined format (e.g., three files - personal data, drug info, drug cost) delivered via a special channel, namely the “debito informativo” channel. Often an actor cannot achieve its goal by itself therefore one might 1) appoint another actor to full the goal using a delegation of execution, or 2) decompose the goal further into more detail/precise subgoals using an AND decomposition relation and assign

164

Y. Asnar and F. Massacci

Obtain the JCI Accrediation

R

Patient

De

Joint Comission

The Healthcare Auhtority

De

Hospital H

De R

Obtain the drug reimbursement

R

De

Get medical service

Get medical service

Provide medical service

Treat Patient

Operational Unit

Record all reimbursed drugs

Deliver the report

Generate the drug reimbursement report

AND De

Treat Patient

Send the report via the debito informativo

Deliver the report in person

AND

Register patient

Examine patient

De

AND

De De

Dispense drugs

De

Prescribe therapy/drugs

De

Accounting Office

Doctor

Generate the drug reimbursement report AND

Dispenser

Medical Director

De

Review the report

Produce the report

Retrieve the dispensed drug data

Fig. 7. The Goal Model of the Drug Reimbursement System

parts of them to other actors. AND-decomposition indicates that all subgoals must be fulfilled by the system in order to achieve the main goal. Sometimes it is useful to specify alternatives to fulfill a goal using several means-end relations indicating alternatives to fulfill the goal. This decomposition is rarely used in practice during an audit or assessment step of existing security mechanisms. However, it is mostly used during the plan phase of a new security mechanism. Example 6. To obtain a drug reimbursement, the hospital “H” needs to do all these (AND-decomposition): record all reimbursed drugs, produce the drug reimbursement report, and deliver the report. To achieve the goal of deliver the report, the hospital can send the report via the “debito informativo” channel managed by the Healthcare Authority or deliver the report in person This analysis is an iterative process, where sub goals need to be refined further with AND-decomposition/means-end relations or being delegated its fulfillment to another actor. This decomposition stops for an actor when all leaf goals (i.e., the lowest subgoals) are have being delegated to other actors via delegation execution or because the actor can fulfill the subgoal by itself.

A Method for Security Governance, Risk, and Compliance (GRC)

165

Later in the analysis of the target system we might refine further this capability by specifying a business process that achieves that. Otherwise, we do not care how the goal is actually achieved (but we know it will be achieved) we can simply mark this assumption in the model. Notice that this is an important assumption that would need to be validated at some point as it enters in the trust model of the system. Example 7. For achieving the goal of generate the drug reimbursement report, the hospital delegates it to the accounting office. However, the accounting office is only capable to retrieve the dispensed drug data and produce the report, but it still needs to delegate further a part of the goal, namely the subgoal review the report to the medical director. The final outcome of this modeling is a goal model, as in Figure 7, which captures a network of actors’ dependencies on fulfilling their goals and the rationales of such goals. 4.3 Process and Services Mapping As mentioned earlier, SI*-GRC allows analysts to use an existing formalization for the process modeling The aim of this modeling to indicate which process, a part of the business process, is a means to fulfill actors’ goals. In the goal modeling, analysts have identified the leaf-goals that actors are able to fulfill; and in the process mapping, analysts map those leaf-goals to the processes using means-end relations in which their executions will fulfill the leaf-goals as depicted in Figure 8.

GOAL MODEL

Doctor

Prescribe therapy/drugs AND

Obtain patient medical record

Prepare prescription

Finalize the prescription sheet

BPMN MODEL Doctor

Select Drugs Medical Info

+

Nurse/ Doctor

Case Details

Retrieve Patient Medical & Personal Info

Identify Patient

Print and Sign Prescription Sheet

Old Prescription

Archive Prescription Sheet

Prescription

Specify Privacy Preferences

Fig. 8. The Process Mapping in the Prescription Phase of the Drug Reimbursement Process

166

Y. Asnar and F. Massacci

Example 8. To achieve the goal prescribe therapy/drugs, a doctor needs to fulfill three subgoals as depicted in Figure 8. To fulfill obtain patient medical record, a doctor/nurse needs to perform two activities retrieve patient medical & personal info and identify patient consecutively. Moreover, those two activities can also be performed by a nurse. By means in this mapping, analysts indirectly define which actors have capabilities to perform particular processes/activities (including sub-process or task in BPMN) to fulfill a goal. Moreover, it can also infer which actors can provide/require particular resources (i.e., data object in BPMN) from which activity produces/consumes some resources. In a SI* model, these capabilities are captured with Provide relations. Example 9. Based on Figure 8, we can infer the capabilities of a doctor and a nurse that can perform the activity of retrieve patient medical & personal info and provide the resource of medical info. As mentioned before, sometime analysts might prefer to indicate actors’ capabilities at the high-level (i.e., goal) without caring how it is actually achieved. As we shall see later this is an important trust assumption. Example 10. A doctor can provide the fulfillment of the goal examine patient. From Figure 8, the complete capabilities of each actor from are listed in Table 2. In SI*-GRC resource is not solely data-related artifacts, but it expands into other physical and logical resource, such as: prescription & dispensation sheet, underlying business web services that supports the execution of some activity, softwares used to process some data while performing some activity, and infrastructures used to collect, store, and transmit the data during the process execution. At the end of this mapping, analysts have completed a three-layer model of the enterprise information system as depicted Figure 4. Table 2. Actors’ Capabilities on the Drug Reimbursement System Capability - Provide relation Performing an Activity/Process • Patient Medical & Personal Info • Identify Patient • Specify Privacy Preferences • Select Drugs • Print & Sign Prescription Sheet • Archive Prescription Sheet Providing a Resource • Medical Info • Case Details • Old Prescription • Prescription

Doctor Nurse X X X X X X

X X X -

X X X

X -

A Method for Security Governance, Risk, and Compliance (GRC)

167

5 Security and Risk Analysis After defining the target system, analysts need to analyze the security and GRC concerns of the system. In SI*-GRC we consider a whole system as an asset, something valuable, therefore we need to protect it from unacceptable events that might harm the security and compliance requirements of the system. This step aims to analyze security and GRC concerns of the system; it starts by identifying critical points that can disrupt the business continuity. Analysts then identify the ownership of processes and resources in the system, and delegation its permission to other actors, and defining the trust relations between actors exist within the organization. Given such settings, we might identify several unwanted events (i.e., threats, failures, errors) that can compromise the security & GRC of the system. To decide which unwanted events need to be controlled, analysts need to estimate the level of risk of such events, and finally take into account the organization policies on controlling them (e.g., risk tolerance, ethics, IT architecture policy). 5.1 Identification of Business Continuity Critical Points The outcome of the target analysis phase is a complex web of relations between actors and business objects. The preliminary analysis of this model is to ensure that all critical points for business continuity have been identified. Intuitively, a business continuity disruption happens when an intention to fulfill a goal (or a part of goal) is passed across actors using delegations of execution relations and it ends to an actor who does not have explicitly the capability to fulfill the goal (or part of it) in term of performing a process or providing a resource. This property can be easily visualized on the model as a path that does not end with either a means-end relation into a business process or with an atomic provide relation between the last actor in-charge and the goal. Example 11. In Figure 7, the Hospital “H” need to delegates the goal register patient to the Operation Unit. Unfortunately, that actor, including any actor part of it, does not have necessary capabilities to fulfill the goal, and not even pass the goal to another actor. This potential disruption can be resolved by reorganizing the delegation of execution to end with a specific business process or by allocating to actor that are explicitly declared to be capable of achieving the desired goal (i.e., outsourcing might be the least thing that an organization can do, when none in the system is capable). In the former case we have specified how the goal will be actually achieved in the system and the business process itself is a critical point for business continuity. In the latter case we have only specified that an actor is in charge of this goal which therefore makes the behavior of this actor critical for the overall achievement. This case can be later analyzed by considering possible unwanted scenarios that lead to the disruption. Analytical techniques are available with the SI* tool. The papers [16,25] describe the formal properties in more details.

168

Y. Asnar and F. Massacci

Dp

O

Hospital H

Patient O

Medical Info

O Prescription Dp

Dp

Doctor Old Prescription

O

Prescription Process +

Dp

Nurse

Fig. 9. The Permission Model in the Prescription Phase of the Drug Reimbursement Process

5.2 Unauthorized Processing Identification So far we have modeled the transfer of responsibility via delegation execution due to actors’ capabilities. However, this concept does not capture the notion of ownership of a particular process or resource. Sometime to execute a process or to access a resource, an actor needs to have a permission from the owner of process/resource. In this modeling phase, we capture the notion of ownership of process/resource and the notion transfer of rights on a business object from one actor to another using a delegation permission relation. The Owner relation depicts a relation between an actor to a process/resource that indicates the actor has full authority concerning the execution of a process or the access of a resource. 4 Example 12. The patient owns the resource of its medical info; and the Hospital “H” has a full authority for all activities in the prescription process depicted in Figure 8. Typically, defining the owner of a resource is simple because a resource is tangible and often some policies/regulations already define the notion of ownership (e.g., EU Data Protection Directive-Directive 95/46/EC or the company property policy). However, it is less trivial for the process ownership. However, in many situation the owner of such process/resource needs to delegate the permission to another actor because the owner does not have capability to execute the process or the other actor needs to access the resource/to execute the process to fulfill its responsibilities. For such situation, the SI* models the action of giving a permission using a delegation permission relation. Example 13. The Hospital “H” delegates the permission to doctor on executing activities in the prescription process (as in Figure 8) because the hospital “H” needs the doctor to fulfill the goal prescribe therapy/drugs that at the end fulfills the top level goal provide medical service. The final result of this modeling activity is represented in term of a SI* model as depicted in Figure 9. This model can then be used to identify more precisely two possible threats which are very important for compliance: 4

Process Owner is defined as an actor that is held accountable and responsible on the performance and improvement of a defined process (including its subprocesses) in http://www. gao.gov/special.pubs/bprag/bprgloss.htm

A Method for Security Governance, Risk, and Compliance (GRC)

169

– lack of Authorization (or unlawful processing) is present when an actor has been assigned a process or managing a resource without a proper authorization path stemming from the owner. This aspect is particularly critical when demonstrating compliance with privacy legislation; Example 14. In Figure 9, the doctor does not have authorization to access the medical info that is required to perform the prescription process – Over-entitlement when an actor has been delegated the permission to access a resource or to execute a process but the latter is not required to achieve the goals assigned to the actor. At minimum this might be a violation of the minimal disclosure principle for the compliance with privacy legislation or might be the source of more serious troubles if the actor can potentially misbehave (e.g., fraud, internaltrading). Example 15. In Figure 9, the nurse have permissions to execute all activities in the prescription process, though in practice the nurse only responsible to perform the retrieve patient medical & personal info activity and the identify patient activity. These properties can be easily visualized on the model as paths. Intuitively, a unlawful processing for a key business objectives is present when there is a path across decompositions and delegations of execution that arrives to an actor. Yet this processing actor, in order to achieve its delegated subgoal make use of some process or some resource whose owner has not delegated the authorization directly or indirectly to the processing actor. This latter properties is simply the absence of some path from the owner to the processor. Also in this case analytical techniques are available with the SI* tool vulnerabilities and threats of a SI* model formalized in Answer-Set Programming (ASP) [16,25]. These potential threats should then be considered in the final mitigation reports because both threats introduce various type risks to the system’s security, governance, and compliance. 5.3 Trusted Computing Base Identification Trust, in SI*-GRC method, captures a social relationship that indicates the belief of one actor (trustor) on another actor (trustee) capability or honesty. The SI*-GRC method distinguishes two types of trust relation: – Trust of execution represents the trustor’s expectations concerning the ability of the trustee accomplishes the trustum (i.e., achieving a goal, executing a process, delivering a resource); Example 16. The operational unit trusts the doctor on fulfilling the goal of prescribe therapy/drugs – Trust of permission models the trustor’s expectations that the trustee is honest/does not misuse the permission on the trustum (i.e., fulfillment of a goal, execution of a process, access to a resource). In other words, by trusting in permission, the trustor believes that the trustee will not use the given permission for accomplishing a purpose different from the one for which the permission has been granted.

170

Y. Asnar and F. Massacci

Te

Prescribe therapy/drugs

Te

Operational Unit

Doctor Tp Retrive patient medical & perosnal info

Play

Tp

Nurse

Play Dave

De

Retrive patient medical & perosnal info

De Alice

Se Se

Fig. 10. Trust Model in the Prescription Phase

Example 17. The doctor believes that the nurse will not misuse the permission to retrieve patient medical and personal info A frequent error in this analysis phase is to assume that each delegation of execution step has to be “completed” by adding the corresponding trust relationship. However, such situation is not always be the appropriate because we might need to delegate actions to people we don’t trust. Example 18. Dr. Dave might not trust Alice because of some personal experience. However, he needs to delegate the goal of retrieve patient medical record because he needs to do other urgent responsibilities and only Alice is available to help. A more frequent example at organizational level is that Example 19. company “X” is known to be unreliable from the point of view of Henri (Head of the Hospital “H”), the hospital “H” needs to archive their prescriptions with the company “X” because they have won a county-wide procurement contracts for all hospitals on the region. Once the model has been specified it can be used for a precise analysis of boundary of the (un)trusted computing base at organizational level: – Potential Unreliability might happen when an actor has been assigned a goal but is not trusted to achieve it. This might generate a potential cascading failure for some of the key goals of the actor who delegated this responsibility. Example 20. Since in Figure 7 the patient might perceived at risk because it does not trust the Hospital “H” to fulfill its goal get medical service – Potential Misuse when an actor has been delegated an authorization but is not trusted not to misuse it.

A Method for Security Governance, Risk, and Compliance (GRC)

171

Example 21. From Figure 9 the Hospital “H” believes a potential misuse on the permission on executing the prescription process given to the Nurse Also in this case analytical techniques are available with the SI* tool vulnerabilities and threats of a SI* model formalized in Answer-Set Programming (ASP) [16,25]. These potential failures indicate some critical points in the trusted computing base. The trusting computing base might be too narrow with respect to our expectations in terms of successfully achieving the goals of the target systems and preventing misuses. At this point we have two alternatives: – identify possible scenarios on a breach of trust for the critical actor and their critical goals (following the steps detailed in the next section and identify possible countermeasures); – or extend our trust models to incorporate the critical actors in our trusted computing base. This last step should not be interpreted as “we just add the trust link in the model” (albeit this is the most common errors that students do). Rather this means that we put in place alternatives techniques for trust building such as those based on legal measures as identified in [26]. Example 22. To extend the trusted computing base in Example 19, Henri includes in the service contract a clause about compensation for punitive damages due to disclosure of the prescription information by the Company “X”. 5.4 Unwanted Scenarios Identification The next step is to identify unwanted events that might disrupt the system in term of its security & GRC relevant properties (e.g., confidentiality, integrity, availability, authenticity, privacy, etc.). In this tutorial, we emphasize on events with negative impact, and a more comprehensive classification can be found in [27]. Looking at the architecture of an enterprise information system in Figure 4 one might consider events that might risk the system into three classes: – Resource level - events that occur because some disruptions at the resource level; Example 23. The Archive service is not functioning so that it cannot archive the prescription sheet that surely disrupts the continuity of the prescription process. – Process level - events that occur because the business process has been performed different from its design; Example 24. The Dr. Dave is too busy to print & sign the prescription sheet therefore he asks Alice to do so. This scenario might compromise the goal maintain audit trail of prescription that is critical to get an accreditation from the Joint Commission. – Objective level - events that prevent the state-of-affairs that the system intends to fulfill.

172

Y. Asnar and F. Massacci

Objective Level Protect patient's privacy

Hospital H Ensure accountability of the medical info

Doctor

AND

Remove senstive patient's privacy

AND

Finalize the prescription sheet

Maintain Audit Trail of Prescription

--

Privacy Leakage from Alliance Partner

...

Process Level Specify Privacy Preference

...

Print and Sign Prescription Sheet

Archive Prescription Sheet

Remove Patient Name in the Report

...

--

Resource Level Prescription

Unauthorized Prescription Signature

Old Prescription

--

Archive Service Disruption

Archive Service

Fig. 11. Unwanted Scenarios in the Drug Reimbursement System

Example 25. A new business policy of making a Hospital Alliance with competitor hospitals require the hospital “H” to give an access to its medical info that might cause a privacy leakage. Analysts starts from eliciting events that might compromise a security & GRC property of the system. We recommend analysts to start the event identification process from the resource level then move up to the process, and finally the objective level. In this manner, we prevent the spurious identification of an event’s impact. If an event disrupts a resource, then certainly it will also produce a disruption effect to the process that uses the resource, and consequently this will affect the goals that the process aim to satisfy. Conversely, in the case of the negative event at the process level (e.g., employee strike), the event cannot be mis-captured as an event originated from a resource disruption. All these is depicted in a SI* model as in Figure 11. Finally, analysts can identifying what other security & GRC properties that might also be compromised. 5.5 Risk Assessment This assessment includes risk analysis (identification and estimation) and risk evaluation [28]. Based on the unwanted scenarios identified in the previous section the

A Method for Security Governance, Risk, and Compliance (GRC)

173

analysts estimate their likelihood and severity [28]. Note that risk estimation can be either qualitative or quantitative depending on the availability of evidence for the occurrence of unwanted scenarios. The SI*-GRC method allows analysts using any risk models to estimate the risk level of the system, such as Probablistic Risk Analysis [29], Bayes Network [30], CORAS [31], Fault-Tree [32], GR-Framework [27]. In the paper [33] we explain how to interplay the goal methodology with other industrial methodologies. For practical reasons our SI* Tool, so far, only implements the automated reasoning from the GR framework. Risk evaluation compares the result of risk the estimation with the risk criteria (e.g., costs, benefits, priorities, acceptable loss) defined by the GRC council (see Section 3.1) . In case the risk level is unacceptable (i.e., it is beyond the risk tolerance that an actor can sustain), the analyst needs to identify appropriate treatments to mitigate the risk either by reducing their likelihood or their severity. Mitigation by control is explained in the next section.

6 Control Analysis Once the risk level of an event is unacceptable, analysts need to formalize a statement of the desired result/state-of-affair or purpose to be achieved by implementing controls. In SI*-GRC we call such statement as control goal. The establishment of control goals and their corresponding implementation by control processes is what ensures the security of business objectives and processes. 6.1 Control Goal Analysis During this step, the analysts must describe the control goals together with the actors responsible to manage them (i.e., the owner). These goals can be refined in a process that mirrors the construction of the target goal model in Section 4. For the refinement of business goals the process can be stopped as soon as we can identify a business process that can fulfill the goal. For control goals we should stop the process of refinement until we can fulfill the following qualities: Complete - all unwanted scenarios leading to unacceptable risks are addressed by at least one control goals ; Appropriate - there must be sufficient evidence that the achievement of a control goal will actually avoid the unwanted scenario or mitigate its effects; Precise - control goals and actors in charge of their achievement must be clearly specified, enabling unambiguous interpretation of the level of compliance or failure of a business process with regard to the control goal. These three qualities are complementary; that is, a set of control goals might be complete, but not appropriate. For example, it covers all relevant business needs, but wrong security assumptions might lead to an unacceptable risk level. The analysis might be appropriate (and determine the right effect in terms of impacts and likelihood of harmful events), but the description of the control is not precise enough to allow for the

174

Y. Asnar and F. Massacci Table 3. Establishing Control Goals

Business Goal: Prescribe therapy/drugs Compliance Requirement: Quality-Attribute: • Ensure accountability of the medical info Q1 Integrity Q2 Accountability Risk: Control Goal: • Compromised Prescription (Risk of Q1 ) • Ensure all prescriptions are correct • Unauthorized modification of the medical • Detect unauthorized modification of the medinfo (Risk of Q1 ) ical info • Unidentifiable clinical case entry in the • Trace any un-accountable entry to the medical medical info (Risk of Q1 ) info

correct implementation or the automation of the solutions, for example because it does not specify who is in charge of achieving the control goals. The control goals specified in Table 3 might be clear and easy to understand by the stakeholders. These control goals are still not precise enough to be machine implemented and monitored in terms of their effectiveness and performance. By taking advantage of repetitive patterns in control design, we can reduce the modeling effort and provide compliance experts with reusable process knowledge through a set of control patterns. Each pattern acts as a generalized description of actions that are frequently used in mitigating similar risks. See [34] for further details and examples of control patterns. At the level of control goals the following patterns are the ones most frequently found in the literature: – avoidance controls try to select an alternative path where the unwanted scenario does not materialize; Example 26. To avoid such unwanted incident in Example 11, the hospital “H” can decide to delegate the goal register patient to another actor that has necessary capabilities (e.g., the Patient Admin Division) – a prevention mechanism aims at preventing unwanted events to occur in the system (or at least reducing their likelihood); Example 27. In Example 25, such initiative might cause a breach on patients’ privacy. However, the sharing is necessary because it brings numerous business opportunities. Therefore, every sharing of medical information to another hospital must be anonymized except the data subject is the patient at the other hospital. – an attenuation mechanism tackle an unwanted events that cannot be prevented and tries to attenuate its severity. Example 28. In Example 25, analysts might decide to obtain an insurance in case the hospital “H” get a privacy-related law-suit.

A Method for Security Governance, Risk, and Compliance (GRC)

175

Table 4. Control Goal Template <modality + verb> – Hospital – should preserve – should ensure – must prevent – need to protect

– privacy – integrity – confidentiality – continuity – reliability

<artefact> – patient personal information – drug reimbursement records – the register patient process – the medical service to patients – all prescriptions

Qualitative-Attribute Reliability

Business Objective

Artefact Prescription & Prescription Archive

Prescribe Therapy/Drugs

More complete

Control Goal

Risk

Ensure all prescriptions are correct

Prevent any modifications to old prescriptions

Compromised prescriptions

Review a new prescription by a Doctor Pharmacy reviews new prescriptions

Protect the archive service from unauthorized accesses

Issue the auth token as the personnel badge

Incorrect prescription -Unauthorized prescription signature More precise

Log all activities done in the archive service

Prevent of having several active sessions for a single operator

control

Attacker obtains an access to the archive service

Attacher hijacks the active session

Unauthorized modification of prescription archive

Server admin modifies the archive entry

Operator misplaces the auth. token

More appropriate

Fig. 12. Control Goal Refinement

Moreover, one might follow the control goal templates (Table 4) in describing control goals in the context of the drug reimbursement system at the hospital “H”. A control goal describing the activity above is at a high-level of abstraction. Therefore, it needs to be further refined into specific definitions. The refinement is performed

176

Y. Asnar and F. Massacci Table 5. KAIs of Control Goals Control Goal

KAI

• Issue the authorization token as the personnel badge • Prevent of having several active sessions for a single operator • Protect the archive service from unauthorized access • Log all activities done in the archive • Pharmacy reviews new prescriptions

The number of system access done by reported missing personnel badges The number of two activities done from two different locations with in less than 30 minutes The number of suspected modification

• Review a new prescription by a doctor • Ensure all prescriptions are correct

The number of identified suspicious access The number of possible adverse drug reactions The drug values from excessive prescriptions The number of near-miss event 5 The number of reported medical error

iteratively, resulting in a “tree” of control goals, similar with the tree in the goal modeling (Section 4.2). Further down the tree, the “leaf” control goals become more precise and specific, and they are designed to provide specific risk mitigation function. This refinement of control goals can be continued in parallel to a refinement of the unwanted scenarios. Each refinement and review iteration of the models leads to an increase in precision, while the broadening of controls increase completeness as shown in Figure 12; where the starred constructs are newly identified during the refinement. Example 29. In Figure 12, we analyze the control goal ensure all prescriptions are correct so that complete, appropriate, and precise. Along the refinement of the risk model (i.e., unwanted event), analysts refine the control goal. into finer (sub) control goals. The event of unauthorized prescription signature, as in Example 24, is newly identified after the 1st iteration because it might affect one of the (sub)control goals (i.e., review a new prescription by a doctor). This situation requires analysts to enrich the control goal model by introducing a new control goal pharmacy reviews new prescriptions. More detailed risk analysis improves the appropriateness of the control and the corresponding mitigation effects. This refinement process is “sufficient” when further refinement of risks do not give new control goals or make the existing control goals more precise. In other words the analyst stops the refinement process because there is no advantage in producing a richer or more detailed risk model. At the end of the refinement process it should be possible to identify Key Assurance Indicators (KAIs for short) which assess the achievement of these objectives. These indicators should not depend on the way in which the control goals are actually implemented by a control process but only on the final outcome in terms of desired and achieved security properties. Intuitively the KAIs play the same role of Key Goal Indicators (KGIs) in COBIT [6]. Some examples from identified control goals, in Figure 12, are illustrated in Table 5.

A Method for Security Governance, Risk, and Compliance (GRC)

177

6.2 Control Process and Services Map A control process is a realization of a detailed control goal (the leaf nodes of Figure 12) and in a SOA environment can be implemented as a wrapper to the business components (as depicted in Figure 136 ) to preserve their quality attributes. Example 30. To implement the control goal prevent of having several active sessions for a single operator, the control process intercepts the message of case-details and consequently blocks the prescription process proceeding to the select drugs activity. The control process triggers an error event of multi-session where there has already an active session (via SessionChace service) for that particular operator, otherwise will release the prescription process. The error event will lead to the current existing process to be aborted and all operations are being rollback.

multi-session

Medical Info

Retrieve Patient Medical & Personal Info

Identify Patient

rollback

Specify Privacy Preferences Prescription

Case Details

Case-details

Select Drugs

Print and Sign Prescription Sheet

+

Archive Prescription Sheet

Old Prescription Prescription

Case-details

Check Active Session Receive Case-details

Prescription Archive

Review Prescription every Friday

Validate a new entry on the Medical Info

New Prescription suspected entry

multi-session

Enterprise Service Bus Legend: Business Process Control Process

Session Cache

Medical review service

Medical entry validator service Log Service

Archive Service

Fig. 13. Interwoven Control Process and Business Process 6

Grayed constructs are the parts that are introduced for implementing a control goal. Moreover, a construct with “mechanical-gear” indicates an underlying technical services supporting the execution of a process (i.e., business or control process).

178

Y. Asnar and F. Massacci Table 6. ESB-related policies – Block every result from the underlying service of Archive Prescription Sheet before storing the prescription at ArchiveService, and forward to MedicalEntryValidatorServices to be validated. MedicalEntryValidatorServices emits a release event to the ESB when the presription is not suspected entry – Release the results of ArchivePrescriptionSheetService after receiving the release event from MedicalEntryValidatorServices; and store the prescription at the ArchiveService. – Trigger the log event each when there is any request to ArchiveService and forward the event log to be saved at LogService – Trigger a suspected entry event when the validate a new entry on the medical info activity, using MedicalEntryValidatorService, suspects the prescription – Trigger a start event, every Friday, for Pharmacy to Review Prescription all prescriptions indicated as suspected entries. This activity is also checked a sample of new prescriptions.

To implement control processes one needs to explicitly specify which service events need to be controlled. So a human mapping is necessary, eventually aggregating conditions on services for achieving a control goal on the specific services on which it is required. The particular forms in which a control process is implemented might vary a lot depending on the architecture that is currently used. However, we can use also in this case a number of security patterns [35]. For a SOA scenario, a possible architecture is based on the Enterprise Service Bus (ESB) [36]. The ESB has the capability to detect when a message (e.g., request, response, notification) arrives and to perform some actions (e.g., block, delete, delay, release modify, forward, trigger). The basic principles for interweaving control and business services are the following: – If a control service is executed before the business service is invoked – (i.e., filter in/out), the ESB will block the request message to the business service and forward the request to the control service. The control service will notify the messaging service whether to remove the blocked request if it is considered to be an inappropriate request, or to release it; as in Example 30. – If a control service is executed after the business service invoked (i.e., verify), the ESB will block the result of the business service invocation before dispatching it to the subsequent service in the business process, and release the results after performing some operations (e.g., modify/add/remove some data items, attach signature) or even remove the result if it violates some policy (e.g., not sending confidential data). Table 6 shows some examples of security control policies for the ESB scenario in implementing the control goal of log all activities done in the archive and the control goal of pharmacy reviews new prescriptions.

A Method for Security Governance, Risk, and Compliance (GRC)

179

Table 7. KSIs of Control Process Control Process Name

KSI-correctness/coverage

• Abort the prescription process Cor: The ration between the occurrences of multiwhen there is already an active session event and the rollback events session Cov: Percentage of the number of check active session execution over the specify privacy preference execution • Log all requests to Cor&Cov: The difference between the number log enArchiveService tries of archiving a prescription and the number of prescription sheet • Friday review done by Pharmacy Cov: The ratio between the number of reviews done by Pharmacy on the suspected prescription entry and the occurrences of suspected entry event Cov: The ratio between the number of prescription reviews done by Pharmacy and the number of prescription sheet • Validate new prescriptions by Cor: The number of prescription which triggers susMedicalEntryValidatorpected entry event but at the end it is benign Service Cor: The number of archived prescription in ArchiveService that is suspected Cov: The ratio between the number of execution of MedicalEntryValidatorService in supporting the Validate a new entery in the medical info and the number of prescription sheet

At the end, each control process must be defined its Key Security Indicators (KSIs for short) which assess the level of correctness and coverage of such controls. By KSI-correctness, these indicators must indicate whether the result of the control is correct according to its design specification; while KSI coverage indicates the level coverage of the control in protecting the business process execution. Note KSI is closely dependent on the technical details how the control is implemented in the system. Intuitively the KSIs play the same role of Key Performance Indicators (KPIs) in COBIT [6]. Some examples from identified control presses, in Figure 13, are illustrated in Table 7. Note that often it is hard to define KSIs of the correctness of (semi-)manual control, because the correctness criteria are in the human knowledge and hardly encode-able for the knowledge of technical system. Besides implementing control processes, designers need to define the events (e.g., a service start/ finish/suspend or messages exchanged among services) that will be used to compute the KAIs and KSIs. To process these events, the business activity monitoring (BAM) [37] can be used, since it allows one to analyze real-time events from the business transaction and, furthermore, to compute KAIs/KSIs following the mathematical formula defined by the designers. BAM-related policies for computing KAI/KSIs related to the control goal of prevent of having several active session for a single operator are illustrated in Table 8.

180

Y. Asnar and F. Massacci Table 8. KAIs/KAIs Policies in BAM

•

Control Goal Prevent of having several active sessions for a single operator KAI The number of two activities done by the same operator from two different locations within less than 30 minutes BAM stores the last location and access’/activity’s timestamp from each operator taken. This information is obtained from the message request to SessionChace. It increments the value of KAI when the most recent activity is within 30 minutes form the previous one and done from different location • Control Process Abort the prescription process when there is already an active session KSIcor The ration between the occurrences of multi-session event and the rollback events Nmulti-session /Nrollback ; where multi-session denotes that Check Active Session founds an active session in SessionCache for the same operator, and rollback is triggered when the prescription process is aborted and some rollback procedures is executed KSIcor Cov: Percentage of the number of check active session execution over the specify privacy preference execution Ncheck-session /Nspecify-privacy × 100; where check-session is an event triggered when Check Active Session is executed, and specify-privacy is triggered when the operator has specified the patient’s privacy preference

7 SI*-GRC In-Vivo Once these controls are in place, we must still assess and monitor them at runtime for ensuring the level of security & compliance of the system; and improving the implemented controls. 7.1 The Check Phase: Monitor and Review Indicators For each control goal and process, analysts need to identify indicators that measure its correctness and effectiveness. For these purposes, we use key assurance indicators (KAIs) and key security indicators (KSIs) as illustrated in Table 5 and Table 7 respectively. Typically, KAIs are the focus of the business analysts, because business analysts are more concerned with the level of compliance rather than how the control is implemented. KSIs, on the other hand, are of interest to risk/security analysts as they measure how well controls are implemented. Both KAIs and KSIs are critical for monitoring, evaluating and improving the GRC implementation. The indicators are computed independently to distinguish between cases in which the KAI of a control goal is “low” but the KSI’s associated control processes are “high”. In the former case, analysts might conclude that there are some risks that have not been mitigated. In the latter, it might be that the compliance of a business process is achieved through external factors (from luck to organizational procedures), rather than deployed controls. The technological infrastructure implementing the controls and indicators identified by the method should then support Continuous Controls Monitoring techniques. For

A Method for Security Governance, Risk, and Compliance (GRC)

181

example, the tools developed in the MASTER project7 makes this possible: monitoring and assessment infrastructures provide one main source of indication and feedback for the review. The monitoring infrastructure gathers and correlates events emitted from monitored services based on monitoring policies defined by control processes. The assessment infrastructure evaluates the relevant KAI/KSI values based on the events triggered by the monitoring infrastructure. The events may be stored in an audit trail or event log database for further processing and assessment as necessary.The KAI/KSI values arethen presented to a (human) control process supervisor through a dashboard or a reporting tool. 7.2 Act Phase: React and Improve In a SOA-based business environment, orchestration of services associated to business processes could change regularly. Each change may violate existing control goals or impact the effectiveness of the relevant controls. It is, therefore, crucial to have a continuous review of the compliance level of existing business processes and the effectiveness of implemented controls. Following the result of the review different types of actions can be performed: augmentative, corrective and preventive actions, can be taken based on the review findings discussed earlier to ensure that actual control processes always reflect the latest business compliance requirements. The methodological steps are detailed below: 1. Identification of changes. Generally, augmentative actions are required when there is an update to existing: (i) legal requirements; (ii) business requirements; or (iii) control activities as the result of a review performed on the associated control processes. Based on these updates, we first identify what potential changes to existing control objectives/activities that need to be made. 2. Planning and implementation of changes. This step repeats the tasks involved in the P LAN and D O phases so that new controls can be put in place or modification to existing controls can be implemented. 3. Documentation and review. All information gathered from performing the above steps should be documented for future references.

8 Final Remarks What we have seen in this tutorial is a summary overview of SI*-GRC method to analyze and design security controls. The gist of the method is to capture the organizational setting of the system making sure that business goals and processes are the main citizen. The basic building blocks of the methods are 1. a modeling framework based on a requirement engineering framework, with some extensions related to security & GRC concerns, such as: trust, permission, risk, and treatment, 2. a analysis process defining systematical steps in analyzing and design security controls, 7

http://www.master-fp7.eu

182

Y. Asnar and F. Massacci

3. a number of analytical techniques to verify that certain security properties are satisfied and the risk level is acceptable, and at last 4. a CASE tool, namely the SI* tool to support analysts in using the method. Many details of the research methods that constitute the building blocks of this methods can be found in our research papers and in the research s of the our co-authors that we list in the acknowledgement section. Acknowledgments. We would like to thank Paolo Giorgini, John Mylopoulos, Nicola Zannone from Trento for many useful discussion on the large and unruly family of security requirements engineering. Volha Bryl, Fabiano Dalpiaz, Federica Paci, Ayda Saidane, Angelo Susi, Minh Sang Tran and many others contributed to the discussion on what really is a good security requirements methodology. Daniela Marino and Andrea Micheletti from Hospital San Raffaele, Valentino Meduri and Alessandra Tedeschi from DeepBlue and Carlo Ricucci and Domenico Presenza from Engineering proved to be an essential soundboard for the applicability of our ideas. Claire Worledge and Bernadette Louat were instrumental to link our methodology to COBIT and other ISACA methodologies. Any mistake is of course ours. This work has been partially funded by EU-FP7-ICT-IP-ANIKETOS, EU-FP7-ICTIP-SecureChange, and EU-FP7-ICT-NoE-NESSoS project.

References 1. Rasmussen, M., Kark, K., Penn, J., McClean, C., Bernhardt, S.: Trends 2007: Governance, risk and compliance: Organizations are motivated to formalize a federated GRC process. Technical report, Forrester Research (April 2007) 2. McClean, C., Whiteley, R., Kark, K., Dill, A.: The Forrester Wave: Enterprise governance, risk, and compliance platforms, Q3 2009. Technical report, Forrester Research (July 2009) 3. ACL: Audit command language, http://www.acl.com/ (last check July 15, 2010 (3020)) 4. Pfleeger, C.P., Pfleeger, S.L.: Security in Computing, 4th edn. Prentice-Hall, Englewood Cliffs (2006) 5. ISACA: The Risk IT Framework. ISACA (November 2009) 6. Institute, T.I.G.: CoBIT - Framework Control Objectives Management Guidelines Maturity Models. 4.1 edn. ISACA.org (2007) 7. Trist, E.: The evolution of Socio-Technical systems. Occasional Paper 2 (1981) 8. Asnar, Y., Lim, H.W., Massacci, F., Worledge, C.: Realizing trustworthy business services through a new GRC approach. ISACA Journal - JOnline 2 (2010) 9. Deming, W.E.: Out of the Crisis. MIT Press, Cambridge (2000) 10. Marino, D., Potral, J.J., Hall, M., Rodriguez, C.B., Rodriguez, P.S., Sobota, J., Jiri, M., Asnar, Y.: Master scenarios. Project Deliverable D1.2.1, MASTER Consortium (2009); This case study has been provided by Hospital San Raffaele Foundation, Its complete description is available at http://www.masterfp7.eu/index.php?option=com_docman& task=doc_details&gid=53&Itemid=60 11. Erl, T.: SOA Principles of Service Design. Prentice-Hall, Englewood Cliffs (2007) 12. Casteleyn, S., Daniel, F., Dolog, P., Matera, M.: Engineering Web Applications. SpringerVerlag New York Inc., New York (2009) 13. Yu, E.: Modelling Strategic Relationships for Process Engineering. PhD thesis, University of Toronto, Department of Computer Science (1995)

A Method for Security Governance, Risk, and Compliance (GRC)

183

14. OMG: Business process modeling notation (January 2009) 15. Hofstede, A.H.M., Aalst, W.M.P., Adams, M., Russell, N. (eds.): Modern Business Process Automation-YAWL and its Support Environment. Springer, Heidelberg (2010) 16. Giorgini, P., Massacci, F., Mylopoulos, J., Zannone, N.: Requirements engineering for trust management: model, methodology, and reasoning. International Journal of Information Security 5(4), 257–274 (2006) 17. Robbins, S.P.: Organizational Behavior, Concepts, Controversies, Applications, 7th edn. Prentice-Hall, Englewood Cliffs (1996) 18. OCC: Management information systems. the comptroller’s handbook, Office of the Comptroller of the Currency (May 1995) 19. Zachman, J.A.: A framework for information systems architecture. IBM Systems Journal 26(3), 276–292 (1987) 20. van der Aalst, W.M.P.: Formalization and verification of event-driven process chains. Information and Software Technology 41(10), 639–650 (1999) 21. Eriksson, H.E., Penker, M.: Business modeling with UML: Business Patterns at Work. John Wiley & Sons, Chichester (2000) 22. van der Aalst, W.M.P., Pesic, M., Schonenberg, H.: Declarative workflows: Balancing between flexibility and support. Computer Science - Research and Development (March 2009) 23. van der Aalst, W.M., Weske, M., Gr¨unbauer, D.: Case handling: a new paradigm for business process support. Data & Knowledge Engineering 53(2), 129–162 (2005) 24. Bresciani, P., Perini, A., Giorgini, P., Giunchiglia, F., Mylopoulos, J.: Tropos: An AgentOriented software development methodology. Autonomous Agents and Multi-Agent Systems 8(3), 203–236 (2004) 25. Massacci, F., Mylopoulos, J., Zannone, N.: Computer-aided support for secure tropos. Automated Software Engineering 14(3), 341–364 (2007) 26. Compagna, L., Khoury, P.E., Krausov´a, A., Massacci, F., Zannone, N.: How to integrate legal requirements into a requirements engineering methodology for the development of security and privacy patterns. Artificial Intelligence and Law 17(1), 1–30 (2008) 27. Asnar, Y., Giorgini, P., Mylopoulos, J.: Goal-driven risk assessment in requirements engineering. Requirements Engineering 16(2), 101–116 (2011), 10.1007/s00766-010-0112-x 28. ISO/IEC: Risk Management-Vocabulary-Guidelines for Use in Standards, Published: ISO/IEC Guide 73 (2002) 29. Vose, D.: Risk Analysis: A Quantitative Guide. Wiley, Chichester (2000) 30. Mosleh, A., Hilton, E.R., Browne, P.S.: Bayesian probabilistic risk analysis. SIGMETRICS Perform. Eval. Rev. 13(1), 5–12 (1985) 31. Lund, M.S., Solhaug, B., Stl?en, K.: Model-Driven Risk Analysis - The CORAS Approach. Springer, Heidelberg (2011) 32. Vesely, W., Goldberg, F., Roberts, N., Haasl, D.: Fault Tree Handbook. U.S Nuclear Regulatory Commission (1981) 33. Delande, O., Felix, E., Massacci, F., Paci, F.: Managing changes with legacy security engineering processes. In: Zeng, D., Yang, C.C., Collberg, C. (eds.) Proc. of IEEE Internat. Conf. on Intelligence and Security Informatics (ISI 2011). IEEE Press, Los Alamitos (2011) 34. Namiri, K.: Model-Driven Management of Internal Controls for Business Process Compliance. PhD thesis, Universit¨at Fridericiana zu Karlsruhe (2008) 35. Schumacher, M., Fernandez-Buglioni, E., Hybertson, D., Buschmann, F., Sommerlad, P.: Security Patterns: Integrating Security and Systems Engineering, 1st edn. Wiley, Chichester (2006) 36. Chappell, D.: Enterprise Service Bus. O’Reilly Media, USA (2004) 37. Kochar, H.: Business Activity Monitoring and Business Intelligence, (December 2005), http://www.ebizq.net/topics/bam/features/6596.html (last access at July 03, 2010)

184

Y. Asnar and F. Massacci

Appendix A - SI* Syntax SI* Diagram Actor

DLV Predicate

Role

Agent

actor(Actor) role(Role) agent(Agent) Goal

goal(Goal) Task

Process +

process(Process) task(Task)

Resource

Service Message

Data Store

resource (Resource)

Unwanted Event

event(UnwatedEvent)

Goal

and Subgoal 1

Subgoal 2

and decomposition(Goal, SubGoal1, SubGoal2) A

De

Goal

De

B

del exec(A,B,Goal) A

Te

Goal

Te

B

trust exec(A,B,Goal) A

Dp

Dp

Process

B

+

del perm(A,B,Process) A

Tp

Tp

Resource

B

trust perm(A,B,Resource)

Resource

R

Goal

`P

O

Actor

Process +

request(Actor,Goal) owner(A,Resource)

provide(Actor,Process)

Note: See BPMN [14] for complete constructs for the Process Modeling.

The Geometry of Lattice Cryptography Daniele Micciancio UC San Diego [email protected]

Abstract. Lattice cryptography is one of the hottest and fastest moving areas in mathematical cryptography today. Interest in lattice cryptography is due to several concurring factors. On the theoretical side, lattice cryptography is supported by strong worst-case/average-case security guarantees. On the practical side, lattice cryptography has been shown to be very versatile, leading to an unprecedented variety of applications, from simple (and eﬃcient) hash functions, to complex and powerful public key cryptographic primitives, culminating with the celebrated recent development of fully homomorphic encryption. Still, one important feature of lattice cryptography is simplicity: most cryptographic operations can be implemented using basic arithmetic on small numbers, and many cryptographic constructions hide an intuitive and appealing geometric interpretation in terms of point lattices. So, unlike other areas of mathematical cryptology even a novice can acquire, with modest eﬀort, a good understanding of not only the potential applications, but also the underlying mathematics of lattice cryptography. In these notes, we give an introduction to the mathematical theory of lattices, describe the main tools and techniques used in lattice cryptography, and present an overview of the wide range of cryptographic applications. This material should be accessible to anybody with a minimal background in linear algebra and some familiarity with the computational framework of modern cryptography, but no prior knowledge about point lattices.

1

Introduction

Lattice cryptography is one of the most fascinating areas of mathematical cryptography. Supported by a solid theoretical foundation, lattice cryptography can also be very attractive in practice, as an alternative to more traditional solutions based on number theory. One of the features that makes lattice cryptography potentially attractive in practice is its simplicity: the most fundamental operation of lattice cryptography is just a modular integer matrix-vector multiplication Ax (mod q). Typically, q is a small integer, which comfortably ﬁts in a machine level register, so that all the basic operations can be eﬃciently performed (and easily implemented) without the need of a “big-num” library for arbitrary precision arithmetic. While many cryptographic functions based on lattices can be easily described using matrix notation (and this is how they are typically described

Supported in part by the National Science Foundation under Grant CNS-0831536.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 185–210, 2011. c Springer-Verlag Berlin Heidelberg 2011

186

D. Micciancio

in most research papers), the operations they perform and the computational problems they hide are fundamentally geometric. The matrix formulation may be all that is needed to specify and implement lattice cryptography. But in order to truly understand it and develop an intuition of what makes lattice cryptography secure, one needs to look under the hood, and explore the geometric structure of the underlying lattices. As we will see, while quite diﬀerent from the mathematics used in more conventional cryptography, the mathematics of lattices (or at least most of the mathematics of lattices used in cryptography) is not that hard to understand. In fact, once you learn the most basic concepts, it is pretty simple! After all, geometry is one of the most intuitive areas of mathematics, well grounded in our everyday experiences in the physical world, and many of the concepts we will introduce can be easily illustrated by pictures.1 If you can draw a picture that captures the intuition of why factoring the product of two large prime numbers is a computationally hard problem, I would like to see your picture! So, let us leave our fears behind, and move on with lattices. In these notes, we will ﬁrst learn about lattices, their geometric structure, and the computational problems they deﬁne. Then we will look at how these simple geometric object can be used for the construction of cryptographic functions. Beside the potential for eﬃcient implementation, lattice cryptography is very attractive because of its versatility: using lattices, in the last few years, researchers were able to develop solutions for an incredibly rich set of security problems, from simple (but eﬃcient) hash functions [57,68,48,51], to hierarchical identity based public key encryption [34,22,1,2], and much more. As of this writing, the latest and greatest discovery in lattice cryptography is the development of fully homomorphic encryption, pioneered by Gentry in [28], and still a very fast moving research target [82,29,65,79,80,32,25,19,31,30]. In these notes, you will not learn about the most complex applications of lattices, including fully homomorphic encryption, but you will learn enough about lattices and lattice cryptography to proceed on your own and read research papers in the area. Beside an introduction to the fundamentals of lattice cryptography, this paper includes a extensive bibliography, that can be used as a source of pointers for further study. Notational conventions. We use Z for the set of integers, and R for the set of real numbers. Elements of these sets are denoted by lowercase letters. Sets are denoted with uppercase letters. We use bold lower case letters x for vectors and bold upper case letters A for matrices. All operations and functions are extended to sets in the usual way, e.g., for any number x ∈ R and sets A, B ⊆ R, we have x · A = {x · a | a ∈ A} and A + B = {a + b | a ∈ A, b ∈ B}. We use standard asymptotic notation O(·), o(·), ω(·), Ω(·) to describe the order of 1

To be clear, all pictures you will ever see (e.g., in the slides that accompany these lecture notes, available from the author’s web page at http://cseweb.ucsd.edu/~daniele) describe 2-dimensional, or in some rare cases 3-dimensional lattices, while for lattice cryptography to be secure one needs to work with lattices in high dimensional space. Still, 2-dimensional lattices often provide good examples to illustrate the geometry of the problems.

The Geometry of Lattice Cryptography

187

growth of functions. Namely, f = O(g) or g = Ω(f ) means that f (n) ≤ c · g(n) for some c and all suﬃciently large n, and f = ω(g) or g = o(f ) means that limn→∞ g(n)/f (n) = 0.

2

Point Lattices: A Primer

Lattices are regular arrangements of points in n-dimensional Euclidean space. The simplest example of a lattice is Zn , the set of all n-dimensional vectors with integer entries. In general, a lattice L ⊂ Rn is a set of points L = BZk = {Bx: x ∈ Zk } ⊂ Rn obtained by applying a non-singular linear transformation B ∈ Rn×k to the integer lattice Zk . (See Figure 1 for a 2-dimensional example.)

b1

b2

Fig. 1. A 2-dimensional lattice B · Z2 generated by the basis B = [b1 , b2 ]. The shaded area is the parallelepiped P(B) = B · [0, 1)2 spanned by the basis vectors. The determinant of the lattice is the volume (area) of the parallelepiped.

The matrix B ∈ Rn×k is called a lattice basis, and non-singular means that its k columns should be linearly independent. For simplicity, in these notes, we focus on full dimensional lattices, i.e., lattices where n = k, and the lattice BZn is generated by a square matrix B ∈ Rn×n with nonzero determinant. n The lattice n BZ generated by a basis B can be equivalently written as the set L = { i=1 bi · xi : ∀i.xi ∈ Z} of all integer linear combinations of the basis vectors B = [b1 , . . . , bn ]. Notice the diﬀerence between a lattice and a vector space spanned by a basis B: the vector space spanned by B is the set of all real linear combinations i bi · xi (with xi ∈ R) of the basis vectors, while in the case of lattices we only take combinations with integer coeﬃcients. A direct consequence of the deﬁnition (very easy to visualize, and only slightly harder to prove) is that lattices are discrete sets: the points in a lattice cannot

188

D. Micciancio

be arbitrarily close to each other, i.e., for every lattice L there is an > 0 such that the distance between any two distinct lattice points is at least . Due to this discrete structure, problems on lattices cannot generally be solved by simple linear algebra. For example, a basic fact from linear algebra is that any vector space has an orthogonal basis, and orthogonal bases are useful to solve a host of other problems. This is not true for lattices: not every lattice admits an orthogonal basis (e.g., see the lattice in Figure 1,) and this is what makes many lattice problems computationally hard to solve, and useful for cryptography. The discrete structure of lattices yields also a more abstract (basis independent) characterization of lattices: a lattice is a discrete additive subgroup of Rn .2 We recall that a subgroup is a set which is closed under addition and subtraction.3 Not every subgroup of Rn is a lattice, i.e., Rn itself is not a lattice because it is not discrete. In computer science and cryptography, it is common to focus on integer lattices, i.e., lattices L ⊆ Zn whose vectors have all integer coordinates. Any set of integer vectors is certainly discrete, so integer lattices can be characterized simply as additive subgroups Zn . 2.1

Fundamental Parameters

As n-dimensional objects, lattices can be hard to visualize, but there are several geometric quantities naturally associated to any lattice that succinctly describe some of its most important properties. These include – The determinant of the lattice, which is the volume of the n-dimensional parallelepiped P(B) = B[0, 1)n spanned by the basis vectors det(BZn ) = vol(P(B)). (See Figure 1.) Intuitively, the determinant of a lattice describes the (inverse) density of lattice points in space, i.e., the number of lattice points in a suﬃciently large and regular region of space S ⊆ Rn , is approximately equal to |S ∩ L| ≈ vol(S)/ det(L). To see this, notice that the space Rn can be tiled with copies of the fundamental parallelepiped P(B), with one copy x+P(B) for every lattice point x ∈ L = BZn . So, there is precisely one lattice point in every region of space x+P(B) of volume det(L) = vol(P(B)). For full dimensional lattices, the lattice determinant simply equals the absolute value of the matrix determinant det(BZn ) = | det(B)|. In general, when B ∈ Rn×k for √ k ≤ n, the determinant can be computed using the formula det(BZk ) = BT B. It is sometime convenient to consider the root determinant of a lattice, which is just the nth root of the determinant det(BZn )1/n . – The minimum distance of a lattice, i.e., the smallest distance between any two lattice points λ(L) = inf{ x − y | x, y ∈ L, x = y}. The minimum distance is always attained by some pair of lattice points, so the infimum in the previous formula is in fact a minimum. 2

3

The fact that any such discrete subgroup admits a basis, and therefore is a lattice, is not at all obvious. But for the lattices considered in this survey, the proof is relatively easy and it is left to the reader as an exercise. Notice that requiring closure under subtraction is enough because x + y = x − ((y − y) − y), and closure under addition is implied.

The Geometry of Lattice Cryptography

189

– More generally, one can deﬁne the successive minima of a lattice λ1 , . . . , λn , where for every i = 1, . . . , n, λi (L) is the smallest positive real r such that the ball B(0, r) = {x: x ≤ r} of radius r centered at the origin contains at least i linearly independent vectors. As a special case, the ﬁrst minimum λ1 (L) is the length of the shortest nonzero lattice vector and equals the minimum distance λ of the lattice. – The covering radius of a lattice ρ(L) is the smallest real number r such that spheres of radius r centered around all lattice points cover the entire space (spanned by the lattice). a useful way to visualize the minimum distance of a lattice is in terms of sphere packings. Think of putting equal spheres (of radius r) around every point in a lattice. When the radius r is very small, the spheres are disjoint. Now make the radius r as large as possible, but still subject to the constraint that the spheres centered around lattice points do not intersect. The maximum value of r is called the packing radius of the lattice, and it clearly equals half the minimum distance λ(L)/2. If we keep increasing the radius of the spheres beyond the packing radius λ/2, the spheres will start intersecting with each other, but they may not cover the entire space, leaving holes to be ﬁlled. But at some point, when r reaches the covering radius of the lattice ρ, all holes will be ﬁlled and the spheres will cover the entire space. All these quantities are lattice invariant, i.e., they depend only on the set of lattice points BZn , and not on the speciﬁc basis B used to represent them. All these quantities are also invariant under rotations, i.e., for any orthogonal matrix Q ∈ Rn we have λ(L) = λ(QL), and similarly for the successive minima, covering radius, determinant and root determinant. All these quantities, with the exception of the determinant, are also linear in the sense that λ(c · L) = |c| · λ(L) for any real scaling factor c, and similarly for λi , ρ and the root determinant. Taken together, all these parameters provide a lot of useful information about the geometry of the lattice. The diligent reader can reﬁne his geometric understanding of lattices by proving the following simple statements. Exercise 1. Show that for any lattice L and target point t (in the linear span of the lattice) there is at most one lattice point within distance (strictly less than) λ(L)/2 from t. Exercise 2. Show that for any lattice L and target point t (in the linear span of the lattice) there is at least one lattice point within distance ρ(L) from t. Exercise 3. Show that for any n-dimensional lattice, the parameters λ1 , . . . , λn , ρ are related by the chain of inequalities4 √ λ1 ≤ λ2 ≤ · · · ≤ λn ≤ 2ρ ≤ n · λn . Far less trivial to prove (but still not too hard, e.g., see [60] for a simple proof) is the following upper bound on the successive minima in terms of the (root)determinant of the lattice. 4

The last two inequalities are not quite trivial. As a starter, the reader may want ﬁrst to prove the weaker inequalities λn ≤ 2nρ ≤ n2 λn .

190

D. Micciancio

Theorem 1 (Minkowski). For any lattice Λ, λ(L) ≤

n

1/n λi (L)

≤

√

γn · det(L)1/n

i=1

where γn ≤ n is a function of the dimension only, and does not depend on the specific lattice. For every n, the optimal value of γn in Minkowski’s theorem is called Hermite’s constant in dimension n. The exact value of γn is known only for n = 1, 2, . . . , 8 and n = 24, but the asymptotic estimate γn = Θ(n) is all that is needed to understand lattice cryptography. Minkowski’s bound provides useful information about the minimum distance of a lattice and it is pretty close to optimal when the lattice is chosen at random according to many natural probability distributions on lattices. (At this point the reader may want to try to come √ up with an explicit construction of an n-dimensional lattice with λ(L) = Ω( n det(L)1/n ). It is not an easy task, and as we will see these are good examples of “cryptographically hard” lattices. See Exercise 13.) On the other hand, the actual minimum distance λ (or the mean value ( i λi )1/n ) can be arbitrarily smaller than Minkowski’s upper bound. Exercise 4. Build a lattice basis B such that λ(L(B)) is smaller than Minkowski’s upper bound by at least a factor c > 1. [Hint: it is enough to consider lattices in dimension n = 2.] On the other √ hand, for the mean value ( i λi )1/n , Minkowski’s bound is loose by at most a n factor. Exercise 5. Show that for any n-dimensional lattice L, ( i λi )1/n ≥ det(L)1/n . [Hint: find a set of linearly independent vectors of length λ1 , . . . , λn , and bound the volume of the parallelepiped they span using Hadamard’s inequality.] Another very useful relation between the fundamental lattice parameters λ and ρ is given by the so-called transference theorems, but in order to present them we need ﬁrst to deﬁne the dual lattice. This is probably one of the more technical and less intuitive concepts used in the study of lattice cryptography, and it takes some time and eﬀort to start developing some geometric intuition about it. But the notion of dual lattice turns out to be so useful that the eﬀort is certainly justiﬁed. Let’s put geometry aside for the moment. The reader is probably familiar with the deﬁnition of the dual of a vector space: the dual of a vector space V ⊆ Rn is the set V † of all linear functions φ from V to R. These functions can be naturally represented as vectors x ∈ V , where φx (v) = v · x, and using this representation V † is isomorphic to the original vector space V . The deﬁnition of dual lattice is analogous, but this time it is natural to consider linear functions φ: L ⊆ Rn → Z that take integer values when evaluated on lattice points. As before, φ can be represented by a vector x in the linear span of L such that φ(v) = v · x. The representation of linear functions as vectors leads to the following deﬁnition: the dual lattice L† is the set of all vectors x (in the linear span of L) such that

The Geometry of Lattice Cryptography

191

v · x ∈ Z for every lattice point v ∈ L. As you may expect, the dual lattice L† is a lattice, and the dual of the dual is the original lattice (L† )† = L. An initial understanding of the geometry of the dual lattice is provided by the following simple observations. Exercise 6. If B ∈ Rn×n is a lattice basis, then a basis of the dual lattice (BZn )† is given by the inverse transpose matrix B−T , i.e., (BZn )† = B−T Zn . More generally, the dual basis of B ∈ Rn×k , is given by B† = B(BT B)−1 . It easily follows that for any lattice L and scaling factor c > 0, the dual lattice of c · L is (c · L)† = (1/c) · L† , i.e., expanding a lattice by a factor c shrinks the dual lattice by the same factor. Also, we have that det(L† ) = 1/ det(L). So, in an informal sense, the dual lattice is the “inverse” of the original lattice. The transference theorems show that the fundamental parameters λ and ρ are also connected by an inverse relation through the dual lattice, although only an approximate one. Theorem 2. For any n-dimensional lattice L, we have 1 ≤ λi (L)·λn−i+1 (L† ) ≤ n and 1 ≤ 2ρ(L) · λ(L† ) ≤ n. So, in an approximate sense (up to a factor n), λ1 (L† ), . . . , λn (L† ) are the inverses of λn (L), . . . , λ1 (L), and λ(L† ) is the inverse of 2ρ(L). This reinforces the geometric intuition behind the informal statement that the dual lattice L† is the inverse of L. To get a better geometric grasp of this statement, consider the following illustrative example. Let L is a lattice such that L† contains a very short vector x of length x = λ† . By deﬁnition, all lattice points in L have integer scalar product with x. So, x can be used to partition L into layers Li = {v ∈ L | x · v = i}. Clearly, the covering radius ρ(L) must be at least as large as half the distance between these layers, but the distance between the layers is precisely 1/ x = 1/λ† . Therefore 1/(2λ† ) ≥ ρ, yielding the lower bound 2λ† · ρ ≥ 1. The upper bound 2ρ · λ† ≤ n is far less trivial to prove, and requires the use of harmonic analysis techniques [11]. 2.2

Computational Problems

We have seen how the determinant of a lattice B can be eﬃciently computed, e.g., by the formula det(L) = det(BT · B). However, the same is not true for the minimum distance λ(L), any of the successive minima λi or the covering radius ρ. In fact, all these parameters give raise to well known hard computational problems on lattices. The best polynomial time approximation algorithms to solve any of the problems described below only achieve approximation factors γ(n) = 2O(n log log n/ log n) almost exponential in the dimension of the lattice. Lattice cryptography is based on the assumption that no eﬃcient algorithm can achieve polynomial approximation factors γ(n) = nO(1) , at least in the worstcase, i.e., no eﬃcient algorithm can achieve γ(n) = nO(1) approximation factor for every possible input lattice. Worst-case hardness assumptions are supported by many NP-hardness results for lattice problems [4,21,15,10,55,26,27,42,43,38],

192

D. Micciancio

although these results only hold for much smaller (sub-polynomial) approximation factors. The main problem underlying lattice cryptography is called the Shortest Independent Vectors Problem, and it is the natural computational problem associated to the parameter λn . Cryptography is based on the hardness of solving this problem even approximately, as deﬁned below. Definition 1. The Shortest Independent Vector Problem (SIVPc ) is the following: given an n-dimensional lattice L (usually represented by a basis B), find n linearly independent lattice vectors b1 , . . . , bn ∈ L such that maxi bi ≤ c · λn (L). In the above deﬁnition, c ≥ 1 is the approximation factor, and it is usually a function of the dimension n. The case c = 1 corresponds to solving the problem exactly, i.e., ﬁnding n linearly independent lattice vectors such that the quantity maxi bi is as small as possible. We remark that n linearly independent lattice vectors are not necessarily a basis for the given lattice. For example, the column vectors 2B are linearly independent vectors in BZn , but they are not a basis for BZn . This is because they only generate the sublattice 2 · L rather than the entire lattice L = BZn . It is usually surprising to the novice to learn that there are lattices L such that any basis must necessarily contain vectors much longer than λn . In particular, there are lattices such that any solution to the SIVP problem will not be a basis for the lattice. Exercise 7. Let L ⊂ Zn be the set of all integer vectors such that in each vector all coordinates have the same parity. (I.e., the coordinates are either all even, or all odd integers.) Assume n ≥ 5. Prove that λ1 = λ2 = · · · = λn = 2. Show that L = BZn has no basis such that maxi bi ≤ λn . If you think this is surprising and counter intuitive, it is not for no reason. In dimension (up to) n = 3, a set of linearly independent vectors of length

bi = λi (for i = 1, 2, 3) is always a lattice basis. Only in dimension 4, they may or may not generate the entire lattice, and as the previous exercise shows, starting in dimension 5, minimizing the length of the vectors may necessarily yield only a basis for a sublattice. So, in order to “see” all this, you need to “look” beyond our 3-dimensional physical world. Fortunately, in most applications (and in particular, in cryptography), a set of short linearly independent vectors is just as good as a basis, and the reader can disregard the subtle distinction between a short basis, and a set of short linearly independent vectors. Moreover, if a basis is really needed, one can always convert a set of linearly independent vectors into a basis of slightly longer vectors. Exercise 8. Show that for any set of n linearly independent lattice √ vectors v1 , . . . , vn ∈ L there is a basis for L = BZn such that maxi bi ≤ n maxi vi . Moreover, such B can be eﬃciently computed from v1 , . . . , vn and an arbitrary basis for L. Another fundamental computational problem on point lattices is the Closest Vector Problem, which may be formulated as follows.

The Geometry of Lattice Cryptography

193

Definition 2. Given a lattice L ⊂ Rn , a target point t ∈ Rn , and a distance bound d, the Closest Vector Problem (CVP) asks for a lattice point v ∈ L at distance t − v ≤ d from the target, provided such a lattice point exists. In the exact version of CVP, the distance bound is taken to be the distance d = μ(t, L) = minv∈L t − v between the target and the lattice. In the approximate problem CVPγ one sets d = γ · μ. Two special versions of CVP that play a prominent role in cryptography are: 1. the Bounded Distance Decoding problem (BDD), where d < λ/2, and 2. the Absolute Distance Decoding problem (ADD), where d ≥ ρ. The importance of these speciﬁc parameter settings is that when d < λ/2, if a solution exists, then it is unique. (See Exercise 1.) On the other hand, when d ≥ ρ, a solution is always guaranteed to exist (for any target t), and it is generally not unique. (See Exercise 2.) Easier variants of BDD and ADD are obtained by introducing a slackness factor γ ≥ 1, and strengthening the constrains on d to d < λ/(2γ) (for BDDγ ) and d ≥ γρ (for ADDγ ). Informally, BDDγ is the problem of ﬁnding the lattice vector closest to a target when the target is very close to the lattice, while ADDγ is the problem of ﬁnding a lattice vector not too far from the target, where far is measured with respect to the absolute bound ρ within which a solution is always guaranteed to exist. As usual, the approximation factor γ can be a function of the dimension. The problems SIVP, ADD and BDD are connected by some very interesting relations, which also play an important role in cryptography. To start with, observe that if the lattice is L = Zn , then CVP can be solved exactly and eﬃciently by rounding each coordinate of the target √ t to the closest integer, and this always results in a lattice vector within distance n/2 from the target. More generally, for an arbitrary lattice L, we can attempt to solve ADD or BDD as follows. Let B be a lattice basis and think of L = BZn as the integer lattice Zn distorted by the linear transformation deﬁned by B. In order to ﬁnd a lattice point close to a target t we may 1. ﬁrst apply the inverse transformation B−1 to get B−1 t, 2. round B−1 t to the closest integer vector B−1 t ∈ Zn , 3. map the the resulting integer vector to the lattice point v = BB−1 t. The quality of this rounding procedure can be analyzed in terms of the two quantities smin (B) = minn Bx / x

x∈R

smax (B) = maxn Bx / x

x∈R

which express by how much the transformation B can shrink or expand the length of a vector. Exercise 9. Show that√the above rounding procedure always returns a lattice point within distance n · smax (B)/2 from t. Moreover, if t is within distance

194

D. Micciancio

smin (B)/2 from the lattice, then the rounding procedure ﬁnds the (necessarily unique) lattice point within distance smin (B)/2 from t. So, in order to solve ADD (resp. BDD) we need to ﬁnd a basis B such that smax (B) is small (resp. smin (B) is large). This can be done by solving SIVPγ , either in the lattice L or in its dual L† . Let’s look Using an SIVPγ √ ﬁrst at ADD. √ oracle we can ﬁnd a basis B with maxi bi ≤ √nγλn ≤ 2 nγρ. (See Exercises 3 and 8.) It is also easy to check that smax (B) ≤ n maxi bi ≤ 2γnρ. This yields a solution to ADDn1.5 γ . In fact, by using a set of linearly independent vectors rather than a basis, in conjunction with a better rounding procedure based on orthogonalized projections, this can be improved to ADDγ √n . Theorem 3. For any γ ≥ 1, there is a polynomial time (dimension preserving) reduction from ADDγ √n to SIVPγ . Let’s now turn to BDD. This time we want smin (B) to be large. Since smin (B) = 1/smax(B† ), this can be achieved by computing a short dual basis B† using SIVPγ . As before, the basis satisﬁes smax (B† ) ≤ 2γnρ(L†), and therefore by the transference theorem smin (B) ≥ 1/(2γnρ† ) ≥ λ/(γn2 ). This yields a solution to BDDγn2 . As before, this can be improved to BDDγn using a better rounding method, and further improvements are probably possible. Theorem 4. For any γ ≥ 1, there is a polynomial time (dimension preserving) reduction from BDDγn to SIVPγ . In summary, Theorems 3 and 4 show that SIVP is the hardest of the three problems, in the sense that both ADD and BDD can be eﬃciently solved given an SIVP oracle, at least approximately, up to polynomial approximation factors. In fact, it is not hard to show that SIVPγ √n reduces to ADDγ . (This is implicit in the proof of the last two inequalities in Exercise 3.) So, ADD and SIVP are equivalent problems, up to small polynomial factors in the quality of the solution. Interestingly, BDD seems an easier problem than ADD and SIVP: BDD reduces to ADD and SIVP, but in the opposite direction reductions are known only under quantum algorithms! Giving a (classical, possibly randomized, but not quantum) reduction from ADD or SIVP to BDD is an important open problem in the complexity of lattice problems, and a problem that, as we will see later, has special cryptographic signiﬁcance. We conclude this section with a brief mention of the standard CVPγ and the Shortest Vector Problem SVPγ , which is the problem of ﬁnding a nonzero lattice vector of length at most cλ1 . These are the two most famous computational problems in the algorithmic study of point lattices. They are also the hardest: they are equivalent to each other (up to polynomial approximation factors), and SIVP reduces to them. However, no reduction (even quantum) is known in the opposite direction. In particular, it is not known how to build cryptographic function based on the conjectured (worst-case) intractability of SVP or CVP. Therefore, another very important open problem in the complexity of lattice problems is to give a reduction from SVP or CVP to SIVP.

The Geometry of Lattice Cryptography

3

195

Random Lattices and Lattice Cryptography

Lattice cryptography is based on the conjectured hardness of SIVP, i.e., the problem of computing (a set of n linearly independent) short vectors in a lattice. Intuitively, one can think of building a cryptosystem where public keys are lattices (e.g., represented by a basis consisting of long vectors), with the corresponding secret key given by the short lattice vectors. If ﬁnding short vectors is hard, then the secret key cannot be easily computed from the public description of the lattice. Moreover, the public/secret key pair can be used to encrypt/decrypt messages as follows.5 – Let L = BZn be a lattice and S a set of linearly independent lattice vectors, serving respectively as the public and secret key of the cryptosystem. – A message m is encrypted by encoding it as a short random vector x, selecting a “random” point v ∈ B† Zn in the dual lattice, and perturbing v by x, to yield ciphertext c = v + x. Notice that the ciphertext is a point within distance x from the dual lattice L† . The reason we use a dual lattice vector here is that a good basis S of L allows to solve BDD in L† . – If x < λ(L† )/2, then recovering x gives raise to a BDD problem. As seen in Theorem 4, the secret key S can be used to ﬁnd the lattice point v closest to the target ciphertext c, provided the error vector x is short enough. The message is then recovered as x = c − v. In summary, recovering the secret key S from the public description of the lattice B corresponds to a SIVP instance, while decrypting without the knowledge of the secret key is an instance of BDD. Of course, this is very informal, and leaves many important questions unanswered. What basis B should be used as the public key? How should the “random” lattice point v ∈ L† be selected? What probability distribution should be used/assumed on the error vector x? In what sense is the above scheme secure? For many of these questions, provably optimal answers are known [54]: both integer lattices L† ⊆ Zn and their cosets x + L† admit easily computable normal forms, or standard representatives.6 Since these standard representatives can be easily computed from any other representation of the lattice and its cosets, they provide the minimum possible amount of information to the adversary, and therefore the highest level of security. But what distribution x should be used to encode the messages, and, more importantly, 5

6

Here we treat encryption somehow informally, like textbook RSA, where the message is assumed to be somehow random, and we identify cryptosystems with the theoretical notion of trapdoor function families. The reader familiar with modern cryptography will certainly know that for cryptosystems to provide a reasonable level of security, they need to be randomized. The example here is only meant to provide the intuitive geometric intuition of how lattice cryptography works, without getting into technical details. For example, given a lattice basis B, a standard representative of the coset x + BZn is given by the unique point of x + BZn inside the parallelepiped P(B). For the basis B one may use the Hermite Normal Form [54].

196

D. Micciancio

what distribution should be used for the lattices L, L† to ensure that SIVP (in L) and BDD (in L† ) are computationally hard? The conjectured (worst-case) intractability of SIVP, ADD and BDD is not directly relevant here. In cryptography one needs computational problems that are hard to solve on the average: when you pick the secret key of a cryptographic function at random (according to an appropriate probability distribution), it is not enough to know that some key is computationally hard to break. You want to be conﬁdent that your randomly chosen key is hard to break with high probability. So, in order to formalize lattice cryptography, we need to introduce a notion of random lattice, so that when L is chosen according to this random distribution, SIVP on L is hard. 3.1

Random Lattices

The two most common distributions over n-dimensional lattices encountered in cryptography are deﬁned as follows. Fix positive integers k ≤ n ≤ q, where k serves as the main security parameter. Typically n is a small multiple of k (e.g., n = O(k) or n = O(k log k)) and q is a small prime with O(log k) bits. Notice that q is very small, not at all like the large primes (with O(k) bits) used in deﬁne number theoretic cryptography. For any matrix A ∈ Zk×n q n Λ⊥ q (A) = {x ∈ Z : Ax = 0 mod q}

Λq (A) = {x ∈ Zn : x = AT s mod q for some s ∈ Zkq }. Intuitively, Λq (A) is the lattice generated by the rows of A modulo q, while Λ⊥ q (A) is the set of solutions of the system of k linear equations modulo q deﬁned by the rows of A. It is easy to check that Λq (A) and Λ⊥ q (A) are subgroups of Zn , and therefore they are lattices. It is also apparent from the deﬁnition that these lattices are q-ary, i.e., they are periodic modulo q: one can take the ﬁnite set Q of lattice points with coordinates in {0, . . . , q − 1}, and recover the whole lattice by tiling the space with copies Q + qZn . The matrix A used to represent them is not a lattice basis. A lattice basis B for the corresponding lattices can be eﬃciently computed from A using linear algebra, but it is typically not needed: cryptographic operations are usually expressed and implemented directly in terms of A. A random lattice is obtained by picking A ∈ Zk×n q uniformly at random. The corresponding distributions are denoted Λq (n, k) and Λ⊥ q (n, k). Regarding the error vector distribution, one possible way to choose x may be to select it uniformly at random among all integer vectors of bounded norm, but for technical reasons a diﬀerent distribution is often more convenient. In lattice cryptography, perturbation vectors are typically chosen according to the Gaussian distribution Dα which picks each x ∈ Zn with probability (roughly) proportional to exp(−π x/α 2 ). The Gaussian distribution has the analytical advantage that the probability of a point x depends only on its norm x , and still the coordinates of x can be chosen independently (namely, each with probability proportional to exp(−π|xi /α|2 )). It can be shown that when x is chosen

The Geometry of Lattice Cryptography

197

√ according to this distribution (over Zn ), Pr{ x > nα} is exponentially small. So, by truncating a negligibly small tail, Dα can be regarded √ as a probability distribution over the integer vectors of norm bounded by α n. Before providing a theoretical justiﬁcation for using these distributions in cryptography, let us try to get a better understanding of the lattices. We begin by observing that Λq (A) and Λ⊥ q (A) are dual to each other, up to a scaling factor q. † ⊥ † Exercise 10. Show that Λq (A) = q · Λ⊥ q (A) and Λq (A) = q · Λq (A) . In par⊥ n k×n ticular, det(Λq (A)) · det(Λq (A)) = q . Moreover, for any A ∈ Zq , we have k n−k det(Λ⊥ . q (A)) ≤ q and det(Λq (A)) ≥ q

To better understand the relation between Λq (n, k) and Λ⊥ q (n, k), it is convenient ⊥ ˜ to deﬁne two auxiliary distributions. Let Λq (n, k) be the conditional distribution of a lattice chosen according to distributions Λ⊥ q (n, k), given that the lattice has determinant exactly q k . Similarly, let Λ˜q (n, k) be the conditional distribution of a lattice chosen according to Λq (n, n − k), given that the determinant of the lattice is q n−k . In both cases, when q is a prime, the condition is equivalent to requiring that the rows of A are linearly independent modulo q. How much do these conditional distributions diﬀer from the original ones? Not much. Exercise 11. Show that if A ∈ Zk×n is chosen uniformly at random, then q k n−k Pr{det(Λ⊥ } ≥ 1 − 1/q n−k . q (A)) = q } = Pr{det(Λq (A)) = q

˜ Moreover, the conditional distributions Λ˜⊥ q (n, k) = Λq (n, n − k) are identical. So, for typical settings of the parameters (e.g., n = 2k), lattices chosen according k to Λ⊥ q (n, k) or Λq (n, n−k) have determinant q except with negligible probability ≤ q −k , and the distributions Λ⊥ q (n, k) and Λq (n, n − k) are almost identical ˜ because they are both statistically close to Λ˜⊥ q (n, k) = Λq (n, n − k). We now move to estimating the parameters of lattices chosen √ according to these distributions. Clearly, we always have λ1 ≤ λn ≤ q and ρ ≤ nq because qI gives a set of n linearly independent vectors of length q. In fact, from Theorem 1 √ and Exercise 10, we know that λ(L) ≤ nq k/n for any L ∈ Λ⊥ q (n, k). This upper bound is essentially tight. Exercise 12. There is a constant that if L is chosen according to √ k/nδ > 0 such Λ⊥ } ≤ 1/2n. [Hint: consider all integer vectors q (n, k), then Pr{λ(L) < δ nq √ of norm at most δ nq k/n and use a union bound.] What about the other parameters λn , ρ? Also these parameters are very close to Minkowski’s upper bound with high probability. Exercise 13. If L is chosen according to Λ⊥ q (n, k), then Pr{λ(L) ≤ 2ρ(L) < 1 √ k/n n · n · q } ≤ 1/2 .[Hint: Prove the bound for Λq (n, n − k) ≈ Λ⊥ q (n, k) instead, δ and use duality and the transference theorems.]

198

D. Micciancio

˜⊥ In summary, when a lattice is chosen according to Λ⊥ q (n, k) ≈ Λq (n, k) = Λ˜q (n, n − k) ≈ Λq (n, n − k), all the parameters λ1 , . . . , λn , ρ are within a con√ stant factor from Minkowski’s bound nq k/n with overwhelming probability. This provides very useful information about what it means to solve ADD or BDD on these random lattices. Let L be a lattice chosen according to Λ⊥ q (n, k), and let t = v + x be a lattice point v ∈ L perturbed by a noise vector x cho√ sen according to distribution Dcqk/n over B(c nq k/n ). Finding a lattice point √ within distance c nq k/n from t is an ADD problem when c > δ and it is a BDD problem when c < 1/δ. 3.2

One Way Functions

We now show that solving ADD and BDD on random lattices can be formulated as the problem of inverting the function fA (x) = Ax

(mod q).

(1)

when the matrix A ∈ Zk×n is chosen uniformly at random and the input is q chosen according to distribution Dcqk/n , for some c > 0. Of course, given a matrix A and a value b = fA (x), recovering a possible preimage of y under fA is just a matter of performing some linear algebra, and it can be eﬃciently accomplished in a variety of ways. In order to get a hard-to-invert function √ from (1), one needs to regard Dcqk/n as a probability distribution over B(c nq k/n ), and consider fA as a function with this domain. The relation between inverting fA (x) = b (i.e., ﬁnding small solutions to the inhomogeneous system Ax = b (mod q)) and lattice problems is easily explained. Using linear algebra, one can eﬃciently ﬁnd an arbitrary solution t ∈ Znq to the system, but this solution will generally have large entries and not belong to the domain of fA . Linear algebra gives us a little more than an arbitrary solution. It tells us that any solution to the inhomogeneous system Ax = b can be expressed as the sum of any ﬁxed speciﬁc solution t to At = b (mod q) and a solution z to the homogeneous system Az = 0 (mod q). But the set of solutions to the homogeneous system is precisely the lattice Λ⊥ q (A). So, ﬁnding a small x = t + z is equivalent to ﬁnding a lattice point −z ∈ Λ⊥ q (A) within distance t − (−z) = x from the target t. In summary, inverting fA is√equivalent to the problem of ﬁnding lattice k/n vectors in Λ⊥ from the target. Depending of the q (A) within distance c nq value of c, this is the ADDc problem (for c > δ ) or the BDD1/c problem (for c < 1/δ). These two diﬀerent ranges for c also correspond to very diﬀerent statistical properties of the function fA . Namely, when c < 1/δ , the function fA is injective with high probability (which corresponds to BDD having at most one solution). √ Exercise 14. Show that for c ≤ δ the function fA with domain B(c nq k/n ) is injective with overwhelming probability over the choice of A. Similarly, when c > δ, the function fA is surjective with high probability. In fact, all the output values in Znq have almost the same probability under fA (Dcqk/n ).

The Geometry of Lattice Cryptography

199

√ Theorem 5 ([34]). Let q be a prime, n ≥ 2k log q and c ≥ ω( log n). Then, with overwhelming probability over the choice of A ∈ Zk×m , if x is chosen q according to distribution Dc·qk/n , then f (x) is statistically close to uniform over Zkq . Now, let us go back to the question: are lattice problems hard on average when the lattice is chosen according to Λ⊥ q (n, k)? A ﬁrst answer to this question was given in a seminal paper of Ajtai [6] which marks the beginning of modern day lattice based cryptography. Ajtai’s result has been subsequently improved and simpliﬁed in a sequence of papers [20,56,63]. The strongest known result is the one due to [63] with some additional reﬁnements in [34]. Theorem 6 (Implicit in [63,34]). For any polynomially bounded n(k) and c(k) > δ, and any ω(k log k) ≤ q(k) ≤ k O(n/k) , the following holds. Assume SIVPγ is hard in the √ worst case (over k-dimensional lattices) for approximation factors γ(k) = ω(c n · k log k√· q k/n ). Then fA is a one-way function with input distribution Dcqk/n (over B(c nq k/n )). Typically, q(k) = k O(1) is a small polynomial, n = k log q = O(k log k), and c > δ a constant. With this parameters, Theorem 6 gives a provably secure oneway function under the assumption that SIVPγ is hard to approximate (in the worst-case, over k-dimensional lattices) within factors γ(k) = ω(k log k) almost linear in the dimension of the lattice. Recall that the one-way function inversion problem from Theorem 6 is an average case version of ADD for lattices distributed according to Λ⊥ q (n, k). Also, in the previous section we have seen that SIVP is equivalent to ADD, up to small polynomial approximation factors. So, Theorem 6 can be interpreted as a connection between the worst-case and average-case complexity of ADD, and justiﬁes using distribution Λ⊥ q (n, k) to select the lattice. If ADD is not hard on average for this distribution, then it is not hard at all! What about BDD, i.e., inverting fA (x) when the input x is distributed according to Dcqk/m for some small c < 1/δ? For these inputs the behavior of the function fA is quite diﬀerent: we have seen that fA is injective (with high probability) and therefore it is trivially collision resistant: there exist no x = y such that fA (x) = fA (y). But for injective functions, collision resistance is a trivial property and does not imply any form of one-wayness. Still, it is possible to prove that fA is a one-way function, using the stronger complexity assumption that SIVP is hard to solve in the worst case even by quantum algorithms. Since we are interested in solving BDD in the dual lattice, we state this result in terms † ⊥ of the (scaled) dual lattice distribution q(Λ⊥ q (n, k)) = Λq (n, k) ≈ Λq (n, n − k). Theorem 7 (Implicit√in [73]). For all c < 1/δ, some polynomial n(k) = k O(1) , and prime q > (2 k/c)n/(n−k) , the following holds true. Assume SIVPγ is hard in the worst case (over k-dimensional lattices) even for quantum algorithms, √ for approximation factor γ(k) = ω( k)·k·q k/n /c. Then fA is a one-way function √ with input distribution Dc·q1−k/n and domain B(c n · q 1−k/n ). Notice that Dc·q1−k/n is the natural distribution to use when the lattice is chosen 1−k/n according to Λ⊥ q (n, n − k) because these lattices have root determinant q

200

D. Micciancio

rather than q k/n . Theorem 7 asserts that fA is a one-way function only when n(k) is some polynomially bounded, but unspeciﬁed, function of k. This is because the BDD problem in Λq (n, k) lattices in [73] is described as a learning problem (the LWE problem, for Learning With Errors), where each coordinate of the lattice corresponds to a noisy sample, and the adversary is assumed to have access to an arbitrary (but polynomially bounded) number of samples. In principle, it should be possible to tighten the proof in [73] to show that a relatively small value of n(k) is suﬃcient for the reduction to go through. But, anyway, this is not the main diﬀerence if we compare the result with Theorem 6. A more fundamental diﬀerence between Theorem 6 and Theorem 7 is that the latter is based on a quantum reduction. The reason the power of quantum computers is needed for establishing the one-wayness of fA in the BDD setting is currently one of the mysteries of lattice cryptography, and it reﬂects our poor understanding of the relation between ADD and BDD problems. Even in the classical setting of worstcase reductions, we know how to approximately reduce BDD to ADD, but no reduction is known in the opposite direction, at least classically. So, BDD seems an easier problem than ADD also in the worst-case. In the average case setting, Theorems 6 and 7 can be summarized as saying that ADD on random q-ary lattices is as hard as approximating SIVP with classical algorithms, while BDD on random q-ary lattices is as hard as approximating SIVP but with quantum algorithms, under which SIVP may be easier to solve. We remark that for certain choices of parameters (e.g., when q = 2O(n) is exponentially large,) it is possible to show that the function fA of Theorem 7 is one-way, assuming the classical worst-case hardness of some lattice problems [67]. The worst-case problems can be BDD, a special version of SVP with “unique solution”, or the problem or approximating the value of λ without necessarily ﬁnding a short lattice vector. All these problems are essentially equivalent to each other [50] and conceivably easier than SIVP. So, in some settings, Theorem 7 can be interpreted as a connection between the worst-case and average-case complexity of BDD under classical algorithms. In summary, assuming that SIVP is hard to solve in the worst case with classic algorithms, is enough to build simple cryptographic functions, like one-way functions, and as will will see in the next section, also collision resistant hash functions, and commitment schemes. The construction of public key encryption and more complex cryptographic primitives requires the stronger assumption that either BDD is hard in the worst-case (if an exponentially large value of q can be used) or SIVP and ADD are hard even under quantum algorithms. Providing a classical reduction from SIVP and ADD to BDD is an important open problem as it would unify the complexity assumptions used for the construction of all lattice cryptography primitives.

4

Applications

In this section we give some representative examples of cryptographic applications based on lattices. The presentation is informal, emphasizing the geometric

The Geometry of Lattice Cryptography

201

ideas behind the constructions, rather than the technical details of the cryptographic deﬁnitions and security proofs. Here we aim primarily at illustrating the wide range of applications enabled by lattice cryptography, and highlight how cryptographic constructions, which are often described simply in terms of matrices and indistinguishability properties, often have a natural geometric interpretation. However, we remark that all the constructions discussed here can be properly analyzed and proved secure based on the conjectured one-wayness of the functions fA discussed in Section 3, or, using Theorems 6 and 7, the worst-case hardness of SIVP under classical or quantum algorithms. For precise statements, security deﬁnitions and formal proofs, the reader is referred to the original papers, cited throughout this section. 4.1

Hash Functions

Hash functions are (keyed) functions that compress long input strings, into shorter digests, and still have the property that they are collision resistant: it is computationally hard to ﬁnd two distinct inputs x = y such that f (x) = f (y). For injective functions, this property holds trivially, so we may regard collision resistance as a computational relaxation of injectivity, and say that fA is pseudo-injective. Now, consider the function fA for A ∈ Zk×m and input x ← Dcqk/n . It q turns out that (when A is chosen at random) the proof of Theorem 6 shows that the function fA is not only one-way, but also collision resistant. By Exercise 14, when c < 1, the function fA is injective with high probability, and collisions fA (x) = fA (y) do not exist. When c grows beyond 1, the function fA is no longer injective over the domain associated to the input distribution. Still, as proved in [63,34], the function fA is collision resistant for the parameters of Theorem 6. The intuitive explanation is that inverting fA for these parameters corresponds to the ADD problem, while√ﬁnding collisions cork/m responds to ﬁnding short vectors Λ⊥ ) ≈ λn . Since q (A) of length O( nq ADD and SIVP are equivalent problems, we can expect that these two problems have roughly the same complexity. In this informal argument, we are hiding the fact that the equivalence between ADD and SIVP is proved for the worst-case formulation of the problems, and that SIVP requires to ﬁnd not just one, but many short vectors. Still, the intuition holds, and fA can be formally shown to be collision resistant, on the average, when A is chosen at random. Hash functions are usually designed using ad-hoc methods, without supporting proofs of security. Lattice based functions like fA are not competitive with such ad-hoc constructions. Still, using special classes of algebraic lattices it is possible to get much more eﬃcient constructions [57,48,68,51] which sill are provably hard to break under worst-case assumptions (albeit over a restricted class of lattices) and at the same time are comparable to ad-hoc constructions in terms of eﬃciency.

202

4.2

D. Micciancio

Pseudorandom Generators

When c > 1 is large enough, the output of fA (x) with x ← Dcqk/n is statistically close to random, and the function compresses its input. When c < 1 is small, the output of fA (x) is certainly not close to uniform, just because the function stretches its input to a longer string. Still, the output of fA (x) can be shown computationally indistinguishable from uniform, i.e., it is pseudo-random. In other words, fA is a pseudo-random generator family, indexed by A. The pseudo-randomness of fA (x) was proved in a loose sense already in [73]. The proof in [73] was loose in the sense that in order to prove the pseudorandomness of the output of fA for certain values of the parameters n, k, q, it required to assume that fA was one-way for a (polynomially related, but) much bigger value of n. Recently [61] it has been shown that such increase is not necessary, and the pseudo-randomness of the output can be proved for the same values of the parameters as in Theorem 7. We recall that inverting fA for this range of parameters corresponds to a BDD problem. On the other hand, distinguishing the output of fA from random corresponds to the decisional version of BDD: the input to BDD is a point close to the lattice, while the uniform distribution in space can be shown to be far from the lattice with high probability [36,37]. So, proving that the output of fA is pseudo-random roughly corresponds to showing that if you can tell if a target is either very close, or very far from a lattice, then given a point very close to the lattice one can ﬁnd the closest lattice point. For a reduction of this kind in the worst-case setting, see [45]. 4.3

Commitment Schemes

A (non-interactive) commitment scheme is a function that allows one party to commit to a value x while keeping it hidden. At a later time the same party can open the commitment, and reveal the value x. The commitment should be binding, i.e., it should be computationally infeasible to commit to a value x and then open the commitment to a diﬀerent value x . The construction of commitment schemes from lattices is very simple: in order to commit to a value represented by a short vector x, choose a short random vector r, and output fA (x, r), where matrix A is wide enough that it can be multiplied by the concatenation of the two vectors (x; r). This commitment scheme was proposed in [41], where it is also used for the construction of a concurrently secure identiﬁcation scheme based on lattices. We remark that, depending on the value of the parameters, this construction gives both commitments that are statistically hiding and computationally binding, and also commitments that are computationally hiding, but statistically binding. The fact that this construction gives a secure commitment scheme is closely related to the fact that fA is both (pseudo-)random and (pseudo-) injective. (Pseudo-)randomness of the output gives the hiding property of the commitment, while the (pseudo-)injective property implies that the commitment

The Geometry of Lattice Cryptography

203

is binding. Depending on the setting of the parameter c in the input distribution Dcqk/n , either the hiding or binding property (but not both) may hold in a strong statistical sense. 4.4

Public Key Encryption

A public key encryption scheme can be built along the lines described at the beginning of Section 3: – The public key is a random lattice Λq (A). The secret key is given by a short basis or set of short vectors S. – Ciphertexts are obtained by encoding a message into a short vector x, and computing a standard representative for the coset x + (Λq (A))† . Such representative can be computed, for example, as fA (x). – Decryption is a BDD problem in the dual lattice (Λq (A))† , and it can be performed using the short basis S. We already know that recovering the secret key from the public one corresponds to an SIVP problem in Λq (A), which by Theorem 6 is hard on average. Similarly, decryption is an average-case BDD problem, and by Theorem 7 it is also hard. All we are missing is a method to generate a random lattice Λq (A) together with a short set of lattice vectors S. In fact, we don’t need a full set of n-linearly independent short lattice vectors in order to build a public key encryption scheme. Assume for simplicity we want to encrypt a single bit message. Then the message 0 may be encoded as a point close to the (dual) lattice, while 1 may be encoded as a random point, which will be far from the lattice with high probability. Then, we can decrypt using a single short (dual) vector s as follows: recall that a lattice vector s ∈ L can be used to partition the dual lattice into layers L†i = {v ∈ L† | v · s = i}. One can check if a point x is either close or far from any of these layers by computing the product x · s and checking if the result is close to an integer. This idea was already present in the cryptosystem of Ajtai and Dwork [7], which didn’t use q-ary lattices. So, how can we obtain this short vector s when the lattice (Λq (A)) is chosen at random? Remember that ﬁnding short vectors in these random lattices is a hard problem, and it has to be as it corresponds to the key recovery problem of our cryptosystem. Fortunately we can plant s in the lattice during key generation process. We recall that Λq (A) is the lattice generated by the columns of AT modulo q. So, we can choose the ﬁrst k − 1 columns of AT = [AT0 |p] truly at random, and then we set the last column to our secret short vector s. Of course, including s in the public key would reveal the key used for decryption. So, we mask s with a random lattice point, and set p = AT0 w + s, where w is chosen at random. Geometrically, instead of revealing s, we reveal its coset s + Λq (A0 ), from which recovering s (or an equally short vector) is a computationally hard task. Clearly, the lattice Λq ([AT0 , p]T ) still contains s, which can be used for decryption.

204

D. Micciancio

Encrypting one bit at a time is not very practical, but a much more eﬃcient system can be obtained along similar lines by embedding not just one, but many short vectors, as ﬁrst proposed in [71], for a syntactically diﬀerent formulation of the encryption scheme. The geometric description presented in these notes can be instantiated in several ways, depending on the length of s. If s is long enough, then p will be distributed almost uniformly at random, and A will follow the distribution Λq (n, k). If s is shorter, then the distribution of A will be far from Λq (n, k), but it will be indistinguishable from it because p is pseudorandom. Either setting of parameters is reasonable, as long as the length of the vector x used during the encryption process is chosen accordingly. If s is longer, then x should be shorter, while if s is shorter, then x can be longer. The corresponding proof of security will invoke a statistical or computational argument when analyzing either the public key or encrypted ciphertext. These two instantiations were originally presented as two diﬀerent cryptosystems in [73] and [34], where they are described using quite diﬀerent matrix formulations. Looking at these two cryptosystems through the lens of geometry and duality, [59] showed that they are in fact the same cryptosystem instantiated for diﬀerent values of the parameters. [59] also suggested that the eﬃciency of the cryptosystem could be improved using intermediate values of the parameters and using a computational argument both during the analysis of the public key and encrypted ciphertext. Signiﬁcant eﬃciency improvements were later demonstrated in [44] which proposed and analyzed a concrete instantiation of the general scheme of [59]. This is the currently most eﬃcient example of a lattice based cryptosystem based on general lattices. 4.5

Identity Based Encryption

We have seen how generating a random lattice together with a single short vector is enough to build a public key encryption scheme. But some applications do require a full basis. For example, this is the case for the identity based encryption (in the random oracle model) of [34]. We recall that an identity based encryption is an encryption scheme where public keys are simply given by the parties’ names or public identiﬁers. The corresponding secret keys are computed by a trusted entity using a master secret key. In the scheme of [34] A is generated together with a full trapdoor basis S, which is used as a master key. Then the (identity based) public key encryption scheme is built as described in the previous subsection, with one twist: instead of choosing the short decryption key s at random, and embedding it in a new independently generated lattice, the short vector s is extracted (sampled) from the lattice Λq (A) using the master key S to solve an instance of the ADD problem. More speciﬁcally, given a target point t that represents a party’s identity, a corresponding secret key is computed using S to ﬁnd a lattice point not too far from t. The problem of generating a random lattice together with a full short basis is studied in [5,8]. A new, much simpler and more eﬃcient method to generate such lattices has been recently discovered in [62].

The Geometry of Lattice Cryptography

205

Lattice based identity based encryption has been further studied in [22,1,2] yielding both solutions in the standard model, and solutions to the more general hierarchical identity based encryption problem, where identities form a hierarchy, and each node can delegate limited decryption capabilities to its children.

5

Conclusion

Lattice cryptography is still a very young and fast developing area, and still it has produced a substantial body of work. We conclude these notes by providing some pointers to the literature that can be used to get a deeper understanding of the concepts touched upon in this tutorial. Other surveys presenting additional perspectives on lattice cryptography include [58,64,74,75]. Work in lattice cryptography can be roughly categorized into two groups: foundational work aimed at establishing that fA is a one way function, and cryptographic applications that use the properties of fA to solve more complex cryptographic tasks. We start with the ﬁrst category. Works in this area tend to be mathematically more involved, but that’s where the real “magic” happens: connecting the average-case and worst-case complexity of lattice problems. The reader interested in getting a feeling of how the proof of Theorem 6 works, is referred to the survey [58]. From there, the reader may proceed to [63,73]. Both papers make use of harmonic analysis and high dimensional Fourier transform, which is very useful both to yield stronger and more concise proofs. Still on the foundational side, is a line of research initiated in [57] aimed at substantially improving the eﬃciency of lattice cryptography, while maintaining a form of provable security, based on the use of lattices with special algebraic structure. We expect these lattices will play a major role in bringing lattice cryptography closer to practice. The original paper [57] is still a good starting point to learn about these lattices. For more recent developments in this direction see [48,69,68,51,53,61]. On the application side, there is a huge variety of cryptographic problems that have been recently solved using lattices. But in the end, all solutions ultimately rely on the results and techniques developed in a small number of foundational papers. In fact, many applications can be described and analyzed simply using as a starting point the one-wayness of the function fA , without even any explicit reference to lattices. But we believe that the geometric perspective advocated in these notes can be of great value to understand those constructions, and interpreting the variety of solutions in the literature in terms of a small number of basic geometric concepts. For the reader interested in applications, here we give some pointers to relevant papers, organized by category. For public key encryption see [73,72,52,81,67,40,44]. For digital signatures see [34,49,47,18]. For group and ring signatures see [35,77,24]. For identity based cryptography see [2,22,1,78]. For (fully) homomorphic encryption see [33,28,80,65,29,3,82,79,32,25,19,31,30]. For zero-knowledge proofs and identiﬁcation protocols see [23,46,76,83,41,70]. For still more cryptographic primitives and protocols see [66,14,13,17,12,39,9,71,16].

206

D. Micciancio

References 1. Agrawal, S., Boneh, D., Boyen, X.: Eﬃcient lattice (H)IBE in the standard model. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 553–572. Springer, Heidelberg (2010) 2. Agrawal, S., Boneh, D., Boyen, X.: Lattice basis delegation in ﬁxed dimension and shorter-ciphertext hierarchical IBE. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 98–115. Springer, Heidelberg (2010) 3. Melchor, C.A., Gaborit, P., Herranz, J.: Additively homomorphic encryption with d-operand multiplications. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 138–154. Springer, Heidelberg (2010) 4. Ajtai, M.: The shortest vector problem in L2 is NP-hard for randomized reductions (extended abstract). In: Proceedings of STOC 1998, pp. 10–19. ACM, New York (1998) 5. Ajtai, M.: Generating hard instances of the short basis problem. In: Wiedermann, J., Van Emde Boas, P., Nielsen, M. (eds.) ICALP 1999. LNCS, vol. 1644, pp. 1–9. Springer, Heidelberg (1999) 6. Ajtai, M.: Generating hard instances of lattice problems. Complexity of Computations and Proofs, Quaderni di Matematica 13, 1–32 (2004); Preliminary version in STOC 1996 7. Ajtai, M., Dwork, C.: A public-key cryptosystem with worst-case/average-case equivalence. In: Proceedings of STOC 1997, pp. 284–293. ACM, New York (1997) 8. Alwen, J., Peiker, C.: Generating shorter bases for hard random lattices. In: Proceedints of STACS, pp. 75–86 (2009) Invited to Theory of Computing Systems, 48(3), 535–553. Prelim. Version in STACS 2009 9. Applebaum, B., Cash, D., Peikert, C., Sahai, A.: Fast cryptographic primitives and circular-secure encryption based on hard learning problems. In: Halevi, S. (ed.) CRYPTO 2009. LNCS, vol. 5677, pp. 595–618. Springer, Heidelberg (2009) 10. Arora, S., Babai, L., Stern, J., Sweedyk, E.Z.: The hardness of approximate optima in lattices, codes, and systems of linear equations. Journal of Computer and System Sciences 54(2), 317–331 (1997); Preliminary version in FOCS 1993 11. Banaszczyk, W.: New bounds in some transference theorems in the geometry of numbers. Mathematische Annalen 296, 625–635 (1993) 12. Barak, B., Haitner, I., Hofheinz, D., Ishai, Y.: Bounded key-dependent message security. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 423–444. Springer, Heidelberg (2010) 13. Bendlin, R., Damg˚ ard, I.: Threshold decryption and zero-knowledge proofs for lattice-based cryptosystems. In: Micciancio, D. (ed.) TCC 2010. LNCS, vol. 5978, pp. 201–218. Springer, Heidelberg (2010) 14. Bendlin, R., Damg˚ ard, I., Orlandi, C., Zakarias, S.: Semi-homomorphic encryption and multiparty computation. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 169–188. Springer, Heidelberg (2011) 15. Bl¨ omer, J., Seifert, J.-P.: On the complexity of computing short linearly independent vectors and short bases in a lattice. In: Proceedings of STOC 1999, pp. 711–720. ACM, New York (1999) 16. Boneh, D., Freeman, D.: Homomorphic signatures for polynomial functions. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 149–168. Springer, Heidelberg (2011)

The Geometry of Lattice Cryptography

207

17. Boneh, D., Freeman, D.: Linearly homomorphic signatures over binary ﬁelds and new tools for lattice-based signatures. In: Catalano, D., Fazio, N., Gennaro, R., Nicolosi, A. (eds.) PKC 2011. LNCS, vol. 6571, pp. 1–16. Springer, Heidelberg (2011) 18. Boyen, X.: Lattice mixing and vanishing trapdoors: A framework for fully secure short signatures and more. In: Nguyen, P.Q., Pointcheval, D. (eds.) PKC 2010. LNCS, vol. 6056, pp. 499–517. Springer, Heidelberg (2010) 19. Brakerski, Z., Vaikuntanathan, V.: Fully homomorphic encryption from ring-LWE and security for key dependent messages. In: CRYPTO (to appear, 2011) 20. Cai, J.-Y., Nerurkar, A.P.: An improved worst-case to average-case connection for lattice problems (extended abstract). In: Proceedings of FOCS 1997, pp. 468–477. IEEE, Los Alamitos (1997) 21. Cai, J.-Y., Nerurkar, A.P.: Approximating the SVP to within a factor (1 + 1/dim ) is NP-hard under randomized reductions. Journal of Computer and System Sciences 59(2), 221–239 (1999); Preliminary version in CCC 1998 22. Cash, D., Hofheinz, D., Kiltz, E., Peikert, C.: Bonsai trees, or how to delegate a lattice basis. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 523–552. Springer, Heidelberg (2010) 23. Cayrel, P.-L., Lindner, R., R¨ uckert, M., Silva, R.: Improved zero-knowledge identiﬁcation with lattices. In: Heng, S.-H., Kurosawa, K. (eds.) ProvSec 2010. LNCS, vol. 6402, pp. 1–17. Springer, Heidelberg (2010) 24. Cayrel, P.-L., Lindner, R., R¨ uckert, M., Silva, R.: A Lattice-Based Threshold Ring Signature Scheme. In: Abdalla, M., Barreto, P.S.L.M. (eds.) LATINCRYPT 2010. LNCS, vol. 6212, pp. 255–272. Springer, Heidelberg (2010) 25. Coron, J.-S., Mandal, A., Naccache, D., Tibouchi, M.: Fully-homomorphic encryption over the integers with shorter public-keys. In: CRYPTO (to appear, 2011) 26. Dinur, I.: Approximating SV P ∞ to within almost-polynomial factors is NP-hard. Theoretical Computer Science 285(1), 55–71 (2002); Preliminary version in CIAC 2000 27. Dinur, I., Kindler, G., Raz, R., Safra, S.: Approximating CVP to within almostpolynomial factors is NP-hard. Combinatorica 23(2), 205–243 (2003); Preliminary version in FOCS 1998 28. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Proceedings of STOC, pp. 169–178. ACM, New York (2009) 29. Gentry, C.: Toward basing fully homomorphic encryption on worst-case hardness. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 116–137. Springer, Heidelberg (2010) 30. Gentry, C.: Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, Report 2011/277 (2011) 31. Gentry, C., Halevi, S.: Fully homomorphic encryption without squashing using depth-3 arithmetic circuits. Cryptology ePrint Archive, Report 2011/279 (2011) 32. Gentry, C., Halevi, S.: Implementing gentrys fully-homomorphic encryption scheme. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 129– 148. Springer, Heidelberg (2011) 33. Gentry, C., Halevi, S., Vaikuntanathan, V.: A simple BGN-type cryptosystem from lwe. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 506– 522. Springer, Heidelberg (2010) 34. Gentry, C., Peikert, C., Vaikuntanathan, V.: Trapdoors for hard lattices and new cryptographic constructions. In: Proceedings of STOC, pp. 197–206. ACM, New York (2008)

208

D. Micciancio

35. Gordon, D., Katz, J., Vaikuntanathan, V.: A group signature scheme from lattice assumptions. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 395–412. Springer, Heidelberg (2010) 36. Guruswami, V., Micciancio, D., Regev, O.: The complexity of the covering radius problem. Computational Complexity 14(2), 90–121 (2005); Preliminary version in CCC 2004 37. Haviv, I., Lyubashevsky, V., Regev, O.: A note on the distribution of the distance from a lattice. Discrete and Computational Geometry 41(1), 162–176 (2009) 38. Haviv, I., Regev, O.: Tensor-based hardness of the shortest vector problem to within almost polynomial factors. In: Proceedings of STOC, pp. 469–477. ACM, New York (2007) 39. Katz, J., Vaikuntanathan, V.: Smooth projective hashing and password-based authenticated key exchange based on lattices. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 636–652. Springer, Heidelberg (2009) 40. Kawachi, A., Tanaka, K., Xagawa, K.: Multi-bit cryptosystems based on lattice problems. In: Okamoto, T., Wang, X. (eds.) PKC 2007. LNCS, vol. 4450, pp. 315– 329. Springer, Heidelberg (2007) 41. Kawachi, A., Tanaka, K., Xagawa, K.: Concurrently secure identiﬁcation schemes and ad hoc anonymous identiﬁcation schemes based on the worst-case hardness of lattice problems. In: Pieprzyk, J. (ed.) ASIACRYPT 2008. LNCS, vol. 5350, pp. 372–389. Springer, Heidelberg (2008) 42. Khot, S.: Hardness of approximating the shortest vector problem in lattices. Journal of the ACM 52(5), 789–808 (2005); Preliminary version in FOCS 2004 43. Khot, S.: Hardness of approximating the shortest vector problem in high Lp norms. J. of Computer Systems Sciences 72(2), 206–219 (2006); Preliminary version in FOCS 2003 44. Lindner, R., Peikert, C.: Better key sizes (and attacks) for LWE-based encryption. In: Kiayias, A. (ed.) CT-RSA 2011. LNCS, vol. 6558, pp. 319–339. Springer, Heidelberg (2011) 45. Liu, Y.-K., Lyubashevsky, V., Micciancio, D.: On bounded distance decoding for general lattices. In: D´ıaz, J., Jansen, K., Rolim, J.D.P., Zwick, U. (eds.) APPROX 2006 and RANDOM 2006. LNCS, vol. 4110, pp. 450–461. Springer, Heidelberg (2006) 46. Lyubashevsky, V.: Lattice-based identiﬁcation schemes secure under active attacks. In: Cramer, R. (ed.) PKC 2008. LNCS, vol. 4939, pp. 162–179. Springer, Heidelberg (2008) 47. Lyubashevsky, V.: Fiat-Shamir with aborts: Applications to lattice and factoringbased signatures. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 598–616. Springer, Heidelberg (2009) 48. Lyubashevsky, V., Micciancio, D.: Generalized compact knapsacks are collision resistant. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 144–155. Springer, Heidelberg (2006) 49. Lyubashevsky, V., Micciancio, D.: Asymptotically eﬃcient lattice-based digital signatures. In: Canetti, R. (ed.) TCC 2008. LNCS, vol. 4948, pp. 37–54. Springer, Heidelberg (2008) 50. Lyubashevsky, V., Micciancio, D.: On bounded distance decoding, unique shortest vectors, and the minimum distance problem. In: Halevi, S. (ed.) CRYPTO 2009. LNCS, vol. 5677, pp. 577–594. Springer, Heidelberg (2009) 51. Lyubashevsky, V., Micciancio, D., Peikert, C., Rosen, A.: SWIFFT: a modest proposal for FFT hashing. In: Nyberg, K. (ed.) FSE 2008. LNCS, vol. 5086, pp. 54–72. Springer, Heidelberg (2008)

The Geometry of Lattice Cryptography

209

52. Lyubashevsky, V., Palacio, A., Segev, G.: Public-key cryptographic primitives provably as secure as subset sum. In: Micciancio, D. (ed.) TCC 2010. LNCS, vol. 5978, pp. 382–400. Springer, Heidelberg (2010) 53. Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 1–23. Springer, Heidelberg (2010) 54. Micciancio, D.: Improving lattice based cryptosystems using the Hermite normal form. In: Silverman, J.H. (ed.) CaLC 2001. LNCS, vol. 2146, pp. 126–145. Springer, Heidelberg (2001) 55. Micciancio, D.: The shortest vector problem is NP-hard to approximate to within some constant. SIAM Journal on Computing 30(6), 2008–2035 (2001); Preliminary version in FOCS 1998 56. Micciancio, D.: Almost perfect lattices, the covering radius problem, and applications to Ajtai’s connection factor. SIAM Journal on Computing 34(1), 118–169 (2004); Preliminary version in STOC 2002 57. Micciancio, D.: Generalized compact knapsacks, cyclic lattices, and eﬃcient oneway functions. Computational Complexity 16(4), 365–411 (2007); Preliminary version in FOCS 2002 58. Micciancio, D.: Cryptographic functions from worst-case complexity assumptions. In: The LLL Algorithm: Survey and Applcations, Springer, Heidelberg (2009) 59. Micciancio, D.: Duality in lattice cryptography. In: Proceedings of PKC. LNCS. IACR, Springer (May 2010) (invited talk); Slides available from author’s web page 60. Micciancio, D., Goldwasser, S.: Complexity of Lattice Problems: a cryptographic perspective. The Kluwer International Series in Engineering and Computer Science, vol. 671. Kluwer Academic Publishers, Boston (2002) 61. Micciancio, D., Mol, P.: Pseudorandom knapsacks and the sample complexity of LEW search-to-decision reductions. In: CRYPTO (to appear, 2011) 62. Micciancio, D., Peikert, C.: Trapdoor for lattices: Simpler, tighter, faster, smaller (manuscript, 2011) 63. Micciancio, D., Regev, O.: Worst-case to average-case reductions based on Gaussian measure. SIAM Journal on Computing 37(1), 267–302 (2007); Preliminary version in FOCS 2004 64. Micciancio, D., Regev, O.: Lattice-based cryptography. In: Post-quantum Cryptography. Springer, Heidelberg (2008) 65. Ogura, N., Yamamoto, G., Kobayashi, T., Uchiyama, S.: An improvement of key generation algorithm for Gentry homomorphic encryption scheme. In: Echizen, I., Kunihiro, N., Sasaki, R. (eds.) IWSEC 2010. LNCS, vol. 6434, pp. 70–83. Springer, Heidelberg (2010) 66. O’Neill, A., Peikert, C., Waters, B.: Bi-deniable public-key encryption. In: CRYPTO (to appear, 2011) 67. Peikert, C.: Public-key cryptosystems from the worst-case shortest vector problem. In: Proceedings of STOC, pp. 333–342. ACM, New York (2009) 68. Peikert, C., Rosen, A.: Eﬃcient collision-resistant hashing from worst-case assumptions on cyclic lattices. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 145–166. Springer, Heidelberg (2006) 69. Peikert, C., Rosen, A.: Lattices that admit logarithmic worst-case to average-case connection factors. In: Proceedings of STOC, pp. 478–487. ACM, New York (2007) 70. Peikert, C., Vaikuntanathan, V.: Noninteractive statistical zero-knowledge proofs for lattice problems. In: Wagner, D. (ed.) CRYPTO 2008. LNCS, vol. 5157, pp. 536–553. Springer, Heidelberg (2008)

210

D. Micciancio

71. Peikert, C., Vaikuntanathan, V., Waters, B.: A framework for eﬃcient and composable oblivious transfer. In: Wagner, D. (ed.) CRYPTO 2008. LNCS, vol. 5157, pp. 554–571. Springer, Heidelberg (2008) 72. Peikert, C., Waters, B.: Lossy trapdoor functions and their applications. In: Proceedings of STOC, pp. 187–196. ACM, New York (2008) 73. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. Journal of ACM 56(6), 34 (2009); Preliminary version in STOC 2005 74. Regev, O.: Learning with errors over rings. In: Hanrot, G., Morain, F., Thom´e, E. (eds.) ANTS-IX. LNCS, vol. 6197, pp. 3–3. Springer, Heidelberg (2010) 75. Regev, O.: The learning with errors problem (invited survey). In: CCC, pp. 191–204 (2010) 76. R¨ uckert, M.: Adaptively secure identity-based identiﬁcation from lattices without random oracles. In: Garay, J.A., De Prisco, R. (eds.) SCN 2010. LNCS, vol. 6280, pp. 345–362. Springer, Heidelberg (2010) 77. R¨ uckert, M.: Lattice-based blind signatures. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 413–430. Springer, Heidelberg (2010) 78. R¨ uckert, M.: Strongly unforgeable signatures and hierarchical identity-based signatures from lattices without random oracles. In: Sendrier, N. (ed.) PQCrypto 2010. LNCS, vol. 6061, pp. 182–200. Springer, Heidelberg (2010) 79. Smart, N., Vercauteren, F.: Fully homomorphic encryption with relatively small key and ciphertext sizes. In: Nguyen, P.Q., Pointcheval, D. (eds.) PKC 2010. LNCS, vol. 6056, pp. 420–443. Springer, Heidelberg (2010) 80. Stehl´e, D., Steinfeld, R.: Faster fully homomorphic encryption. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 377–394. Springer, Heidelberg (2010) 81. Stehl´e, D., Steinfeld, R., Tanaka, K., Xagawa, K.: Eﬃcient public key encryption based on ideal lattices. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 617–635. Springer, Heidelberg (2009) 82. van Dijk, M., Gentry, C., Halevi, S., Vaikuntanathan, V.: Fully homomorphic encryption over the integers. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 24–43. Springer, Heidelberg (2010) 83. Xagawa, K., Tanaka, K.: Zero-knowledge protocols for NTRU: Application to identiﬁcation and proof of plaintext knowledge. In: Pieprzyk, J., Zhang, F. (eds.) ProvSec 2009. LNCS, vol. 5848, pp. 198–213. Springer, Heidelberg (2009)

Quantitative Information Flow and Applications to Diﬀerential Privacy M´ ario S. Alvim, Miguel E. Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi INRIA and LIX, Ecole Polytechnique, France

Abstract. Secure information ﬂow is the problem of ensuring that the information made publicly available by a computational system does not leak information that should be kept secret. Since it is practically impossible to avoid leakage entirely, in recent years there has been a growing interest in considering the quantitative aspects of information ﬂow, in order to measure and compare the amount of leakage. Information theory is widely regarded as a natural framework to provide ﬁrm foundations to quantitive information ﬂow. In this notes we review the two main information-theoretic approaches that have been investigated: the one based on Shannon entropy, and the one based on R´enyi min-entropy. Furthermore, we discuss some applications in the area of privacy. In particular, we consider statistical databases and the recently-proposed notion of diﬀerential privacy. Using the information-theoretic view, we discuss the bound that diﬀerential privacy induces on leakage, and the trade-oﬀ between utility and privacy.

1

Introduction

In the last few decades the amount of information ﬂowing through computational systems has increased dramatically. Never before in history has a society been so dependent on such a huge amount of information being generated, transmitted and processed. It is expected that this vertiginous trend of increase will continue in the near future, reinforcing the need for eﬃcient and safe ways to cope with this reality. One of the concerns in the use of computational systems is to avoid the leakage of secret information through public observables. If some information is supposed to be conﬁdential, then unauthorized users should not be allowed to infer such information from the output or the behavior of the system. Ideally we would like systems to be completely secure, i.e. protect the secret information entirely, but in practice this goal is usually impossible to achieve. The following example illustrates some of the issues.

This work has been partially supported by the project ANR-09-BLAN-0169-01 PANDA and by the INRIA DRI Equipe Associ´ee PRINTEMPS. The work of Miguel E. Andr´es has been supported by the LIX-Qualcomm postdoc fellowship.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 211–230, 2011. c Springer-Verlag Berlin Heidelberg 2011

212

M.S. Alvim et al.

Example 1. Consider the password-checker algorithm Table 1. Passworddeﬁned in Table 1, where K1 K2 . . . KN is the passchecker pseudocode word, and x1 x2 . . . xN is the string entered by the user. An attacker may obtain the password by simout := OK ply trying a string, if she is so lucky to guess the for i = 1, ..., N do string that matches the password. if xi = Ki then Furthermore, even if the attacker makes the wrong out := FAIL guess, she still obtains some (small) amount of inforexit() mation: the information that the password is not the end if string she just entered, thus restricting the guessing end for range of potential further attempts. Worse yet, the algorithm is subject to timing attacks: the attacker may be able to determine from the duration of the execution how many iterations are performed, thus inferring a preﬁx of the password even in case of failure. Thus it is important to express the amount of leakage in quantitative terms, so to be able to assess whether a system is better than another, although they may both be insecure. Another reason to consider the quantitative aspects is that the system may have a probabilistic behavior. This is the case, for instance, of anonymity protocols, which use randomization to obfuscate the link between the identity of the culprit and the observable outcome. Example 2. DC-Net is a protocol for anonymous broadcasting based on the paradigm of the dining 1 0 1 cryptographers [7]. 1 The participants in the protocol are assumed to 1 1 be the vertexes of a graph, whose edges are associ1 0 culprit 0 ated to binary coins which are visible to the adjab=1 0 1 1 cent vertexes. The protocol works as follows: sup0 0 0 pose that a participant x wants to broadcast one bit 1 b. Then, all coins get ﬂipped, and each participant 0 1 0 reads the value of the coins which are visible to her and computes the binary sum of these values. Then Fig. 1. Example of a all participants but x declare the result, while x (bidc-net narily) adds b to her result, and then declares it. Fig. 1 illustrates the situation. It is easy to see that, since each coin is counted twice, the global contribution of all coins is 0. Hence the binary sum of all the declarations gives b, thus achieving the goal of making b public. Furthermore thanks to the “noise” created by the coins, it is generally impossible for an adversary to determine with certainty the culprit, i.e. the vertex who is broadcasting b. We will see that a stronger property actually holds: if the coins are fair, and the graph is connected, then for an external adversary the probability of each vertex to be the culprit does not change after she observes the declarations.

Quantitative Information Flow and Applications to Diﬀerential Privacy

213

Several authors have proposed to use concepts from information theory to model information ﬂow and to deﬁne the leakage in a quantitative way. So far, most of the approaches were based on Shannon entropy [25,24,30,8,21,22,5]. This is probably due to the fact that Shannon entropy is the most established and useful notion of entropy in information theory, because of its mathematical properties and its correspondence with the channel transmission rate. Nevertheless, other notions have been considered, and argued to be more appropriate for security in certain scenarios. These include: R´enyi min-entropy [27,29], guessing entropy [23], and marginal guesswork [26]. The common idea in these information-theoretic approaches is that a system can be seen as a channel in the information-theoretic sense, where the secret is the input and the observables are the output. The entropy of the input provides a measure of its vulnerability, i.e. how easy is for an attacher to discover the secret. Therefore, as argued in [19], the notion of entropy should be chosen according to the model of attacker, and to the way we extimate the success of the attack. Normally, the entropy of a random variable represents its uncertainty, hence the vulnerability is anti-monotonic on the entropy. Independently from the intended model of attacker, the notion of leakage can be expressed in a uniform way as the diﬀerence between the secret’s initial uncertainty i.e. the uncertainty before we run the system, and the secret’s remaining uncertainty, i.e. the uncertainty after we run the system and observe its outcome: information leakage = initial uncertainty − remaining uncertainty

(1)

In general the observation of the outcome should increase the probabilistic knowledge, and consequently decrease the uncertainty of the secret. Therefore we expect the result of (1) to be non-negative. This is indeed the case, for all the notions of entropy mentioned above, when the channel is deterministic. In the more general case of a probabilistic channel, however, the non-negativeness is ensured only for Shannon entropy and for R´enyi min-entropy. In these notes we aim at encompassing also the probabilistic case, hence we will restrict to the latter two notions of entropy. Recently, the above notions have been applied also to the problem of statistical disclosure control in the area of databases. The goal is to learn properties of the population as a whole, while maintaining the privacy of individuals. For example, in medical research it is desirable to collect the personal medical information of a large number of individuals. Researchers or public authorities can calculate a series of statistics from the sample and decide, for instance, how much money the health care system should spend next year in the treatment of a speciﬁc disease. It desirable, however, that the participation in the sample will not harm the privacy of any individual: usually people do not want to have disclosed their speciﬁc status with respect to a given disease, or other personal matters. Some studies show indeed that when individuals are guaranteed anonymity and privacy they tend to be more cooperative in giving in personal information [18]. The fact that the answer is publicly available, however, constitutes a threat for the privacy of the individuals. For instance, assume that we are interested

214

M.S. Alvim et al.

in the query “what is the percentage of individuals with a given disease?”. The addition of an individual to the database will modify the percentage, and reveal whether the individual has the disease or not. A common solution to the above problem is to introduce some output perturbation mechanism based on randomization: instead of the exact answer to the query, we report a “noisy” answer. Namely, we use some randomized function K which produces values according to some probability distribution. Of course, depending oin the distribution, it may still be possible to guess the value of an individual with a high probability of success, i.e. there may still be a risk of violating privacy. The notion of diﬀerential privacy, due to Dwork [12,15,13,14], is a proposal to control such risk. The idea is to say that K satisﬁes -diﬀerential privacy (for some > 0) if the ratio between the probabilities that two adjacent databases give the same answer is bound by e , where by “adjacent” we mean that the databases diﬀer for only one individual. Often we will abbreviate “-diﬀerential privacy” as -d.p. Obviously, the smaller is , the greater is the privacy protection. In particular, when is close to 0 the output of K is nearly independent from the input (all distributions are almost equal). Unfortunately, such K is practically useless. The utility, i.e. the capability to retrieve accurate answers from the reported ones, is the other important characteristic of K, and it is clear that there is a trade-oﬀ between utility and privacy. On the other hand, these two notions are not the complete opposite of each other, because utility concerns the relation between the reported answer and the real answer, while privacy is concerns the relation between the reported answer and the information in the database. This asymmetry makes more interesting the problem of ﬁnding a good compromise between the two. At this point, we would like to remark an intriguing analogy between the area of diﬀerential privacy and that of quantitative information ﬂow (QIF), both in the motivations and in the conceptual framework. At the motivational level, the concern about privacy is akin the concern about information leakage. At the conceptual level, the randomized function K can be seen as an informationtheoretic channel, and the limit case of = 0, for which the privacy protection is total, corresponds to the case in which the answer to the query does not add any information about the input distribution, and therefore the leakage is 0. In this notes we recall the notion of diﬀerential privacy and its implications, in light of the min-entropy framework developed for QIF. In particular, we explore the relation between -d.p., leakage, and utility. The rest of the paper is organized as follows: in the next section we discuss and compare Shannon entropy and R´enyi min-entropy, and their interpretation in terms of attack model. In Section 3 we illustrate the interpretation of systems as channels, from the point of view of leakage, and we review the main results concerning the information-theoretic approaches based on these two notions of entropy. In Section 4 we show that -diﬀerential privacy induces a bound on the leakage of statistical databases. In Section 5 we show that -diﬀerential privacy induces a bound also on their utility, and we present a method to deﬁne a

Quantitative Information Flow and Applications to Diﬀerential Privacy

215

randomization mechanism that gives the maximum utility while providing diﬀerential privacy. In Section 6 we discuss some related work on the relation between diﬀerential privacy and QIF.

2

Adversary Models and Entropy Notions

In this section we review the two main notions of entropy used in the literature on quantitative information ﬂow, and we discuss the relation with the model of attacker. In general we consider the kind of threats that in the model of [19] are called brute-force guessing attacks, which can be summarized as follows: The goal of the adversary is to determine the value of a random variable. He can make a series of queries to an oracle. Each query must have a yes/no answer. In general the adversary is adaptive, i.e. he can choose the next query depending on the answer to the previous ones. We assume that the adversary knows the probability distribution. In the following, X, Y denote two discrete random variables with carriers X = {x1 , . . . , xn }, Y = {y1 , . . . , ym }, and probability distributions pX (·), pY (·), respectively. We will use X ∧ Y to represent the random variable with carrier X × Y and joint probability distribution pX∧Y (x, y) = pX (x) · pY |X (y | x), where pY |X (y | x) is the conditional probability that Y = y when X = x. The notation X ·Y will denote the random variable with carrier X ×Y and probability distribution deﬁned as product, i.e. pX·Y (x, y) = pX (x) · pY (y). Clearly if X and Y are independent we have X ∧ Y = X · Y . We shall omit the subscripts on the probabilities when they are clear from the context. In general, pX (·) is called a priori distribution of X, and the conditional probability pX|Y (· | ·) is called a posteriori distribution. We will also refer to the individual probabilities p(x) and p(x|y) as the a priori an the a posteriori probabilities of x. 2.1

Shannon Entropy

In this section we illustrate a model of adversary that corresponds to Shannon entropy. The ideas discussed here are based on the works [26,20]. Deﬁnition 1. A Shannon adversary is characterized by questions are of the form “is X ∈ X ?” where X ⊆ X . Her goal is to determine exactly the value of X, and to do it as fast as possible, i.e. to ask as few questions as possible. Clearly the strategy of the adversary in choosing the sets X will have an inﬂuence on the number of queries necessary to determine the value of X. Intuitively the best strategy is to choose X so that its mass probability is as close as possible to that of X \ X , where X is the set of values that are currently determined as possible for X. The next example illustrates the situation. Example 3. Consider the set X = {a, b, c, d, e, f, g, h}, and assume that the probability is distributed as follows: 1 1 1 p(a) = p(b) = p(c) = p(d) = p(e) = p(f ) = p(g) = p(h) = 4 8 16

216

M.S. Alvim et al.

Fig. 2 illustrates a possible strategy of the adversary. The leafs represent the secrets, i.e. the elements of X . Each internal node is labeled with a set X ⊆ X , representing the query “is X ∈ X ?”. Depending on the answer to the query, the adversary will continue searching in the left or in the right subtree. In this particular case, at each step the left and Fig. 2. Search space the right subtrees have the same probability mass, and as we will see later this means that the strategy is optimal. For a given strategy σ of the adversary, let nσ (x) be the number of questions that are needed to determine the value of X when X = x. For instance, in the example above, we have nσ (a) = 2, and nσ (e) = 4. The expected value of nσ is: 1 1 1 11 Enσ = p(a) nσ (a) + . . . + p(f ) nσ (f ) = 2 × 2 + 2 × 3 + 4 × 4 = 4 8 16 4 Note that if we replace nσ (·) by log2 p(·) in the above deﬁnition of Enσ , we obtain the formula for Shannon entropy (cf. Deﬁnition 2). Coming back to Fig. 2, we wish remark that each element x ∈ X is uniquely associated to the path from the root to the leaf associated to x. If we label the arcs of the tree with 0 (left) and 1 (right), then each x ∈ X can be represented by the binary string relative to the path. The problem of minimizing the expected value of nσ corresponds therefore to that of ﬁnding the optimal coding, i.e. the coding that minimize the expected number of bits needed to represent the elements of X . Shannon entropy is also related to the optimal coding, and more precisely it corresponds to the expected length of the code. Deﬁnition 2 ([28]). The Shannon entropy of a random variable X is deﬁned as H(X) = − p(x) log2 p(x) x∈X

The minimum value H(X) = 0 is obtained when p(·) is concentrated on a single value (i.e. when p(·) is a Dirac measure). The maximum value H(X) = log2 |X | is obtained when p(·) is the uniform distribution. Usually the base of the logarithm is set to be 2 and, correspondingly, the entropy is measured in bits. Proposition 1. H(X) is a lower bound for the expected value of nσ (·), with respect to all possible strategies σ. Note that the lover bound mentioned in the above proposition (i.e. the entropy) is actually achieved in Example 3. Indeed, a simple calculation shows that H(X) = 11 4 . In general the lower bound H(X) can be achieved when for each x ∈ X we have that p(x) is a power of 2, and we can organize the questions as a tree, with the characteristic that each node (question) splits the probability mass in two. We end this section with the deﬁnition of conditional entropy.

Quantitative Information Flow and Applications to Diﬀerential Privacy

Deﬁnition 3. The conditional entropy of X given Y is H(X | Y ) = p(y) H(X | Y = y) y∈Y

where H(X | Y = y) = −

217

(2)

p(x|y) log2 p(x|y)

x∈X

It is possible to prove that 0 ≤ H(X | Y ) ≤ H(X). The minimum value, 0, is obtained when X is completely determined by Y . The maximum value, H(X), is obtained when Y reveals no information about X, i.e. when X and Y are independent. 2.2

R´ enyi Min-Entropy

In this section we illustrate a model of adversary that corresponds to R´enyi min-entropy. This material is based on [29]. Deﬁnition 4. A one-try adversary is a model of adversary in which the adversary can ask exactly one question of the form: “is X = x?”. Her goal is to maximize the probability of guessing the right element in just one single try. Of course, the best strategy for the adversary consists in choosing the x with the maximum probability. Therefore the measure of success of this kind of adversary is maxx∈X p(x). We discuss now how the probability of success is related to the R´enyi minentropy. Let us ﬁrst give a quick overview of the context in which this concept originated. In [27], R´enyi introduced a one-parameter family of entropy measures, intended as a generalization of Shannon entropy. The R´enyi entropy of order α (α > 0, α = 1) of a random variable X is deﬁned as 1 Hα (X) = log2 p(x)α 1−α x∈X

R´enyi’s motivations were of axiomatic nature: Shannon entropy satisﬁes four axioms, namely symmetry, continuity, value 1 on the Bernoulli uniform distribution, and the chain rule1 : H(X ∧ Y ) = H(X | Y ) + H(Y )

(3)

Shannon entropy is also the only function that satisﬁes those axioms. However, if we replace (3) with a weaker property representing the additivity of entropy for independent distributions: H(X · Y ) = H(X) + H(Y ) 1

The original axiom, called the grouping axiom, does not mention the conditional entropy. However it corresponds to the chain rule if the conditional entropy is deﬁned as in (2).

218

M.S. Alvim et al.

then there are more functions satisfying the axioms, among which all those of the R´enyi’s family. Shannon entropy is obtained by taking the limit of Hα as α approaches 1. In fact we can easily prove, using l’Hˆ opital’s rule, that def H1 (X) = lim Hα (X) = − p(x) log2 p(x) α→1

x∈X

We are particularly interested in the limit of Hα as α approaches ∞. This is called min-entropy. It can be proven that def

H∞ (X) =

lim Hα (X) = − log2 max p(x)

α→∞

x∈X

which gives the connection with the model of adversary described in Deﬁnition 4. As for the α-generalization of the conditional entropy, R´enyi did not deﬁne it, and there is no agreement on what it should be. Various researchers, including Cachin [4], have considered the following deﬁnition, based on (2): p(y) Hα (X | Y = y) HαCachin (X | Y ) = y∈Y

which, as α → ∞, becomes Cachin (X | Y ) = − H∞

y∈Y

p(y) log2 max p(x | y) x∈X

An alternative proposal for H∞ (· | ·) came from [11,10], and was again advocated by Smith in his seminal paper [29]: Smith H∞ (X | Y ) = − log2 y∈Y maxx∈X p(x, y) (4) The most interesting version of H∞ (X | Y ), in terms of security, seems to be that of Smith: Indeed, y∈Y maxx∈X p(x, y) represents the expected value of the a posteriori probability of the adversary’s success. The complement of this expected value is also known as probability of error or Bayes risk, and has been used as a measure of information ﬂow also by other authors [6]. Observing Y decreases the probability of error, and consequently we can prove formally Smith that H∞ (X | Y ) ≤ H∞ (X), with equality if X and Y are independent. This inequality will ensure that the leakage is always nonnegative, and it is therefore Smith Cachin another reason to choose H∞ (· | ·): the alternative H∞ (· | ·) does not have this feature. For the sake of simplicity, in the following we will omit the superscript “Smith” and simply write H∞ (· | ·).

3

Information Leakage and Channels

In this section we consider the interpretation of the leakage in computational systems in terms of channels, and we review some results concerning the Shannon and the min-entropy approaches.

Quantitative Information Flow and Applications to Diﬀerential Privacy

219

The main idea is that if we focus on the problem of the leakage of secret information trough public observables, then we can regard a computational system as an information-theoretic channel, where the secrets are the inputs, and the observables are the outputs. The following examples illustrate this precept revisiting the examples in the introduction. Example 4. Consider again the password-checker algorithm deﬁned in Example 1. Assume that the password is 110. Depending on whether the adversary is supposed to be able to detect the number of iterations or not, we have two diﬀerent channels, as illustrated in the ﬁgure below. Note that the program is deterministic, and this is reﬂected by the fact that, in both channels, each input corresponds to only one output.

000 001 010 011 100 101 110 111

Fail

OK

(a)

No iteration count

000 001 010 011 100 101 110 111

(b)

(Fail,1) (Fail,2) (Fail,3) OK

Counting the iterations

Fig. 3. Channels associated with the program in Example 1. The password is 110.

Example 5. Consider the dc-nets of Example 2, and consider a particularly simple case: a net with two nodes only, connected by an arc, as illustrated in Fig. 4(a). The channel, illustrated in Fig. 4(b), takes as input the identity of the agent and the bit that she wants to broadcast2 . On the right-hand side, the binary strings represent the declarations of the nodes. Note that this program is not deterministic. The input (a, 1), for instance, can produce two diﬀerent outcomes: the declaration 10, produced when the coin is 0 (assuming that the ﬁrst bit from left to right is the declaration of a). The other possible declaration, 01, is produced when the coin is 1. Deﬁnition 5. An information-theoretic channel is a triple (X, Y, M ) where X and Y are two random variables representing the input and the output, and M is a matrix (called the channel matrix) which contains the conditional probabilities p(y|x) for each x ∈ X and each y ∈ Y. Example 6. The matrices associated to the channels in Figures 3(a) and 3(b) are given in Tables 2(a) and 2(b), respectively. Note that they contain only 0’s and 1’s. This is characteristic of deterministic channels. 2

One may argue that the bit to be broadcasted should not be considered as a secret, and that having it as input will alter the computation of the leakage. This is a valid observation, but there is no harm in representing this bit as input, because we can still remove its contribution to the leakage when we make the calculation.

220

M.S. Alvim et al.

a (a)

b

coin

(a, 1)

10

(b, 1)

01

(a, 0)

00

(b, 0)

11

(b)

A dc-net with

Channel associated with

the two-nodes dc-net

two nodes

Fig. 4. A simple dc-net and its channel Table 2. Channel matrices for the password-checker algorithm (a) 000 001 010 011 100 101 110 111

(b)

No iter. c.

Fail 1 1 1 1 1 1 0 1

OK 0 0 0 0 0 0 1 0

Counting the iterations

(Fail, 1) (Fail, 2) (Fail, 3) OK 000 001 010 011 100 101 110 111

1 1 1 1 0 0 0 0

0 0 0 0 1 1 0 0

0 0 0 0 0 0 0 1

0 0 0 0 0 0 1 0

Example 7. Consider again Example 2. The channel matrix depends on the coins: whether they are biased, and how much biased they are. Here we consider the simple two-nodes net of Example 5 and two particular situations: the case in which the coins are unbiased (cf. Table 3(a)) and the case in which they are all biased in favor of 0, and more precisely, they give 0 with probability 23 (cf. Table 3(b)). Table 3. Channel matrices for the two-node dc-net of Example 5 (b) (a)

Biased

(p(0) =

Fair coin

10 01 00 11

2 3,

coin

p(1) =

1 3)

10 01 00 11

(a, 1)

1 2

1 2

0 0

(a, 1)

2 3

1 3

0 0

(b, 1)

1 2

1 2

0 0

(b, 1)

1 3

2 3

0 0

(a, 0) 0 0

1 2

1 2

(a, 0) 0 0

2 3

1 3

(b, 0) 0 0

1 2

1 2

(b, 0) 0 0

2 3

1 3

We consider now the deﬁnition of leakage for the two approaches.

Quantitative Information Flow and Applications to Diﬀerential Privacy

3.1

221

Leakage in the Shannon Approach

As explained in the introduction, we regard the leakage as the diﬀerence between the initial uncertainty and the remaining uncertainty after the observation. Since the uncertainty is represented by the entropy, the leakage can be represented by the well-known (information-theoretic) notion of mutual information, which is the diﬀerence between the entropy of the input and the conditional entropy of the input given the output: Deﬁnition 6. The mutual information between A and B is deﬁned as I(X; Y ) = H(X) − H(X | Y ) and it measures the amount of information about X that we gain by observing Y. It can be shown that I(X; Y ) = I(Y ; X) and 0 ≤ I(X; Y ) ≤ H(X). The notion of capacity, deﬁned as C = max I(X; Y ) pX (·)

is often used to represent the worst-case leakage. We end this section with some relevant properties of the Shannon capacity: Proposition 2. The capacity of a channel is 0 if and only if all the rows of the matrix are identical. The capacity is maximum (i.e. equal to H(X)) if and only if each column has only one positive value. 3.2

Leakage in the Min-entropy Approach

Also in the case of min-entropy the leakage can be expressed using a concept analogous to mutual information3 : I∞ (X; Y ) = H∞ (X) − H∞ (X | Y ) and analogously for the capacity: C∞ = max I∞ (X; Y ) pX (·)

This min-entropy variant of the capacity is called min-capacity. Only a weak form of Proposition 2 holds for C∞ , namely, only the if part. Proposition 3. The min-capacity of a channel is 0 if all the rows of the matrix are identical. The capacity is maximum (i.e. equal to H∞ (X)) if each column has only one positive value. 3

Some authors, including Smith, prefer avoiding the notation I∞ and referring to it as “mutual information” because some of the properties of Shannon mutual information are not satisﬁed in the min-entropy case. In particular, symmetry is not satisﬁed, i.e. I∞ (X; Y ) = I∞ (Y ; X) in general.

222

4

M.S. Alvim et al.

Diﬀerential Privacy: Leakage

In this section we discuss the notion of diﬀerential privacy and we show the relation with the notion of leakage. Let us start with some deﬁnitions. Let Ind be a ﬁnite set of individuals that may participate in a database and Val a ﬁnite set of possible values for the attribute of interest of these individuals. In order to capture in a uniform way the presence/absence of an individual in the database, as well as its value, we enrich the set of possible values with an element a representing the absence of the individual. Thus the set of all possible databases is the set X = V Ind , where V = Val ∪ {a}. We will use u and v to denote the cardinalities of Ind and V , |Ind | and |V |, respectively. Hence we have that |X | = v u . A database x can be represented as a u-tuple v0 v1 . . . vu−1 where each vi ∈ V is the value of the corresponding individual. Two databases x, x are adjacent (or neighbors), written x ∼ x , if they diﬀer for the value of exactly one individual. For instance, for u = 3, v0 v1 v2 and v0 w1 v2 , with w1 = v1 , are adjacent. The structure (X , ∼) forms an undirected graph. Intuitively, diﬀerential privacy is based on the idea that a randomized query function provides suﬃcient protection if the ratio between the probabilities of two adjacent databases to give a certain answer is bound by e , for some given > 0. Formally: Deﬁnition 7 ([14]). A randomized function K from X to Z satisﬁes diﬀerential privacy if for all pairs x, x ∈ X , with x ∼ x , and all S ⊆ Z, we have that: Pr [K(x) ∈ S] ≤ e × Pr [K(x ) ∈ S] The above deﬁnition takes into account the possibility that Z is a continuous domain. In our case, since Z is ﬁnite, the probability distribution is discrete, and we can rewrite the property of -d.p. more simply as (using the notation of conditional probabilities, and considering both quotients): 1 Pr [Z = z|X = x] ≤ ≤ e e Pr [Z = z|X = x ]

for all x, x ∈ X with x ∼ x , and all z ∈ Z

where X and Z represent the random variables associated to X and Z, respectively. 4.1

Graph Symmetries

In this section we explore some classes of graphs that allow us to derive a strict correspondence between -d.p. and the a posteriori entropy of the input. Let us ﬁrst recall some basic notions. Given a graph G = (V, ∼), the distance d(v, w) between two vertices v, w ∈ V is the number of edges in a shortest path connecting them. The diameter of G is the maximum distance between any two vertices in V. The degree of a vertex is the number of edges incident to it. G is

Quantitative Information Flow and Applications to Diﬀerential Privacy

223

called regular if every vertex has the same degree. A regular graph with vertices of degree k is called a k-regular graph. An automorphism of G is a permutation σ of the vertex set X , such that for any pair of vertices x, x , if x ∼ x , then σ(x) ∼ σ(x ). If σ is an automorphism, and v a vertex, the orbit of v under σ is the set {v, σ(v), . . . , σ k−1 (v)} where k is the smallest positive integer such that σ k (v) = v. Clearly, the orbits of the vertices under σ deﬁne a partition of V. The following two deﬁnition introduce the classes of graphs that we are interested in. The ﬁrst class is well known in literature. Deﬁnition 8. Given a graph G = (V, ∼), we say that G is distance-regular if there exist integers bi , ci , i = 0, ..., d such that for any two vertices v, w in V with distance i = d(v, w), there are exactly ci neighbors of w in Gi−1 (x) and bi neighbors of v in Gi+1 (x), where Gi (x) is the set of vertices y of G with d(x, y) = i. The next class is a variant of the VT (vertex-transitive) class: Deﬁnition 9. A graph G = (V, ∼) is VT+ (vertex-transitive +) if there are n automorphisms σ0 , σ1 , . . . σn−1 , where n = |V|, such that, for every vertex v ∈ V, we have that {σi (v) | 0 ≤ i ≤ n − 1} = V. In particular, the graphs for which there exists an automorphism σ which induces only one orbit are VT+ : in fact it is suﬃcient to deﬁne σi = σ i for all i from 0 to n − 1. From graph theory we know that neither of the two classes subsumes the other. They have however a non-empty intersection, which contains in particular all the structures of the form (V Ind , ∼), i.e. the database domains. Proposition 4. The structure (X , ∼) = (V Ind , ∼) is both a distance-regular graph and a VT+ graph. The two symmetry classes deﬁned above, distance-regular and VT+ , will be used in the next section to transform a generic channel matrix into a matrix with a symmetric structure, while preserving the a posteriori min entropy and the -d.p.. This is the core of our technique to establish the relation between diﬀerential privacy and quantitive information ﬂow, depending on the structure induced by the database adjacency relation. 4.2

Deriving the Relation between Diﬀerential Privacy and QIF on the Basis of the Graph Structure

This section discusses a general technique for determining the relation between -diﬀerential privacy and leakage, and between -diﬀerential privacy and utility, depending on the graph structure induced by ∼ and f . The idea is to use the symmetries of the graph structure to transform the channel matrix into an equivalent matrix with certain regularities, which allow to establish the link between -diﬀerential privacy and the a posteriori min entropy.

224

M.S. Alvim et al.

Let us illustrate brieﬂy this transformation. Consider a channel whose matrix M has at least as many columns as rows. First, we transform M into a matrix M in which each of the ﬁrst n columns has a maximum in the diagonal, and the remaining columns are all 0’s. Second, under the assumption that the input domain is distance-regular or VT+ , we transform M into a matrix M whose diagonal elements are all the same, and coincide with the maximum element of M , which we denote here by maxM . We are now going to present formally our the technique. Let us ﬁrst ﬁx some notation: In the rest of this section we consider channels with input A and output B, with carriers A and B respectively, and we assume that the probability distribution of A is uniform. Furthermore, we assume that |A| = n ≤ |B| = m. We also assume an adjacency relation ∼ on A, i.e. that (A, ∼) is an undirected graph structure. With a slight abuse of notation, we will also write i ∼ h when i and h are associated to adjacent elements of A, and we will write d(i, h) to denote the distance between the elements of A associated to i and h. We note that a channel matrix M satisﬁes -d.p. if for each column j and for each pair of rows i and h such that i ∼ h we have that: 1 Mi,j ≤ ≤ e . e Mh,j M The a posteriori entropy of a channel with matrix M will be denoted by H∞ (A|B).

Theorem 1. Consider a matrix M , and let r be a row of M . Assume that (A, ∼) is either distance-regular or VT+ , and that M satisﬁes -d.p. For each distance d from 0 to the diameter of (A, ∼), let nd be the number of nodes j at distance d from r. Then we have that: 1 M H∞ (A|B) ≥ − log2 nd e d

(5)

d

Note that this bound is tight, in the sense that we can build a matrix for which (5) holds with equality. In the next section, we will see how to use this theorem for establishing a bound on the leakage and on the utility. 4.3

Application to Leakage

As already hinted in the introduction, we can regard K as a channel with input X and output Z. From Proposition 4 we know that (X , ∼) is both distance-regular and VT+ , we can therefore apply Theorem 1. Let us ﬁx a particular database x ∈ X . The number of databases at distance d from x is u nd = (v − 1)d (6) d

Quantitative Information Flow and Applications to Diﬀerential Privacy

225

where u = |Ind | and v = V . In fact, recall that x can be represented as a u-tuple with values in V . We need to select d individuals in the u-tuple and then change their values, and each of them can be changed in v − 1 diﬀerent ways. Using the nd from (6) in Theorem 1 we obtain a binomial expansion in the denominator, namely: M H∞ (X|Z) ≥ − log2

u d=0

u d

1 (v − 1)d

e

(u−d)

= −u log2

e v − 1 + e

e u

which gives the following result: Theorem 2. If K satisﬁes -d.p., then for the uniform input distribution the information leakage is bound from above as follows: I∞ (X; Z) ≤ u log2

5

v e v − 1 + e

Diﬀerential Privacy: Utility

We turn now our attention to the issue of utility. We focus on the case in which K is oblivious, which means that it depends only on the (exact) answer to the query, i.e. on the value of f (x), and not on x. An oblivious function can be decomposed in the concatenation of two channels, one representing the function f , and the other representing the randomization mechanism H added as output perturbation. The situation is illustrated in Figure 5. The standard way to deﬁne utility is by means of guess and gain functions. The functionality of the ﬁrst is guess : Z → Y, and it represents the user’s strategy to retrieve the correct answer form the reported one. The functionality Utility

X dataset

Y

f

real answer

query

H randomization mechanism

Z reported answer

K (-diff. priv. randomized function) Leakage Fig. 5. Schema of an oblivious randomized function

226

M.S. Alvim et al.

of the latter is gain : Y × Y → R. the value gain(y, y ) represents the reward for guessing the answer y when the correct answer is y . The utility U can then be deﬁned as the expected gain: U(Y, Z) = p(y, z) gain(guess(z), y) y,z

We focus here on the so-called binary gain function, which is deﬁned as 1 if y = y gain(y, y ) = 0 otherwise This kind of function represents the case in which there is no reason to prefer an answer over the other, except if it is the right answer. More precisely, we get a gain if and only if we guess the right answer. If the gain function is binary, and the guess function represents the user’s best strategy, i.e. it is chosen to optimize utility, then there is a well-known correspondence between U and the Bayes risk / the a posteriori min entropy. Such correspondence is expressed by the following proposition: Proposition 5. Assume that gain is binary and guess is optimal. Then: U(Y, Z) = max(p(z|y) p(y)) = 2−H∞ (Y |Z) z

y

In order to analyze the implications of the -d.p. requirement on the utility, we need to consider the structure that the adjacency relation induces on Y. Let us deﬁne ∼ on Y as follows: y ∼ y if there are x, x ∈ X such that y = f (x), y = f (x ), and x ∼ x,. Note that K satisﬁes -d.p. if and only if H satisﬁes -d.p. If (Y, ∼) is distance-regular or VT+ , then we can apply Theorem 1 to ﬁnd a bound on the utility. In the following, we assume that the distribution of Y is uniform. Theorem 3. Consider a randomized mechanism H, and let y be an element of Y. Assume that (Y, ∼) is either distance-regular or VT+ and that H satisﬁes -d.p. For each distance d from 0 to the diameter of (Y, ∼), let nd be the number of nodes y at distance d from y. Then we have that: 1 U(Y, Z) ≤ nd d

(7)

e d

The above bound is tight, in the sense that (provided (Y, ∼) is distance-regular or VT+ ) we can construct a mechanism H which satisﬁes (7) with equality. More precisely, deﬁne 1 c = nd e d d

Quantitative Information Flow and Applications to Diﬀerential Privacy

227

Then deﬁne H (here identiﬁed with its channel matrix for simplicity) as follows: Hi,j =

c e

d(i,j)

(8)

Theorem 4. Assume (Y, ∼) is distance-regular or VT+ . Then the matrix H deﬁned in (8) satisﬁes -d.p. and has maximal utility: 1 U(Y, Z) = nd e d d

Note that we can always deﬁne H as in (8): the matrix so deﬁned will be a legal channel matrix, and it will satisfy -d.p.. However, if (Y, ∼) is neither distanceregular nor VT+ , then the utility of such H is not necessarily optimal. We end this section with an example (borrowed from [1]) to illustrate our technique. Example 8. Consider a database with electoral information where each row corresponds to a voter and contains the following three ﬁelds: – Id : a unique (anonymized) identiﬁer assigned to each voter; – City: the name of the city where the user voted; – Candidate: the name of the candidate the user voted for. Consider the query “What is the city with the greatest number of votes for a given candidate cand ?”. For such a query the binary utility function is the natural choice: only the right city gives some gain, and all wrong answers are equally bad. It is easy to see that every two answers are neighbors, i.e. the graph structure of the answers is a clique. Let us consider the scenario where City = {A, B, C, D, E, F } and assume for simplicity that there is a unique answer for the query, i.e., there are no two cities with exactly the same number of individuals voting for candidate cand . Table 4 shows two alternative mechanisms providing -diﬀerential privacy (with = log 2). The ﬁrst one, M1 , is based on the truncated geometric mechanism method used in [16] for counting queries (here extended to the case where every pair of answers is neighbor). The second mechanism, M2 , is obtained by applying the deﬁnition (8). From Theorem 4 we know that for the uniform input distribution M2 gives optimal utility. For the uniform input distribution, it is easy to see that U(M1 ) = 0.2242 < 0.2857 = U(M2 ). Even for non-uniform distributions, our mechanism still provides better utility. For instance, for p(A) = p(F ) = 1/10 and p(B) = p(C) = p(D) = P (E) = 1/5, we have U(M1 ) = 0.2412 < 0.2857 = U(M2 ). This is not too surprising: the geometric mechanism, as well as the Laplacian mechanism proposed by Dwork, perform very well when the domain of answers is provided with a metric and the utility function is not binary4 . It also works well when 4

In the metric case the gain function can take into account the proximity of the reported answer to the real one, the idea being that a close answer, even if wrong, is better than a distant one.

228

M.S. Alvim et al.

Table 4. Mechanisms for the city with higher number of votes for candidate cand (a) M1 : truncated geometric mechanism In/Out A B C D E F

A 0.535 0.465 0.405 0.353 0.307 0.267

B 0.060 0.069 0.060 0.053 0.046 0.040

C 0.052 0.060 0.069 0.060 0.053 0.046

D 0.046 0.053 0.060 0.069 0.060 0.052

E 0.040 0.046 0.053 0.060 0.069 0.060

F 0.267 0.307 0.353 0.405 0.465 0.535

(b) M2 : our mechanism In/Out A B C D E F

A 2/7 1/7 1/7 1/7 1/7 1/7

B 1/7 2/7 1/7 1/7 1/7 1/7

C 1/7 1/7 2/7 1/7 1/7 1/7

D 1/7 1/7 1/7 2/7 1/7 1/7

E 1/7 1/7 1/7 1/7 2/7 1/7

F 1/7 1/7 1/7 1/7 1/7 2/7

(Y, ∼) has low connectivity, in particular in the cases of a ring and of a line. But in this example, we are not in these cases, because we are considering binary gain functions and high connectivity.

6

Related Work

As far as we know, the ﬁrst work to investigate the relation between diﬀerential privacy and information-theoretic leakage for an individual was [2]. In this work, a channel is relative to a given database x, and the channel inputs are all possible databases adjacent to x. Two bounds on leakage were presented, one for teh R´enyi min entropy, and one for Shannon entropy. Barthe and K¨opf [3] were the ﬁrst to investigates the (more challenging) connection between diﬀerential privacy and the R´enyi min-entropy leakage for the entire universe of possible databases. They consider the “end-to-end diﬀerentially private mechanisms”, which correspond to what we call K in our paper, and propose, like we do, to interpret them as information-theoretic channels. They provide a bound for the leakage, but point out that it is not tight in general, and show that there cannot be a domain-independent bound, by proving that for any number of individual u the optimal bound must be at least a certain expression f (u, ). Finally, they show that the question of providing optimal upper bounds for the leakage of -diﬀerentially private randomized functions in terms of rational functions of is decidable, and leave the actual function as an open question. In our work we used rather diﬀerent techniques and found (independently) the same function f (u, ) (the bound in Theorem 1), but we actually proved that f (u, ) is the optimal bound5 . Another diﬀerence is that [3] captures the case in which the focus of diﬀerential privacy is on hiding participation of individuals in a database. In our work, we consider both the participation and the values of the participants. Clarkson and Schneider also considered diﬀerential privacy as a case study of their proposal for quantiﬁcation of integrity [9]. There, the authors analyze 5

When discussing our result with Barthe and K¨ opf, they said that they also conjectured that f (u, ) is the optimal bound.

Quantitative Information Flow and Applications to Diﬀerential Privacy

229

database privacy conditions from the literature (such as diﬀerential privacy, kanonymity, and l-diversity) using their framework for utility quantiﬁcation. In particular, they study the relationship between diﬀerential privacy and a notion of leakage (which is diﬀerent from ours - in particular their deﬁnition is based on Shannon entropy) and they provide a tight bound on leakage. Heusser and Malacaria [17] were among the ﬁrst to explore the application of information-theoretic concepts to databases queries. They proposed to model database queries as programs, which allows for statical analysis of the information leaked by the query. However [17] did not attempt to relate information leakage to diﬀerential privacy. In [16] the authors aim at obtaining optimal-utility randomization mechanisms while preserving diﬀerential privacy. The authors propose adding noise to the output of the query according to the geometric mechanism. Their framework is very interesting in the sense it provides a general deﬁnition of utility for a mechanism M that captures any possible side information and preference (deﬁned as a loss function) the users of M may have. They prove that the geometric mechanism is optimal in the particular case of counting queries. Our results do not restrict to counting queries, but on the other hand we only consider the case of binary loss function.

References 1. Alvim, M.S., Andr´es, M.E., Chatzikokolakis, K., Degano, P., Palamidessi, C.: Differential privacy: on the trade-oﬀ between utility and information leakage. Technical report (2011), http://hal.inria.fr/inria-00580122/en/ 2. Alvim, M.S., Chatzikokolakis, K., Degano, P., Palamidessi, C.: Diﬀerential privacy versus quantitative information ﬂow. Technical report (2010) 3. Barthe, G., K¨ opf, B.: Information-theoretic bounds for diﬀerentially private mechanisms. In: Proc. of CSF (to appear, 2011) 4. Cachin, C.: Entropy Measures and Unconditional Security in Cryptography. PhD thesis (1997) 5. Chatzikokolakis, K., Palamidessi, C., Panangaden, P.: Anonymity protocols as noisy channels. Inf. and Comp. 206(2-4), 378–401 (2008) 6. Chatzikokolakis, K., Palamidessi, C., Panangaden, P.: On the Bayes risk in information-hiding protocols. J. of Comp. Security 16(5), 531–571 (2008) 7. Chaum, D.: The dining cryptographers problem: Unconditional sender and recipient untraceability. Journal of Cryptology 1, 65–75 (1988) 8. Clark, D., Hunt, S., Malacaria, P.: Quantitative information ﬂow, relations and polymorphic types. J. of Logic and Computation 18(2), 181–199 (2005) 9. Clarkson, M.R., Schneider, F.B.: Quantiﬁcation of integrity, Tech. Rep. (2011), http://hdl.handle.net/1813/22012 10. Dodis, Y., Ostrovsky, R., Reyzin, L., Smith, A.: Fuzzy extractors: How to generate strong keys from biometrics and other noisy data. SIAM Journal of Computing 38(1), 97–139 (2008) 11. Dodis, Y., Reyzin, L., Smith, A.: Fuzzy extractors: How to generate strong keys from biometrics and other noisy data. In: Cachin, C., Camenisch, J.L. (eds.) EUROCRYPT 2004. LNCS, vol. 3027, pp. 523–540. Springer, Heidelberg (2004)

230

M.S. Alvim et al.

12. Dwork, C.: Diﬀerential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006) 13. Dwork, C.: Diﬀerential privacy in new settings. In: Proc. of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, Austin, Texas, USA, January 17-19, pp. 174–183. SIAM, Philadelphia (2010) 14. Dwork, C.: A ﬁrm foundation for private data analysis. Communications of the ACM 54(1), 86–96 (2011) 15. Dwork, C., Lei, J.: Diﬀerential privacy and robust statistics. In: Proc. of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31-June 2, pp. 371–380. ACM, New York (2009) 16. Ghosh, A., Roughgarden, T., Sundararajan, M.: Universally utility-maximizing privacy mechanisms. In: Proc. of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, pp. 351–360. ACM, New York (2009) 17. Heusser, J., Malacaria, P.: Applied quantitative information ﬂow and statistical databases. In: Degano, P., Guttman, J.D. (eds.) FAST 2009. LNCS, vol. 5983, pp. 96–110. Springer, Heidelberg (2010) 18. Joinson, A.N.: Self-disclosure in computer-mediated communication: The role of self-awareness and visual anonymity. Eur. J. Soc. Psychol. 31(2), 177–192 (2001) 19. K¨ opf, B., Basin, D.A.: An information-theoretic model for adaptive side-channel attacks. In: Proc. of CCS, pp. 286–296. ACM, New York (2007) 20. K¨ opf, B., Basin, D.A.: An information-theoretic model for adaptive side-channel attacks. In: Proc. of CCS, pp. 286–296. ACM, New York (2007) 21. Malacaria, P.: Assessing security threats of looping constructs. In: Proc. of POPL, pp. 225–235. ACM, New York (2007) 22. Malacaria, P., Chen, H.: Lagrange multipliers and maximum information leakage in diﬀerent observational models. In: Proc. of PLAS, pp. 135–146. ACM, New York (2008) 23. Massey: Guessing and entropy. In: Proc. of ISIT, p. 204. IEEE, Los Alamitos (1994) 24. Moskowitz, I.S., Newman, R.E., Crepeau, D.P., Miller, A.R.: Covert channels and anonymizing networks. In: Proc. of PES, pp. 79–88. ACM, New York (2003) 25. Moskowitz, I.S., Newman, R.E., Syverson, P.F.: Quasi-anonymous channels. In: Proc. of CNIS, pp. 126–131. IASTED (2003) 26. Pliam, J.O.: On the incomparability of entropy and marginal guesswork in bruteforce attacks. In: Roy, B., Okamoto, E. (eds.) INDOCRYPT 2000. LNCS, vol. 1977, pp. 67–79. Springer, Heidelberg (2000) 27. R´enyi, A.: On Measures of Entropy and Information. In: Proc. of the 4th Berkeley Symposium on Mathematics, Statistics, and Probability, pp. 547–561 (1961) 28. Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423, 625–656 (1948) 29. Smith, G.: On the foundations of quantitative information ﬂow. In: de Alfaro, L. (ed.) FOSSACS 2009. LNCS, vol. 5504, pp. 288–302. Springer, Heidelberg (2009) 30. Zhu, Y., Bettati, R.: Anonymity vs. information leakage in anonymity systems. In: Proc. of ICDCS, pp. 514–524. IEEE, Los Alamitos (2005)

Risk Analysis of Changing and Evolving Systems Using CORAS Mass Soldal Lund1 , Bjørnar Solhaug1 , and Ketil Stølen1,2 1

SINTEF ICT, P.O. box 124 Blindern, 0314 Oslo, Norway Department of Informatics, University of Oslo, Norway {mass.s.lund,bjornar.solhaug,ketil.stolen}@sintef.no 2

Abstract. Risk analysis is the identiﬁcation and documentation of risks with respect to an organisation or a target system. Established risk analysis methods and guidelines typically focus on a particular system conﬁguration at a particular point in time. The resulting risk picture is then valid only at that point in time and under the assumptions made when it was derived. However, systems and their environments tend to change and evolve over time. In order to appropriately handle change, risk analysis must be supported with specialised techniques and guidelines for modelling, analysing and reasoning about changing risks. In this paper we introduce general techniques and guidelines for managing risk in changing systems, and then instantiate these in the CORAS approach to model-driven risk analysis. The approach is demonstrated by a practical example based on a case study from the Air Traﬃc Management (ATM) domain. Keywords: Risk management, risk analysis, change management, Air Traﬃc Management, security.

1

Introduction

Risk management is coordinated activities to direct and control an organisation with regard to risk [15]. Risk analysis is a core part of risk management, and involves the identiﬁcation and documentation of risks with respect to the organisation or the target system in question. When deriving the risk picture for the target of analysis, established risk analysis methods and guidelines typically focus on a particular system conﬁguration at a particular point in time. The resulting risk picture is then valid only at that point in time and under the assumptions made when it was derived. However, systems and their environments tend to change and evolve over time. In order to appropriately handle change, each of the risk management activities must be supported with specialised techniques and guidelines. Figure 1 is adapted from the ISO 31000 risk management standard [15] and illustrates the seven activities of the risk management process. The ﬁve activities in the middle constitute the core activities of a risk analysis, and are described as follows: A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 231–274, 2011. c Springer-Verlag Berlin Heidelberg 2011

232

M.S. Lund, B. Solhaug, and K. Stølen

Estimate risks

Monitor and review

Identify risks Risk assessment

Communicate and consult

Establish the context

Evaluate risks

Treat risks

Fig. 1. Risk management process

– Establish the context is to deﬁne the external and internal parameters to be accounted for when managing risk, and to set the scope and risk criteria for the risk management policy. – Risk identification is to ﬁnd, recognise and describe risks. – Risk estimation is to comprehend the nature of risk and to determine the risk level. – Risk evaluation is to compare the risk estimation results with the risk criteria to determine whether the risk and its magnitude are acceptable or tolerable. – Risk treatment is the process of modifying the risk. The remaining two activities are continuous activities of the overall risk management process, and are described as follows: – Communicate and consult are the continual and iterative processes an organisation conducts to provide, share or obtain information, and to engage in dialogue with stakeholders about risk management. – Monitoring involves the continuous checking, supervising and critically observing the risk status in order to identify changes from the performance level required or expected, whereas review focuses on the activity undertaken to determine the suitability, adequacy and eﬀectiveness of the subject matter necessary to achieve established objectives. When targeting changing and evolving systems the main challenge is to ensure that the analysis results are kept valid under change. A straightforward way to

Risk Analysis of Changing and Evolving Systems Using CORAS

233

ensure this is to conduct a full risk analysis from scratch when faced with a potentially risk relevant change. Needless to say, such a strategy is not to prefer as it is time and resource consuming, and as it often implies conducting exactly the same analysis again to the extent that the risk picture is persistent. Instead, a customised analysis method for changing systems should provide guidelines and techniques for how to systematically trace the relevant changes from the target to the risks, and thereby for how to update only the part of the risk picture that is aﬀected by the changes. How to adequately deal with changes in a risk analysis depends, however, on the nature of the changes in each speciﬁc case. We have for this reason identiﬁed three diﬀerent perspectives on change, each with its speciﬁc methodological needs. These are the maintenance perspective, the before-after perspective, and the continuous evolution perspective. The approach of this paper is to take as starting point the risk management principles and guidelines of the ISO 31000 standard and generalise these to the setting of changing and evolving systems. This standard is quite general, and most of the established risk management methodologies can be understood as instantiations of the activities it prescribes. The objective of such an approach is to understand and explain how to deal with changing risks without restricting to a speciﬁc methodology. ISO 31000 comes with no guidance on risk analysis techniques. While a risk analysis method provides methodological advice on how to carry out the various activities of risk management, a risk analysis technique is more narrow in the sense that it addresses only some aspects of the risk analysis process. A risk analysis method typically makes use of one or more risk analysis techniques. Risk modelling refers to techniques that are used to support the activities of risk identiﬁcation and risk estimation. In this paper we take the risk graphs of [3], which can be understood as a common abstraction of diﬀerent risk modelling techniques, as the starting point for generalising risk modelling to the setting of changing systems. The idea is, as is also demonstrated in the paper, that the extensions we make to risk graphs carry over to other risk modelling techniques. In order to demonstrate and validate our approach, we present a case study from the domain of Air Traﬃc Management (ATM). The speciﬁc risk analysis method applied in this case study is CORAS [20] used as an instantiation of the general approach to risk analysis of changing systems. CORAS is a modeldriven approach to risk analysis that consists of a method, a language and a tool to support the risk analysis process. The ATM case study thus demonstrates a concrete instantiation of the approach in this paper in both the CORAS method and the CORAS language. The structure of the paper is as follows. In Sect. 2 we make a classiﬁcation of change by introducing the perspectives of maintenance, before-after and continuous evolution. As the methodological needs are strongly situation dependent, we discuss these in relation to the mentioned perspectives before we present our approach to risk analysis of changing and evolving systems in Sect. 3. Section 4 introduces the formal foundation for our approach by presenting the syntax and

234

M.S. Lund, B. Solhaug, and K. Stølen

semantics of risks graphs and their generalisation to the setting of changing and evolving risks. The section moreover presents a calculus with rules for reasoning about risk graphs, and a set of guidelines for how to do consistency checking of risk graphs. In Sect. 5 we instantiate our approach with CORAS, thus generalising CORAS to the setting of changing and evolving systems. In Sect. 6 we demonstrate our approach by applying this generalisation of CORAS for changing and evolving systems to the practical example of the ATM case study. Finally, we present related work in Sect. 7 before we conclude in Sect. 8.

2

Classification of Change

The appropriate way of handling change when managing, modelling, analysing and documenting risks depends greatly on the context and on the kind of changes we face [19]. For example, do the changes result from maintenance or from bigger, planned changes? Do the changes comprise a transition from one stable state of the target of analysis to another stable state, or do they reﬂect the continuous evolution of a target designed to change over time? Do the changes occur in the target system or in the environment of the target? The answers to such questions have implications on how to methodologically approach the problem. In the following we present three main classes of change by introducing three respective perspectives on change: maintenance, before-after, and continuous evolution. After this presentation, the remainder of the paper focuses on the before-after perspective. This is the perspective of the case study from which the practical example is provided, and is perhaps the perspective that most clearly illustrates the need for specialised methods when dealing with risk analysis of changing and evolving systems. 2.1

The Maintenance Perspective

The changes we address from the maintenance perspective are those that accumulate over time and gradually make any previous risk analysis results invalid. The challenge for the risk analysts can be described by the following example scenario: The risk analysts conducted a risk analysis three years ago and are now requested by the same client to analyse the same target system anew so as to update the risk picture and reﬂect any changes to the target or its environment, thus restoring the validity of the analysis. Typical changes in this setting can be bug ﬁxes and security patches, an increase in network traﬃc, and increase in the number of attacks. For such changes, the risk picture remains more or less the same, but risk values may have changes such that previously acceptable risks are now unacceptable, or vice versa. The objective is then to maintain the documentation of the previous risk analysis by conducting an update.

Risk Analysis of Changing and Evolving Systems Using CORAS

235

Updates Old target

Current target

Old risks

Current risks

Old risk picture

Current risk picture Risk analyst

Fig. 2. The maintenance perspective

The illustration in Fig. 2 shows the principles by which the risk analysts conduct the analysis from the maintenance perspective. Assuming that we have available the documentation of the previous risk analysis, as well as the description of the old and current target of analysis, we start by identifying the changes that have occurred. We then use the relevant changes as input when deriving the current risk picture based on the previous risk analysis results. Methodologically, the main challenge is how to reuse the old risk analysis results and not conducting a full risk analysis from scratch. This requires a systematic way of identifying the updates of the target and which parts of the risk picture that are aﬀected such that the risk picture can be updated without addressing and spending eﬀort on the unaﬀected parts. 2.2

The Before-After Perspective

The changes we address from the before-after perspective are those that are planned or anticipated, and perhaps of a radical kind. The challenge for the risk analysts can be described by the following scenario: The risk analysts are asked to predict the eﬀect on the risk picture of implementing the changes to the target of analysis. Examples of changes in the before-after perspective are the rolling out of a new system, or making major organisational changes such as implementing a merger agreement between two companies. We must thus understand the current risk picture and how this will change to the future risk picture as a consequence of implementing the changes to the target of analysis. As the planned or anticipated process of change may involve risks in itself, the before-after perspective may require a risk analysis of this process. The illustration in Fig. 3 shows the principles by which the risk analysts conduct the analysis from the before-after perspective. Assuming that we have available the description of the current target of analysis as well as the change

236

M.S. Lund, B. Solhaug, and K. Stølen Planned changes Current target

Current risks

Future target

Risks due to change process

Future risks

Risk picture

Risk analyst

Fig. 3. The before-after perspective

process to bring the target from the current to the future state, we can work out a coherent risk picture for the future target and for the change process. Methodologically, the main challenges are how to obtain and present a risk picture that describes the current and the future risks, as well as the impact of the change process itself. This requires an approach to making a description of the target “as-is” and a description of the target “to-be”, describing the process of change in suﬃcient detail, identifying and documenting the current and future risks without doing double work, and identifying risks due to the change process. 2.3

The Continuous Evolution Perspective

The changes we address from the continuous evolution perspective are predictable and gradual evolutions that can be described as functions of time. Such predictions can be based on well-founded forecasts or planned developments. The challenge for the risk analysts can be described by the following scenario: The risk analysts are asked to predict the future evolutions of risk, which mandates that they conduct a risk analysis that establish a dynamic risk picture that reﬂects the expected evolution of the target. Examples of predictable changes are the slow increase in the number of components working in parallel or gradually including more sites in a system. Examples of well-founded forecasts are the expected steady increase of end-users, adversary attacks, and annual turnover.

Risk Analysis of Changing and Evolving Systems Using CORAS

Target at time t0

Evolution

Risks at time t0

Target at time t1

Evolution

…

Risks at time t1

Target at time tn

237

…

Risks at time tn

Risk picture

Risk analyst

Fig. 4. The continuous evolution perspective

The illustration in Fig. 4 shows the principles by which the risk analysts conduct the analysis from the continuous evolution perspective. Assuming that we have available a description of the target of analysis as a function of time, such that we can derive the target at any given point in time, we use this as input to the risk analysis. Understanding how the target and its environment evolve, we seek to work out a risk picture as a function of time that shows how risks evolve. Methodologically, the main challenges are how to identify evolving risks and present them in a dynamic risk picture. This requires us to generalise the target description to capture evolution and to identify and likewise generalise the risks aﬀected by evolution.

3

Our Approach

As explained in Sect. 2 this paper focuses on the before-after perspective. Conﬁning to this perspective, we present in this section our approach to risk analysis of changing systems by adequately generalising the risk management process of ISO 31000. Our main concern is the ﬁve activities in the middle of Fig. 1 as this comprise the actual risk analysis activities. We moreover focus on what is needed for handling change, referring to the ISO 31000 standard [15] for the principles and guidelines for risk management in the traditional setting. The reader is also referred to the practical example in Sect. 6 for further details about the various activities and tasks.

238

M.S. Lund, B. Solhaug, and K. Stølen

In the following we go through each of the ﬁve activities of the risk analysis process. Subsequently, in Sect. 4, we introduce language support for this process by a generalisation of risk graphs to a risk graph notation with the expressiveness to explicitly model changing risks in the before-after perspective. 3.1

Context Establishment

The activity of establishing the context involves making the premises for the subsequent risk analysis. Establishing the context should result in a target description, which is the documentation of all the information that serves as the input to and basis for the risk analysis, and a set of risk evaluation criteria. Any information about the target of analysis that is relevant for the risk analysis and its outcome needs to be included in the target description. This means that any risk relevant change in the target of analysis must be reﬂected by changes to the target description. Establishing the context of the analysis also includes articulating the goals and objectives of the analysis and deciding its focus and scope. In particular, we need to determine precisely what the target of analysis is and what the assets that need to be protected are. The risk analysis is conducted with respect to the identiﬁed assets, and it is only by precisely understanding what the assets are that we can conduct a risk analysis that meets the overall goals and objectives of the analysis. Establishing the Target Description. The target description includes the documentation of the target of analysis, the focus and scope of the analysis, the environment of the target, the assumptions of the analysis, the parties and assets of the analysis, and the context of the analysis. The UML [22] class diagram of Fig. 5 gives an overview of the elements of a target description. In a risk analysis, the notions of party, asset and risk are closely related. A party is an organisation, company, person, group or other body on whose behalf

Assumption *

Context *

1 1

1 Environment

Party 1..*

1

1

1 Target of analysis

1 1..*

* 1..*

*

Asset within

1 Scope

1 part of

Focus

Fig. 5. Target description

Risk Analysis of Changing and Evolving Systems Using CORAS

239

the risk analysis is conducted. An asset is something to which a party assigns value and hence for which the party requires protection. A risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. This means that if there is no party, it makes no sense to speak of assets. And without assets there can moreover be no risks. The target of the analysis is the system, organisation, enterprise, or the like that is the subject of a risk analysis. The focus of the analysis is the main issue or central area of attention. The focus is within the scope of the analysis, which the extent or range of the analysis. The scope deﬁnes the border of the analysis, i.e. what is held inside and what is held outside of the analysis. The environment of the target is the surrounding things of relevance that may aﬀect or interact with the target; in the most general case the environment is the rest of the world. The assumptions are something we take as granted or accept as true, although it may not be so; the results of a risk analysis are valid only under the assumptions. The context of the analysis is the premises for and the background of the analysis. This includes the purposes of the analysis and to whom the analysis is addressed. For a given target of analysis we assume that we can use a traditional risk analysis method to conduct the context establishment while not taking into account changes. The additional task of establishing the changes in the context includes making a description of the target of analysis when the changes have been taken into account. This extended target description should include both a description of the changes and the result of implementing them, although it should be possible to deduce the latter from the current target description and the description of the changes. Changes that concern the target of analysis can be new or diﬀerent work processes, the introduction of new services or applications, changes in users or roles, etc. These may imply changes the risk picture, and therefore imply the need for a new risk analysis of parts of the target. There may, however, also be changes in parties, changes in assets or asset priorities, changes in the environment or in the assumptions, changes in the focus or scope, and so on, that must be documented in the description of the changing target of analysis. Establishing the Risk Evaluation Criteria. The risk evaluation criteria are a speciﬁcation of the risk levels that the parties of the risk analysis are willing to accept. The criteria will later be used to evaluate the signiﬁcance of risk, and should reﬂect the values, objectives and resources of the parties in question. When we are deciding the risk evaluation criteria we need to take into account not only the views of the parties, but also the nature of the assets, the types of consequences and how they should me measured and described. Speciﬁcally, we need for each asset to deﬁne a consequence scale where each consequence value describes a level of impact of an unwanted incident on an asset in terms of harm or reduced asset value. We furthermore need to deﬁne a likelihood scale, the values of which will be used to describe the frequency or probability of unwanted incidents and threat scenarios to occur. Recall that a risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. The risk level is the level or value of a risk as derived from

240

M.S. Lund, B. Solhaug, and K. Stølen

its likelihood and consequence. The risk level of each combination of a likelihood and a consequence is calculated by a risk function. Since it is only the party of a given asset that can determine the severity of a risk, it is the party that must determine an adequate risk function. Essentially, the risk evaluation criteria are a speciﬁcation of the level at which risks become unacceptable. Some changes are of a kind that does not aﬀect the assets or other values, objectives or resources of the parties. In this case, there is no need to reconsider the risk evaluation criteria. For other changes, the value or priorities of assets may change, new assets may arise, the parties may become more or less risk averse, and so forth. In that case we need a new iteration on establishing and documenting the risk evaluation criteria. 3.2

Risk Identification

The risk identiﬁcation involves identifying and documenting unwanted incidents with respect to the identiﬁed assets, as well as identifying vulnerabilities and sources of risk. The risk identiﬁcation should involve people with appropriate expert knowledge about the target of analysis. The activity of extracting the relevant information relies on techniques and tools for identifying risk relevant information, for structuring the information in a sensible way, and for adequately documenting the information. While the documentation of the risks that are identiﬁed should serve as a means for reporting the ﬁndings to the relevant stakeholders, it should at the same time facilitate the subsequent estimation and evaluation. In this section we focus on the methodological guidelines for risk identiﬁcation of changing and evolving systems. In Sect. 4 we discuss more closely the required risk modelling techniques. As mentioned above, it is the target description that serves as the input to and basis for the subsequent risk analysis. The objective of the risk identiﬁcation is to identify and document the changing risks given the description of the changing target. A main principle is that to the extent that we have identiﬁed and documented the risks for the target of analysis before changes, we only address the parts of the target that are aﬀected by the change when identifying the changing risks. This means that when considering the target description without the changes, the risk identiﬁcation and the risk documentation are conducted according to traditional risk analysis methods. When this is completed we need to update the resulting risk documentation according to the changes. This is conducted by making a walkthrough of the current target description and risk documentation. The risks that are persistent under change can immediately be included in the new documentation with no further investigation. The risks that may be aﬀected by change need to be considered again: Previous scenarios, incidents, etc. may change, new may arise, and others may disappear.

Risk Analysis of Changing and Evolving Systems Using CORAS

241

The methodological guidelines for risk identiﬁcation of a changing target of analysis are summarised as follows: 1. Identify and document risks by using as input the target description before changes have been taken into account. 2. Establish and document the traceability between the target description before change and the risk documentation resulting from the previous step. 3. Based on the traceability and the description of the changed target, identify the parts of the risk documentation that are persistent under change. 4. Conduct the risk identiﬁcation of the changed target only with respect to the parts of the target and the risks that are aﬀected by the change. In order to conduct the activity of risk identiﬁcation of a changing target of analysis, there is thus a need for techniques for identiﬁcation and modelling of risks that change, as well as techniques for establishing and modelling traceability between target description and risk models, both which are topics of Sect. 4. 3.3

Risk Estimation

The objective of the risk estimation is to establish an understanding of the severity of identiﬁed risks, and to provide the basis for the subsequent risk evaluation and risk treatment. By considering the causes and sources of risk, the risk estimation amounts to estimating and documenting the likelihoods and consequences of the identiﬁed unwanted incidents. It is the likelihoods of unwanted incidents and their consequences for assets that constitute risks, and by making estimates of the likelihoods and consequences we can understand which risks are the most in need of treatment and which risks are less relevant. Given the documentation of identiﬁed risks from the previous activity, including the documentation of the changing risks, the risk estimation of a changing and evolving target is quite similar to traditional risk analysis: The estimation is conducted by a walkthrough of the risk documentation addressing each of the relevant elements in turn. To the extent that risks are persistent under the changes, the estimation is not repeated. The estimates need to be continuously documented, which means that there must be adequate support for including the estimates in the risk models. In order to conduct the activity of risk analysis and the documentation of the results, there is hence a need for techniques for making estimates of likelihoods and consequences of changing risks, as well as modelling support for documenting the results. As we shall see, such techniques and support is provided by the approach to the modelling of changing risk presented in Sect. 4. 3.4

Risk Evaluation

The objective of the risk evaluation is to determine which of the identiﬁed risks that need treatment, and to make a basis for prioritising the treatment options. Basically, the risk evaluation amounts to estimating the risk levels based on the

242

M.S. Lund, B. Solhaug, and K. Stølen

likelihood and consequence estimates, and to compare the results with the risk evaluation criteria. The risk evaluation of a changing and evolving target is conducted in the same way as risk evaluation of a traditional risk analysis. Given the risk documentation of the changing risks with the risk estimates, the risk evaluation is conducted by calculating the risk level of each pair of an unwanted incident and asset harmed by the incident. The calculation is straightforwardly done by using the risk function deﬁned during the context establishment. For changing systems, the criteria may of course be diﬀerent before and after the changes in question. 3.5

Risk Treatment

The risk treatment succeeds the risk identiﬁcation, estimation and evaluation activities, and the objective is to identify and select a set of treatment options for the risks that are not acceptable according to the risk evaluation criteria. A treatment option is thus an appropriate measure to reduce risk level, and the implementation of the selected treatments should bring the risk level down to an acceptable level. For changes that are planned or predicted, it may be that we are only concerned about the future risks and in ensuring that implementation of the changes results in a system with an acceptable risk level. For changes that are planned to be implemented over a longer time span it may, however, be that we also need to identify treatments for current unacceptable risks and treatments that are consecutively implemented in the future in order to maintain an acceptable risk level under the period of change.

4

Foundations

Risk analysis involves the process of understanding the nature of risks and determining the level of risk [15]. Risk modelling refers to techniques that are used to aid the process of identifying, documenting and estimating likelihoods and consequences of unwanted incidents. In this section we present an approach to risk modelling referred to as risk graphs [3]. Once we have introduced the syntax and semantics of risk graphs, we generalise these to enable modelling and reasoning about changing risks. We then provide a calculus for analysing likelihoods in risk graphs and a means for relating risk graphs to target models. 4.1

Risk Graphs

A risk model is a structured way of representing an unwanted incident and its causes and consequences by means of graphs, trees or block diagrams [24]. We introduce risk graphs as an aid for structuring events and scenarios leading to incidents and estimating likelihoods of incidents. There exist several modelling techniques that can be used for such structuring of scenarios and incidents, and for the reasoning about likelihoods of incidents, for example fault trees [13],

Risk Analysis of Changing and Evolving Systems Using CORAS

v1 [P1]

243

Pa

v2 [P2]

Pc

v3 [P3]

Pb

v4 [P4]

Pd

Pf Pe

v5 [P5]

v7 [P7]

v6 [P6]

Fig. 6. Risk graph

event trees [14], attack trees [25], cause-consequence diagrams [21], Bayesian networks [2] and CORAS threat diagrams (see Sect. 5). Risk graphs can be understood as a common abstraction of these modelling techniques [3]. By giving formal semantics to risk graphs, we thereby also provide a risk model semantics that can be used to explain and reason about several established approaches to risk modelling. A risk graph consists of vertices (representing threat scenarios) and a ﬁnite set of directed relations (representing the “leads to” relationship) between them. An example risk graph is shown in Fig. 6. Each vertex in a risk graph is assigned a set of likelihood values representing the estimated likelihood for the scenario to occur. The assignment of several likelihood values, typically a likelihood interval, represents underspeciﬁcation of the likelihood estimate. A relation from vertex v to vertex v means that v may lead to v . Also the relations can be assigned likelihood sets. These are conditional likelihoods that specify the likelihood for a scenario leading to another scenario when the former occurs. One threat scenario may lead to several other threat scenarios, so the probabilities on the relations leading from a threat scenario may add up to more than 1. A risk graph is furthermore allowed to be incomplete in the sense that a given threat scenario may lead to more scenarios than what is accounted for in the risk graph. The probabilities of the relations leading from a threat scenario may for this reason also add up to less than 1. The Syntax of Risk Graphs. Formally a risk graph is a set D of elements e. An element is a vertex v or a relation v − → v . Let P ⊆ [0, 1] denote a probability set. We then write v(P ) to indicate that the probability set P is assigned to v. P Similarly, we write v − → v to indicate that the probability set P is assigned to the relation from v to v . If no probability set is explicitly assigned, we assume the probability set assigned to the element to be [0, 1]. Using this textual notation, the risk graph shown in Fig. 6 can be represented by D = {v1 (P1 ), v2 (P2 ), v3 (P3 ), v4 (P4 ), v5 (P5 ), v6 (P6 ), v7 (P7 ), P

P

P

P

P

Pf

a b c d e v1 −−→ v3 , v2 −→ v3 , v3 −→ v4 , v4 − −→ v7 , v5 −→ v6 , v6 −−→ v7 }

244

M.S. Lund, B. Solhaug, and K. Stølen

The Semantics of Risk Graphs. Risk graphs are used for the purpose of documenting and reasoning about risks, particularly the documentation and analysis of threat scenarios and unwanted incidents and their likelihoods. The approach of [3] assumes that scenarios and their probabilities are represented by a probability space [4] on traces of events. We let H denote the set of all traces (both ﬁnite and inﬁnite) and HN the set of all ﬁnite traces. A probability space is a triple (H, F, µ). H is the sample space, i.e. the set of possible outcomes, which in our case is the set of all traces. F is the set of measurable subsets of the sample space, and µ is a measure that assigns a probability to each element in F . The semantics of a risk graph is statements about the probabilities of the trace sets that represent vertices or the composition of vertices. In other words, the semantics is a set of statements about the measure µ. For composition of vertices, v v denotes the occurrence of both v and v where the former occurs before the latter. We let v v denote the occurrence of at least one of v and v . A vertex is atomic if it is not of the form v v or v v . We use lower case v as the naming convention for arbitrary vertices, and upper case V as the naming convention for the set of ﬁnite traces representing the vertex v. When deﬁning the semantics of risk graphs we use the auxiliary function tr( ) that yields a set of ﬁnite traces from an atomic or combined vertex. Intuitively, tr(v) is the set of all possible traces leading up to and through the vertex v, without continuing further. The function is deﬁned by def

tr(v) = HN V when v is an atomic vertex def tr(v v ) = tr(v) tr(v ) def tr(v v ) = tr(v) ∪ tr(v )

where is the operator for sequential composition of trace sets, for example weak sequencing in UML sequence diagrams [10]. Notice that the deﬁnition of the composition v v does not require v to occur immediately before v . The deﬁnition implies that tr(v v ) includes traces from v to v via ﬁnite detours. A probability interval P assigned to v, denoted v(P ), means that the likelihood of going through v is a value p ∈ P , independent of what happens before or after v. The semantics of a vertex is deﬁned by def

[[v(P )]] = µc (tr(v)) ∈ P where the expression µc (S) denotes the probability of any continuation of the trace set S ⊆ H, and is deﬁned as def

µc = µ(S H)

A probability interval P assigned to a relation v − → v means that the likeli hood of v occurring after an occurrence of v is a value in P . This likelihood is referred to as the conditional likelihood. The semantics of a relation is deﬁned by def P [[v − → v ]] = µc (tr(v v )) ∈ µc (tr(v)) · P

Risk Analysis of Changing and Evolving Systems Using CORAS

245

def

[pi , pj ] · [pk , pl ] = [pi · pk , pj · pl ] def

[pi , pj ] · p = [pi · p, pj · p] def

[pi , pj ] + [pk , pl ] = [pi + pk , pj + pl ] def

[pi , pj ] + p = [pi + p, pj + p] def

[pi , pj ] − [pk , pl ] = [pi − pk , pj − pl ] def

[pi , pj ] − p = [pi − p, pj − p] def

p − [pk , pl ] = [p − pk , p − pl ] Fig. 7. Interval arithmetic

Our deﬁnitions of interval arithmetic in the setting of risk graphs are given in Fig. 7. The semantics [[D]] of a risk graph is the conjunction of the expressions deﬁned by the elements in D, formally deﬁned as def

[[D]] =

e∈D [[e]]

A risk graph is said to be correct (with respect to the world or a speciﬁcation of the relevant part of the world) if each of the conjuncts of [[D]] is true. We say that D is inconsistent if it is possible to deduce False from [[D]]. 4.2

Risk Graphs for Changing Risks

In order to support the modelling of changing risks we need to generalise risk graphs to allow the simultaneous modelling of risks both before and after the implementation of some given changes. For this purpose we extend the risk graph notation to three kinds of vertices and three kinds of relations, namely before, after and before-after. When an element (vertex or relation) is of kind before it represents risk information before the changes, when it is of kind after it represents risk information after the changes, and when it is of kind before-after it represents risk information that holds both before and after the changes. The Syntax of Risk Graphs with Change. A risk graph with change is represented by a pair (Db , Da ) of sets of elements, the former consisting of the vertices and relations of kind before and the latter consisting of vertices and relations of kind after. Table 1 gives an overview of the language constructs and the naming conventions we use for referring to them. The symbols written in sans serif and the arrows denote speciﬁc language constructs, whereas v denotes an arbitrary vertex of any kind. Table 2 gives an overview of the various ways of specifying likelihoods. Recall that any of the likelihoods can be undeﬁned, in which case they are completely underspeciﬁed.

246

M.S. Lund, B. Solhaug, and K. Stølen Table 1. Naming conventions Variable Diagram construct v vb va v v− → v v− →b v v− →a v

Vertex before-after Vertex before Vertex after Vertex Relation before-after Relation before Relation after

Table 2. Denoting likelihoods Likelihood spec. Interpretation v(P P )

v occurs with likelihood P before, and v occurs with likelihood P after vb occurs with likelihood P before va occurs with likelihood P after

vb(P ) va(P )

P P

v −−−→ v

v leads to v with conditional likelihood P before, and v leads to v with conditional likelihood P after

P

v− →b v P

v− →a v

v leads to v with conditional likelihood P before

v leads to v with conditional likelihood P after

Figure 8 shows an example of the visual representation of a risk graph with change. Solid lines, like on the vertex v1 and the relation from v1 to v3 , indicates elements that only exists before, while dashed lines indicates elements that exists after. The vertices with a white shadow, like v2 , are those that exist both before and after, while those with black shadows, like v5 , exist only after. The dashed relations with a single probability set, like the relation from v5 to v6 , exist only after, while those with double probability sets, like the relation from v3 to v4 , exist both before and after. v1 [P1]

v2 [P2]/[P2’]

Pa Pb/Pb’

v3 [P3]/[P3’]

Pc/Pc’

v4 [P4]/[P4’]

Pd/Pd’

Pf v5 [P5]

Pe

v6 [P6]

Fig. 8. Risk graph for changing risks

v7 [P7]/[P7’]

Risk Analysis of Changing and Evolving Systems Using CORAS

v1 [P1]

v2 [P2]

v2 [P2’]

Pa Pb

Pc

v3 [P3]

v4 [P4]

Pd v7 [P7]

Pb’

Pc’

v3 [P3’]

v4 [P4’]

Pd’

Pf v5 [P5]

247

Pe

v7 [P7’]

v6 [P6]

Fig. 9. Two views on risk graphs for changing risk

Since we are operating with vertices and relations of kind before-after as language element of their own, we also allow the representation of risk graphs with change as a single set D of vertices and relations, where each element is of one of the kinds before, after or before-after. This single set of elements is then syntactic sugar for the equivalent representation of a pair of sets of elements. For such a combined representation D we use the functions before( ) and after( ) to ﬁlter the combined risk graph with respect to the elements of kind before and after, respectively. The following deﬁne the function before( ) for singleton sets of elements. def

before({v(P P )}) = {vb(P )} def before({vb(P )}) = {vb(P )} def before({va(P )}) = ∅ P P

def

P

P

def

P

P

def

before({v −−−→ v }) = {v − →b v } before({v − →b v }) = {v − →b v } before({v − →a v }) = ∅ The ﬁltering of a risk graph with change D with respect to the before elements is then deﬁned as def before(D) = e∈D before({e}) The deﬁnition of the function after( ) is symmetric. For a risk graph with change D of elements of the three diﬀerent kinds, the representation as a pair of elements of kind before and elements of kind after is then given by (before(D), after(D)). Figure 9 shows the graphical representation of the two risk graph before(D) (top) and after(D) (bottom) where D is the risk graph shown in Fig. 8.

248

M.S. Lund, B. Solhaug, and K. Stølen

The Semantics of Risk Graphs with Change. Given the syntax of risk graphs with change as deﬁned above, we can deﬁne the semantics as a straightforward generalisation of the semantics of regular risk graphs. The semantics [[(Db , Da )]] of a risk graph with change is deﬁned as def

[[(Db , Da )]] = [[Db ]] ∧ [[Da ]] For a combined representation D of a risk graph with change, the semantics is deﬁned as def [[D]] = [[before(D), after(D)]] 4.3

Reasoning about Likelihoods in Risk Graphs

In this section we introduce rules for calculating probabilities of vertices in risk graphs, and we provide guidelines for consistency checking probabilities that are assigned to risk graphs. The ﬁrst rule is referred to as the relation rule, and captures the conditional likelihood semantics of a risk graph relation. For a vertex v that leads to v , the vertex v v denotes the occurrences of v that happen after an occurrence of v. Rule 1 (Relation). If there is a direct relation from v to v , we have: P

v(P ) v −→ v (v v )(P · P ) The second rule is referred to as the mutual exclusive vertices rule, and yields the probability of either v or v occurring when the two vertices are mutually exclusive: Rule 2 (Mutually exclusive vertices). If the vertices v and v are mutually exclusive, we have: v(P ) v (P ) (v v )(P + P ) The third rule is referred to as the statistically independent vertices rule, and yields the probability of either v or v occurring when the two vertices are statistically independent. Rule 3 (Statistically independent vertices). If vertices v and v are statistically independent, we have: v(P ) v (P ) (v v )(P + P − P · P ) As a small example of probability calculation consider the risk graph in Fig. 6 and assume we want to calculate the probability of v3 from v1 and v2 . By Rule 1

Risk Analysis of Changing and Evolving Systems Using CORAS

249

Table 3. Guidelines for consistency checking likelihoods How to check consistency of likelihoods in risk graphs Exact values in complete diagrams Assigned value: v(p) Calculated value: v(p ) Consistency check: p = p Exact values in incomplete diagrams Assigned value: v(p) Calculated value: v(p ) Consistency check: p ≥ p Intervals in complete diagrams Assigned interval: v([pi , pj ]) Calculated interval: v([pi , pj ]) Consistency check: [pi , pj ] ⊆ [pi , pj ] or, equivalently, pi ≤ pi and pj ≥ pj Intervals in incomplete diagrams Assigned interval: v([pi , pj ]) Calculated interval: v([pi , pj ]) Consistency check: pj ≥ pj

we calculate (v1 v3 )(P1 · Pa ) and (v2 v3 )(P2 · Pb ). Assuming that v1 and v2 , as well as v1 v3 and v2 v3 , are statistically independent, we use Rule 3 to calculate ((v1 v3 ) (v2 v3 ))(P1 · Pa + P2 · Pb − P1 · Pa · P2 · Pb ). Assuming that the likelihood estimates in Fig. 6 are correct, there is still one issue to consider before we can conclude about the likelihood of the vertex v3 . The issue is whether or not the risk graph is complete. If the risk graph is complete, the graph shows all the possible ways in which v3 may occur. In that case we have that v3 = (v1 v3 )(v2 v3 ) and that P3 = P1 ·Pa +P2 ·Pb −P1 ·Pa ·P2 ·Pb is the correct likelihood of this vertex. If the risk graph is incomplete, there may be further scenarios that can lead to v3 . In that case we only know that P3 is the lower bound of the probability of v3 . Consistency checking of risk models is important, as it is a useful means for detecting errors or misunderstandings of the risk estimates that are documented during a risk analysis. The basis for the consistency checking is the likelihood values that are already assigned to the vertices and relations of a risk graph. The guidelines for consistency checking depend on whether the risk graph in question is complete, and whether the likelihoods are given as exact probabilities or as probability intervals. The guidelines are given in Table 3. As an example of consistency checking, consider the risk graph in Fig. 6, assuming ﬁrst that the graph is complete. By the above example, we know that the probability of the vertex v3 is P3 = P1 · Pa + P2 · Pb − P1 · Pa · P2 · Pb given the vertices and relations that lead to this vertex. The assigned probability P3 must therefore equal the calculated probability P3 in order to be consistent with the preceding probability estimates if we are working with exact values. If the P ’s are intervals, we must have that P3 ⊆ P3 for the risk graph to be consistent.

250

M.S. Lund, B. Solhaug, and K. Stølen

Discarding the assumption of the completeness of the graph gives the consistency requirement that the assigned probability P3 must be greater than or equal to the calculated probability P3 , i.e. that P3 ≥ P3 , if we have exact values. On the other hand, if P3 and P3 are intervals P3 = [pi , pj ] and P3 = [pi , pj ], the requirement is that pj ≥ pj . 4.4

Relating Risk Model to Target Description

Risk analysis of changing systems requires means for identifying the parts of a risk picture that are aﬀected by changes to a speciﬁc part of the target (and therefore need to be reassessed), as well as identifying the parts of the risk picture that are not aﬀected (and therefore valid also after the changes). Thus we need techniques for identifying and documenting the relation between the target description and the risk models in a way that gives us traceability between target elements (elements of the target description) and risk model elements. Two key artifacts in a risk analysis are the target model and the risk model. The target model is the core part of the overall target description and documents the events, scenarios and actors that are the subject for the risk analysis. Given these two artifacts we introduce a third artifact in order to establish the relation between the former two: a trace model. The trace model is of a table format that allows the tracing from target model elements to risk model elements, and vice versa. Initially we can think of the trace model as a set of pairs (uid , vid ) of target model identiﬁers uid and risk model identiﬁers vid representing the rows of the table. This of course require that each of the elements have a unique identiﬁer. In the following we assume that we already have a target model and a risk model of elements with unique identiﬁers, since obtaining such models by indexing the elements is a trivial task. From a pragmatic point of view, there are two obvious shortcomings of the table format given above. To make eﬃcient use of the trace model it should convey information about the relations in an intuitive way; the use of possibly tool generated indexes for the model elements is not intuitively informative. Furthermore, in many cases several target model elements are logically understood as a whole. Without some means of grouping several rows of the table into one compound relation, such structures of the target model will be obscured. To mitigate this we introduce a third column in the table for tagging the target model element/risk model element pairs. The grouping of pairs is then conducted by inserting the same tag on several rows. The name of the tag should be chosen by the end-user, and should be a unique name that conveys intuitive information about the grouping. More formally, the trace model is now a set of tuples (uid , vid , t) of a target model identiﬁer, a risk model identiﬁer and a tag. We extend the risk graph notation with a language construct for explicitly specifying the relation to the target model. The construct is used for annotating risk graphs with the tags of the trace model. We understand this construct as a mere visualisation of the trace model in the risk graphs, and not as part of the semantics of risk graphs. An example of the visualisation of a trace model in a risk graph with change is shown in Fig. 10. As with the other elements of

Risk Analysis of Changing and Evolving Systems Using CORAS

251

t1 t2

v1 [P1]

Pa

v2 [P2]/[P2’]

Pb/Pb’

v3 [P3]/[P3’]

Pc/Pc’

v4 [P4]/[P4’]

Pd/Pd’

Pf Pe

v5 [P5]

v7 [P7]/[P7’]

v6 [P6]

t3

Fig. 10. Relating target model and risk model

risk models with change, the target model relations can be speciﬁed as existing before the change only, (for example t1 ), after the change only (for example t3 ), or both before and after the change (for example t2 ).

5

Instantiation of CORAS

In a CORAS risk analysis, threat diagrams are used intensively to facilitate risk identiﬁcation and risk estimation. The diagrams are furthermore used as a part of the documentation and reporting of the analysis results. Figure 11 depicts an example of a threat diagram. In fact, this threat diagram shows the scenarios that are modelled by means of the risk graph in Fig. 10. Threat diagrams describe how threats may exploit vulnerabilities to initiate threat scenarios, how threat scenarios may lead to unwanted incidents or other threat scenarios, and the assets harmed by the unwanted incidents. The language constructs are threats (deliberate, accidental and non-human), vulnerabilities, threat scenarios, unwanted incidents and assets. Only threat scenarios and unwanted incidents may be assigned likelihoods. There are furthermore three kinds of relations in threat diagrams, namely initiates relations, leads-to relations and impacts relations. An initiates relation has a threat as source and a threat scenario or unwanted incidents as target. It can be annotated with a likelihood that describes the likelihood for the threat to initiate the related scenario or incident. A leads-to relation has a threat scenario or unwanted incident as both source and target. It can be annotated with a conditional likelihood. An impacts relation has an unwanted incident as source and an asset as target, and can be annotated with a consequence value that describes the harm of the incident on the asset when the incident occurs.

r3

r2

r1

v5 [P5]

v2 [P2]/[P2’]

v1 [P1]

t3

Pa

Pb/Pb’

t1

Pe

v3 [P3]/[P3’]

t2

Pc/Pc’

v6 [P6]

v4 [P4]/[P4’]

Pf

Pd/Pd’ v7 [P7]/[P7’]

c/c’ a

252 M.S. Lund, B. Solhaug, and K. Stølen

Fig. 11. Instantiation of risk graphs in CORAS

Risk Analysis of Changing and Evolving Systems Using CORAS

253

While all scenarios and relations in Fig. 10 are present in Fig. 11, there are some signiﬁcant diﬀerences between the two diagrams. The threat diagram explicitly shows the initiating threats, distinguishes v7 from the other scenarios as an unwanted incident, and explicitly shows the asset that is harmed. The diﬀerences between threat diagrams and risk graphs are summarised as follows: – Initiate relations and leads-to relations in threat diagrams can be annotated with vulnerabilities, while the relations in risk graphs cannot. – Threat diagrams distinguish between four kinds of vertices, namely threats, threat scenarios, unwanted incidents and assets, while risk graphs only have scenarios. – Threat diagrams distinguish between three kinds of relations, namely initiates relations, leads-to relations and impacts relations, while risk graphs only have leads-to relations. Given the diﬀerences between threat diagrams and risk graphs, the techniques for reasoning about likelihoods nevertheless carry over to the CORAS instantiation. The vulnerabilities are mere annotations on relations, and can be ignored in the formal representation of the diagrams. Moreover, the various vertices and relations of threat diagrams can be interpreted as special instances of the risk graph vertex and relation: – An unwanted incident of a threat diagram is interpreted as a scenario of a risk graph. – A set of threats r1 , . . . , rn with initiates relations to the same threat scenario s is interpreted as follows: The threat scenario s is decomposed into n parts, where each resulting sub-scenario sj , j ∈ {1, . . . , n}, corresponds to the part of s that is initiated by threat rj . The threat rj with initiates relation of likelihood lj to sub-scenario sj is then combined into the risk graph scenario Threat rj initiates sj and the scenario is assigned likelihood lj . – An impacts relation from unwanted incident u to asset a with consequence c in a threat diagram is interpreted as follows: The impacts relation is interpreted as a risk graph relation with likelihood 1; the asset a is interpreted as the risk graph scenario Incident u harms asset a with consequence c.

With this interpretation, we refer to Sect. 4.3 for the techniques for reasoning about likelihoods in CORAS threat diagrams. However, notice that Rule 1 (Relation) applies to the CORAS leads-to relations only and that Rule 2 (Mutually exclusive vertices) and Rule 3 (Independent vertices) apply to the CORAS threat scenarios and unwanted incidents. In order to allow all likelihood reasoning to be conducted directly in CORAS diagrams, we introduce a separate rule for the initiates relation. We let r denote a threat, v denote a vertex (threat scenario or unwanted incident), r − → v denote the initiates relation from threat r to vertex v, and r v denote the occurrences of vertex v that are initiated by the threat r.

254

M.S. Lund, B. Solhaug, and K. Stølen Table 4. List of acronyms Acronym

Meaning

ACC ADS-B AMAN AOIS ATCO ATM ATS CWP FDPS OPS Room SUP

Area Control Center Automatic Dependent Surveillance-Broadcast Arrival Manager Aeronautical Operational Information System Air Traﬃc Controller Air Traﬃc Management Air Traﬃc System Controller Working Position Flight Data Processing System Operation room Supervisor

Rule 4 (Initiates). If there is an initiates relation from threat r to vertex v, we have: P r− →v (r v)(P )

6

Practical Example

In this section we present a practical example of a risk analysis of a changing system under the before-after perspective. As the running example we use an Air Traﬃc Management (ATM) risk analysis case study conducted within the SecureChange project.1 European Air Traﬃc Management is currently undergoing huge changes with introduction of new information systems and decision support systems, as well as the reorganisation of ATM services, as part of the ATM 2000+ strategic agenda [6] and the Single European Sky ATM Research (SESAR) [26]. The case study presented in the following focuses in particular on the introduction of two new systems: Arrival Manager (AMAN) and Automatic Dependent SurveillanceBroadcast (ADS-B). The results of the case study are fully documented in a SecureChange deliverable [17]; in this paper only a selection of the target and the analysis is presented. A list of acronyms from the ATM domain used in the example is given in Table 4. 6.1

Context Establishment

The context establishment includes articulating the overall goals and objectives of the risk analysis, and deciding its focus and scope. This includes making a description of the target of analysis, identifying the assets and deciding the risk evaluation criteria. 1

The case study was conducted in close interaction with ATM personnel with expert knowledge about the target of analysis and the planned changes. For more information on the SecureChange project, see http://www.securechange.eu/

Risk Analysis of Changing and Evolving Systems Using CORAS

255

Goals and Objectives. An important part of Air Traﬃc Management is the services provided by ground-based Air Traﬃc Controllers (ATCOs) located at Area Control Centers (ACCs). One of the main responsibilities of Air Traﬃc Controllers is to maintain horizontal and vertical separation among aircrafts and between aircrafts and possible obstacles. They must ensure an orderly and expeditious air traﬃc ﬂow by issuing instructions and information to aircrafts, and by providing ﬂight context information to pilots, such as routes to waypoints and weather conditions. An important characteristic of the ATM domain of today is that there are limited interactions with the external world, and therefore also limited security problems in relation to information ﬂow to and from the environment. A further characteristic is that humans are at the center of the decision and work processes and the role of automated decision support systems and tools is limited. The planned and ongoing changes raise new security issues and security concerns with immediate impact on safety issues. The overall objective of this risk analysis is to understand, document and assess security risks of ATM with particular focus on the arrival management process with the involved activities, tasks, roles, components and interactions. The party of the analysis is the ATM service provider. Target Description. The target of the analysis is a speciﬁc Area Control Center and the activities of the Air Traﬃc Controllers in the arrival management process. The Area Control Center is a ground-based center with the responsibility of managing the traﬃc of a given airspace. The actual traﬃc management is conducted from the Operation room (OPS Room), which is the operational environment of the Air Traﬃc Controllers. The Air Traﬃc Controllers have different roles, some of which have their own Controller Working Position (CWP). The Controller Working Positions makes a range of tools for surveillance, communication and planning available to the Air Traﬃc Controllers. In the following we ﬁrst document the target of analysis before the introduction of AMAN and ADS-B. Thereafter we describe the planned changes, before we document the target description where the introduction of AMAN and ADSB is reﬂected. The selected part of the target is documented by use of UML structured classiﬁers and activity diagrams. Before Changes. The structured classiﬁer of Fig. 12 shows the structure and communication links of the ATM components, while the structured classiﬁer of Fig. 13 shows the Operation room as consisting of a number of ACC islands that are connected to the ACC network. Each ACC island consists of a number of Controller Working Positions, each of which are operated by exactly one Air Traﬃc Controller, and is divided into sector teams with the responsibility of assigned sectors of the airspace. The Operation room furthermore has a number of Supervisors (SUPs) that communicate with the ACC islands, and that also are connected to the ACC Network via the Controller Working Position CWP SUP. The UML activity diagram of Fig. 14 gives a high-level overview of the various tasks of the arrival management process.

256

M.S. Lund, B. Solhaug, and K. Stølen

class ATM

: Meteostations

: FDPS

: Radar

: AOIS

: Surveillance

: ACC network

: Technical room[1..*]

: OPS room[1..*]

: Adjacent ATS unit[*]

: Aircraft[*]

Fig. 12. ATM before changes

class OPS Room : ACC network

: CWP_SUP

: SUP[1..*]

: ACC island[1..*]

Fig. 13. Operation room before changes

Planned Changes. The changes we are addressing are, in particular, the introduction of the Arrival Manager (AMAN) in the managing of air traﬃc, as well as the introduction of the Automatic Dependent Surveillance-Broadcast (ADS-B). The AMAN is a decision support tool for the automation of Air Traﬃc Controllers’ tasks in the arrival management, such as the computation of arrival sequences for aircrafts approaching an airport. The introduction of the AMAN aﬀects the Controller Working Positions, as well as the Area Control Center as a whole. The main foreseen changes from an operational and organisational point of view are the automation of tasks (i.e. the usage of the AMAN for the computation of the arrival sequence) that currently are carried out by Air Traﬃc Controllers.

Risk Analysis of Changing and Evolving Systems Using CORAS

257

Arrival management tasks

Aircraft data analysis for starting the sequence creation

Controlling the aircraft in the sector

Sequence finalisation

Clearances to the aircraft for building the planned sequence Progressive transfer of the whole sequence to the adjacent sector

Fig. 14. Arrival management tasks before changes

The introduction of the ADS-B is actually independent of the AMAN, but is introduced during the same time frame. ADS-B is a cooperative GPS-based surveillance technique for air traﬃc control where the aircrafts constantly broadcasts their position to the ground and to other aircrafts. After Changes. In order to highlight the changes in the diagrams, we use grey shading to indicate elements that are introduced. The UML structured classiﬁer of Fig. 15 shows the structure and communication links of the ATM components after the changes. At this level we only see the introduction of the ADS-B. The diagram of Fig. 16 shows the internal structure of the Operation room after the introduction of the AMAN. The AMAN is connected to the ACC network, and thereby also to the ACC islands and the Controller Working Positions. The UML activity diagram of Fig. 17 gives a high-level overview of the arrival management tasks after the changes. Assets. The purpose of asset identiﬁcation is to identify the parts, aspects or properties of the target with respect to which the risk analysis will be conducted.

258

M.S. Lund, B. Solhaug, and K. Stølen class ATM

: Meteostations

: FDPS

: ADS-B

: Radar

: AOIS

: Surveillance

: ACC network

: Technical room[1..*]

: OPS room[1..*]

: Adjacent ATS unit[*]

: Aircraft[*]

Fig. 15. ATM after changes class OPS Room : AMAN : ACC network

: CWP_SUP

: SUP[1..*]

: ACC island[1..*]

Fig. 16. Operation room after changes

An asset is something to which a party assigns value, and hence for which the party requires protection. In this analysis, the party is the ATM service provider who owns the Area Control Center in question. The risk analysis addresses security issues. Before the changes, the focus is on the security property Information Provision: The provisioning of information regarding queue management sensitive data by speciﬁc actors (or systems) must be guaranteed 24 hours a day, 7 days a week, taking into account the kind of data shared, their conﬁdentiality level and the diﬀerent actors involved. As explained above the ATM of today has little interaction with the external world, but the planned changes will raise new security issues. Therefore, in the analysis of the situation after the changes, the focus is extended to also include the security property Information protection: Unauthorised actors (or systems) are not allowed to access conﬁdential queue management information.

Risk Analysis of Changing and Evolving Systems Using CORAS

Arrival management tasks

Acquisition of the AMAN provided sequence

Controlling the aircraft in the sector

AMAN sequence monitoring and verification Clearances to the aircraft for building the planned sequence Progressive transfer of the whole sequence to the adjacent sector

Fig. 17. Arrival management tasks after changes

ATM service provider

Compliance

Availability of arrival sequences

Availability of aircraft position data

Airlines’ trust

Confidentiality of ATM information

Fig. 18. Assets before and after the changes

259

260

M.S. Lund, B. Solhaug, and K. Stølen

In the asset identiﬁcation these security properties are covered by the two corresponding concepts of conﬁdentiality and availability. More precisely we deﬁne the assets of the analysis to be Availability of arrival sequences and Availability of aircraft position data before the changes, with the addition of Confidentiality of ATM information, after the changes. In addition we deﬁne the indirect assets Compliance and Airlines’ trust. Indirect assets are assets that, with respect to the target and scope at hand, is harmed only via harm to other assets. Assets that are not indirect we often refer to as direct assets. We use CORAS asset diagrams to document the assets and the relations between them. The asset diagram showing assets both before and after is given in Fig. 18. Availability of arrival sequences and Availability of aircraft position data, as well as the indirect asset Compliance, shown with white shadows, are relevant both before and after, while Confidentiality of ATM information and the indirect asset Airlines’ trust, shown with black shadow, are relevant only after. The arrows of the diagram specify harm relations. One asset is related to another if harm to the former may lead to harm to the latter. In this case, harm to all direct assets may lead to harm to the indirect asset Compliance, but only harm no Confidentiality of ATM information may lead to harm to Airlines’ trust. Risk Evaluation Criteria. The risk evaluation criteria deﬁne the level of risk that the party, i.e. the ATM service provider, is willing to accept for the given target of analysis. Basically, the criteria are a mapping from risk levels to the decision of either accepting the risk or evaluating the risk further for possible treatment. In order to speak of risk levels, we need ﬁrst to deﬁne consequence and likelihood scales and a risk function. The consequence and likelihood scales are partly based on requirements and advisory material provided by EUROCONTROL [5,7], and are given in Tables 5 through 7. The risk function is a mapping from pairs of consequence and likelihood to risk levels and is documented by the risk matrix shown in Fig. 19. We use three risk levels, namely low (light grey), medium and high (dark grey). With these deﬁnitions we deﬁne the risk evaluation criteria as follows: – High risk : Unacceptable and must be treated. – Medium risk : Must be evaluated for possible treatment. – Low risk : Must be monitored. In this analysis, the scales, risk function and risk evaluation criteria apply both before and after the changes, while in the general this is not always case as both scales, risk functions and risk evaluation criteria may change in the transition from before to after. 6.2

Risk Identification

Risk identiﬁcation is conducted as a structured brainstorming involving personnel with ﬁrst hand knowledge about the target of analysis. By conducting a

Risk Analysis of Changing and Evolving Systems Using CORAS

261

Table 5. Consequence scale for availability Consequence Description Catastrophic Major Moderate Minor Insigniﬁcant

Catastrophic accident Abrupt manoeuvre required Recovery from large reduction in separation Increasing workload of ATCOs or pilots No hazardous eﬀect on operations

Table 6. Consequence scale for conﬁdentiality Consequence Description Catastrophic Major Moderate Minor Insigniﬁcant

Loss of data that can be utilised in terror Data loss of legal implications Distortion of air company competition Loss of aircraft information data (apart from aircraft position data) Loss of publicly available data

Table 7. Likelihood scale Likelihood Description Certain Likely Possible Unlikely Rare

A very high number of similar occurrences already on record; has occurred a very high number of times at the same location/time A signiﬁcant number of similar occurrences already on record; has occurred a signiﬁcant number of times at the same location Several similar occurrences on record; has occurred more than once at the same location Only very few similar incidents on record when considering a large traﬃc volume or no records on a small traﬃc volume Has never occurred yet throughout the total lifetime of the system

Likelihood

Insignificant

Minor

Consequence Moderate

Rare Unlikely Possible Likely Certain

Fig. 19. Risk function

Major

Catastrophic

Availability of aircraft position data

M.S. Lund, B. Solhaug, and K. Stølen

Software error

CWP

Duplication of labels

Creation of false alarms The consolidation of data from several radar sources fails

Technical room

ATCO

Lack of awareness

ATCO fails to comply with arrival management procedures

Delays in sequence provisioning

Degradation of aircraft position data

Availability of arrival sequences

262

Fig. 20. Threat diagram before changes

walkthrough of the target description, the risk are identiﬁed by systematically identifying unwanted incidents, threats, threat scenarios and vulnerabilities. The results are documented on-the-ﬂy by means of CORAS threat diagrams. While the various parts of the threat diagrams are modelled and documented, the relations to the target of analysis are identiﬁed and documented at the same time. The approach to risk identiﬁcation is to ﬁrst identify and document risks for the target of analysis before the changes. Once this is completed, we

Fig. 21. Threat diagram before and after changes

Attacker

ADS-B transponder

Surveillance

Software error

Dependence on broadcasting

ADS-B

CWP

The consolidation of data from several radar sources fails

Technical room

Lack of awareness

Spoofing of ADS-B data

Degradation of aircraft position data

Delays in sequence provisioning

ATCO

Critical aircraft position data leakes to unauthorised third parties

ATCO fails to comply with AMAN sequence

Eavesdropping ADS-B communication

ADS-B transponders not transmitting correct information

Duplication of labels

Creation of false alarms

ATCO fails to comply with arrival management procedures

Confidentiality of ATM information

Availability of aircraft position data

Availability of arrival sequences

Risk Analysis of Changing and Evolving Systems Using CORAS 263

264

M.S. Lund, B. Solhaug, and K. Stølen

proceed by identifying and documenting the risks after the changes. Based on the documented relations to the target description, i.e. the trace model, we identify the parts of the threat scenarios that are not aﬀected by the changes and therefore do not have to be addressed again from scratch. Before Change. The threat diagram of Fig. 20 documents unwanted incidents that may arise due to duplication of labels on the Controller Working Position interface. A label depicts an aircraft with its position data, and is derived from radar data. When several radar sources are used, the label is generated by automatically consolidating (merging) the data from the various sources. In some cases, software errors may yield duplicated labels that may lead Air Traﬃc Controllers to believe there are two aircrafts. The duplication may also lead to false near miss alarms. This may lead to two unwanted incidents Delays in sequence provisioning and Degradation of aircraft position data. After Change. Figure 21 is the result of the risk identiﬁcation where the changes have been taken into account. The unwanted incidents documented in Fig. 20 persist under the changes and we therefore ﬁnd them as “two-layered” before-after elements in Fig. 21. However, the incidents Delays in sequence provisioning may be caused by the threat scenario ATCO fails to comply with arrival management procedures only before the changes and by the threat scenario ATCO fails to comply with AMAN sequence only after the changes. Due to the introduction of ADS-B as a means for surveillance, there are also further threats and threat scenarios that are relevant for the unwanted incident Degradation of aircraft position data after the changes. This is documented by the threats ADS-B transponder and Attacker, and the threat scenarios ADS-B transponders not transmitting correct information and Spoofing of ADS-B data. The threat Attacker also initiates a new threat scenario Eavesdropping ADS-B communication which leads to a new unwanted incident Critical aircraft position data leaks to unauthorised third parties that may cause harm to the new asset Confidentiality of ATM information. 6.3

Risk Estimation

The risk estimation basically amounts to estimating likelihoods and consequences for the unwanted incidents. Usually, we also estimate likelihoods for threat scenarios in order to get a better basis for estimating the likelihood of unwanted incidents and to understand the most important sources of risks. To the extent that threat scenarios and unwanted incidents before changes are completely unaﬀected by the changes, the risk estimates need not be conducted twice. However, when scenarios and incidents are aﬀected by the changes, the value of these must be reassessed. Likewise, we must estimate likelihoods and consequences for scenarios and incidents that only are relevant after the changes. Before Change. The estimation of likelihoods and consequences before the changes is documented in Fig. 22.

minor

Availability of aircraft position data

265

Software error

CWP

Duplication of labels [possible]

Degradation of aircraft position data [possible]

Availability of arrival sequences Creation of false alarms [possible]

The consolidation of data from several radar sources fails [possible]

Technical room

ATCO

Lack of awareness

ATCO fails to comply with arrival management procedures [rare]

Delays in sequence provisioning [possible]

minor

Risk Analysis of Changing and Evolving Systems Using CORAS

Fig. 22. Risk estimation before changes

After Change. The threat diagram of Fig. 23 documents the estimation of likelihoods and consequences when the changes are taken into account. The issues in relation to ADS-B are relevant only after the changes and are therefore only assigned after values. The likelihood of the threat scenario Creation of false alarms and the unwanted incident Delays in sequence provisioning furthermore changes under the changes to the target of analysis. The threat scenario ATCO fails to comply with arrival management procedures is not relevant after the changes and are thus only assigned a before value.

Fig. 23. Risk estimation before and after changes

Attacker

ADS-B transponder

Surveillance

Software error

Dependence on broadcasting

ADS-B

CWP

The consolidation of data from several radar sources fails [possible]/[possible]

Technical room

Lack of awareness

Spoofing of ADS-B data [rare]

ATCO fails to comply with AMAN sequence [rare]

Eavesdropping ADS-B communication [certain]

ADS-B transponders not transmitting correct information [likely]

Duplication of labels [possbile]/[possible]

Creation of false alarms [possbile]/[unlikely]

Degradation of aircraft position data [possible]/[possible]

Confidentiality of ATM information

Availability of aircraft position data

Availability of arrival sequences

minor/ minor

major

minor/ minor

Delays in sequence provisioning [possible]/[unlikely]

ATCO

Critical aircraft position data leakes to unauthorised third parties [rare]

ATCO fails to comply with arrival management procedures [rare]

266 M.S. Lund, B. Solhaug, and K. Stølen

Risk Analysis of Changing and Evolving Systems Using CORAS

6.4

267

Risk Evaluation

The purpose of risk evaluation is to decide which of the identiﬁed risks are acceptable and which must be evaluated for treatments. But before we evaluate the risks, we also need to estimate risks to the indirect assets, which until this point have not been dealt with. Indirect assets are assets that are harmed only through harm to other assets. This activity therefore consists of following the harm relations from direct to indirect assets documented in the asset diagram (recall Fig. 18) and estimating how the harm caused by unwanted incidents propagate. The result of this is documented in Fig. 24. A risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. From the risk estimation documented in Figs. 23 and 24, we get seven risks when coupling unwanted incidents and assets. – R1:Delays in sequence provisioning (risk before-after toward Availability of arrival sequences) – R2:Degradation of aircraft position data (risk before-after toward Availability of aircraft position data) – R3:Critical aircraft position data leaks to unauthorised third part (risk after toward Confidentiality of ATM information) – R4:Delays in sequence provisioning (risk before-after toward Compliance) – R5:Degradation of aircraft position data (risk before-after toward Compliance) – R6:Critical aircraft position data leaks to unauthorised third part (risk after toward Compliance) – R7:Critical aircraft position data leaks to unauthorised third part (risk after toward Airlines’ trust)

Delays in sequence provisioning [possible]/[unlikely]

Degradation of aircraft position data [possible]/[possible]

Critical aircraft position data leakes to unauthorised third parties [rare]

minor/ minor Availability of arrival sequences minor/ minor

insignificant/ insignificant

minor/ minor

Availability of aircraft position data major

minor

Compliance

moderate Confidentiality of ATM information

Fig. 24. Risk estimation for indirect assets

Airlines’ trust

268

M.S. Lund, B. Solhaug, and K. Stølen

Likelihood

Insignificant Rare Unlikely Possible Likely Certain

R4 R4

Minor R6 R1 R1, R2, R5

Consequence Moderate R7

Major R3

Catastrophic

Fig. 25. Risk evaluation

ATCO

R1: Delays in sequence provisioning [low]/[low] Software error

Availability of arrival sequences

Technical room Surveillance

ADS-B transponder

R2: Degradation of aircraft position data [low]/[low]

ADS-B

Availability of aircraft position data

CWP

Attacker

R3: Critical aircraft position data leakes to unauthorised third parties [medium]

Confidentiality of ATM information

Fig. 26. Risk diagram before and after changes – direct assets

Risk Analysis of Changing and Evolving Systems Using CORAS

Software error

ADS-B transponder

Attacker

269

R4: Delays in sequence provisioning [low]/[low]

R5: Degradation of aircraft position data [low]/[low]

Compliance

R6: Critical aircraft position data leakes to unauthorised third parties [low]

R7: Critical aircraft position data leakes to unauthorised third parties [moderate]

Airlines’ trust

Fig. 27. Risk diagram before and after changes – indirect assets

The ﬁrst step of the risk evaluation is to calculate the risk levels. These are obtained by plotting the risks in the risk function deﬁned during the context establishment (recall Fig. 19) using the estimated likelihoods and consequences. The risk evaluation is conducted separately for the risks before the changes and the risk after the changes. Figure 25 shows the risks plotted in the risk function; we use italic to indicate risks before and bold to indicate risks after. We use CORAS risk diagrams to document the results of calculating the risk levels. These diagrams show the risks together with the threats that initiate them and the assets they harm. The three risks toward direct assets (R1–R3) are documented in Fig. 26, while the risks toward indirect assets (R4–R7) are documented in Fig. 27. By the risk evaluation criteria deﬁned in the context establishment, risks R3 and R7 must be evaluated for possible treatments, while the rest must be monitored. 6.5

Risk Treatment

For changes that are planned and/or anticipated, the risk treatment should ensure that the risk level is maintained at an acceptable level through the changes, and that the risk level of the resulting system is acceptable. Whether or not

Attacker

ADS-B transponder

Surveillance

Dependence on broadcasting

ADS-B Spoofing of ADS-B data [rare] Implement encryption of ADS-B signals

Eavesdropping ADS-B communication [certain]

ADS-B transponders not transmitting correct information [likely]

Implement backup or improve maintenance of the transponder

Fig. 28. Treatment diagram after changes R7: Critical aircraft position data leakes to unauthorised third parties [moderate]

R3: Critical aircraft position data leakes to unauthorised third parties [medium]

R2: Degradation of aircraft position data [low]/[low]

Airlines’ trust

Confidentiality of ATM information

Availability of aircraft position data

270 M.S. Lund, B. Solhaug, and K. Stølen

Risk Analysis of Changing and Evolving Systems Using CORAS

271

the current risks should be subject to treatment depends on the time frame of the change, as well as the priorities of the parties and other stakeholder. If the planned or anticipated changes are very immediate, it may not make sense to invest in treatments for risks that disappear after the changes. Identiﬁed treatment options are documented using CORAS treatment diagrams, as shown in Fig. 28. The identiﬁed treatments focus mainly on the risk picture after the changes and, because only risks R3 and R7 must be evaluated for possible treatments, on the introduction of the ADS-B. One of the treatment options, namely ADS-B encryption, can ensure both conﬁdentiality and authentication, while the second turns out to be relevant only for availability and thus not for R3 and R7.

7

Related Work

The extent to which existing risk analysis methods support the analysis and representation of changing risks depends of course on the relevant perspective of change; recall from Sect. 2 the classiﬁcation of changes into the maintenance, before-after and continuous evolution perspectives. Somewhat more thorough treatments than provided in this paper of the maintenance and continuous evolution perspectives in CORAS are given in [17,20], in particular an approach to the continuous evolution perspective inspired by an application of CORAS in risk monitoring [23]. In principle, any risk analysis method can be used to analyse changing risks from the maintenance and the before-after perspective by staring from scratch and doing complete reassessments. For the management of changing risks to be eﬃcient, however, there should be methodical support for dealing with change in a systematic and focused manner. Most of the established risk analysis methods provide little or no support for analysing changing and evolving risks. The ISO 31000 risk management standard [15] prescribes change detection and identiﬁcation for emerging risks, but provides no guidelines for how to do this in practice. A state-of-the-art methodology like OCTAVE [1] recommends reviewing risks and critical assets, but how the risk analysis results should be updated is not speciﬁed. Some approaches have support for associating elements of risk models to parts of the target description, which may facilitate the identiﬁcation and documentation of risk changes due to target changes. UML based approaches such as misuse cases [28] may utilise built-in mechanisms in the UML for relating elements from diﬀerent UML diagrams. ProSecO [12] relates risks to elements of a functional model of the target. With respect to the before-after perspective, ProSecO provides some support for modelling the various phases or states of a change process; when the models change, elements of the model may be transferred to states that indicate the need for additional risk analysis. ProSecO furthermore provides some support for the modelling of the change process by means of state machines. Other academic studies have focused on either maintenance [18,27] or variants of reassessment [9,16].

272

M.S. Lund, B. Solhaug, and K. Stølen

The continuous evolution perspective is the most general of the three perspectives on change as it incorporates time into the picture and thereby refers to the instance of the evolving risk picture at any given point in time. Among the existing approaches the support for this perspective is virtually non-existent. Some risk modelling approaches do provide support for updating the values that are annotated to diagrams in the sense that by changing the input values, the derived output values can be automatically updated. These includes fault trees [13], Markov models [11], and Bayesian networks [2,8]. Still, CORAS is to our knowledge the sole approach to provide means for predicting the evolution of risks related to an evolving target.

8

Conclusion

A traditional risk analysis considers the target of analysis at a particular point in time and in a particular conﬁguration. The result is the documentation of a risk picture valid for that conﬁguration at that point in time. However, systems and their environment tend to change and evolve over time and the risks toward the system may shift. If changes are not captured and reﬂected in the risk documentation the established risk picture may no longer be valid, with the consequence that we no longer have a full and correct overview of the risks of our target. Conducting a new risk analysis from scratch when the target or its environment has changed is not an approach to be preferred as risk analyses are time and resource consuming and parts of the risk documentation is likely to still be valid. In order to appropriately handle change, risk analysis methods should be supported with techniques and guidelines that are specialised toward modelling, analysing and reasoning about changing risks. However, the nature of the changes have implications the methodological needs. In this paper we have categorised changes into three perspectives referred to as maintenance, before-after and continuous evolution and characterised the methodological needs for each of them. The focus in this paper has been on the before-after perspective which comprise planned and anticipated changes of a radical and extensive nature. The challenge in this perspective is to obtain and present a risk picture that describes both current and future risks without doing double work. This requires methods and guidelines for distinguishing the parts of the risk documentation that are aﬀected by changes from the parts that are unaﬀected. In this paper we handle this by providing modelling support for changing risks and means for relating risk models to target models in such a way that changes in the target models can be traced to the risk models. These are given as extensions to risk graphs, which may be seen as a common abstraction of a variety of diﬀerent risk modelling techniques. In this way, the extension may be instantiated into these diﬀerent techniques. We demonstrate this by providing an instantiation into the CORAS risk modelling language. Further, we demonstrate our approach by a practical example from a risk analysis case study from the Air Traﬃc Management (ATM) domain.

Risk Analysis of Changing and Evolving Systems Using CORAS

273

In diﬀerence from other approaches, we provide a systematic approach to change management in the context of risk analysis. This includes methodological guidelines on how to maintain and update risk analysis documentation to reﬂect both past and future changes to the target without starting from scratch and doing a full reassessment, something that to a large extent is missing in other risk analysis methodologies. Acknowledgements. The work on which this paper reports has been funded by the European Commission through the projects SecureChange (Contract no. 231101) and NESSoS (Contract no. 256980). Thanks to Massimo Felici, Frank Innerhofer-Oberperﬂer, Valentino Meduri, Alessandra Tedeschi, and the ATM experts participating in the analysis, for their contributions to the ATM case study.

References 1. Alberts, C.J., Davey, J.: OCTAVE criteria version 2.0. Technical report CMU/SEI2001-TR-016. Carnegie Mellon University (2004) 2. Ben-Gal, I.: Bayesian networks. In: Ruggeri, F., Kenett, R.S., Faltin, F.W. (eds.) Encyclopedia of Statistics in Quality and Reliability. John Wiley & Sons, Chichester (2007) 3. Brændeland, G., Refsdal, A., Stølen, K.: Modular analysis and modelling of risk scenarios with dependencies. J. Syst. Softw. 83(10), 1995–2013 (2010) 4. Dudely, R.M.: Real analysis and probability. Cambridge Studies in Advanced Mathematics, Cambridge (2002) 5. EUROCONTROL: EUROCONTROL safety regulatory requirements (ESARR) 4 – Risk assessment and mitigation (2001) 6. EUROCONTROL: Air Traﬃc Management Strategy for the years 2000+. vol. 1,2 (2003) 7. EUROCONTROL: ESARR advisory material/Guidance document (EAM 2/ GUI 5) – Harmonisation of safety occurrences severity and risk assessment (2005) 8. Fenton, N., Neil, M.: Combining evidence in risk analysis using Bayesian networks. Agena White Paper W0704/01, Agena (2004) 9. Goel, S., Chen, V.: Can business process reengineering lead to security vulnerabilities: Analyzing the reengineered process. Int. J. Prod. Econ. 115(1), 104–112 (2008) 10. Haugen, Ø., Husa, K.E., Runde, R.K., Stølen, K.: STAIRS towards formal design with sequence diagrams. Softw. Syst. Modeling 4(4), 355–367 (2005) 11. Howard, R.A.: Dynamic Probabilistic Systems. Markov Models, vol. I. John Wiley & Sons, Chichester (1971) 12. Innerhofer-Oberperﬂer, F., Breu, R.: Using an enterprise architecture for IT risk management. In: Information Security South Africa Conference, ISSA 2006 (2006) 13. International Electrotechnical Commission: IEC 61025 Fault Tree Analysis (FTA) (1990) 14. International Electrotechnical Commission: IEC 60300-9 Dependability management - Part 3: Application guide - Section 9: Risk analysis of technological systems - Event Tree Analysis (ETA) (1995)

274

M.S. Lund, B. Solhaug, and K. Stølen

15. International Organization for Standardization: ISO 31000 Risk management – Principles and guidelines (2009) 16. Lee, E., Park, Y., Shin, J.G.: Large engineering project risk management using a Bayesian belief network. Expert Syst. Appl. 36(3), 5880–5887 (2009) 17. Lund, M.S., et al.: SecureChange Deliverable D5.3 Assessment method (2011), http://www.securechange.eu/sites/default/files/deliverables/ D5.3.Assessment Methods.pdf 18. Lund, M.S., den Braber, F., Stølen, K.: Maintaining results from security assessments. In: 7th European Conference on Software Maintenance and Reengineering (CSMR 2003), pp. 341–350. IEEE Computer Society, Los Alamitos (2003) 19. Lund, M.S., Solhaug, B., Stølen, K.: Evolution in relation to risk and trust management. Comput. 43(5), 49–55 (2010) 20. Lund, M.S., Solhaug, B., Stølen, K.: Model-Driven Risk Analysis – The CORAS Approach. Springer, Heidelberg (2011) 21. Nielsen, D.S.: The cause/consequence diagram method as basis for quantitative accident analysis. Technical report RISO-M-1374, Danish Atomic Energy Commission (1971) 22. Object Management Group: Uniﬁed Modeling Language: Superstructure, version 2.1.1 (non-change bar). OMG Document: formal/2007-02-05 (2005) 23. Refsdal, A., Stølen, K.: Employing key indicators to provide a dynamic risk picture with a notion of conﬁdence. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (eds.) IFIPTM 2009. IFIP Advances in Information and Communication Technology, vol. 300, pp. 215–233. Springer, Heidelberg (2009) 24. Robinson, R.M., Anderson, K., Browning, B., Francis, G., Kanga, M., Millen, T., Tillman, C.: Risk and Reliability – An Introductory Text. R2A, 5th edn (2001) 25. Schneider, S.: Attack trees: Modeling security threats. Dr. Dobb’s J. 24, 21–29 (1999) 26. SESAR Consortium: The ATM Target Concept. SESAR Deﬁnition Phase - Deliverable 3 (2007) 27. Sherer, S.A.: Using risk analysis to manage software maintenance. J. Softw. Maint.: Res. Pract. 9(6), 345–364 (1997) 28. Sindre, G., Opdahl, A.L.: Eliciting security requirements by misuse cases. In: 37th International Conference on Technology of Object-Oriented Languages and Systems (TOOLS Paciﬁc 2000), pp. 120–131. IEEE Computer Society, Los Alamitos (2000)

Author Index

Alvim, M´ ario S. 211 Andr´es, Miguel E. 211 Asnar, Yudistira 152 Basin, David 101 Bhargavan, Karthikeyan

Livraga, Giovanni 1 Luccio, Flaminia L. 35 Lund, Mass Soldal 231 Massacci, Fabio 152 Micciancio, Daniele 185

66

Chatzikokolakis, Konstantinos Clavel, Manuel 101

211

Dania, Carolina 101 De Capitani di Vimercati, Sabrina Egea, Marina

Ortiz, Gonzalo 1

101

Palamidessi, Catuscia

211

101 Samarati, Pierangela 1 Solhaug, Bjørnar 231 Steel, Graham 35 Stølen, Ketil 231

Focardi, Riccardo 35 Foresti, Sara 1 Fournet, C´edric 66 Garc´ıa de Dios, Miguel A. Gordon, Andrew D. 66 Gorrieri, Roberto 125

101 Valdazo, Javier Vernali, Matteo

101 125

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany

6858

Alessandro Aldini Roberto Gorrieri (Eds.)

Foundations of Security Analysis and Design VI FOSAD Tutorial Lectures

13

Volume Editors Alessandro Aldini Università degli Studi di Urbino “Carlo Bo” Dipartimento di Scienze di Base e Fondamenti Piazza della Repubblica 13 61029 Urbino, Italy E-mail: [email protected] Roberto Gorrieri Università degli Studi di Bologna Dipartimento di Scienze dell’Informazione Mura Anteo Zamboni 7 40127 Bologna, Italy E-mail: [email protected]

ISSN 0302-9743 e-ISSN 1611-3349 ISBN 978-3-642-23081-3 e-ISBN 978-3-642-23082-0 DOI 10.1007/978-3-642-23082-0 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011934268 CR Subject Classiﬁcation (1998): D.4.6, C.2, K.6.5, K.4, D.3, F.3, E.3 LNCS Sublibrary: SL 4 – Security and Cryptology

© Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, speciﬁcally the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microﬁlms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientiﬁc Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

International School on Foundations of Security Analysis and Design

This book is the sixth in a series of volumes collecting tutorial papers accompanying lectures presented at FOSAD, the International Summer School on Foundations of Security Analysis and Design, which has been held yearly since 2000 at the University Residential Center of Bertinoro, Italy. FOSAD has been one of the foremost educational events established with the goal of disseminating knowledge in the critical area of security in computer systems and networks. Every year, FOSAD oﬀers a good spectrum of current research in foundations of security – ranging from programming languages to analysis of protocols, from cryptographic algorithms to access control policies and trust/identity management – that can be of help for graduate students and young researchers from academia or industry who intend to approach the ﬁeld. The spirit of FOSAD is also characterized by the “open session”, which represents a series of presentations given by selected participants about their ongoing work. The objective of this initiative is to encourage discussions, propose new ideas, comment on open problems, and favor novel scientiﬁc collaborations. The topics covered in this book include privacy and data protection, security APIs, cryptographic veriﬁcation by typing, model-driven security, noninterference analysis, security in governance, risk, and compliance, lattice cryptography, quantitative information ﬂow analysis, and risk analysis. The opening paper presented by Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati gives an overview of the techniques developed for protecting data and ensuring privacy. Riccardo Focardi, Flaminia Luccio, and Graham Steel discuss the subtleties behind the design of secure application program interfaces (security APIs) and show that their analysis through formal techniques has recently proved highly successful both in ﬁnding new ﬂaws and verifying security properties of improved designs. The tutorial paper by C´edric Fournet, Karthikeyan Bhargavan, and Andrew Gordon shows the use of types for verifying authenticity properties of cryptographic protocols. The paper by David Basin, Manuel Clavel, Marina Egea, Miguel Garc´ıa de Dios, Carolina Dania, Gonzalo Ortiz, and Javier Valdazo is a survey of a very promising instance of model-driven security. The authors present an approach and a toolkit supporting the construction of security, data, and graphical user interface (GUI) models together with related Web applications. Roberto Gorrieri and Matteo Vernali extend the notion of intransitive noninterference by Rushby to the frameworks of deterministic labelled transition systems, nondeterministic automata, and the class of Petri nets called elementary net systems. Yudistira Asnar and Fabio Massacci describe a methodology to design systems trading information security with governance, risk, and compliance (GRC) management. Daniele Micciancio gives an introduction to the mathematical theory and appli-

VI

Foundations of Security Analysis and Design

cation of lattice cryptography, which is one of the hottest and fast-moving areas in mathematical cryptography today. M´ ario Alvim, Miguel Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi review two information-theoretic approaches to the quantitative analysis of information ﬂows, namely, the one based on Shannon entropy, and the one based on R´enyi min-entropy. In the last paper, Mass Soldal Lunda, Bjørnar Solhauga, and Ketil Stølen introduce general techniques and guidelines for dealing with risk analysis in systems evolving over time. In particular, the authors propose the CORAS approach to model-driven risk analysis. This year, FOSAD was organized in cooperation with the Network of Excellence on Engineering Secure Future Internet Software Services and Systems (EU FP7 Project NESSoS). Obviously, we would like to thank all the institutions that have promoted and founded FOSAD in the last few years. Finally, we also wish to thank all the staﬀ of the University Residential Center of Bertinoro for the organizational and administrative support. August 2011

Alessandro Aldini Roberto Gorrieri

Table of Contents

Foundations of Security Analysis and Design Protecting Privacy in Data Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati

1

An Introduction to Security API Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . Riccardo Focardi, Flaminia L. Luccio, and Graham Steel

35

Cryptographic Veriﬁcation by Typing for a Sample Protocol Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C´edric Fournet, Karthikeyan Bhargavan, and Andrew D. Gordon Model-Driven Development of Security-Aware GUIs for Data-Centric Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Basin, Manuel Clavel, Marina Egea, Miguel A. Garc´ıa de Dios, Carolina Dania, Gonzalo Ortiz, and Javier Valdazo On Intransitive Non-interference in Some Models of Concurrency . . . . . . . Roberto Gorrieri and Matteo Vernali A Method for Security Governance, Risk, and Compliance (GRC): A Goal-Process Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yudistira Asnar and Fabio Massacci The Geometry of Lattice Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniele Micciancio Quantitative Information Flow and Applications to Diﬀerential Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M´ ario S. Alvim, Miguel E. Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi

66

101

125

152 185

211

Risk Analysis of Changing and Evolving Systems Using CORAS . . . . . . . Mass Soldal Lund, Bjørnar Solhaug, and Ketil Stølen

231

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

275

Protecting Privacy in Data Release Sabrina De Capitani di Vimercati, Sara Foresti, Giovanni Livraga, and Pierangela Samarati Dipartimento di Tecnologie dell’Informazione, Universit` a degli Studi di Milano, Via Bramante 65 - 26013 Crema, Italy [email protected]

Abstract. The evolution of the Information and Communication Technology has radically changed our electronic lives, making information the key driver for today’s society. Every action we perform requires the collection, elaboration, and dissemination of personal information. This situation has clearly brought a tremendous exposure of private and sensitive information to privacy breaches. In this chapter, we describe how the techniques developed for protecting data have evolved in the years. We start by providing an overview of the ﬁrst privacy deﬁnitions (k-anonymity, -diversity, t-closeness, and their extensions) aimed at ensuring proper data protection against identity and attribute disclosures. We then illustrate how changes in the underlying assumptions lead to scenarios characterized by diﬀerent and more complex privacy requirements. In particular, we show the impact on privacy when considering multiple releases of the same data or dynamic data collections, ﬁne-grained privacy deﬁnitions, generic privacy constraints, and the external knowledge that a potential adversary may exploit for inferring sensitive information. We also brieﬂy present the concept of diﬀerential privacy that has recently emerged as an alternative privacy deﬁnition. Keywords: Privacy, microdata protection, data release.

1

Introduction

The advancements in the Information and Communication Technology (ICT) have revolutionized our lives in a way that was unthinkable until few years ago. We live in the Globalization era, where everything we need to do is available within “one mouse click”. Global infrastructure, digital infrastructure, digital society are only few examples of terms used at diﬀerent times for concisely referring to our “computer-based” society. The term that better represents the today’s society is however information society (or information age) since the information has a key role in the daily life activities of everyone. Every time we browse Internet, perform an online transaction, ﬁll in forms to, for example, enter contests or participate in online games, and spend our time online in social networks, information about us is collected, stored, analyzed, and sometimes A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 1–34, 2011. c Springer-Verlag Berlin Heidelberg 2011

2

S. De Capitani di Vimercati et al.

shared with third parties. Furthermore, public and private companies have often the need of publishing aggregate statistical data (macrodata) as well as detailed data (microdata) for research or statistical purposes. The complexity and variety of the today’s information society introduce therefore new risks and pose new research challenges. In fact, the vast amount of personal (user-generated) data collected, stored, and processed, the unclear data ownership, and the lack of control of the users on their own data are creating unprecedented risks of privacy breaches. The problem of properly protecting the privacy of the users is clearly not new and has received (and receives) considerable attention from the research and development communities. In the past, the restricted access to information and its expensive processing represented a form of protection that does not hold anymore. In fact, with the rate at which technology is developing, it is now becoming easier and easier to access huge amount of data by using, for example, portable devices (e.g., PDAs, mobile phones) and ubiquitous network resources. Also, the availability of powerful techniques for analyzing and correlating data coming from diﬀerent information sources makes it simple to infer information that was not intended for disclosure. It is interesting to observe how the problem of guaranteeing privacy protection is changing over the years, in line with the evolution of the ICT. Data were principally released in the form of macrodata, that is, tables (often of two dimensions), where each cell contains aggregate information about users or companies, called respondents. The macrodata protection techniques were principally based on the identiﬁcation and obfuscation of sensitive cells [11]. With the growing importance and use of microdata, the research community dedicated many eﬀorts in designing microdata protection techniques able to preserve the privacy of the respondents while limiting the disclosure risks. Traditionally, the disclosure risks are related to the possibility, for an adversary, to use the microdata for determining conﬁdential information on a speciﬁc individual (attribute disclosure) or for identifying the presence of an individual in the microdata table itself (identity disclosure). To limit the disclosure risks, names, addresses, phone numbers, and other identifying information are removed (or encrypted) from the microdata. For instance, in the microdata table in Figure 1, which contains medical data, the names of the patients as well as their Social Security Numbers are removed, thus obtaining the de-identiﬁed medical data in Figure 2(a). Although a de-identiﬁed microdata table apparently protects the identities of the respondents represented in the table, there is no guarantee of anonymity. The de-identiﬁed microdata may contain other information, called quasi-identiﬁer , such as birth date and ZIP code that in combination can be linked to publicly available information to re-identify individuals. As an example, consider the deidentiﬁed medical data in Figure 2(a) and the voter list for the San Francisco area, publicly released by the local municipality, in Figure 2(b). It is easy to see that the values of attributes DoB, Sex, and ZIP can be exploited for linking the tuples in the microdata with the voter list, thus possibly re-identifying individuals and revealing their illnesses. For instance, in the microdata in Figure 2(a) there is only one female born on 1958/12/11 living in the 94142 area.

Protecting Privacy in Data Release SSN

Name

DoB

Sex

ZIP

123-45-6789 234-56-7890 345-67-8901 456-78-9012 567-89-0123 678-90-1234 789-01-2345 890-12-3456 901-23-4567 012-34-5678 134-56-7890 245-67-8901 356-78-9012 467-89-0123 578-90-1234 689-01-2345 790-12-3456

Diana Smith Nathan Johnson Eric Williams Liz Jones John Brown Luke Davis Barbara Miller Fay Wilson Anthony Moore Matt Taylor Jane Doe Anna Anderson Carol Thomas Gabrielle White Lorna Harris Rob Martin Bob Thompson

1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1958/12/11 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F F M M

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94142 94145 94136 94134 94147 94148

3

Disease H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Pneumonia Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

Fig. 1. An example of microdata table with identifying information

This combination, if unique in the external world as well, uniquely identiﬁes the corresponding tuple in the table as pertaining to Jane Doe, 300 Main St., San Francisco, revealing that she suﬀers from Pneumonia. From a study performed on the data collected for the 2000 US Census, Golle showed that 63% of the US population can be uniquely identiﬁed combining their gender, ZIP code, and full date of birth [21]. This percentage decreases if the gender is combined with the County of residence instead of the ZIP code, and with the month/year of birth (see Figure 3). In the 1990s, several microdata protection techniques were developed [11]. Such techniques can be classiﬁed in two main categories: masking techniques that transform the original data in a way that some statistical analysis on the original and transformed data produce the same or similar results; synthetic data generation techniques that replace the original data with synthetic data that preserve some statistical properties of the original data. Among the microdata protection techniques, k-anonymity [37] is probably one of the most popular, which has inspired the development of algorithms and techniques for both enforcing k-anonymity and for complementing it with other forms of protection (e.g., -diversity [29], and t-closeness [26]). These techniques are based on the assumptions that: quasi-identiﬁers are the only attributes that can be used for inferring the respondents to whom information refers; the same microdata table is published only once; and potential adversaries do not have any external knowledge. Clearly, such assumptions do not hold anymore in the today’s society, where any information can be used to re-identify anonymous data [33]. Two well-known examples of privacy violations, which testiﬁed how ensuring proper privacy protection is becoming a diﬃcult task, are the America OnLine (AOL) and Netﬂix incidents [3,32]. AOL is an Internet services and media company that in 2006 released around 20 millions of search records of 650,000 of its customers. To protect the privacy of its customers, AOL de-identiﬁed such records by substituting personal identiﬁers with numerical identiﬁers. A sample of such data is the following:

4

S. De Capitani di Vimercati et al. SSN Name

DoB

Sex

ZIP

1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1958/12/11 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F F M M

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94142 94145 94136 94134 94147 94148

Disease H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Pneumonia Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

(a) De-identiﬁed medical data Name

Address

City

ZIP

DoB

Sex

... ... ... ... ... ... Jane Doe 300 Main St. San Francisco 94142 58/12/11 female ... ... ... ... ... ...

Education ... secondary ...

(b) Voter list Fig. 2. An example of de-identiﬁed microdata table (a) and of publicly available non de-identiﬁed dataset (b)

ZIP County

year 0.2% 0%

Date of Birth month, year full date 4.2% 63.3% 0.2% 14.8%

Fig. 3. Identiﬁability of the US population in the 2000 US Census data [21]

116874 thompson water seal 2006-05-24 11:31:36 1 http://www.thompsonwaterseal.com 116874 knbt 2006-05-31 07:57:28 116874 knbt.com 2006-05-31 08:09:30 1 http://www.knbt.com 117020 texas penal code 2006-03-03 17:57:38 1 http://www.capitol.state.tx.us 117020 homicide in hook texas 2006-03-08 09:47:35 117020 homicide in bowle county 2006-03-08 09:48:25 6 http://www.tdcj.state.tx.us

that shows records related to two diﬀerent users (116874 and 117020) containing the ID, the term(s) used for the search, the timestamp, whether the user clicked on a result, and the corresponding visited website. With these data, two reporters of the New York Times newspaper were able to identify AOL customer no. 4417749 as Thelma Arnold, a 62 yeas old widow living in Lilburn [3]. In the same year, the on-line movies renting service Netﬂix publicly released 100 millions records, showing the ratings given by 500,000 users to the movies they rent. The records were released within the “Netﬂix Prize” competition that oﬀered $1

Protecting Privacy in Data Release

5

million to anyone who could improve the algorithm used by Netﬂix to suggest movies to its customers. Also in this case, records were de-identiﬁed by replacing personal identiﬁers with numerical identiﬁers. However, some researchers were able to de-anonymize the data by comparing the Netﬂix data against publicly available ratings on the Internet Movie Database (IMDB). For instance, the release of her movie preferences damaged a lesbian mother since she was reidentiﬁed, thus causing the disclosure of her sexual orientation [32]. From the discussion, it is clear that the protection of microdata against improper disclosure is a key issue in today’s information society. The main objective of this chapter is then to provide an overview of how the techniques for protecting microdata releases have evolved in the years, according to the evolution and complexity of the scenarios considered. We will start with a description of the ﬁrst solutions aimed at protecting microdata against identity and attribute disclosures (Section 2). We will then describe recent approaches that removed some of the assumptions characterizing the ﬁrst proposals. In particular, we will describe novel solutions for supporting: i) multiple data releases and the insertion/removal of tuples into/from the released microdata table (Section 3); ii) the deﬁnition of ﬁne-grained privacy preferences by each data respondent (Section 4); iii) generic sensitive associations among released data that need to be kept conﬁdential (Section 5); and iv) external knowledge by malicious data recipients that may cause information leakage (Section 6). We will ﬁnally illustrate a new privacy notion, called diﬀerential privacy, that deﬁnes when a computation is privacy-preserving (Section 7).

2

Privacy in Microdata Publishing

Goal of this section is to present the privacy models speciﬁcally targeted to the protection of microdata from identity and attribute disclosures (i.e., kanonymity [37], -diversity [29], and t-closeness [26]) that have inﬂuenced the work performed by the research community in the data protection area during the last two decades. All these privacy models are based on the following assumptions. First, the attributes composing a microdata table are classiﬁed in four categories, as follows. – Identiﬁers: attributes that uniquely identify a respondent (e.g., Name and SSN). – Quasi-identiﬁers (QI): attributes that, in combination, can be linked with external information to re-identify (all or some of) the respondents to whom information refers, or to reduce the uncertainty over their identities (e.g., DoB, Sex, and ZIP). – Conﬁdential attributes: attributes that represent sensitive information (e.g., Disease). – Non-conﬁdential attributes: attributes that are not considered sensitive by the respondents and whose release is not harmful (e.g., FavoriteColor). A microdata table is then protected by applying microdata protection techniques [11] that transform the values of the quasi-identiﬁer attributes. Second,

6

S. De Capitani di Vimercati et al.

there is a one-to-one correspondence between tuples in the table and data respondents. Third, a microdata table is released only once. Fourth, each table is characterized by a unique quasi-identiﬁer. In the remainder of this section, after a description of k-anonymity, -diversity, and t-closeness, we also brieﬂy discuss their recent enhancements that remove some of the assumptions on which they are based. 2.1

k-Anonymity

k-Anonymity has been proposed for protecting microdata from identity disclosure [37]. It captures the well-known requirement, traditionally applied by statistical agencies, stating that any released data should be indistinguishably related to no less than a certain number of respondents. Basically, it characterizes the protection degree against re-identiﬁcation caused by linking the released dataset with external data sources. Due to the assumption that linking attacks exploit quasi-identiﬁers only, in [37] the general requirement described above has been translated into the following k-anonymity requirement: Each release of data must be such that every combination of values of quasi-identiﬁers can be indistinctly matched to at least k respondents. A microdata table then satisﬁes the k-anonymity requirement if each tuple in the table cannot be related to less than k respondents in the population, and each respondent in the population cannot be related to less than k tuples in the released table. The k-anonymity requirement assumes that the data holder knows any possible external source of information that may be exploited by a malicious recipient for respondents re-identiﬁcation. This assumption is however limiting and highly impractical in most scenarios. k-anonymity then takes a safe approach by requiring that each combination of values of the quasi-identiﬁer attributes appears with at least k occurrences in the released table. The deﬁnition of k-anonymity represents a suﬃcient (not necessary) condition for the k-anonymity requirement. In fact, since each combination of values of the quasi-identiﬁer attributes appears with at least k occurrences, each respondent cannot be associated with less than k tuples in the microdata table. Analogously, each tuple in the table cannot be related to less than k respondents in the population. As a consequence, the risk of re-identiﬁcation is reduced by at least a factor k, independently from the external datasets available to a malicious recipient. Traditional approaches for guaranteeing k-anonymity transform the values of the attributes composing the quasi-identiﬁer and leave the sensitive and nonsensitive attributes unchanged. In particular, k-anonymity relies on generalization and suppression, which have the advantage of preserving the truthfulness of the released information, while producing less precise and less complete tables. Generalization consists in substituting the original values with more general values. For instance, the date of birth can be generalized by removing the day, or the day and the month, of birth. Suppression consists in removing data from the microdata table. The intuition behind the combined use of generalization and suppression is that suppression can reduce the amount of generalization

Protecting Privacy in Data Release

7

necessary to guarantee k-anonymity. In fact, if a limited number of outliers (i.e., quasi-identiﬁer values with less than k occurrences in the table) would force a large amount of generalization to satisfy k-anonymity, these values can be more conveniently removed from the table. For instance, consider the microdata table in Figure 2(a) and assume that the quasi-identiﬁer is attribute DoB only. Since there is only one person born in 1958, attribute DoB should be generalized to the decade of birth to guarantee 4-anonymity. Alternatively, removing the date of birth associated with the eleventh tuple in the table, 4-anonymity can be achieved by generalizing the date of birth to the month of birth, thus reducing the information loss. Both generalization and suppression can be applied at different granularity levels. More precisely, generalization can be applied at the cell or attribute level, while suppression can be applied at the cell, attribute, or tuple level. The combined use of generalization and suppression at diﬀerent granularity levels leads to diﬀerent classes of approaches for guaranteing k-anonymity [10]. However, most of the solutions proposed in the literature adopt attribute generalization and tuple suppression (e.g., Samarati’s algorithm [37], Incognito [23], and k-Optimize [4]). The reason is that cell generalization produces a table where the values in the cells of the same column may be non homogeneous, since they belong to diﬀerent domains (e.g., some tuples report the complete date of birth, while other tuples only report the year of birth). On the other hand, cell generalization has the advantage of causing a reduced information loss if compared to attribute generalization. Therefore, recently also solutions adopting cell generalization have been analyzed (e.g., Mondrian [24]). Consider the microdata table in Figure 2(a) and assume that the quasiidentiﬁer is composed of attributes DoB, Sex, and ZIP. Figure 4 illustrates the 4-anonymous table obtained combining tuple suppression (the eleventh tuple of the table in Figure 2(a) is suppressed) and attribute generalization (DoB has been generalized removing the day of birth, Sex has been generalized to a unique value, denoted *, and ZIP has been generalized removing the last two digits). Since generalization and suppression cause information loss, it is necessary to compute an optimal k-anonymous microdata table that maximizes the utility while preserving the privacy of the respondents. The computation of an optimal k-anonymous table is an NP-hard problem, independently from the granularity level at which generalization and suppression are applied [1,12,31]. As a consequence, both exact and heuristic algorithms have been proposed [12]. The exact algorithms have computational time complexity exponential in the number of quasi-identifying attributes, which is however a small value compared with the number of tuples in the microdata table. 2.2

-Diversity

Although k-anonymity represents an eﬀective solution for protecting respondents’ identities, it has not been designed to protect the released microdata table against attribute disclosure. Given a k-anonymous table it may then be possible to infer (or reduce the uncertainty about) the value of the sensitive attribute associated with a speciﬁc respondent.

8

S. De Capitani di Vimercati et al.

SSN Name

DoB

Sex

ZIP

1950/06 1950/06 1950/06 1950/06

* * * *

941** 941** 941** 941**

H1N1 Gastritis Dyspepsia Pneumonia

Disease

1940/04 1940/04 1940/04 1940/04

* * * *

941** 941** 941** 941**

Peptic Peptic Peptic Peptic

1940/06 1940/06 1940/06 1940/06

* * * *

941** 941** 941** 941**

Broken Leg Short Breath Broken Leg Stomach Cancer

1950/05 1950/05 1950/05 1950/05

* * * *

941** 941** 941** 941**

H1N1 Flu Stomach Cancer Gastritis

Ulcer Ulcer Ulcer Ulcer

Fig. 4. An example of 4-anonymous table

Two well-known attacks that may lead to attribute disclosure in a kanonymous table are the homogeneity attack [29,37] and the external knowledge attack [29]. To illustrate the homogeneity attack, consider a k-anonymous table where all the tuples composing an equivalence class (i.e., all the tuples having the same value for the quasi-identifying attributes) have the same value also for the sensitive attribute. If a data recipient knows that an individual is represented in the microdata table and the value of her quasi-identiﬁer, the recipient can easily identify the equivalence class representing the target respondent. Under the above homogeneity assumption, the recipient can also infer the value of the sensitive attribute of the target respondent. For instance, consider the 4-anonymous table in Figure 4 and suppose that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. Since all the tuples in the equivalence class with quasi-identiﬁer value equal to 1940/04,*,941** have Peptic Ulcer as value for attribute Disease, Alice can infer that her friend suﬀers from Peptic Ulcer. The external knowledge attack occurs when the data recipient may reduce her uncertainty about the sensitive attribute value of a target respondent, exploiting some additional (external) knowledge about the respondent. For instance, consider the 4-anonymous table in Figure 4 and suppose that Alice knows that her neighbor Carol is a female living in 94145 area and born on 1940/06/30. Observing the 4-anonymous table, Alice can only infer that her neighbor suﬀers either from Short Breath or Stomach Cancer, or has a Broken Leg. Suppose that Alice sees Carol running every day: as a consequence, Alice can infer that Carol can neither have broken her leg nor suﬀer from short breath. Hence, Alice discovers that Carol suﬀers from Stomach Cancer. The deﬁnition of -diversity has been proposed to counteract homogeneity and external knowledge attacks [29]. -diversity extends k-anonymity requiring that each equivalence class must be associated with at least well-represented values for the sensitive attribute. In [29], the authors propose diﬀerent deﬁnitions for “well-represented” values. The simplest formalization of -diversity requires

Protecting Privacy in Data Release

SSN Name DoB Sex

ZIP

9

Disease

1940 1940 1940 1940

M M M M

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Short Breath

1950 1950 1950 1950

F F F F

941** 941** 941** 941**

H1N1 Pneumonia H1N1 Flu

1940 1940 1940 1940

F F F F

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Stomach Cancer

1950 1950 1950 1950

M M M M

941** 941** 941** 941**

Gastritis Dyspepsia Stomach Cancer Gastritis

Fig. 5. An example of 4-anonymous and 3-diverse table

that each equivalence class includes at least diﬀerent values for the sensitive attribute. Even with this straightforward deﬁnition, -diversity counteracts the homogeneity attack, and reduces the eﬀectiveness of external knowledge attacks, since the data recipient needs more external knowledge to be able to associate a single sensitive value with a target respondent. For instance, consider the microdata table in Figure 5, which is 4-anonymous and 3-diverse and suppose that Alice knows that her neighbor Carol is a female living in 94145 area and born on 1940/06/30. Observing the 3-diverse table, Alice can infer that her neighbor either suﬀers from Peptic Ulcer , Stomach Cancer , or has a Broken Leg. Since Alice only knows that Carol does not have a Broken Leg, she cannot be certain about her neighbor’s disease. Although any traditional k-anonymity algorithm can be easily adapted to guarantee -diversity [29], in [44] the authors propose a speciﬁc approximation algorithm. This algorithm is based on cell suppression, and aims at minimizing the number of suppressed cells in computing a microdata table that satisﬁes -diversity. 2.3

t-Closeness

Even if -diversity represents a ﬁrst step for counteracting attribute disclosure, this solution may still produce a table that is vulnerable to privacy breaches. In fact, the deﬁnition of -diversity does not take into consideration diﬀerent factors that might be exploited by a malicious data recipient for attribute disclosure, such as: i) the frequency distribution of the values in the sensitive attribute domain; ii) the possible semantic relationships among sensitive attribute

10

S. De Capitani di Vimercati et al.

values; and iii) the diﬀerent sensitivity degree associated with diﬀerent values of the sensitive attribute domain (e.g., HIV is usually considered more sensitive than Flu). Two attacks that may cause attribute disclosure in an -diverse table are the skewness attack and the similarity attack [26]. Skewness attack exploits the possible diﬀerence in the frequency distribution of sensitive attribute values within an equivalence class, with respect to the frequency distribution of sensitive attribute values in the whole population (or in the released microdata table). Indeed, a difference in the distribution may change the probability with which a respondent in the equivalence class is associated with a speciﬁc sensitive value. For instance, consider the 3-diverse table in Figure 5 and suppose that Alice knows that her friend Diana is a female living in 94141 area and born on 1950/06/02. Since two out of the four tuples in the equivalence class with quasi-identiﬁer value equal to 1950,F,941** have value H1N1 for attribute Disease, Alice can infer that her friend has 50% probability of suﬀering from this disease, compared to the 12.5% of the whole released table. Also, although -diversity guarantees that each equivalence class has at least diﬀerent values for the sensitive attribute, it does not impose constraints on their semantics. As a consequence, the sensitive attribute values within an equivalence class may be diﬀerent, but semantically similar, thus causing attribute disclosure. For instance, consider the 3-diverse table in Figure 5 and suppose that Alice knows that her friend Eric is a male living in 94137 area and born on 1950/06/12. Observing the 3-diverse table, Alice can infer that her friend suﬀers from either Gastritis, Dyspepsia, or Stomach Cancer and, therefore, that Eric suﬀers from a stomach-related disease. The deﬁnition of t-closeness has been proposed to counteract skewness and similarity attacks [26]. t-Closeness extends the deﬁnition of k-anonymity, requiring that the frequency distribution of the sensitive attribute values in each equivalence class has to be close (i.e., with distance less than a ﬁxed threshold t) to the frequency distribution of the sensitive attribute values in the released microdata table. In this way, the frequency distribution of sensitive attribute values in each equivalence class is similar to the frequency distribution characterizing the released microdata table. As a consequence, the knowledge of the quasi-identiﬁer value for a target respondent does not change the probability for a malicious recipient of correctly guessing the sensitive value associated with the respondent. The eﬀect of similarity attacks is also mitigated since the presence of semantically similar values in an equivalence class can only be due to the presence, with the same relative frequencies, of these values in the microdata table. To determine if a microdata table satisﬁes the t-closeness property, it is necessary to measure the distance between the frequency distributions of sensitive attribute values characterizing each equivalence class and the whole microdata table. In [26], the authors present diﬀerent techniques to measure the distance between two frequency value distributions, and propose to adopt the Earth Mover Distance (EMD) technique.

Protecting Privacy in Data Release

2.4

11

Extensions

k-anonymity, -diversity, and t-closeness are based on some assumptions that make them not always applicable in speciﬁc scenarios. Such assumptions can be summarized as follows. – Each respondent is represented by only one tuple in the microdata table. In many real-world scenarios diﬀerent tuples in a microdata table may refer to the same respondent. For instance, consider the relation in Figure 2(a): each respondent may be associated with as many tuples in the table as the number of diseases she suﬀers from. k-anonymity is not suited for these scenarios, since an equivalence class composed of k tuples may represent less than k respondents. As a consequence, a k-anonymous table may however violate the k-anonymity requirement. – A unique microdata table is released. Data may be stored in diﬀerent relations, characterized by (functional) dependencies among them. Since different portions of these data may be released at diﬀerent times (e.g., upon request) or to diﬀerent recipients, it is necessary to guarantee that the combined view over all the released pieces of information does not cause privacy violations. Traditional solutions may not be applicable in these scenarios since they assume that the data to be released are stored in a unique microdata table. – The microdata table is characterized by one quasi-identiﬁer . Although traditional solutions assume that the released microdata table is characterized by a unique quasi-identiﬁer, diﬀerent data recipients may posses diﬀerent external data collections to be linked with the released table. The release of a diﬀerent microdata table to each data recipient, anonymized considering the speciﬁc knowledge of the recipient, is however not eﬀective since collusion among recipients would violate respondents’ privacy. On the other hand, a solution that considers the quasi-identiﬁer as the union of the quasiidentiﬁers of any possible data recipient would cause an excessive information loss. – Respondents can be re-identiﬁed through a predeﬁned quasi-identiﬁer . As discussed in Section 1, data respondents can be re-identiﬁed through any piece of information that uniquely (or almost uniquely) pertain to them. As an example, in transactional data (e.g., AOL and Netﬂix datasets) a pattern of items in a transaction (or a set thereof related to the same respondent) may re-identify a respondent. Traditional anonymization solutions are however based on a preliminary identiﬁcation of a quasi-identiﬁer, and are therefore not applicable in this scenario, where the information exploited for reidentiﬁcation cannot be determined a-priori and may not be represented by a set of attributes. In the following, we present a brief overview of some solutions that extend kanonymity, -diversity, and t-closeness by removing (some of) the above assumptions (see Figure 6).

12

S. De Capitani di Vimercati et al. (X,Y )MultiR Butterﬂy km Privacy k-anonymity Anonymity multiple tuples per respondent multiple tables multiple quasi-identiﬁers non-predeﬁned quasi-identiﬁers

× ×

× × × ×

Fig. 6. Extended scenarios (rows) and applicable techniques (columns)

(X,Y )-Privacy [39] addresses the problem of protecting the privacy of the respondents when multiple relations are released and each respondent is possibly associated with a set of tuples in the released tables. In particular, in [39] the authors assume that diﬀerent tables, obtained as the projection of a subset of attributes of a unique microdata table, are sequentially released. To guarantee that a data release, either singularly taken or combined with previous releases, does not compromise the privacy of the respondents, the deﬁnitions of (X,Y )anonymity and (X,Y )-linkability have been introduced, which are both referred with the term (X,Y )-privacy. Given a microdata table and two disjoint sets of attributes X and Y , the microdata table satisﬁes (X,Y )-anonymity if each value of X is linked to at least k diﬀerent values of Y . Note that the deﬁnition of (X,Y )-anonymity is more general than the original k-anonymity requirement. Indeed, the k-anonymity requirement can be satisﬁed by a (X,Y )-anonymous table where X is the set of attributes composing the quasi-identiﬁer and Y is the set of identifying attributes. For instance, consider the microdata table in Figure 2(a) and assume that X = {DoB,Sex,ZIP} and Y = {SSN}. A (X,Y )anonymous table guarantees that each combination of values for attributes DoB, Sex, ZIP is associated with at least k diﬀerent values for attribute SSN. (X,Y )linkability states that no value of Y can be inferred from a value of X with conﬁdence higher than a ﬁxed threshold. To guarantee privacy protection in sequential releases, the join among all the released tables has to satisfy (X,Y )privacy (i.e., it must satisfy both (X,Y )-anonymity and (X,Y )-linkability). MultiR k-anonymity [35] extends the deﬁnition of k-anonymity to multiple relations, while guaranteeing a limited information loss. The considered scenario is characterized by a set S = {T1 , . . . , Tn } of relations that preserves the lossless join property and that includes a person speciﬁc table P T , where each row corresponds to a diﬀerent respondent. The release of a set S of relations satisﬁes multiR k-anonymity if each view over the join JT = P T T1 ... Tn is k-anonymous, meaning that it includes either 0 or at least k occurrences for each combination of values for the attributes in the quasi-identiﬁer. In this scenario, the quasi-identiﬁer is deﬁned as a subset of the attributes of the join relation JT , which can be exploited to re-identify the respondents in the person speciﬁc table

Protecting Privacy in Data Release

13

P T . To protect the privacy of the respondents against attribute (besides identity) disclosure, in [35] the authors propose to extend the deﬁnition of -diversity to the multirelational scenario. A release of a set S of relations satisﬁes multiR ... Tn among all the -diversity if each view over the join JT = P T T1 released relations satisﬁes -diversity with respect to the considered sensitive attribute. Butterfly [36] aims at protecting the privacy of the respondents, while minimizing information loss, when the microdata table to be released is characterized by diﬀerent quasi-identiﬁers. Traditional approaches consider, as a quasi-identiﬁer, the set of all the attributes that may be externally available (although not in combination) to some data recipient. Even if eﬀective, this assumption causes an excessive information loss that can be reduced if the set of attributes available to each recipient is considered separately. Indeed, as shown in [36], a table that is k-anonymous with respect to quasi-identiﬁer QI1 and with respect to QI2 may not be k-anonymous with respect to QI1 ∪ QI2 . In [36] the authors then introduce the k-butterﬂy principle that allows the anonymization of a microdata table so that the k-anonymity requirement is satisﬁed with respect to multiple quasi-identiﬁers while reducing the information loss. More precisely, given a microdata table and two quasi-identiﬁers QI1 and QI2 , a subset of tuples with the same value for the attributes in QI1 ∩ QI2 satisﬁes k-butterﬂy with respect to QI1 and QI2 if the equivalence classes induced by QI1 \ QI2 (QI2 \ QI1 , resp.) include at least k tuples each. A microdata relation satisﬁes k-butterﬂy if all the equivalence classes induced by QI1 ∩ QI2 satisfy k-butterﬂy. It is important to note that a set of tuples that satisﬁes k-butterﬂy with respect to QI1 and QI2 is k-anonymous with respect to both QI1 and QI2 . km -Anonymity [38] aims at protecting the privacy of respondents of transactional data, where a subset of the items composing a transaction may represent a quasi-identiﬁer. For instance, consider a transactional dataset including all the items purchased in a supermarket and assume that Alice has seen her neighbor Bob buying parmesan cheese, strawberries, and soy milk. If the supermarket dataset includes only one transaction with this combination of items, Alice can ﬁnd out the complete list of items purchased by Bob, which may be sensitive (e.g., if it contains a medicine). The k m -anonymity concept solves this problem by requiring that each combination of at most m items must appear in at least k transactions, where m is the maximum number of items per transaction that may be known to a malicious data recipient.

3

Multiple Releases and Data Streams

The proposals described in Section 2 assume that only one instance of the microdata is published. There are however many scenarios where data are subject to frequent changes, due to the insertion, deletion, and update of tuples in the microdata, and need to be published at a regular basis. For instance, a hospital may be forced by law to publish an anonymized version of its data every six months.

14

S. De Capitani di Vimercati et al.

Data may also be continuously generated and may need to be immediately released. This happens, for example, when a credit card company outsources to a third party the transactions generated by its customers whenever they use their credit cards. Such transactions form a microdata stream that has to be continuously monitored by the third party to detect possible misbehaviors. In this section, we ﬁrst describe the privacy issues that arise in the case of multiple data releases and data streams, highlighting why the traditional approaches for protecting microdata are not suitable. We then illustrate some recent approaches that counteract such privacy issues. 3.1

Problem

The multiple releases of a microdata table whose content is updated over time may cause information leakage since a malicious data recipient can correlate the released datasets to gain information about respondents. Also, if a combination of quasi-identiﬁer values appears in one of the released tables only, it may be easier for a malicious recipient to correctly associate it with one of those respondents who have been removed from the subsequent releases (possibly exploiting the information available through external sources). To illustrate, consider the two subsequent releases of the medical data of the patients of a hospital illustrated in Figure 7. Some patients are included in both tables, while others are represented in one of the two tables only (tuples marked with a bullet in Figures 7(a) and 7(b)). Both the ﬁrst and the second release satisfy 4-anonymity and 3-diversity. However, suppose that Alice knows that her friend Fay, who is a female living in the 94130 area, born on 1940/04/20, was in the hospital when both tables have been released. Alice then knows that Fay is represented by a tuple in the tables of Figure 7, and she can identify the two equivalence classes to which Fay belongs in the two tables. In particular, in the ﬁrst release Fay is included in the third class of the table in Figure 7(a), and hence her disease could be either Peptic Ulcer, Broken Leg, or Stomach Cancer. Instead, in the second release Fay is included in the second class of the table in Figure 7(b), and hence her disease could be either Gastritis, Short Breath, Pneumonia, or Peptic Ulcer. By comparing the two sets of possible illnesses, Alice can easily infer that Fay suﬀers from Peptic Ulcer, since this is the unique disease that appears in both sets. When we consider a data stream scenario where only new tuples can be inserted into the released microdata table, it is not necessary to compute diﬀerent releases of the dataset. In fact, it may be more convenient to release only the new tuples. In this case, traditional approaches cannot be adopted, since they need the whole dataset that is instead not available in advance. Furthermore, a data stream is possibly an inﬁnite sequence of tuples, which must be released as soon as they are available since data utility decreases as time passes. As a consequence, it is not possible to assume that the whole dataset can be collected before its publication. For instance, consider a credit card company that releases the stream of data generated by purchases represented as tuples in a microdata table. Such data should be immediately published to check, for example,

Protecting Privacy in Data Release DoB Sex ZIP Disease

15

DoB Sex ZIP Disease

1940 1940 • 1940 1940

M M M M

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Short Breath

1940 1940 • 1940 1940

* * * *

9414* 9414* 9414* 9414*

Peptic Ulcer Peptic Ulcer Measles Stomach Cancer

1950 1950 1950 • 1950

F F F F

941** 941** 941** 941**

H1N1 Pneumonia H1N1 Flu

• 1940 1940 • 1940 1940

* * * *

9413* 9413* 9413* 9413*

Gastritis Short Breath Pneumonia Peptic Ulcer

• 1940 1940 • 1940 1940

F F F F

941** 941** 941** 941**

Peptic Ulcer Peptic Ulcer Broken Leg Stomach Cancer

1950 1950 • 1950 1950

* * * *

9414* 9414* 9414* 9414*

H1N1 Pneumonia Measles Gastritis

• 1950 1950 • 1950 1950

M M M M

941** 941** 941** 941** (a)

Gastritis Dyspepsia Stomach Cancer Gastritis

• 1950 1950 1950 • 1950

* * * *

9413* 9413* 9413* 9413* (b)

Infract Dyspepsia H1N1 Thrombosis

Fig. 7. An example of two subsequent releases of a table including the medical data of a hospital

possible frauds, while preserving the privacy of the card holders. We note that data streams may include more than one tuple for each respondent (e.g., a card holder may perform multiple purchases). 3.2

Solutions

Recently, the scientiﬁc community has proposed diﬀerent approaches addressing the privacy issues previously discussed. These solutions can be classiﬁed in two categories, depending on whether they consider the re-publication of data or the publication of data streams. In the following, we summarize some of the approaches proposed in both scenarios. Data re-publication (e.g., [39,43]). Most of the solutions proposed to protect the privacy of data respondents in data re-publication scenarios only focus on supporting the insertion of new tuples in the dataset and implicity assume that no tuple is removed from the microdata table. m-Invariance [43] is the ﬁrst technique addressing the problem of data re-publication that takes both insertion and deletion of tuples into account. The possible removal of tuples from the dataset can cause information leakage, due to the critical absence phenomenon. To illustrate, consider the two subsequent releases in Figure 7 and suppose that Alice knows that Gabrielle is represented in both microdata tables and that she suﬀers from either Flu or H1N1 . The value Flu, however, does not appear in the second release. As a consequence, Alice can conclude with certainty that Gabrielle contracted H1N1 . It is important to note that this inference can be drawn independently from how the two released datasets have been generalized to prevent disclosure. To counteract this privacy breach, in [43] the authors introduce the m-invariance property. A sequence T1 , . . . , Tn of released microdata tables satisﬁes m-invariance if the following properties hold: i) each equivalence

16

S. De Capitani di Vimercati et al.

class in Ti , i = 1, . . . , n, includes at least m tuples; ii) no sensitive value appears more than once in each equivalence class in Ti , i = 1, . . . , n; and iii) for each tuple t, the equivalence classes to which t belongs in the sequence [Ti , Tj ], 1 ≤ i ≤ j ≤ n, are characterized by exactly the same set of sensitive values. The rationale of m-invariance is that all the equivalence classes to which a published tuple t belongs must be associated with exactly the same set of (at least) m diﬀerent sensitive values. In this way, the correlation of the tuples in T1 , . . . , Tn does not permit a malicious recipient to associate less than m diﬀerent sensitive values with each respondent in the released datasets. The technique proposed in [43] to achieve m-invariance is incremental, meaning that the n-th release Tn can be determined taking into account only the previous release Tn−1 , and is based on the possible insertion of counterfeits, when it is needed to prevent critical absences. Data streams (e.g., [25,40,45]). One of the most important requirements when releasing data as a stream is timeliness, since these data are usually time critical and need to be published in a timely fashion to be useful for the recipients. The proposed solutions mainly aim at satisfying the k-anonymity requirement. To this purpose, they rely on generalization and on the introduction of a limited delay in data publication. The ﬁrst solution in this direction has been proposed in [45] and is based on the principle that all the tuples in an equivalence class must be published at the same time. Therefore, the data holder locally maintains a set of equivalence classes, which are all initially empty. As a new tuple is generated by the stream, it is inserted into a suitable equivalence class, if such class exits; a new equivalence class suitable for the tuple is generated, otherwise. We note that each equivalence class can include at most one tuple for each respondent. As soon as one of the equivalence classes includes k tuples that, by construction, are related to k diﬀerent respondents, these tuples are generalized to the same quasi-identiﬁer value and published. This technique, although simple, guarantees privacy protection and a timely data publication, at the price of a possibly high information loss. To limit this information loss, in [45] the authors introduce an improvement of their technique that makes a probabilistic estimation of the tuples that will be generated by the data stream, to choose the most convenient generalization strategy. An alternative approach for protecting the privacy of respondents in data streams has recently been proposed in [40], where the authors assume that data are generated and published as “snapshots” (i.e., sets of records available at a given moment of time) of d tuples each. This technique combines generalization and tuple suppression with tuple relocation to guarantee -diversity. Relocation consists in moving a tuple from one snapshot to a more recent one, if this delay in data publishing could be useful to satisfy the -diversity principle. The approach illustrated in [40] guarantees that window queries (i.e., queries that need to be evaluated only on the data released in a speciﬁc time window) evaluated on the released dataset produce the same

Protecting Privacy in Data Release

17

result as if they were evaluated on the original data stream. Another approach for protecting data streams has been illustrated in [25]. This approach is based on noise addition and therefore is not suitable for all those scenarios that require truthfulness of released data.

4

Fine-Grained Privacy Preferences

The privacy-aware publishing techniques illustrated in previous sections guarantee the same amount of privacy to all the respondents represented in the released microdata table. Privacy requirements may however depend on respondents’ preferences, or on the sensitivity of the released values. In the following of this section, we illustrate both the issues that may arise when enforcing the same protection degree to all the respondents, and the advantages of solutions that permit a ﬁne-grained speciﬁcation of privacy preferences. We also describe some recent approaches supporting ﬁne-grained privacy preference speciﬁcations. 4.1

Problem

Privacy is an individual concept [27], since each individual may have her own privacy needs that may be diﬀerent from the requirements of another individual. However, traditional privacy protection techniques provide all the data respondents with the same amount of privacy, without considering their preferences. For instance, the k-anonymity requirement demands that each tuple in a released table cannot be associated with less than k respondents in the population, and viceversa. The anonymity threshold k is ﬁxed by the data holder, without considering the speciﬁc privacy requirements of the respondents. As a consequence, this value may be adequate for some respondents and inadequate for others. For instance, consider the 4-anonymous microdata table in Figure 4 and assume that Gabrielle is a female born on 1950/05/02 and living in the 94136 area, who suﬀers from H1N1. A data recipient knowing her quasi-identiﬁer can infer that Gabrielle suﬀers from either H1N1, Flu, Stomach Cancer, or Gastritis, being these values all equally likely (25% of probability each). Although the microdata table satisﬁes 4-anonymity, Gabrielle may not want people to know, with a probability of 25%, that she suﬀers from H1N1 . On the other hand, Lorna, born on 1950/05/05 and living in the 94134 area and suﬀering from Flu, may agree to release her disease without the need to protect the corresponding tuple in the table. Therefore, the 4-anonymous table in Figure 4 does not protect Gabrielle’s privacy, while it over-protects Lorna’s sensitive value. -diversity, t-closeness, m-invariance, and all the other approaches illustrated in the previous sections suﬀer from this problem, since they are based on a unique protection threshold that is adopted to guarantee the privacy of all the respondents. As a consequence, these techniques cannot be adopted for supporting the following privacy requirements.

18

S. De Capitani di Vimercati et al.

– Value-speciﬁc privacy: diﬀerent values of the sensitive attribute should enjoy a diﬀerent protection degree. For instance, consider the medical microdata table in Figure 2(a). Stomach Cancer is usually considered more sensitive than Flu. As a consequence, tuples representing patients suﬀering from Stomach Cancer should be more carefully protected than tuples of patients suﬀering from Flu. – Respondent-speciﬁc privacy: diﬀerent respondents may have a diﬀerent perception of their privacy, independently from the values of the sensitive attribute. As a consequence, respondents may be willing to specify a diﬀerent threshold for the protection of their data. As an example, consider two patients suﬀering from the same disease. A patient may consider the protection oﬀered by a 3-diverse table adequate, while the other patient may require at least a 5-diverse table. – Sensitive presence: the presence of a respondent in the microdata table may violate her privacy. For instance, consider a microdata table representing patients suﬀering from rare diseases. Even the fact that an individual is represented in the table may violate her privacy. To satisfy the privacy requirements of all the respondents, the privacy parameter used by the techniques previously discussed should be ﬁxed to the most restrictive threshold (e.g., to the highest value of k among the preferences of the respondents), thus causing an excessive information loss. 4.2

Solutions

Recently, diﬀerent solutions supporting ﬁne-grained privacy speciﬁcations have been proposed. In the following, for each of the privacy requirements described above, we illustrate a speciﬁc protection technique addressing it. Value-specific privacy (e.g., [18]). To permit the data holder to specify diﬀerent privacy thresholds for diﬀerent values of the domain of the sensitive attribute, in [18] the authors propose (αi , βi )-closeness. (αi , βi )-closeness is an extension of the t-closeness principle that, instead of adopting a unique threshold value t, associates a range [αi , βi ] with each value si in the domain of the sensitive attribute. An equivalence class satisﬁes (αi , βi )-closeness if, for each sensitive value si , the percentage of tuples in the class with value si for the sensitive attribute is in the range [αi , βi ]. A microdata table satisﬁes (αi , βi )-closeness if each equivalence class in the table satisﬁes (αi , βi )-closeness. (αi , βi )-closeness presents diﬀerent advantages over t-closeness. First, (αi , βi )-closeness is ﬂexible, since it deﬁnes a diﬀerent threshold for each sensitive value. As a consequence, only the values that are considered highly sensitive are associated with a small range, while the frequency of non-sensitive values is possibly not bounded, thus better preserving data utility. Second, (αi , βi )-closeness is easy to check, since it does not require to compute the distance between two frequency distributions, but only to evaluate the number of tuples in the equivalence class associated with each sensitive value. Finally, (αi , βi )-closeness is easy to use, since the deﬁnition

Protecting Privacy in Data Release

19

Any

Stomach-Related

Peptic Ulcer

Stomach Cancer

...

Respiratory

Short Breath

Pneumonia

...

Fig. 8. An example of partial taxonomy tree for attribute Disease

of a range [αi , βi ] for each sensitive value is more intuitive than the choice of the maximum distance t from a known frequency distribution. (αi , βi )-closeness is enforced by applying generalization at the level of attributes [18]. In particular, the authors propose to enforce the (αi , βi )-closeness requirement by extending traditional algorithms used for solving the k-anonymity problem (see Section 2) and, in particular, the Incognito algorithm [23]. Respondent-specific privacy (e.g., [42]). In [42] the authors observed that diﬀerent respondents may have diﬀerent privacy requirements, independently from the value of the sensitive attribute associated with them. As a consequence, the adoption of a unique privacy protection threshold to the whole dataset may result in over-protecting or under-protecting respondents. To overcome this issue, in [42] the authors introduce the concept of personalized anonymity that aims at releasing as much information as possible, while satisfying the speciﬁc privacy requirement deﬁned by each respondent. Personalized anonymity is based on the deﬁnition of a taxonomy tree over the domain of each sensitive attribute in the microdata table. The tree has a leaf for each value in the sensitive attribute domain, and each internal node summarizes the speciﬁc values in its subtree. For instance, Figure 8 illustrates an example of a partial taxonomy tree for attribute Disease in the microdata table in Figure 2(a). Each respondent speciﬁes her privacy preference by choosing a guarding node along the path from the root of the taxonomy tree to the leaf node representing the respondent’s sensitive attribute value. The technique proposed in [42] for enforcing the privacy preferences of the respondents relies on the application of generalization, performed in two steps. In the ﬁrst step, the attributes composing the quasi-identiﬁer are generalized following traditional generalization schemes [37]. In the second step, each equivalence class generated in the ﬁrst step is further modiﬁed to respect the privacy level required by each respondent. In particular, for each tuple in an equivalence class, the value of the sensitive attribute is generalized (based on the taxonomy tree deﬁned for the attribute) if needed to respect the guarding node chosen by the tuple respondent. A table to be released satisﬁes personalized anonymity if, for each respondent, a malicious recipient cannot infer with probability higher than p that the respondent is associated with a speciﬁc value in the subtree rooted at the guarding node chosen by the respondent herself. The value of p

20

S. De Capitani di Vimercati et al.

represents the maximum conﬁdence with which a recipient is allowed to infer sensitive attribute values and is set by the data holder. For instance, consider the taxonomy tree in Figure 8 and suppose that Carol , who suﬀers from Stomach Cancer , chooses Stomach-Related disease as her guarding node. Carol ’s privacy is violated if a data recipient can infer with probability higher than p that Carol suﬀers from a disease among Peptic Ulcer , Stomach Cancer , and the other values children of the Stomach-Related node in the taxonomy tree. Another respondent, Matt , who suﬀers from Short Breath, does not consider a privacy violation the release of his disease. As a consequence, he sets his guarding node as ∅ (i.e., a special guarding node denoting that a respondent believes that the release of her sensitive attribute value does not violate her privacy). Sensitive presence (e.g., [34]). In [34], the authors propose δ-presence as a metric to evaluate the risk that a data recipient can identify the presence of an individual in the released table. The released dataset includes a subset of the tuples of a larger data collection D. The released microdata table T ∗ , obtained by generalizing (at the level of attributes) T ⊆ D, satisﬁes δ-presence if δmin ≤ P (t ∈ T |T ∗ ) ≤ δmax , for all t ∈ D, where P (t ∈ T |T ∗ ) is the probability that a data recipient correctly guesses that tuple t belongs to the released dataset, observing the released microdata table T ∗ . If the released dataset satisﬁes δpresence, a malicious recipient cannot determine the inclusion of a respondent in T ∗ with probability lower than δmin or higher than δmax . We note that, by tuning the values of δmin and δmax , it is possible to ﬁnd a good trade-oﬀ between data utility and privacy of the released generalized microdata table T ∗ . In fact, a small [δmin ,δmax ] range favors privacy, while a large [δmin ,δmax ] range favors data utility.

5

Group-Based Approaches for Protecting Sensitive Associations

The approaches described in the previous sections typically apply generalization and suppression for guaranteeing k-anonymity, -diversity, t-closeness, or other (more enhanced) privacy requirements. Generalization and suppression, however, cause information loss that could be reduced by adopting diﬀerent protection techniques. In the following of this section, we highlight the disadvantages caused by generalization and illustrate an alternative approach that overcomes these shortcomings, while protecting sensitive data against disclosure. 5.1

Problem

The adoption of well-known generalization and suppression techniques results in tables that are less complete and less detailed than the original microdata tables. In fact, the released table is composed of a set of equivalence classes, including all the tuples that have been generalized to the same value for the quasi-identiﬁer. The values of the quasi-identiﬁer in the released table are then less precise than

Protecting Privacy in Data Release

21

the values in the original data collection, thus destroying the correlation among the values of the quasi-identifying attributes and the sensitive attribute. The generalization-based approaches are therefore not suitable for those publishing scenarios where the precision of aggregate queries is of paramount importance and the exact distribution of the values of the quasi-identiﬁer must be released. For instance, if the correlation existing among Sex, ZIP, and Disease is important for the analysis of the impact of an infectious disease, the release of the 4-anonymous table in Figure 4 is of limited utility for the ﬁnal recipient, since both Sex and ZIP have been generalized to a unique value. 5.2

Solutions

An alternative technique to generalization that permits the release of the exact distribution of the quasi-identiﬁer values, while guaranteeing to preserve the privacy of the respondents, is fragmentation. Basically, fragmentation consists in splitting the original microdata table in vertical fragments, such that the attributes composing the quasi-identiﬁer and the sensitive attribute are not represented in the same fragment. In the following, we illustrate two diﬀerent solutions adopting a group-based approach to protect the privacy of the respondents. Anatomy [41] is a group-based proposal addressing the issue of guaranteing diversity in microdata release without resorting to generalization. Anatomy ﬁrst partitions the tuples in the microdata table in groups that satisfy the -diversity principle (i.e., each group includes at least well-represented values for the sensitive attribute). Each group is then associated with a unique group identiﬁer and the microdata table is split into two fragments, F1 and F2 , including the attributes composing the quasi-identiﬁer and the sensitive attribute, respectively. For each tuple, both F1 and F2 report the identiﬁer of the group to which it belongs. For simplicity, each group in the fragment storing the sensitive attribute has a tuple for each sensitive value appearing in the group, and reports the frequency with which the value is represented in the group. For instance, consider the microdata table in Figure 2(a) and assume that the data holder is interested in releasing a 3-diverse table. Figure 9 illustrates the two fragments F1 and F2 obtained by partitioning the tuples in the table in Figure 2(a) in groups that satisfy 3-diversity. Although a malicious recipient may know the quasi-identiﬁer value of a target respondent, she can only infer that the respondent belongs to one group (say, g1 ) in F1 , and that the sensitive value of the target respondent is one of the values in the group in F2 that is in relation with g1 . To illustrate, assume that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. Alice can easily infer that her friend is represented by the ninth tuple of the table in Figure 9(a). However, since the tuples in the third group in Figure 9(a) are in relation with the tuples in the third group in Figure 9(b), Alice can only infer that Barbara suﬀers from either Peptic Ulcer , Broken Leg, or Stomach Cancer . Note that the privacy guarantee oﬀered by Anatomy is exactly the same oﬀered by traditional generalization-based approaches. In fact, a malicious data recipient cannot associate less than diﬀerent

22

S. De Capitani di Vimercati et al.

DoB

Sex ZIP GroupID

GroupID Disease

Count

1940/04/01 1940/04/02 1940/06/07 1940/06/05

M M M M

94143 94142 94130 94131

1 1 1 1

1 1 1

Peptic Ulcer Broken Leg Short Breath

2 1 1

1950/06/02 1950/06/05 1950/05/02 1950/05/05

F F F F

94141 94144 94136 94134

2 2 2 2

2 2 2

H1N1 Pneumonia Flu

2 1 1

1940/04/10 1940/04/20 1940/06/25 1940/06/30

F F F F

94139 94130 94142 94145

3 3 3 3

3 3 3

Peptic Ulcer Broken Leg Stomach Cancer

2 1 1

1950/06/20 1950/06/12 1950/05/10 1950/05/30

M M M M

94132 94137 94147 94148

4 4 4 4

4 4 4

Gastritis Dyspepsia Stomach Cancer

2 1 1

(a) F1

(b) F2

Fig. 9. An example of two fragments satisfying 3-diversity obtained adopting the Anatomy approach

sensitive values with each respondent in the released table. On the other hand, by releasing the exact distribution of the values of the attributes composing the quasi-identiﬁer, the evaluation of aggregate queries can be more precise [41]. Loose associations [15] represent a more ﬂexible solution to guarantee privacy in data publication without adopting generalization. Loose associations have been proposed to protect generic sensitive associations among the attributes in a data collection. For instance, consider the microdata table in Figure 1 and suppose that attribute Treatment is also represented in the table. A possible set of sensitive associations deﬁned among attributes {SSN,Name,DoB, Sex,ZIP,Disease,Treatment} could include: i) both the association between the values of attributes SSN and Disease, and the association between the values of attributes Name and Disease; ii) the association between the values of quasiidentifying attributes DoB, Sex, ZIP and the values of sensitive attribute Disease; iii) the association between the values of attributes Disease and Treatment. Given a set of sensitive associations deﬁned among the attributes included in a microdata table, they are broken by publishing a set of diﬀerent fragments. It is easy to see that the problem of protecting the association of a sensitive attribute with the respondents’ quasi-identiﬁer can be modeled through the deﬁnition of a sensitive association among the sensitive attribute and quasi-identifying attributes. Like Anatomy, the original microdata table can then be split in diﬀerent fragments in such a way that the sensitive attribute is not stored together with all the attributes composing the quasi-identiﬁer. It is in fact suﬃcient to store a subset of the quasi-identifying attributes in a fragment F1 , and all the other quasi-identifying attributes in another fragment F2 , together with the sensitive attribute. For instance, consider the microdata table in Figure 2(a). A fragmentation that would protect against identity and attribute disclosures could be

Protecting Privacy in Data Release 1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/06/07 1940/04/10 1940/06/30 1940/04/02 1940/06/05 1940/06/25 1940/04/20 1950/05/02 1950/05/05 1950/05/10 1950/05/30

F M M F M M F F M M F F F F M M

94141 94132 94137 94144 94143 94130 94139 94145 94142 94131 94142 94130 94136 94134 94147 94148

•ZZZZZZZZ r• ZZZZZZZZZrZrZrZrZ• • rZZZZZ• •ZZZZZZrZrZ r ZZZZZ r •UUUr • UUUU rr UUUU rr• •r U •ZZZZZZZZ rrr U• rZZZZZ• •UUUU rr rUUU U r r • UUUU • rrr U• • •ZZZZZZZZ ZZZmmm• Z • ZZZZZZmZmmm ZZ• m ZZZZZ• •ZZZZmm mmmm ZZZZZZZZZ • • d• d •ZZZZZZd d d d Z d Z d Z ddddd ZZZZ• • • •

23

Peptic Ulcer H1N1 Gastritis Dyspepsia Peptic Ulcer Pneumonia Broken Leg Stomach Cancer Peptic Ulcer H1N1 Short Breath Broken Leg Peptic Ulcer Stomach Cancer Flu Gastritis

Fig. 10. An example of loose association deﬁned on the table in Figure 2(a)

composed of the following two fragments: F1 (DoB,Sex,ZIP) and F2 (Disease). Note that a fragmentation is not unique: F1 (DoB,Sex) and F2 (ZIP,Disease) is another solution that still protects the association between the sensitive attribute and the quasi-identiﬁer. To provide the data recipient with some information on the associations between the quasi-identiﬁer and the sensitive attribute values existing in the original relation, provided a given privacy degree of the association is respected, in [15] the authors propose to publish a loose association between the tuples composing F1 and F2 . The tuples in F1 and in F2 are independently partitioned in groups of size at least k1 and k2 , respectively. Each group in F1 and in F2 is then associated with a diﬀerent group identiﬁer. For each tuple, both F1 and F2 report the identiﬁer of the group to which the tuple belongs. The group-level relationships between the tuples in F1 and in F2 are represented by an additional table A that includes, for each tuple t in the original microdata table, a tuple modeling the relationship between the group where t appears in F1 and the group where t appears in F2 . For instance, Figure 10 represents two fragments F1 and F2 for the microdata table in Figure 2(a). Both the fragments have been partitioned into groups of 2 tuples each and the lines between the tuples in F1 and F2 represent their relationships in the original microdata table. Figure 11 illustrates the three relations, F1 , A, and F2 that are released instead of the original microdata. It is easy to see that, even if a malicious recipient knows the quasi-identiﬁer of a respondent, she can only identify the tuple related to the target respondent in F1 , but not the corresponding Disease in F2 . For instance, assume that Alice knows that her friend Barbara is a female living in 94139 area and born on 1940/04/10. By looking at the released tables, Alice discovers that her friend is represented by the seventh tuple in F1 , which belongs to group dsz4. However, since group dsz4 is associated in A with two diﬀerent groups in F2 (i.e., d4 and d5) Alice cannot identify the illness Barbara suﬀers from, since it could be either Peptic Ulcer , Broken Leg, Stomach Cancer , or H1N1.

24

S. De Capitani di Vimercati et al. F1 DoB 1950/06/02 1950/06/20 1950/06/12 1950/06/05 1940/04/01 1940/04/02 1940/04/10 1940/04/20 1940/06/07 1940/06/05 1940/06/25 1940/06/30 1950/05/02 1950/05/05 1950/05/10 1950/05/30

A

Sex ZIP F M M F M M F F M M F F F F M M

(a)

94141 94132 94137 94144 94143 94142 94139 94130 94130 94131 94142 94145 94136 94134 94147 94148

G dsz1 dsz1 dsz2 dsz2 dsz3 dsz5 dsz4 dsz6 dsz3 dsz5 dsz6 dsz4 dsz7 dsz7 dsz8 dsz8

F2

G1 G2

Disase

G

dsz1 dsz1 dsz2 dsz2 dsz3 dsz3 dsz4 dsz4 dsz5 dsz5 dsz6 dsz6 dsz7 dsz7 dsz8 dsz8

H1N1 Gastritis Dyspepsia Pneumonia Peptic Ulcer Peptic Ulcer Peptic Ulcer Peptic Ulcer Broken Leg Short Breath Broken Leg Stomach Cancer H1N1 Flu Stomach Cancer Gastritis

d1 d2 d2 d3 d1 d3 d5 d7 d4 d6 d6 d4 d5 d8 d7 d8

d1 d2 d2 d3 d1 d4 d5 d4 d3 d6 d7 d6 d5 d8 d7 d8

(b)

(c)

Fig. 11. An example of 4-loose association

The partitioning of the tuples in the two fragments should be carefully designed to guarantee an adequate protection degree. In fact, a loose association enjoys a degree k of protection if every tuple in A indistinguishably corresponds to at least k distinct associations among tuples in the two fragments (i.e., it could have been generated starting from k diﬀerent tuples in the microdata table). The release of F1 , F2 , and A satisﬁes k-looseness, with k ≤ k1 · k2 , if for each group g1 in F1 (group g2 in F2 , respectively), the union of the tuples in all the groups with which g1 (g2 , respectively) is associated in A is a set of at least k diﬀerent tuples. Figure 11 represents an example of a 4-loose association. This implies that it is not possible, for a malicious data recipient, to associate with each quasi-identiﬁer value in F1 less than 4 diﬀerent diseases in F2 .

6

Microdata Publishing with Adversarial External Knowledge

Another source of complication for the protection of microdata is the external (or background) knowledge that an adversary may exploit for inferring information about the respondents represented in the microdata. The research community has recently dedicated many eﬀorts for counteracting this problem whose diﬃculty lies in the fact that the data holder is unaware of the type of knowledge an adversary may have. In this section, we illustrate the privacy problems that may arise due to the adversarial external knowledge, and we present recent protection techniques speciﬁcally designed to consider such knowledge. 6.1

Problem

When publishing a microdata table it is necessary to take into consideration the fact that a malicious recipient may exploit, besides the released table, also

Protecting Privacy in Data Release

25

additional information to infer the sensitive attribute value associated with a target respondent. This knowledge can be obtained by similar data collections released by other organizations or competitors, by social networking sites, or by personal knowledge. For instance, consider the microdata table in Figure 5 and suppose that Alice knows that her neighbor Gabrielle is a female born on 1950/05/02 and living in the 94136 area. If Alice does not have any additional knowledge, by looking at the released microdata table she can only infer that her friend suﬀers from either H1N1 , Pneumonia, or Flu. However, Alice can improve this inference by exploiting external sources of information. For instance, Alice is a close friend of Liz (who appears in the same equivalence class as Gabrielle in the table in Figure 5) and therefore knows that she suﬀers from Pneumonia. As a consequence, Alice can infer that Gabrielle either suﬀers from Flu or H1N1 . However, Gabrielle’s sister Hellen has been recently hospitalized for having contracted H1N1 , and she posted it on her Facebook proﬁle. Since H1N1 is a contagious disease, Alice can infer with high probability that also Gabrielle contracted H1N1 . The problem of protecting released data against malicious recipients exploiting external knowledge has been acknowledged since the ﬁrst introduction of proposals addressing privacy issues in microdata release (see Section 2). However, traditional approaches only consider a few speciﬁc types of external knowledge. For instance, k-anonymity assumes that data recipients only know publicly available datasets associating the identity of respondents with their quasi-identiﬁer. -diversity considers also the fact that a recipient may have additional (personal) knowledge about a subset of the respondents, which permits her to discard a subset of the sensitive values in the equivalence class of a target respondent. Taking external knowledge into consideration when releasing a microdata collection requires the deﬁnition of an adequate modeling of the knowledge that a malicious recipient may possess. This task is complicated by the fact that it is not realistic to assume that data holders have complete knowledge of all the data available to recipients. Furthermore, information is collected and publicly released every day and, consequently, the external information that could be exploited for re-identifying respondents is continuously changing. The external knowledge modeling should therefore be ﬂexible enough to possibly capture any kind of information that might be available to the data recipient. 6.2

Solutions

We summarize some of the most important results modeling the external knowledge of an adversary. (c,k)-Safety [30] introduces a formal modeling of external knowledge assuming a worst case scenario, where the malicious recipient knows the set of respondents represented by a tuple in the published microdata table, and the values of both quasi-identifying and non-sensitive attributes associated with each respondent.

26

S. De Capitani di Vimercati et al.

Besides this identiﬁcation information, a malicious recipient may also possess additional information modeled with the concept of basic unit of knowledge. A basic unit of knowledge is deﬁned as an implication formula (∧i Ai ) → (∨j Bj ), where Ai and Bj are atoms of the form tp [S] = s that represent the fact that respondent p is associated with value s for sensitive attribute S. For instance, suppose that Alice knows that Bob and Carol , who are married, are both sick. Since Flu is highly contagious, Alice knows that if Bob has Flu also Carol suﬀers from the same disease. This knowledge can be modeled through the following basic unit: tBob [Disease]=Flu→tCarol[Disease]=Flu. To avoid unrealistic scenarios where the malicious recipient has unbounded external knowledge, in [30] the authors assume that the knowledge available to a data recipient is composed of at most k basic units of knowledge. Therefore, the overall external knowledge of a recipient is represented as the conjunction ϕ of all her basic units of knowledge. Given the released microdata table T (either obtained using a generalization-based or a group-based approach), the maximum disclosure risk to which the released table is exposed can be computed as the maximum probability that a data recipient with knowledge ϕ can infer that respondent p is associated with value s for sensitive attribute S. More formally, the maximum disclosure of table T is deﬁned as max P (tp [S] = s|T ∧ ϕ), computed on the tuples tp of table T that are consistent with external knowledge ϕ, and that take value s in the domain of S. A released table T satisﬁes (c, k)-safety if the maximum disclosure risk of T is lower than a threshold 0 ≤ c ≤ 1 ﬁxed by the data holder, assuming that malicious recipients have at most k basic units of external knowledge. Given a microdata table, the goal is therefore to determine a corresponding (c, k)-safety microdata table that minimizes the loss of information due to the adoption of generalization-based or group-based approaches. In [30] the authors note that traditional algorithms proposed to guarantee kanonymity can be easily adapted to guarantee (c, k)-safety, since the maximum disclosure is monotonic with respect to generalization. The authors also note that Anatomy [41] can be easily adapted to guarantee (c, k)-safety, since the disclosure decreases merging the groups of tuples in the fragments. Privacy skyline [6] introduces a more intuitive and usable approach than (c, k)safety to measure the external knowledge available to a data recipient. In fact, the deﬁnition of an adequate value of k modeling the adversarial knowledge is not intuitive. Also, (c, k)-safety is based on the assumption that each data respondent is associated with a unique sensitive value and that all the values in the sensitive attribute domain are equally sensitive. Privacy skyline tries to overcome these limitations. The basic idea is that external knowledge can be seen as composed of several categories, which need to be quantiﬁed adopting diﬀerent measures. As a consequence, the disclosure risk cannot be a numeric value, but it is decomposed according to the external knowledge components that are considered relevant for the speciﬁc scenario. In [6], the authors classify the external knowledge in the following three categories:

Protecting Privacy in Data Release

27

– the knowledge about a target individual (e.g., Alice knows that Bob does not suﬀer from Cancer ); – the knowledge about individuals other than the target individual (e.g., Alice knows that Carol , who is also a respondent of the considered microdata table, suﬀers from H1N1 ); – the knowledge about same-value families that model the fact that a group (or family) of respondents have the same value for the sensitive attribute (e.g., Alice knows that David and Hellen are married and, therefore, if any of them suﬀers from Flu, it is highly probable that also the other person suﬀers from Flu). It is important to note that the composition of a same-value family is value speciﬁc, meaning that the relationship among users may be diﬀerent depending on the speciﬁc sensitive value s considered. For instance, the same-value family for an infectious disease may be composed of colleagues and people leaving in the same house, while the same-value family for HIV may only include a married couple. The external knowledge available to a malicious recipient is expressed as a triple (,k,m) that quantify, for each of the categories illustrated above, the amount of knowledge held by the recipient for each sensitive value s. A triple (,k,m) indicates that the recipient knows: sensitive values that the target respondent does not have; the sensitive value associated with k individuals diﬀerent from the target respondent; and m individuals in the same-value family of the target respondent. For instance, suppose that, with respect to respondent Ron and sensitive value H1N1, Alice knows that Ron does not suﬀer from Ovarian Cancer and Measles ( = 2). Also, suppose that Alice already knows the diseases of other three respondents represented in the table where Ron is represented (k = 3). Finally, suppose that Alice knows that other ﬁve people work in Ron’s oﬃce (m = 6). As a consequence, the triple expressing such external knowledge of Alice is (2,3,6). The release of a microdata table T is safe if the maximum probability that a data recipient can infer that an arbitrary respondent p is associated with sensitive value s in the original dataset is lower than a threshold c ﬁxed by the data holder, assuming the recipient’s external knowledge to be bounded by (, k, m). For instance, assume that the external knowledge of a malicious recipient for an arbitrary target respondent p and sensitive value H1N1 is represented by the triple (2,5,1) and that c=0.5. A released table is considered safe if the malicious recipient cannot determine with probability higher than 0.5 that p suﬀers from H1N1 , provided the malicious user knows: at most 2 sensitive attribute values that p does not have; the sensitive value of at most 5 respondents other than p; and at most 1 member in the same-value family as p. To provide the data holder with a more ﬂexible privacy measure, in [6] the authors propose to bound the external knowledge not through a unique triple (,k,m), but through a set of triples (representing incomparable points in the three-dimensional space ,k,m) that form a skyline. The release of a microdata table T , protected applying generalization or group-based techniques, is safe if the probability that the malicious recipient violates the privacy of the respondents is lower than

28

S. De Capitani di Vimercati et al.

4

(1,4)

≤1 (3,3)

3 m 2 (5,1)

1

1

2

3 k

4

5

Fig. 12. An example of privacy skyline obtained ﬁxing three points {(1,1,4), (1,3,3),(1,5,1)}

threshold c, assuming that the external knowledge of the recipient is bounded by the skyline (e.g., the grey area in Figure 12, depicted in only two dimensions for simplicity, represents the area in which the external knowledge of the recipient is bounded). In [6] the authors extend their deﬁnition of safe release to take into account the fact that each respondent may be associated with more than one sensitive attribute value (e.g., a respondent that suﬀers from more than one disease). -Privacy [28] is aimed at deﬁning a realistic adversarial model. Indeed, traditional approaches either assume that the adversary has a very limited knowledge, or an inﬁnite knowledge. Both these assumptions are however unrealistic. To overcome this issue, in [28] the authors propose to measure the privacy degree oﬀered to a respondent p as the diﬀerence in the recipient’s belief about the respondent’s sensitive attribute value when p belongs to the released table and when p does not belong to the same. This measure is based on the observation that looking at the released dataset, a data recipient can learn information about the sensitive value of a respondent p, even if the tuple representing p is not released [22]. Therefore, it is necessary to consider this unavoidable gain of information when evaluating the quality of a released table. In particular, -privacy states that the release of a table T obtained through generalization exposes the in privacy of a respondent if the ratio ppout is greater than a threshold ﬁxed by the data holder, where pin (pout , respectively) is the probability that a recipient infers the sensitive value associated with an arbitrary respondent p when p is not represented in T (belongs to T , respectively). To compute pin and pout , the authors consider two diﬀerent kinds of external knowledge: i) full information on a subset of the tuples in T ; and ii) information on the distribution of the sensitive attribute values in the dataset. We note that, although both (c, k)-safety and privacy skyline consider the ﬁrst kind of external knowledge, they do not make any assumption on the second kind of knowledge modeled by -privacy.

Protecting Privacy in Data Release

7

29

Diﬀerential Privacy

The proposals described so far typically measure the disclosure risk associated with the release of a microdata table as the increase of the probability that an adversary may correctly guess the identity or the values of sensitive attributes of a respondent represented in the table. Such approaches do not consider that the microdata table can also be exploited for inferring information of respondents that are not represented in the table. In the remainder of this section, we ﬁrst illustrate in more details such a privacy issue, and then brieﬂy describe diﬀerential privacy, a recent privacy notion that is becoming popular in the data protection community. 7.1

Problem

One of the ﬁrst deﬁnitions of privacy in data publishing scenarios states that: anything that can be learned about a respondent from the statistical database should be learnable without access to the database [13]. Although this deﬁnition has been thought for statistical databases, it is also well suited for the microdata publishing scenario. Unfortunately, this deﬁnition of ideal privacy cannot be achieved by any privacy-aware microdata publication technique that is aimed at preserving data utility. In fact, as proved in [16], only an empty dataset can guarantee absolute disclosure prevention. This is also due to the fact that the release of a dataset may violate the privacy of any respondent, independently of whether the respondent is represented in the dataset. For instance, suppose that the released dataset permits to compute the average annual income of people living in city A for each ethnic group, and suppose that this information is not publicly available (and therefore a malicious recipient can only gain this information by looking at the released table). Assume also that Alice knows that Bob’s annual income is 1,000$ more than the average annual income of Asian people living in city A. Although this piece of information alone does not permit Alice to gain any information about Bob’s annual income, if combined with the released dataset, it allows Alice to infer Bob’s annual income. It is important to note that the disclosure of Bob’s annual income does not depend on his representation in the released dataset. The solutions proposed in the literature for protecting microdata tables implicitly assume that the privacy of individuals not included in the dataset is not at risk. As a consequence, they cannot be adopted to prevent the attack described above. 7.2

Solution

Diﬀerential privacy [16] is a novel privacy notion whose goal is to guarantee that the release of a microdata table does not disclose sensitive information about any individual, represented or not by a tuple in the table. In particular, a data release is considered safe if the inclusion in the dataset of tuple tp , related to respondent p, does not change the probability that a malicious recipient can

30

S. De Capitani di Vimercati et al.

correctly identify the sensitive attribute value associated with p. More formally, given two datasets T and T diﬀering only for one tuple tp , an arbitrary randomized function K operating on the dataset satisﬁes -diﬀerential privacy if and only if P (K(T ) ∈ S) ≤ exp() · P (K(T ) ∈ S), where S is a subset of the possible outputs of function K and is a public privacy parameter. Intuitively, -diﬀerential privacy holds if the removal (insertion, respectively) of one tuple tp from (into, respectively) the dataset does not signiﬁcatively aﬀect the result of the evaluation of function K. As an example, consider an insurance company that consults a medical dataset to decide whether an individual p is eligible for an insurance contract. If diﬀerential privacy is satisﬁed, the presence or absence of tuple tp representing p in the dataset does not signiﬁcantly aﬀect the ﬁnal decision by the insurance company. It is also important to note that the external knowledge that an adversary may possess cannot be exploited for breaching the privacy of individuals. In fact, the knowledge that the recipient gains looking at the released dataset is bounded by a multiplicative factor exp(), for any individual either represented or not in the released microdata table. Diﬀerential privacy is applicable to both the non-interactive publishing scenario (i.e., public release of a dataset) and the interactive publishing scenario (i.e., evaluation of queries over a private dataset). The techniques proposed in the literature to guarantee -diﬀerential privacy are based on the addition of noise, and therefore do not preserve data truthfulness. To achieve diﬀerential privacy, by deﬁnition, it is necessary to hide the presence (or absence) of the tuple associated with an individual. Therefore, considering simple count queries, the query result returned to the requesting recipient may diﬀer by at most one (0 if the tuple is not removed; 1 otherwise) from the result computed on the original dataset. To compute the query result, the solution proposed in [17] consists in adding random noise to the query result evaluated on the original dataset. The distribution considered for the random noise is the Laplace distribution Lap(Δ(f )/) with probability density function P (x|b) = exp(−|x|/b)/2b, where b = Δ(f )/ and Δ(f ) is the maximum diﬀerence between the query result evaluated over T and over T (which is equal to 1 for count queries, since T and T diﬀer for one tuple). The addition of independently generated noise, with distribution Lap(Δ(f )/), to the query result guarantees -diﬀerential privacy [17]. This strategy can also be adopted in a non-interactive scenario, where the data holder releases the frequency matrix representing the dataset and each cell in the matrix is the result of a count query. The frequency matrix has a dimension for each attribute in the table and the entries in each dimension are labeled with the values in the attribute domain. Each cell in the matrix reports the number of tuples in the table with value, for each attribute, equal to the label of the corresponding entry in the frequency matrix.

8

Conclusions

The public and semi-public release of large microdata collections is becoming increasingly popular, thanks to the high availability of computational power at

Protecting Privacy in Data Release

31

low prices, which makes data analysis an easy task for most data recipients. Although microdata collections represent a valuable resource for data recipients, the release of ﬁne-grained information related to single individuals may put the privacy of respondents at risk. In this chapter, we ﬁrst illustrated the traditional approaches designed for preventing identity and attribute disclosure in microdata publishing. We then discussed the privacy risks that may arise when changing the underlying assumptions, and described some techniques recently proposed in the literature to overcome these privacy risks. Acknowledgments. This work was supported in part by the EU within the 7FP project “PrimeLife” under grant agreement 216483, by the Italian Ministry of Research within the PRIN 2008 project “PEPPER” (2008SY2PH4), and by the Universit` a degli Studi di Milano within the “UNIMI per il Futuro - 5 per Mille” project “PREVIOUS”.

References 1. Aggarwal, G., Feder, T., Kenthapadi, K., Motwani, R., Panigrahy, R., Thomas, D., Zhu, A.: Approximation algorithms for k-anonymity. Journal of Privacy Technology (November 2005) 2. Azzini, A., Marrara, S., Sassi, R., Scotti, F.: A fuzzy approach to multimodal biometric continuous authentication. Fuzzy Optimization and Decision Making 7(3), 215–302 (2008) 3. Barbaro, M., Zeller, T.: A face is exposed for AOL searcher no. 4417749. New York Times, August 9 (2006) 4. Bayardo, R.J., Agrawal, R.: Data privacy through optimal k-anonymization. In: Proc. of the 21st IEEE International Conference on Data Engineering (ICDE 2005), Tokyo, Japan (April 2005) 5. Bezzi, M., De Capitani di Vimercati, S., Livraga, G., Samarati, P.: Protecting privacy of sensitive value distributions in data release. In: Proc. of the 6th Workshop on Security and Trust Management (STM 2010), Athens, Greece (September 2010) 6. Chen, B.C., LeFevre, K., Ramakrishnan, R.: Privacy skyline: Privacy with multidimensional adversarial knowledge. In: Proc. of the 33rd International Conference on Very Large Data Bases (VLDB 2007), Vienna, Austria (September 2007) 7. Cimato, S., Gamassi, M., Piuri, V., Sassi, R., Scotti, F.: Privacy-aware biometrics: Design and implementation of a multimodal veriﬁcation system. In: Proc. of the 24th Annual Computer Security Applications Conference (ACSAC 2008), Anaheim, CA, USA (December 2008) 8. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Fragmentation design for eﬃcient query execution over sensitive distributed databases. In: Proc. of the 29th International Conference on Distributed Computing Systems (ICDCS 2009), Montreal, Canada (June 2009) 9. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Combining fragmentation and encryption to protect privacy in data storage. ACM Transactions on Information and System Security (TISSEC) 13(3), 22:1–22:33 (2010)

32

S. De Capitani di Vimercati et al.

10. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: k-Anonymity. In: Yu, T., Jajodia, S. (eds.) Secure Data Management in Decentralized Systems. Springer, Heidelberg (2007) 11. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: Microdata protection. In: Yu, T., Jajodia, S. (eds.) Secure Data Management in Decentralized Systems. Springer, Heidelberg (2007) 12. Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P.: Theory of privacy and anonymity. In: Atallah, M., Blanton, M. (eds.) Algorithms and Theory of Computation Handbook, 2nd edn. CRC Press, Boca Raton (2009) 13. Dalenius, T.: Towards a methodology for statistical disclosure control. Statistik Tidskrift 15, 429–444 (1977) 14. De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Encryption policies for regulating access to outsourced data. ACM Transactions on Database Systems (TODS) 35(2), 12:1–12:46 (2010) 15. De Capitani di Vimercati, S., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Fragments and loose associations: Respecting privacy in data publishing. Proc. of the VLDB Endowment 3(1), 1370–1381 (2010) 16. Dwork, C.: Diﬀerential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006) 17. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating Noise to Sensitivity in Private Data Analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006) 18. Frikken, K.B., Zhang, Y.: Yet another privacy metric for publishing micro-data. In: Proc. of the 7th Workshop on Privacy in Electronic Society (WPES 2008), Alexandria, VA, USA (October 2008) 19. Gamassi, M., Lazzaroni, M., Misino, M., Piuri, V., Sana, D., Scotti, F.: Accuracy and performance of biometric systems. In: Proc. of the 2004 IEEE Instrumentation & Measurement Technology Conference (IMTC 2004), Como, Italy (May 2004) 20. Gamassi, M., Piuri, V., Sana, D., Scotti, F.: Robust ﬁngerprint detection for access control. In: Proc. of the 2nd RoboCare Workshop (RoboCare 2005), Rome, Italy (May 2005) 21. Golle, P.: Revisiting the uniqueness of simple demographics in the US population. In: Proc. of the 5th Workshop on Privacy in the Electronic Society (WPES 2006), Alexandria, VA, USA (October 2006) 22. Kifer, D.: Attacks on privacy and deFinetti’s theorem. In: Proc. of the 2009 ACM SIGMOD International Conference on Management of Data (SIGMOD 2009), Providence, RI, USA (June 2009) 23. LeFevre, K., DeWitt, D.J., Ramakrishnan, R.: Incognito: Eﬃcient full-domain kanonymity. In: Proc. of the 2005 ACM SIGMOD International Conference on Management of Data (SIGMOD 2005), Baltimore, MD, USA (June 2005) 24. LeFevre, K., DeWitt, D.J., Ramakrishnan, R.: Mondrian multidimensional kanonymity. In: Proc. of the 22nd IEEE International Conference on Data Engineering (ICDE 2006), Atlanta, GA, USA (April 2006) 25. Li, F., Sun, J., Papadimitriou, S., Mihaila, G., Stanoi, I.: Hiding in the crowd: Privacy preservation on evolving streams through correlation tracking. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 26. Li, N., Li, T., Venkatasubramanian, S.: t-closeness: Privacy beyond k-anonymity and -diversity. In: Proc.of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007)

Protecting Privacy in Data Release

33

27. Li, T., Li, N.: On the tradeoﬀ between privacy and utility in data publishing. In: Proc. of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2009), Paris, France (June-July 2009) 28. Machanavajjhala, A., Gehrke, J., G¨ otz, M.: Data publishing against realistic adversaries. Proc. of the VLDB Endowment 2(1), 790–801 (2009) 29. Machanavajjhala, A., Kifer, D., Gehrke, J., Venkitasubramaniam, M.: -diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD) 1(1), 3:1–3:52 (2007) 30. Martin, D.J., Kifer, D., Machanavajjhala, A., Gehrke, J., Halpern, J.Y.: Worstcase background knowledge for privacy-preserving data publishing. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 31. Meyerson, A., Williams, R.: On the complexity of optimal k-anonymity. In: Proc. of the 23rd ACM-SIGMOD-SIGACT-SIGART Symposium on the Principles of Database Systems (PODS 2004), Paris, France (June 2004) 32. Narayanan, A., Shmatikov, V.: Robust de-anonymization of large sparse datasets. In: Proc. of the 2008 IEEE Symposium on Security and Privacy (SP 2008), Berkeley/Oakland, CA, USA (May 2008) 33. Narayanan, A., Shmatikov, V.: Myths and fallacies of personally identiﬁable information. Communications of the ACM (CACM) 53, 24–26 (2010) 34. Nergiz, M.E., Atzori, M., Clifton, C.: Hiding the presence of individuals from shared databases. In: Proc. of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD 2007), Beijing, China (June 2007) 35. Nergiz, M., Clifton, C., Nergiz, A.: Multirelational k-anonymity. In: Proc. of the 23rd IEEE International Conference on Data Engineering (ICDE 2007), Istanbul, Turkey (April 2007) 36. Pei, J., Tao, Y., Li, J., Xiao, X.: Privacy preserving publishing on multiple quasiidentiﬁers. In: Proc. of the 25th IEEE International Conference on Data Engineering (ICDE 2009), Shanghai, China (March-April 2009) 37. Samarati, P.: Protecting respondents’ identities in microdata release. IEEE Transactions on Knowledge and Data Engineering (TKDE) 13(6), 1010–1027 (2001) 38. Terrovitis, M., Mamoulis, N., Kalnis, P.: Privacy-preserving anonymization of setvalued data. Proc. of the VLDB Endowment 1, 115–125 (2008) 39. Wang, K., Fung, B.C.M.: Anonymizing sequential releases. In: Proc. of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2006), Philadelphia, PA, USA (August 2006) 40. Wang, K., Xu, Y., Wong, R., Fu, A.: Anonymizing temporal data. In: Proc. of the 2010 IEEE International Conference on Data Mining (ICDM 2010), Sydney, Australia (December 2010) 41. Xiao, X., Tao, Y.: Anatomy: Simple and eﬀective privacy preservation. In: Proc. of the 32nd International Conference on Very Large Data Bases (VLDB 2006), Seoul, Korea (September 2006) 42. Xiao, X., Tao, Y.: Personalized privacy preservation. In: Proc. of the 2006 ACM SIGMOD International Conference on Management of Data (SIGMOD 2006), Chicago, IL, USA (June 2006) 43. Xiao, X., Tao, Y.: m-invariance: Towards privacy preserving re-publication of dynamic datasets. In: Proc. of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD 2007), Beijing, China (June 2007)

34

S. De Capitani di Vimercati et al.

44. Xiao, X., Yi, K., Tao, Y.: The hardness and approximation algorithms for diversity. In: Proc. of the 13th International Conference on Extending Database Technology (EDBT 2010), Lausanne, Switzerland (March 2010) 45. Zhou, B., Han, Y., Pei, J., Jiang, B., Tao, Y., Jia, Y.: Continuous privacy preserving publishing of data streams. In: Proc. of the 12th International Conference on Extending Database Technology (EDBT 2009), Saint Petersburg, Russia (March 2009)

An Introduction to Security API Analysis Riccardo Focardi1 , Flaminia L. Luccio1 , and Graham Steel2 1 2

1

DAIS, Universit` a Ca’ Foscari Venezia, Italy {focardi,luccio}@dsi.unive.it LSV, ENS Cachan & CNRS & INRIA, France [email protected]

Introduction

A security API is an Application Program Interface that allows untrusted code to access sensitive resources in a secure way. Examples of security APIs include the interface between the tamper-resistant chip on a smartcard (trusted) and the card reader (untrusted), the interface between a cryptographic Hardware Security Module, or HSM (trusted) and the client machine (untrusted), and the Google maps API (an interface between a server, trusted by Google, and the rest of the Internet). The crucial aspect of a security API is that it is designed to enforce a policy, i.e. no matter what sequence of commands in the interface are called, and no matter what the parameters are, certain security properties should continue to hold. This means that if the less trusted code turns out to be malicious (or just faulty), the carefully designed API should prevent compromise of critical data. Designing such an interface is extremely tricky even for experts. A number of security ﬂaws have been found in APIs in use in deployed systems in the last decade. In this tutorial paper, we introduce the subject of security API analysis using formal techniques. This approach has recently proved highly successful both in ﬁnding new ﬂaws and verifying security properties of improved designs. We will introduce the main techniques, many of which have been adapted from language-based security and security protocol veriﬁcation, by means of two case studies: cryptographic key management, and Personal Identiﬁcation Number (PIN) processing in the cash machine network. We will give plenty of examples of API attacks, and highlight the areas where more research is needed.

2

Background

The ﬁrst security vulnerability that may properly be called an ‘API attack’, and thus highlighted the security API as a critical design point, was discovered by Longley and Rigby in the early 1990s [41]. Their article showed how the logic programming language Prolog could be used to analyse a key management interface of a cryptographic device. Although the device was not identiﬁed at

Work partially supported by the RAS Project “TESLA: Techniques for Enforcing Security in Languages and Applications”.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 35–65, 2011. c Springer-Verlag Berlin Heidelberg 2011

36

R. Focardi, F.L. Luccio, and G. Steel

the time, it later became known that it was an HSM manufactured by Eracom and used in the cash machine network. In 2000, Anderson published an attack on key loading procedures on another similar module manufactured by Visa [5], and the term ‘security API’ was coined by Bond and Anderson [12,13] in two subsequent papers giving more attacks. Clayton and Bond showed how one of their more computationally intensive attacks could be implemented against a real IBM device using programmable FPGA hardware [21]. Independently from the Cambridge group, an MSc thesis by Clulow gave more examples of attacks, mostly speciﬁc to the PIN translation and veriﬁcation commands oﬀered by the API of Prism HSMs [22]. Clulow also published attacks on the industry standard for cryptographic key management APIs, RSA PKCS#11 [23]. Up until this point all the attacks had been discovered by manual analysis or by ad-hoc semi-formal techniques speciﬁc to the particular API under consideration. A ﬁrst eﬀort to apply more general formal tools, speciﬁcally the automatic ﬁrst-order theorem prover Otter, was not especially successful, and the results remain unpublished (though they are available in a technical report [52]). The researchers were unable to discover any new attacks, and because the modelling lacked formal groundwork, when no attacks were found they were unable to conclude anything about the security of the device. One member of the team later remarked that “It ended up being more about how to use the tools than about analysing the device” [35]. Meanwhile, the formal analysis of protocols for e.g. cryptographic key exchange and authentication had become a mature ﬁeld. A particularly successful approach had centred around the so-called Dolev-Yao (DY) abstract model, where bitstrings are modelled as terms in an abstract algebra, and cryptographic functions are functions on these terms [27]. Together with suitable abstractions, lazy evaluation rules and other heuristics, this model has proved highly amenable to automated analysis, by model checking or theorem proving techniques [8,11,28,42]. Modern automated tools can check secrecy and authentication properties of (abstract models of) widely-used protocols such as Kerberos and TLS in a few seconds. The idea of applying protocol analysis techniques to the analysis of security APIs seemed very attractive. However, initial experiments showed that existing tools were not suitable for the problem [15,38] for a number of reasons: In particular many of the attacks pertinent to security APIs are outside the scope of the normal DY model. For example, they might involve an attacker learning a secret value, such as a cash machine PIN, by observing error messages returned by the API (a so-called error oracle attack). Furthermore, the functionality of security APIs typically depends on global mutable state which may loop, a feature which invalidates many abstractions and optimisations made by protocol analysis tools, particularly when freshly generated nonces and keys are considered. There are also problems of scale - a protocol might describe an exchange of 5 messages between two participants, while an API will typically oﬀer dozens of commands.

An Introduction to Security API Analysis

37

In the sections that follow, we will show how formal analysis techniques have been adapted to security API problems. The aim of this paper is to serve as an introduction to the ﬁeld. As such, the technical material is introduced by means of two case studies. The ﬁrst is based around the RSA standard for cryptographic interfaces PKCS#11, while the second concerns commands for PIN processing using HSMs in the cash machine network. We follow these two case studies with a discussion of open problems.

3

Key Management with RSA PKCS#11

Cryptographic key management, i.e. the secure creation, storage, backup, use and destruction of keys has long been identiﬁed as a major challenge in applied cryptography. Indeed, Schneier calls it “the hardest part of cryptography and often the Achilles’ heel of an otherwise secure system.” [48]. In real-world applications, key management often involves the use of HSMs or cryptographic tokens, since these are considered easier to secure than commodity hardware, and indeed are mandated by standards in certain sectors [37]. There is also a growing trend towards enterprise-wide schemes based around key management servers oﬀering cryptographic services over open networks [18]. All these solutions aim to enforce security by dividing the system into trusted parts (HSM, server) and untrusted parts (host computer, the rest of the network). The trusted part makes cryptographic functions available via an API. Hence we arrive at our ﬁrst security API problem: how to design an interface for a key management device so that we can create, delete, import and export keys from the device, as well as permitting their use for encryption, decryption, signature and veriﬁcation, all so that if the device comes into contact with a malicious application we can be sure the keys stay secure. This is far from trivial, as well will see from our case study of the most widely used standard for such interfaces, RSA PKCS#11. 3.1

The PKCS#11 Standard

RSA Public Key Cryptography Standards (PKCS) aim to standardise various aspects of cryptography to promote interoperability and security. PKCS#1, for example, deﬁnes the RSA asymmetric encryption and signing algorithm. PKCS#11 describes the ‘Cryptoki’ API for cryptographic hardware. Version 1.0 was published in 1995. The latest oﬃcial version is v2.20 (2004) which runs to just under 400 pages [47]. Adoption of the standard is almost ubiquitous in commercial cryptographic tokens and smartcards, even if other interfaces are frequently oﬀered in addition. In a PKCS#11-based API, applications initiate a session with the cryptographic token, by supplying a PIN. Note that if malicious code is running on the host machine, then the user PIN may easily be intercepted, e.g. by a keylogger or by a tampered device driver, allowing an attacker to create his own sessions with the device, a point conceded in the security discussion in the standard [47, p. 31]. PKCS#11 is intended to protect its sensitive cryptographic keys even when connected to a compromised host.

38

R. Focardi, F.L. Luccio, and G. Steel

Once a session is initiated, the application may access the objects stored on the token, such as keys and certiﬁcates. However, access to the objects is controlled. Objects are referenced in the API via handles, which can be thought of as pointers to or names for the objects. In general, the value of the handle, e.g. for a secret key, does not reveal any information about the actual value of the key. Objects have attributes, which may be bitstrings e.g. the value of a key, or Boolean ﬂags signalling properties of the object, e.g. whether the key may be used for encryption, or for encrypting other keys. New objects can be created by calling a key generation command, or by ‘unwrapping’ an encrypted key packet. In both cases a new handle is returned. When a function in the token’s API is called with a reference to a particular object, the token ﬁrst checks that the attributes of the object allow it to be used for that function. For example, if the encrypt function is called with the handle for a particular key, that key must have its encrypt attribute set. To protect a key from being revealed, the attribute sensitive must be set to true. This means that requests to view the object’s key value via the API will result in an error message. Once the attribute sensitive has been set to true, it cannot be reset to false. This gives us the principal security property stated in the standard: attacks, even if they involve compromising the host machine, cannot “compromise keys marked ‘sensitive’, since a key that is sensitive will always remain sensitive”, [47, p. 31]. Such a key may be exported outside the device if it is encrypted by another key, but only if its extractable attribute is set to true. An object with an extractable attribute set to false may not be read by the API, and additionally, once set to false, the extractable attribute cannot be set to true. Protection of the keys essentially relies on the sensitive and extractable attributes. 3.2

The Wrap-Decrypt Attack

Clulow ﬁrst published attacks on PKCS#11 based APIs in 2003 [23], where he gave many examples of ways in which keys with the sensitive attribute set to true could be read in clear outside the device. The most straightforward of these is the ‘key separation’ attack, where the attributes of a key are set in such a way as to give a key conﬂicting roles. Clulow gives the example of a key with the attributes set for decryption of ciphertexts, and for ‘wrapping’, i.e. encryption of other keys for secure transport. To determine the value of a sensitive key, the attacker simply wraps it and then decrypts it, as shown in Fig. 1. The attack is described with a notation for PKCS#11 based APIs brieﬂy deﬁned in the following and more formally in the next section: h(n1 , k1 ) is a predicate stating that there is a handle encoded by n1 for a key k1 stored on the device. The symmetric encryption of k1 under key k2 is represented by {|k1}| k2 . Note also that according to the wrapping formats deﬁned in PKCS#11, the device cannot tell whether an arbitrary bitstring is a cryptographic key or some other piece of plaintext. Thus when it executes the decrypt command, it has no way of telling that the packet it is decrypting contains a key.

An Introduction to Security API Analysis

39

Initial knowledge: The intruder knows h(n1 , k1 ) and h(n2 , k2 ). The name n2 has the attributes wrap and decrypt set whereas n1 has the attribute sensitive and extractable set. Trace: Wrap: h(n2 , k2 ), h(n1 , k1 ) → {|k1}| k2 SDecrypt: h(n2 , k2 ), {|k1}| k2 → k1

Fig. 1. Wrap/Decrypt attack

It might appear easy to prevent such an attack, but as we shall see, it is in fact rather diﬃcult within the conﬁnes of PKCS#11. Before treating this in detail, we will introduce our language for formal modelling of the API. 3.3

Formal Model

Our model follows the approach used by Delaune, Kremer and Steel (DKS) [25]. The device is assumed to be connected to a host under the complete control of an intruder, representing a malicious piece of software. The intruder can call the commands of the API in any order he likes using any values that he knows. We abstract away details of the cryptographic algorithms in use, following the classical approach of Dolev and Yao [27]. Bitstrings are modelled as terms in an abstract algebra. The rules of the API and the abilities of an attacker are written as deduction rules in the algebra. Basic Notions. We assume a given signature Σ, i.e. a ﬁnite set of function symbols, with an arity function ar : Σ → N, a (possibly inﬁnite) set of names N and a (possibly inﬁnite) set of variables X . Names represent keys, data values, nonces, etc. and function symbols model cryptographic primitives, e.g. {|x|}y representing symmetric encryption of plaintext x under key y, and {x}y representing public key encryption of x under y. Function symbols of arity 0 are called constants. This includes the Boolean constants true () and false (⊥). The set of plain terms PT is deﬁned by the following grammar: t, ti := x | n | f (t1 , . . . , tn )

x∈X n∈N f ∈ Σ and ar (f ) = n

We also consider a ﬁnite set F of predicate symbols, disjoint from Σ, from which we derive a set of facts. The set of facts is deﬁned as F T = {p(t, b) | p ∈ F , t ∈ PT , b ∈ {, ⊥}} In this way, we can explicitly express the Boolean value b of an attribute p on a term t by writing p(t, b). For example, to state that the key referred to by n has the wrap attribute set we write wrap(n, ).

40

R. Focardi, F.L. Luccio, and G. Steel

The description of a system is given as a ﬁnite set of rules of the form new n ˜

T ; L −−−→ T ; L where T, T ⊆ PT are sets of plain terms L, L ⊆ F are sets of facts and n ˜⊆N is a set of names. The intuitive meaning of such a rule is the following. The rule can be ﬁred if all terms in T are in the intruder knowledge and if all the facts in L hold in the current state. The eﬀect of the rule is that terms in T are added to the intruder knowledge and the valuation of the attributes is updated to satisfy L . The new n ˜ means that all the names in n ˜ need to be replaced by fresh names in T and L . This allows us to model nonce or key generation: if the rule is executed several times, the eﬀects are diﬀerent as diﬀerent names will be used each time. Example 1. The following rule models wrapping: h(x1 , y1 ), h(x2 , y2 ); wrap(x1 ,), extract(x2 ,) → {|y2 }| y1 Intuitively, h(x1 , y1 ) is a handle x1 for key y1 while term {|y2 }| y1 represents a key y2 wrapped with y1 . Since the attribute wrap for key y1 is set, noted as wrap(x1 ,), and key y2 is extractable, written extract(x2 ,), then we can wrap y2 with y1 , creating {|y2 }| y1 . The semantics of the model is deﬁned in a standard way in terms of a transition system. Each state in the model consists of a set of terms in the intruder’s knowledge, and a set of global state predicates. The intruder’s knowledge increases monotonically with each transition (he can only learn more things), but the global state is non-monotonic (attributes may be set on and then oﬀ). For a formal semantics, we refer to the literature [25]. We now present in Fig. 2 a subset of PKCS#11 commands suﬃcient for some basic symmetric key management commands (the asymmetric key has also been treated in the literature [26]). This will suﬃce to demonstrate the modelling technique and some attacks. 3.4

Using the Formal Model

We ﬁrst add some rules to the model for the intruder that allow him to encrypt and decrypt using his own keys (see Fig. 3). The intruder is assumed not to be able to crack the encryption algorithm by brute-force search or similar means, thus he can only read an encrypted message if he has the correct key. We analyse security as reachability, in the model, of attack states, i.e. states where the intruder knows the value of a key stored on the device with the sensitive attribute set to true, or the extractable attribute set to false. We give the intruder some initial knowledge, typically just some key ki that is not loaded on the device, and then use a tool such as a model checker to search the model for a chain of commands and intruder steps that leads to an attack state. Unfortunately we immediately encounter both theoretical and practical problems. First, we know that the reachability problem in general for languages like

An Introduction to Security API Analysis

41

Fig. 2. PKCS#11 Symmetric Key Fragment

x, y → {|x|}y {|x|}y , y → x Fig. 3. Intruder rules for symmetric key cryptography

this is undecidable: even with ﬁxed message size there is a reduction to the Post correspondence problem [29, table 12, page 298]. Even so we might hope to ﬁnd some attacks in practice. But in fact the model checker is quickly swamped by the combinatorial possibilities, many of which at ﬁrst sight seem unlikely to lead to an attack. For example, the intruder can take an already encrypted term {|k1}| k2 , and use it as input to the encrypt command along with some handle h(n, k3 ) to obtain {|{|k1}| k2 }| k3 . Unfortunately one cannot in general just delete these terms from the intruder knowledge: they may be a necessary step for some ﬁendish attack. Fortunately we can address both the the theoretical and practical problems at once, by means of a well-modedness result [26]. There it is shown that each function symbol such as h(., .), {|.|}. can be given a unique interpretation in terms of modes. For example, h(., .) has mode Nonce × Key → Handle. We assign to each constant symbol a mode. A term is well-moded if all the function symbols in it are applied to symbols such that the modes are respected. Furthermore, any reachability query that can be expressed with well-moded terms is satisﬁable if and only if it it is reachable by a sequence of steps where the intruder learns only well-moded terms. This allows us to prune the search space dramatically, since any branch that results in the creation of an ill-moded term can be ignored. If

42

R. Focardi, F.L. Luccio, and G. Steel

the function symbols can be moded acyclically, we can also show decidability provided we bound fresh handles and keys. Consult the paper for details and proofs [26]. 3.5

A Suite of Attacks

Equipped with a suitable formal model and a model checker, we can attempt to ﬁnd secure conﬁgurations of the standard. Clulow’s ﬁrst suggestion for preventing the attack in Fig. 1 is to prevent attribute changing operations from allowing a stored key to have both wrap and decrypt set. Note that in order to do this, it is not suﬃcient merely to check that decrypt is unset before setting wrap, and to check wrap is unset before setting decrypt. One must also add wrap and decrypt to a list of ‘sticky’ attributes which once set, may not be unset, or the attack is not prevented, [51]. Eﬀectively this means the unset rules will be omitted from the model for these attributes. Having applied these measures, we discover the attack given in Fig. 4. The intruder imports his own key k3 by ﬁrst encrypting it under k2 , and then unwrapping it. He can then export the sensitive key k1 under k3 to discover its value. Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ) and the key k3 ; n1 has the attributes sensitive and extract set whereas n2 has the attributes unwrap and encrypt set. Trace: Encrypt: h(n2 , k2 ), k3 → {|k3}| k2 new n3 Unwrap: h(n2 , k2 ), {|k3}| k2−−−→h(n 3 , k3 ) Set wrap: h(n3 , k3 ) → wrap(n3 , ) Wrap: h(n3 , k3 ), h(n1 , k1 ) → {|k1}| k3 Intruder: {|k1}| k3 , k3 → k1

Fig. 4. Attack using encrypt and unwrap

To prevent the attack shown in Fig. 4, we add encrypt and unwrap to the list of conﬂicting attribute pairs. Another new attack is discovered (see Fig. 5) of a type discussed by Clulow, [23, Section 2.3]. Here the key k2 is ﬁrst wrapped under k2 itself, and then unwrapped, gaining a new handle h(n4 , k2 ). The intruder then wraps k1 under k2, and sets the decrypt attribute on handle h(n4 , k2 ), allowing him to obtain k1. One can attempt to prevent the attack in Fig. 5 by adding wrap and unwrap to our list of conﬂicting attribute pairs. Now in addition to the initial knowledge from the ﬁrst three experiments, we give the intruder an unknown key k3 encrypted under k2. Again he is able to aﬀect an attack similar to the one above, this time by unwrapping {|k3}| k2 twice (see Fig. 6).

An Introduction to Security API Analysis

43

Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ); n1 has the attributes sensitive, extract and whereas n2 has the attribute extract set. Trace: Set wrap: h(n2 , k2 ) → Wrap: h(n2 , k2 ), h(n2 , k2 ) → Set unwrap: h(n2 , k2 ) → new n4 Unwrap: h(n2 , k2 ), {|k2}| k2 −−−→ Wrap: h(n2 , k2 ), h(n1 , k1 ) → Set decrypt: h(n4 , k2 ) → Decrypt: h(n4 , k2 ), {|k1}| k2 →

wrap(n2 , ) {|k2}| k2 unwrap(n2 , ) h(n4 , k2 ) {|k1}| k2 decrypt(n4 , ) k1

Fig. 5. Re-import attack 1

Initial state: The intruder knows the handles h(n1 , k1 ), h(n2 , k2 ); n1 has the attributes sensitive, extract and whereas n2 has the attribute extract set. The intruder also knows {|k3}| k2 . Trace: Set unwrap: h(n2 , k2 ) → new n3 Unwrap: h(n2 , k2 ), {|k3}| k2 −−−→ new n4 Unwrap: h(n2 , k2 ), {|k3}| k2 −−−→ Set wrap: h(n3 , k3 ) → Wrap: h(n3 , k3 ), h(n1 , k1 ) → Set decrypt: h(n4 , k3 ) → Decrypt: h(n4 , k3 ), {|k1}| k3 →

unwrap(n2 , ) h(n3 , k3 ) h(n4 , k3 ) wrap(n3 , ) {|k1}| k3 decrypt(n4 , ) k1

Fig. 6. Re-import attack 2

This sample of the attacks found show how diﬃcult PKCS#11 is to conﬁgure in a safe way, and indeed there are several more attacks documented in the literature [17,23,26,33] and perhaps more to discover. Another line of research has consisted of trying to propose secure subsets of the API together with a suitable security proof. An obstacle here is the fresh generation of keys and handles: if there are no attacks in the model after generating n fresh keys, how do we know there are no attacks after generating n+1? To address this problem, Fr¨oschle and Steel proposed abstractions for handles and keys that allow security proofs for unbounded fresh data [33]. In particular, they showed the security of a symmetric key management subset based around the proprietary extensions to PKCS#11 made by Eracom, where keyed hashes are used to bind attributes to keys under wrapping, ensuring that they are re-imported with the same attributes.

44

R. Focardi, F.L. Luccio, and G. Steel

Fig. 7. Tookan system diagram

3.6

Finding Attacks on Real Devices

Attacks on the standard are interesting in themselves, but in reality every device implements a diﬀerent subset of PKCS#11 with diﬀerent restrictions on the use of each command. How can one know whether a particular device is vulnerable? To address this, Bortolozzo, Centenaro, Focardi and Steel developed the Tookan1 tool [17]. Tookan functions as shown in Fig. 7. First, Tookan extracts the capabilities of the token following a reverse engineering process (1). The results of this task are written in a meta-language for PKCS#11 models. Tookan uses this information to generate a model the language described above (2), which is encoded for input to the SATMC model checker [9]. If SATMC ﬁnds an attack, the attack trace (3) is sent to Tookan for testing directly on the token (4). Changes to the model. There are several diﬀerences between Tookan’s model and the model of section 3.3. One is that Tookan takes into account key templates. In section 3.3, the key generation commands create a key with all attributes unset (see Fig. 2). Attributes are then enabled one by one using the SetAttribute command. In our experiments with real devices, we discovered that some tokens do not allow attributes of a key to be changed. Instead, they use a key template specifying settings for the attributes which are given to freshly generated keys. Templates are used for the import of encrypted keys (unwrapping), key creation using CreateObject and key generation. The template to be used in a speciﬁc command instance is speciﬁed as a parameter, and must come from a set of valid templates, which we label G, C and U for the valid templates for key generation, creation and unwrapping respectively. Tookan can construct the set of templates in two ways: the ﬁrst, by exhaustively testing the commands using templates for all possible combinations of attribute settings, which may be very time consuming, but is necessary if we aim to verify the security of a token. The second method is to construct the set of templates that should be allowed based on the reverse-engineered attribute policy (see next paragraph). This is an approximate process, but can be useful for quickly ﬁnding attacks. Indeed, in our experiments, we found that these models reﬂected well the operation of the token, i.e. the attacks found by the model checker all executed on the tokens without any ‘template invalid’ errors.

1

Tool for cryptoki analysis.

An Introduction to Security API Analysis

45

Attribute Policies. Most tokens tested attempt to impose some restrictions on the combinations of attributes that can be set on a key and how these may be changed. There are four kinds of restriction that Tookan can infer from its reverse engineering process: Sticky on These are attributes that once set, may not be unset. The PKCS #11 standard lists some of these [47, Table 15]: sensitive for secret keys, for example. The UnsetAttribute rule is only included for attributes which are not sticky on. To test if a device treats an attribute as sticky on, Tookan attempts to create a key with the attribute on, and then calls SetAttribute to change the attribute to oﬀ. Sticky off These are attributes that once unset may not be set. In the standard, extractable is listed as such an attribute. The SetAttribute rule is only included for attributes which are not sticky oﬀ. To test if a device treats an attribute as sticky on, Tookan attempts to create a key with the attribute oﬀ, and then calls SetAttribute to change the attribute to on. Conflicts Many tokens (appear to) disallow certain pairs of attributes to be set, either in the template or when changing attributes on a live key. For example, some tokens do not allow sensitive and extractable to be set on the same key. The SetAttribute rule is adjusted to prevent conﬂicting attributes from being set on an object or on the template. When calculating the template sets C, G, U (see above), we forbid templates which have both the conﬂicting attributes set. To test if a device treats an attribute pair as a conﬂict, Tookan attempts to generate a key with the the pair of attributes set, then if no error is reported, it calls GetAttribute to check that the token really has created a key with the desired attributes set. Tied Some tokens automatically set the value of some attributes based on the value of others. For example, many tokens set the value of always sensitive based on the value of the attribute sensitive. The SetAttribute and UnsetAttribute rules are adjusted to account for tied attributes. The template sets C, G, U are also adjusted accordingly. To test if a device treats an attribute pair as tied, Tookan attempts to generate a key with some attribute a on and all other attributes oﬀ. It then uses GetAttribute to examine the key as it was actually created, and tests to see if any other attributes were turned on. Limitations of Reverse Engineering. Tookan’s reverse engineering process is not complete: it may result in a model that is too restricted to ﬁnd some attacks possible on the token, and it may suggest false attacks which cannot be executed on the token. This is because in theory, no matter what the results of our ﬁnite test sequence, the token may be running any software at all, perhaps even behaving randomly. However, if a token implements its attribute policy in the manner in which we can describe it, i.e. as a combination of sticky on, sticky oﬀ, conﬂict and tied attributes, then our process is complete in the sense that the model built will reﬂect exactly what the token can do (modulo the usual Dolev-Yao abstractions for cryptography).

46

R. Focardi, F.L. Luccio, and G. Steel Table 1. Summary of results on devices

Company

Device Model

Supported Functionality Attacks found sym asym cobj chan w ws a1 a2 a3 a4 a5 mc

Aladdin Athena Bull Eutron Feitian USB Feitian Feitian Gemalto MXI Security RSA SafeNet Sata ACS Card Athena Gemalto Gemalto Gemalto Siemens

eToken PRO ASEKey Trustway RCI Crypto Id. ITSEC StorePass2000 ePass2000 ePass3003Auto SEG Stealth MXP Bio SecurID 800 iKey 2032 DKey ACOS5 ASE Smartcard Cyberﬂex V2 Classic TPC IS V1 Classic TPC IS V2 CardOS V4.3 B

Eracom IBM

HSM simulator opencryptoki 2.3.1

Soft

a1

a1

a3 a3 a3

a3 a3

a2 a3 a4

a1 a1

In our testing, the model performed very well: the Tookan consistently found true attacks on ﬂawed tokens, and we were unable to ﬁnd ‘by hand’ any attacks on tokens which the model checker deemed secure. This suggests that real devices do indeed implement their attribute policies in a manner similar to our model. 3.7

Results

Table 1 summarises the results obtained by Tookan on a number of devices as well as two software simulators. Supported functionality and attacks are summarized in Table 2 and described below. Implemented functionality. Columns ‘sym’ and ‘asym’ respectively indicate whether or not symmetric and asymmetric key cryptography are supported. Column ‘cobj’ refers to the possibility of inserting external, unencrypted, keys on the device via C CreateObject PKCS#11 function. This is allowed by almost all of the analysed tokens, although it wasn’t included in the original model of the standard used by Delaune, Kremer and Steel [26]. The next column, ‘chan’, refers to the possibility of changing key attributes through C SetAttributeValue. The following two columns, ‘w’ and ‘ws’, respectively indicate whether the token permits wrapping of nonsensitive and sensitive keys.

An Introduction to Security API Analysis

47

Table 2. Key for table 1 Acronym Description

Supported functionality

Attacks

sym asym cobj chan w ws a1 a2 a3 a4 a5 mc

symmetric-key cryptography asymmetric-key cryptography inserting new keys via C CreateObject changing key attributes wrapping keys wrapping sensitive keys wrap/decrypt attack based on symmetric keys wrap/decrypt attack based on asymmetric keys sensitive keys are directly readable unextractable keys are directly readable (forbidden by the standard) sensitive/unextractable keys can be changed into nonsensitive/extractable ﬁrst attack found by Tookan

Attacks. Attack a1 is a wrap/decrypt attack as discussed in section 3.2. The attacker exploits a key k2 with attributes wrap and decrypt and uses it to attack a sensitive key k1 . Using our notation from section 3.3: Wrap: h(n2 , k2 ), h(n1 , k1 ) →{|k1}| k2 Decrypt: h(n2 , k2 ), {|k1}| k2 →k1 As we have discussed above, the possibility of inserting new keys in the token (column ‘cobj’) might simplify further the attack. It is suﬃcient to add a known wrapping key: new n

2 CreateObject: k2 −−−−→h(n 2 , k2 ) Wrap: h(n2 , k2 ), h(n1 , k1) →{|k1}| k2

The attacker can then decrypt {|k1 }| k2 since he knows key k2 . SATMC discovered this variant of the attack on several vulnerable tokens. Despite its apparent simplicity, this attack had not appeared before in the PKCS#11 security literature [23,25]. Attack a2 is a variant of the previous ones in which the wrapping key is a public key pub(z) and the decryption key is the corresponding private key priv(z): Wrap: h(n2 , pub(z)), h(n1 , k1 ) →{k1 }pub(z) ADecrypt: h(n2 , priv(z)), {k1 }pub(z) →k1 In this case too, the possibility of importing key pairs simpliﬁes even more the attacker’s task by allowing him to import a public wrapping key while knowing the corresponding private key. Once the wrap of the sensitive key has been performed, the attacker can decrypt the obtained ciphertext using the private key.

48

R. Focardi, F.L. Luccio, and G. Steel

Attack a3 is a clear ﬂaw in the PKCS#11 implementation. It is explicitly required that the value of sensitive keys should never be communicated outside the token. In practice, when the token is asked for the value of a sensitive key, it should return some “value is sensitive” error code. Instead, we found that some of the analysed devices just return the plain key value, ignoring this basic policy. Attack a4 is similar to a3: PKCS#11 requires that keys declared to be unextractable should not be readable, even if they are nonsensitive. If they are in fact readable, this is another violation of PKCS#11 security policy. Finally, attack a5 refers to the possibility of changing sensitive and unextractable keys respectively into nonsensitive and extractable ones. Only the Sata and Gemalto SafeSite Classic V2 tokens allow this operation. However, notice that this attack is not adding any new ﬂaw for such devices, given that attacks a3 and a4 are already possible and sensitive or unextractable keys are already accessible. Model-checking results. Column ‘mc’ reports which of the attacks was automatically rediscovered via model-checking. SATMC terminates once it has found an attack, hence we report the attack that was found ﬁrst. Run-times for ﬁnding the attacks vary from a couple of seconds to just over 3 minutes. 3.8

Finding Secure Configurations

As we observed in the last section, none of the tokens we tested are able to import and export sensitive keys in a secure fashion. In particular, all the analysed tokens are either insecure or have been drastically restricted in their functionality, e.g. by completely disabling wrap and unwrap. In this section, we present CryptokiX, a software implementation of a Cryptoki token which can be ﬁXed by selectively enabling diﬀerent patches. The starting point is openCryptoki [45], an open-source PKCS#11 implementation for Linux including a software token for testing. As shown in Table 1, the analysis of openCryptoki software token has revealed that it is subject to all the non-trivial attacks. This is in a sense expected, as it implements the standard ‘as is’, i.e., with no security patches. CryptokiX extends openCryptoki with: Conflicting attributes. We have seen, for example, that it is insecure to allow the same key to be used for wrapping and decrypting. In CryptokiX it is possible to specify a set of conﬂicting attributes. Sticky attributes. We know that some attributes should always be sticky, such as sensitive. This is also useful when combined with the ‘conﬂicting attributes’ patch above: if wrap and decrypt are conﬂicting, we certainly want to avoid that the wrap attribute can be unset so as to allow the decrypt attribute to be set. Wrapping formats. It has been shown that specifying a non-conﬂicting attribute policy is not suﬃcient for security [23,25]. A wrapping format should also be used to correctly bind key attributes to the key. This prevents attacks where the key is unwrapped twice with conﬂicting attributes.

An Introduction to Security API Analysis

49

Secure templates. We limit the set of admissible attribute combinations for keys in order to avoid that they ever assume conﬂicting roles at creation time. This is conﬁgurable at the level of the speciﬁc PKCS#11 operation. For example, we can deﬁne diﬀerent secure templates for diﬀerent operations such as key generation and unwrapping. A way to combine the ﬁrst three patches with a wrapping format that binds attributes to keys in order to create a secure token has already been demonstrated [33] and discussed in section 3.5. One can also use the fourth patch to produce a secure conﬁguration that does not require any new cryptographic mechanisms to be added to the standard, making it quite simple and cheap to incorporate into existing devices. We consider here a set of templates with attributes sensitive and extractable always set. Other attributes wrap, unwrap, encrypt and decrypt are set as follows: Key generation. We allow three possible templates: 1. wrap and unwrap, for exporting/importing other keys; 2. encrypt and decrypt, for cryptographic operations; 3. neither of the four attributes set, i.e. the default template if none of the above is speciﬁed. Key creation/import. We allow two possible templates for any key created with CreateObject or imported with Unwrap: 1. unwrap,encrypt set and wrap,decrypt unset; 2. none of the four attributes set. The templates for key generation are rather intuitive and correspond to a clear separation of key roles, which seems a sound basis for a secure conﬁguration. The rationale behind the single template for key creation/import, however, is less obvious and might appear rather restrictive. The idea is to allow wrapping and unwrapping of keys while ‘halving’ the functionality of created/unwrapped keys: these latter keys can only be used to unwrap other keys or to encrypt data, wrapping and decrypting under such keys are forbidden. This, in a sense, oﬀers an asymmetric usage of imported keys: to achieve full-duplex encrypted communication two devices will each have to wrap and send a freshly generated key to the other device. Once the keys are unwrapped and imported in the other devices they can be used to encrypt outgoing data in the two directions. Notice that imported keys can never be used to wrap sensitive keys. Note also that we require that all attributes are sticky on and oﬀ, and that we assume for bootstrapping that any two devices that may at some point wish to communicate have a shared long term symmetric key installed on them at personalisation time. This need only be used once in each direction. Our solution works well for pairwise communication, where the overhead is just one extra key, but would be more cumbersome for group key sharing applications. The developed solution has been implemented and analysed by extracting a model using Tookan. A model for SATMC was constructed using the abstractions

50

R. Focardi, F.L. Luccio, and G. Steel

described above (see end of section 3.5). Given the resulting model, SATMC terminates with no attacks in a couple of seconds, allowing us to conclude the patch is safe in our abstract model for unbounded numbers of fresh keys and handles. Note that although no sensitive keys can be extracted by an intruder, there is of course no integrity check on the wrapped keys that are imported. Indeed, without having an encryption mode with an integrity check this would seem to be impossible. This means that one cannot be sure that a key imported on to the device really corresponds to a key held securely on the intended recipient’s device. This limitation would have to be taken into account when evaluating the suitability of this conﬁguration for an application. CryptokiX is available online2 . 3.9

Summary

We have seen how RSA PKCS#11 describes an API for key management where the usage policy for each key is described by a set of attributes. We have seen how this interface, if not conﬁgured carefully, can be vulnerable to a variety of attacks, and we have seen that these vulnerabilities aﬀect not just theoretical models but real devices. We have also seen how to use formal modelling techniques to systematically ﬁnd attacks and then verify the security of models. However there are still many open questions about designs of a secure API and cryptographic soundness of models, for example, which we will discuss in section 5.

4

PIN Processing

We now consider our second case study, which addresses the problem of protecting a user’s PIN when withdrawing some money at an Automated Teller Machine (ATM). International bank networks are structured in such a way that an access to an ATM implies that the user’s PIN is sent from the ATM to the issuing bank for the veriﬁcation. While travelling, the PIN is decrypted and reencrypted by special tamper-resistant HSMs which are placed on each network switch, as illustrated in Fig. 8. Indeed, international standards mandate the use of these devices to keep PINs secure [37]. The ﬁrst PIN encryption is performed by the ATM keypad which is an HSM itself, using a symmetric key k1 shared with the neighbour acquiring bank. While travelling from node to node, the encrypted PIN is decrypted and re-encrypted by the HSM located in the switch with another key shared with the destination node. This is done using a so called translation API. The ﬁnal veriﬁcation and acceptance/refusal of the PIN is done by the issuing bank. This check is performed via a veriﬁcation API. At ﬁrst sight this setup seems highly secure. However, several API-level attacks have been discovered on these HSMs in recent years [10,16,22]. These attacks work by assuming that the attacker is an insider who has gained access to the HSM at some bank switch, a threat considered in-scope by the standard 2

http://secgroup.ext.dsi.unive.it/cryptokix

An Introduction to Security API Analysis

51

Fig. 8. Bank network

[37]. The attacker performs some unexpected sequence of API calls from which he is able to deduce the value of a PIN. API-level attacks on PINs have recently attracted attention from the media [1,3]. This has increased the interest in developing formal methods for analysing PIN recovery attacks. We will brieﬂy survey the literature here. Examining the PIN veriﬁcation API, Bond and Zielinski [16] and Clulow [22] discovered the so called decimalization attack that we will discuss in section 4.1 at more or less the same time. The original version of the attack requires an average of 16.5 API calls to obtain a 4-digit PIN [16]. Later variations proposed by Steel [49] and Focardi and Luccio [31] reduced that value to 16.145 and 14.47 respectively. The decimalisation table attack is similar to the Mastermind game, and in section 4.2, we will illustrate how to reuse known techniques to solve Mastermind in order to ﬁnd eﬃcient attacks on bank PINs. Bond and Clulow have also demonstrated an attack that works even if the decimalisation table is ﬁxed [14], although it requires many more calls to the API (some 10s of thousands). Attacks on the PIN translation API have been presented by Clulow [22] and subsequently by Berkman and Ostrovsky [10]. These attacks often exploit details of the formats used to pad a PIN into a 64 bit block for encryption. Berkman and Ostrovsky’s attacks also exploit commands for changing a customer PIN, though they generally require a known plaintext-ciphertext pair (i.e. a clear PIN and an encryption of the same PIN) to be introduced into the system. The ﬁrst formal analysis work on PIN recovery attacks is due to Steel [49], who showed hos a combination of logic programming and probabilistic model checking could be used to ﬁnd new attacks and analyse possible patches to the APIs. Recently Centenaro and the authors of this survey presented a

52

R. Focardi, F.L. Luccio, and G. Steel Table 3. The veriﬁcation API

PIN V(EPB , len, oﬀset, vdata, dectab ) { x1 := encpdk (vdata); x2 := left(len, x1 ); x3 := decimalize(dectab, x2 ); x4 := sum mod10(x3 , oﬀset ); x5 := deck (EPB ); x6 := fcheck(x5 ); if (x6 =⊥) then return( format wrong ); if (x4 = x6 ) then return(PIN correct ); else return(PIN wrong )}

language-based setting for analysing PIN processing API via a type-system [20]. We have formally modelled existing attacks, proposed some ﬁxes and proved them correct via type-checking. These ﬁxes typically require to reduce and modify the HSM functionality by, e.g., deciding on a single format of the transmitted PIN or adding MACs for the integrity of user data. This will be the subject of section 4.3. As upgrading the bank network HSMs worldwide is complex and very expensive in [30] the authors have proposed a method for the secure upgrade of HSMs in wide networked systems. This method incrementally upgrades the network so to obtain upgraded, secure subnets, while preserving the compatibility towards the legacy system. Finally, in [32] we have also have proposed a low-impact, easily implementable ﬁx that involves very little change to the existing ATM network infrastructure. Attacks are still possible, but they are 50000 times slower. 4.1

API-Level Attacks on PIN Verification

In this section we show in detail a real API-level attack on cash machine PINs. We focus our attention on the PIN veriﬁcation API (PIN V for short) and we specify its code in Table 3. It takes as input the encrypted PIN block EPB , i.e. the encrypted PIN arriving from the ATM, the PIN length len and oﬀset oﬀset, the validation data vdata and the decimalization table dectab. The API returns the result of the veriﬁcation or an error code. PIN V behaves as follows: – The user PIN of length len is computed by ﬁrst encrypting validation data vdata with the PIN derivation key pdk (x1 ) and obtaining a 16 hexadecimal digit string. Then, the ﬁrst len hexadecimal digits are chosen (x2 ), and decimalized through dectab (x3 ), obtaining the “natural” PIN assigned by the issuing bank to the user. decimalize is a function that associates to each possible hexadecimal digit (of its second input) a decimal one as speciﬁed by its ﬁrst parameter (dectab). Finally, if the user wants to choose her own PIN, an oﬀset is calculated by digit-wise subtracting (modulo 10) the natural PIN from the user-selected one (x4 ).

An Introduction to Security API Analysis

53

– To recover the trial PIN EPB is ﬁrst decrypted with key k (x5 ), then the PIN is extracted by the formatted decrypted message (x6 ). This last operation depends on the speciﬁc PIN format adopted by the bank. In some cases, for example, the PIN is padded with random digits so to make its encryption immune from codebook attacks. In this case, extracting the PIN involves removing this random padding. – If the format in incorrect (⊥ represents failure) then an error message is returned. Otherwise, the equality between the user PIN and the trial PIN is veriﬁed. Example 2. For simplicity we consider the ‘default’ decimalisation table: 0 1 2 3 4 5 6 7 8 9 A B C D E F 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 We will write it as dectab = 0123456789012345. Let also EPB = {|5997, r|}k , len=4, oﬀset =4732 and x1 = encpdk (vdata) = BC6595FDE32BA101 then: x2 x3 x4 x5 x6 x6 x4

= left(4, BC6595F DE32BA101) = BC65 = decimalize(dectab, BC65) = 1265 = sum mod10(1265, 4732) = 5997 = deck ({|5997, r|}k ) = (5997, r) = fcheck(5997, r) = 5997 =⊥ = x6 return(PIN correct )

Since x6 is diﬀerent from ⊥ and x4 = x6 , the API returns PIN correct . The decimalization attack. We now illustrate a real attack on PIN V ﬁrst reported in [16,22], called the decimalization table attack. It consists of deducing the PIN digits by modifying the way numbers are decimalized and by observing whether this aﬀects the result of PIN V. The position of the guessed PIN digits is reconstructed by manipulating the oﬀset, which is a public parameter. By combining all this information the attacker is able to reconstruct the whole PIN. More speciﬁcally, the attack works by iterating the following two steps, until the whole PIN is recovered: 1. To discover whether or not a decimal digit d is present in the intermediate value contained in x3 the intruder picks digit d, changes the dectab function so that values previously mapped to d now map to d + 1 mod 10, and then checks whether the system still returns PIN correct . If this is the case digit d does not appear in x3 . 2. It the API call returns PIN wrong the attacker discovers that d is one of the digits of x3 . To locate the position of the digit the intruder also decreases the oﬀset digits by 1, position by position, until the API returns again that the PIN is correct.

54

R. Focardi, F.L. Luccio, and G. Steel

We illustrate the attack through a simple example. Example 3. Consider again Example 2. The attacker, unaware of the value of the PIN, ﬁrst tries to discover whether or not 0 appears in x3 , so it changes the dectab as dectab = 1123456789112345 i.e., he replaces the two 0’s by 1’s. Invoking the API with dectab he obtains decimalize(dectab , BC65) = decimalize(dectab, BC65) = 1265 that is x3 remains unchanged and so the attacker deduces 0 does not appear in the PIN. The attacker proceeds by replacing the 1’s of dectab by 2’s: dectab = 0223456789022345 which gives x3 x4 x5 x6 x6 x4

= decimalize(dectab , BC65) = sum mod10(2265, 4732) = deck ({|5997, r|}k ) = fcheck(5997, r) =⊥ = x6

= 2265 = 1265 = 6997 = 5997 = (5997, r) = 5997 return(PIN wrong )

The intruder now knows that digit 1 occurs in x3 . To discover its position and multiplicity, he now varies the oﬀset so as to “compensate” for the modiﬁcation of the dectab. In particular, he tries to decrement each oﬀset digit by 1. For example, testing the position of one occurrence of one digit amounts to trying the following oﬀset variations: 3732, 4632, 4722, 4731 Notice that, in this speciﬁc case, oﬀset value 3732 makes the API return again PIN correct . The attacker now knows that the ﬁrst digit of x3 is 1. Given that the oﬀset is public, he also calculates the ﬁrst digit of the user PIN as 1 + 4 mod 10 = 5. He then proceeds using the same technique to discover the other PIN digits. 4.2

PIN Attacks as a Mastermind Game

As observed in [15,31], the above attack resembles the Mastermind game, invented in 1970 by the Israeli postmaster and telecommunication expert Mordecai Meirowitz [46, page 442]. The game is shown in Fig. 9 which is taken from [2]. It is played as a board game between two players, a codebreaker and a codemaker [50]. The codemaker chooses a linear sequence of coloured pegs and conceals them behind a screen. Duplicates of coloured pegs are allowed. The codebreaker has to guess, in diﬀerent trials, both the colour and the position of the pegs. During each trial he learns something and based on this he decides the next guess: in

An Introduction to Security API Analysis

55

Fig. 9. Guessing a code of a Mastermind game is similar to guessing a PIN

particular, a response consisting of a red marker represents a correct guess of the colour and the position of a peg (but the marker does not indicate which one is correct), a response consisting of a white marker represents only the correct guess of a colour but at the wrong position. The analogy between the game and the attack is as follows: each API call represents a trial of the codebreaker and the API return value is the corresponding answer of the codemaker. Modifying the dectab corresponds to disclosing the presence of certain digits in the PIN, like the white marker in the Mastermind Game. On the other hand, manipulating the dectab and the oﬀset together is similar to asking the codemaker to conﬁrm a guess of both the value and the position of one PIN digit, like the red marker of the game. An extended Mastermind game. To make the above analogy more precise, it is in fact necessary to extend the Mastermind game so as to allow the codebreaker to pose a guess of sets of coloured pegs, instead of just single pegs. Intuitively, the sets represent alternative guesses, i.e., it is suﬃcient that one of the pegs in the set is correct to get a red or a white marker. E.g., given the secret (1, 5, 3, 1) and the (extended) guess ({1}, {3, 4, 5}, {1, 2}, {0}), then the result is two red markers for the ﬁrst two positions of the guess and one white marker for the third position of the guess. The correspondence between the values of the secret and the values in the guess are shown as underlined values in ({1}, {3, 4, 5}, {1, 2}, {0}). In [31] it is proved that the decimalization attack is equivalent to playing an extended Mastermind game, only focusing on answers containing 4 red markers. Intuitively, given a modiﬁed dectab and oﬀset, we construct the sets for the extended guess by picking all digits whose modiﬁcation in the oﬀset are correctly compensated by the variation in the dectab. The ﬁrst set corresponds to the

56

R. Focardi, F.L. Luccio, and G. Steel

ﬁrst oﬀset digit, the second set to the second digit and so on. For example, consider an original oﬀset (1, 5, 4, 2) modiﬁed as (0, 3, 1, 0), giving a variation of (−1, −2, −3, −2). Suppose now that the modiﬁed dectab turns the mapping of digit 4 into 5, with a variation of +1, then digit 4 will be in the ﬁrst set of the extended guess, since the oﬀset has the correct (opposite) variation -1 in the ﬁst position. As an example, consider the following default dectab and a modiﬁed one, named dectab : dectab 0 1 2 3 4 5 6 7 8 9 dectab 0 2 4 3 5 8 6 9 1 9 variation +1 +2 +1 +3 +2 +3

0 0

1 2 3 4 5 2 4 3 5 8 +1 +2 +1 +3

Since the variation on the ﬁrst oﬀset digit is −1 we collect, in the ﬁrst set of the guess, all digits with variation +1 in the dectab, i.e., {1, 4}. For the second and fourth oﬀset digits, both with variation −2, we obtain set {2, 7}. Finally, for the third oﬀset digit, with variation −3, we have set {5, 8}. The resulting extended guess is ({1, 4}, {2, 7}, {5, 8}, {2, 7}). Let us now see how the answer to this guess in the Mastermind game relates to the result of an API call with the above modiﬁed oﬀset and dectab. Recall that we start from a dectab and oﬀset that give PIN correct . Thus, we will still have PIN correct only if the variations in the dectab are correctly compensated for by the ones in the oﬀset. For example, if digit 1 originally occurred in the ﬁrst place after decimalization (x3 ), with dectab it will now be a 2. Summing 1 with the ﬁrst digit of the original oﬀset we had 1 + 1 = 2. With the modiﬁed oﬀset we now have 2 + 0 = 2. In fact the variation +1 in the dectab is correctly compensated for by the oﬀset. Notice that 1 occurs in the ﬁrst set of the guess. Observe also that 1 does not occur in the other sets of the guess. Thus, if 1 originally occurred in the second place after decimalization it would still be turned into 2 by dectab but the change in the oﬀset this time would not compensate for this variation. In fact, with the original dectab and oﬀset we would get 1 + 5 = 6 (where 5 is the second value of the oﬀset) while with the modiﬁed ones we would get 2 + 3 = 5 (where 3 is the second value of the new oﬀset). Thus, the modiﬁcation of +1 in the dectab this time would not correspond to the variation −2 in the second oﬀset digit. By iterating this reasoning on all digits, we obtain that the API call gives PIN correct if and only if the extended guess in the Mastermind game gives 4 red markers, meaning that the four secret digits, correct after decimalization and before summing the oﬀset, are in the four sets of the extended guess. This correspondence has been formally proved [31]. Improving attack strategies by playing Mastermind. Now that we have shown a strict analogy between PIN cracking and an extension of the Mastermind game, we can ﬁnd eﬃcient attacks for recovering PINs by simply devising minimal strategies for winning the Mastermind game. The solution proposed by Knuth [39] is to build a balanced search tree by minimizing the maximum number of solutions surviving after a guess. A guess, in our case, gives a Boolean

An Introduction to Security API Analysis

57

Table 4. An algorithm for ﬁnding PIN cracking strategies do_guess ( S ) : if | S | != 1: # not a s i n g l e s o l u t i o n yet min = | S |

# s t a r t i n g value , we want to d e c r e a s e it

# d e p e n d i n g on the round and the n u m b e r of s u r v i v i n g solutions , # g e n e r a t e s the g u e s s e s ( h e u r i s t i c s here ...) guesses = g e n e r a t e _ g u e s s e s( S ) for g in guesses : # for each guess g # i n t e r s e c t S and s u r v i v i n g s o l u t i o n s for g M_SOLS = S & s u r v i v i n g( g ) # count m a t c h i n g and non - m a t c h i n g and take the max n_sol = max (| m_sols | ,| S | -| m_sols |) # if we got a minimum , we store it if mas < min : min = n_sol MIN_SOLS = M_SOLS if min < | S |: # we are d e c r e a s i n g the s u r v i v i n g s o l u t i o n set size : # let us p e r f o r m the guess # case1 : guess was right , e x p l o r e s o l u t i o n s in M I N _ S O L S do_guess ( MIN_SOLS ) # case2 : guess was wrong , e x p l o r e s o l u t i o n s in S - M I N _ S O L S do_guess ( S - MIN_SOLS ) else : FAIL

# we are l o o p i n g

answer which partitions the set of possible solutions into two disjoint sets: matching or non-matching ones. The guess that is more balanced, i.e., that minimizes the maximum of the two sets, is picked and the procedure is recursively applied until it reaches a unique solution. This method does not guarantee an optimal solution as an unbalanced guess might produce a more balanced subtree later on, and vice versa, but it has been shown to work well in practice [31,39]. In Table 4, we report an algorithm inspired by the above idea, adapted to our setting. The recursive function do guess is initially invoked with the whole S set of possible secrets. It then ﬁnds a guess that minimizes the maximum number of surviving solutions on the two possible answers and it descends recursively into the two sets in which the actual set is partitioned. A critical issue is how the set of candidate guesses are picked (function generate guesses(S)). In our setting, considering all extended guesses consisting of all possible combinations of digit subsets for each position is computationally intractable. In [31] it is observed that the ‘shape’ of the guesses used in the strategies found is quite regular. This makes it possible to devise a few heuristics that incrementally extend the set of

58

R. Focardi, F.L. Luccio, and G. Steel

Fig. 10. Noninterference

guesses when needed. As a result, the existing bounds on the average number of API calls required to recover a PIN has been reduced to 14.47 and a new bound of 19.3 has been given for 5-digits PINs. Both cases are close to the optimum, i.e., the depth of a perfectly balanced search tree. 4.3

Devising Robust PIN Management APIs

In the previous sections, we described one of the known attacks on PIN management APIs and its similarity to the Mastermind game. We now focus on remedies and we ﬁrst try to understand what is the source of the attacks, i.e., what security property is missing. This will allow us to propose formal models and tools supporting a more ‘robust’ development of PIN processing APIs. Preventing information flow. Since the PIN recovery attacks involve the leakage of a secret value, a ﬁrst attempt to understand the problem might be in terms of classical noninterference [34]. In this setting, data items are labelled with security levels taken from a lattice. Noninterference requires that data from a higher level, e.g. the secret level, should never aﬀect the value of data with a lower level, e.g. the public level. The idea is depicted in Fig. 10. The system is (highly) simpliﬁed as a box containing two smaller boxes, with a switch for the input and a display for the output. The top box (high level input/output) contains the secret information while the bottom box (low level input/output) is public and possibly under the control of the attacker. Noninterference holds if a variation on the high input does not produce any change in the low level output. The noninterference property is very strong and ensures that no secret information will ever be leaked, but this is too much for our setting. In fact, PIN veriﬁcation needs to leak 1 bit of information about the correctness of the PIN. To see how the property is broken, it is enough to consider an API call with the correct encrypted PIN block (giving PIN correct ), and another API call with

An Introduction to Security API Analysis

59

Fig. 11. Robust declassiﬁcation

a wrong encrypted PIN block (giving PIN wrong ). This for example happens if the user types the wrong PIN at the ATM. By changing the secret data, i.e., the encrypted PIN, we have two diﬀerent outcomes of the API, thus two diﬀerent low level outputs. Clearly a leak of information about the correctness of the PIN is inevitable, but we would somehow like to limit this to prevent attacks. Admitting controlled information leakage. One approach to relaxing noninterference while still controlling information release is called robustness, and was proposed by Myers, Sabelfeld and Zdancewic (MSZ) [43]. This property admits some form of declassiﬁcation (or downgrading) of conﬁdential data, but requires that an attacker cannot inﬂuence the secret information which is declassiﬁed. In our case study of section 4.1, PIN V returns the correctness of the typed PIN which is a one-bit leakage of information about a secret datum. Robustness requires that an attacker cannot abuse such a declassiﬁcation in order to gain more information than intended. The idea is illustrated in Fig. 11: at the top of the ﬁgure, moving from left to right, we see that a change in the high, i.e., secret input is observable at the low, i.e., public level, meaning that noninterference does not hold and some information is leaked. However, moving from the top to the bottom, we notice that a change in the low level input does not aﬀect the leaked information. This amounts to stating that what is leaked is never aﬀected by untrusted, low level users. If we consider the decimalization attack we notice that the attacker, by manipulating public data such as the dectab and the oﬀset, can aﬀect what is declassiﬁed, i.e., the outcome of the API call. By forbidding this, the attack should be prevented.

60

R. Focardi, F.L. Luccio, and G. Steel

PIN V M(EPB , len, oﬀset, vdata, dectab, MAC ) { if (macak (EPB , len, oﬀset, vdata, dectab) == MAC ) then PIN V(EPB , len, oﬀset, vdata, dectab ) ; else return(“integrity violation ); }

Fig. 12. A ﬁxed PIN veriﬁcation API

A robust PIN verification API. In [20] we have formalized the above property in the setting of PIN management APIs, and we have given a patched version of PIN V that prevents attacks by checking the integrity of the public data. The idea is to have a Message Authentication Code (MAC) of such data, i.e., an unforgeable cryptographic signature that once checked by the HSM, guarantees the integrity of data. If the attacker tries to tamper with the decimalization table or oﬀset, the MAC check fails and the HSM aborts the call with an error message. Fig. 12 reports the code of the ﬁxed PIN veriﬁcation API, called PIN V M, which recomputes and checks the MAC before calling the original PIN V.3 In the paper we have also devised a type-system to statically prove the robustness of the new API. Even if the proposed ﬁx has been proved to be secure, the infrastructure changes needed to add full MAC calculation to ATMs and switches are seen as prohibitive by the banks [6]. We have proposed a way to implement a weaker version of our scheme whilst minimising changes to the existing infrastructure [32]. In the paper, we observe that cards used in the cash machine network already have an integrity checking value: the card veriﬁcation code (CVC), or card veriﬁcation value (CVV), is a 5 (decimal) digit number included in the magnetic stripe of most cards in use and contains a MAC of the customer’s PAN, the expiry date of the card and some other data. The current purpose of the CVV is to make card cloning more diﬃcult, since the CVV is not printed in the card and does not appear on printed POS receipts.4 Our idea is to use the CVV calculation method twice, in the manner of a hashed MAC or HMAC function. We calculate the CVV of a new set of data, containing the decimalisation table and oﬀset. Then, we insert the result of the original CVV calculation to produce a ﬁnal 5-digit MAC. The scheme is easily implementable but we loose some security, since our MACs now have an entropy of only 5 decimal digits (216 < 105 < 217 ).

3

4

For technical reasons, the speciﬁcation in [20] has that the Personal Account Number is in the list of parameters. Here we consider it as part of the validation data vdata. The CVV/CVC should not be confused with the CVC2 or CVV2, which is printed on the back of the card and is designed to counteract “customer not present” fraud when paying over the Internet or by phone.

An Introduction to Security API Analysis

4.4

61

Summary

We have seen how bank networks use special HSMs to secure ATM transactions. These sophisticated devices perform cryptographic operations such as PIN encryption/decryption and PIN veriﬁcation via a security API. We have discussed attacks on this API and we have shown in detail the so called decimalization table attack. We have seen how these attacks progressively leak partial information about PIN digits similarly to what happens in the Mastermind game. We have shown that algorithms for the solution of Mastermind can be adapted to search for eﬃcient attacking strategies on bank PINs. Finally, we have illustrated how the security of PIN management APIs can be modelled in terms of an information ﬂow property called robustness and we have described a ﬁxed API that can be proved to be robust. Intuitively, a robust API can leak partial information on a secret, e.g., whether the PIN typed at the ATM is correct or not, but this leakage should never be under the control of the attacker, as occurs in the decimalization attack.

5

Conclusions

There are many open research problems in security APIs, both theoretical and practical. In the domain of key management, we have seen that the current standard for APIs, PKCS#11, is very diﬃcult to conﬁgure securely. This is evidenced by a number of attacks on models of the standard as well as attacks on real PKCS#11 security tokens: in our sample of 18 devices, we found 6 tokens that trivially gave up their sensitive keys in complete disregard of the standard, 3 that were vulnerable to a variety of key separation attacks, and a further smartcard that allowed unextractable keys to be read in breach of the standard. The remainder provide no functionality for secure transport of sensitive keys. We have seen that it is possible to propose secure conﬁgurations, [17,33], but security proofs here are only in the symbolic model of cryptography: there is more work to be done to reconcile these proofs to the cryptographic details of real implementations, though work in this direction is underway [40]. Given the current situation it is perhaps not surprising that there have been articles proposing completely new designs for key management APIs [19,24]. Indeed at least two new industrial standards which address key management are currently at the draft stage: IEEE 1619.3 [36] (for secure storage) and OASIS Key Management Interoperability Protocol (KMIP) [44]. It remains to be seen how these latter designs will address security concerns. We have described attacks on bank HSMs for the ATM network. We have discussed possible ﬁxes for the decimalization attack but the extent to which these ﬁxes can be implemented in real devices is still unclear. There is a great deal of existing infrastructure in the ATM network that would be costly to replace. Since the attacks involve obtaining PINs in order to withdraw money using closed magnetic stripe cards, one possibility would be to upgrade the whole network to the new chip-based cards [4]. This seems unlikely to happen in the short term, and even if a decision were taken to do so it might take years to cover

62

R. Focardi, F.L. Luccio, and G. Steel

the whole world-wide network. In the meanwhile the network will fall-back to the magnetic stripe protocol whenever the chip is not supported by the ATM, and well-organised attackers will continue to ‘cash out’ their stolen PINs in countries where magnetic stripe ATMs are still the norm. We have seen that the problem of PIN processing API attacks, in a sense, an excessive parametrization of some of the functionalities. Since these parameters are public, and attacker can manipulate them in order to aﬀect the information that is declassiﬁed by the PIN check. Some of these parameters could be ‘lockeddown’: the dectab, for example, could be ﬁxed on the HSMs of a particular bank, assuming that one bank only uses one dectab for all users. As a more general solution, we have proposed adding a MAC of (a subset of) the function parameters, but this of course requires to change HSMs to support this new operation. A cheap low-impact ﬁx could be obtained using CVVs but this is not completely satisfactory from a security point of view, as it only mitigates the attacks. In our opinion, the direction that bank industry will take is still unclear. The ﬁeld of security API analysis is highly active with an annual international workshop (ASA - Analysis of Security APIs) and a chapter of Anderson’s essential textbook [7] devoted to the theme. Recent articles at ASA and elsewhere have proposed security API techniques for solving problems in anything from social networks to browser Javascript engines to utility meters. Thanks to the ASA forum, the formal methods and security community has come closer to industrial users of security APIs, leading to research results that are applicable in the short term, and hopefully, in the longer term, inﬂuencing the way secure devices are developed, programmed and used. It is, in fact, a very exciting time to be researching security APIs.

References 1. Hackers crack cash machine PIN codes to steal millions. The Times online, http:// www.timesonline.co.uk/tol/money/consumer_affairs/article4259009.ece 2. Mastermind, http://commons.wikimedia.org/wiki/File:Mastermind.jpg 3. PIN Crackers Nab Holy Grail of Bank Card Security. Wired Magazine Blog ’Threat Level’, http://blog.wired.com/27bstroke6/2009/04/pins.html 4. The EMV Standard, http://www.emvco.com/ 5. Anderson, R.: The correctness of crypto transaction sets. In: 8th International Workshop on Security Protocols (April 2000), http://www.cl.cam.ac.uk/ftp/ users/rja14/protocols00.pdf 6. Anderson, R.: What we can learn from API security (transcript of discussion). In: Security Protocols, pp. 288–300. Springer, Heidelberg (2003) 7. Anderson, R.: Security Engineering, 2nd edn. Wiley, Chichester (2007) 8. Armando, A., Basin, D.A., Boichut, Y., Chevalier, Y., Compagna, L., Cu´ellar, J., Hankes Drielsma, P., H´eam, P., Kouchnarenko, O., Mantovani, J., M¨ odersheim, S., von Oheimb, D., Rusinowitch, M., Santiago, J., Turuani, M., Vigan` o, L., Vigneron, L.: The AVISPA tool for the automated validation of internet security protocols and applications. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 281–285. Springer, Heidelberg (2005)

An Introduction to Security API Analysis

63

9. Armando, A., Compagna, L.: SAT-based model-checking for security protocols analysis. Int. J. Inf. Sec. 7(1), 3–32 (2008), Software available at http://www. ai-lab.it/satmc, Currently developed under the AVANTSSAR project, http:// www.avantssar.eu 10. Berkman, O., Ostrovsky, O.M.: The unbearable lightness of PIN cracking. In: Dietrich, S., Dhamija, R. (eds.) FC 2007 and USEC 2007. LNCS, vol. 4886, pp. 224–238. Springer, Heidelberg (2007) 11. Blanchet, B.: From secrecy to authenticity in security protocols. In: Hermenegildo, M.V., Puebla, G. (eds.) SAS 2002. LNCS, vol. 2477, pp. 342–359. Springer, Heidelberg (2002) 12. Bond, M.: Attacks on cryptoprocessor transaction sets. In: Ko¸c, C ¸ .K., Naccache, D., Paar, C. (eds.) CHES 2001. LNCS, vol. 2162, pp. 220–234. Springer, Heidelberg (2001) 13. Bond, M., Anderson, R.: API level attacks on embedded systems. IEEE Computer Magazine 34(10), 67–75 (2001) 14. Bond, M., Clulow, J.: Encrypted? randomised? compromised (when cryptographically secured data is not secure). In: Cryptographic Algorithms and their Uses, pp. 140–151 (2004) 15. Bond, M., Clulow, J.: Extending security protocol analysis: New challenges. Electronic Notes in Theoretical Computer Science 125(1), 13–24 (2005) 16. Bond, M., Zielinski, P.: Decimalization table attacks for pin cracking. Technical Report UCAM-CL-TR-560, University of Cambridge, Computer Laboratory (2003), http://www.cl.cam.ac.uk/TechReports/UCAM-CL-TR-560.pdf 17. Bortolozzo, M., Centenaro, M., Focardi, R., Steel, G.: Attacking and ﬁxing PKCS#11 security tokens. In: Proceedings of the 17th ACM Conference on Computer and Communications Security (CCS 2010), pp. 260–269. ACM Press, Chicago (2010) 18. Cachin, C., Camenisch, J.: Encrypting keys securely. IEEE Security & Privacy 8(4), 66–69 (2010) 19. Cachin, C., Chandran, N.: A secure cryptographic token interface. In: Computer Security Foundations (CSF-22), pp. 141–153. IEEE Computer Society Press, Long Island (2009) 20. Centenaro, M., Focardi, R., Luccio, F.L., Steel, G.: Type-based analysis of PIN processing APIs. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 53–68. Springer, Heidelberg (2009) 21. Clayton, R., Bond, M.: Experience using a low-cost FPGA design to crack DES keys. In: Kaliski Jr., B.S., Ko¸c, C ¸ .K., Paar, C. (eds.) CHES 2002. LNCS, vol. 2523, pp. 579–592. Springer, Heidelberg (2003) 22. Clulow, J.: The design and analysis of cryptographic APIs for security devices. Master’s thesis, University of Natal, Durban (2003) 23. Clulow, J.: On the security of PKCS#11. In: Walter, C.D., Ko¸c, C ¸ .K., Paar, C. (eds.) CHES 2003. LNCS, vol. 2779, pp. 411–425. Springer, Heidelberg (2003) 24. Cortier, V., Steel, G.: A generic security API for symmetric key management on cryptographic devices. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 605–620. Springer, Heidelberg (2009) 25. Delaune, S., Kremer, S., Steel, G.: Formal analysis of PKCS#11. In: Proceedings of the 21st IEEE Computer Security Foundations Symposium (CSF 2008), pp. 331–344. IEEE Computer Society Press, Pittsburgh (2008) 26. Delaune, S., Kremer, S., Steel, G.: Formal analysis of PKCS#11 and proprietary extensions. Journal of Computer Security 18(6), 1211–1245 (2010)

64

R. Focardi, F.L. Luccio, and G. Steel

27. Dolev, D., Yao, A.: On the security of public key protocols. IEEE Transactions in Information Theory 2(29), 198–208 (1983) 28. Durante, A., Focardi, R., Gorrieri, R.: A compiler for analyzing cryptographic protocols using noninterference. ACM Transactions on Software Engineering and Methodology 9(4), 488–528 (2000) 29. Durgin, N.A., Lincoln, P., Mitchell, J.C.: Multiset rewriting and the complexity of bounded security protocols. Journal of Computer Security 12(2), 247–311 (2004) 30. Focardi, R., Luccio, F.L.: Secure upgrade of hardware security modules in bank networks. In: Armando, A., Lowe, G. (eds.) ARSPA-WITS 2010. LNCS, vol. 6186, pp. 95–110. Springer, Heidelberg (2010) 31. Focardi, R., Luccio, F.L.: Guessing bank pins by winning a mastermind game. Theory of Computing Systems (to appear, 2011) 32. Focardi, R., Luccio, F.L., Steel, G.: Blunting diﬀerential attacks on PIN processing APIs. In: Jøsang, A., Maseng, T., Knapskog, S.J. (eds.) NordSec 2009. LNCS, vol. 5838, pp. 88–103. Springer, Heidelberg (2009) 33. Fr¨ oschle, S., Steel, G.: Analysing PKCS#11 key management APIs with unbounded fresh data. In: Degano, P., Vigan` o, L. (eds.) ARSPA-WITS 2009. LNCS, vol. 5511, pp. 92–106. Springer, Heidelberg (2009) 34. Goguen, J.A., Meseguer, J.: Security policies and security models. In: IEEE Symposium on Security and Privacy, pp. 11–20 (1982) 35. Herzog, J.: Applying protocol analysis to security device interfaces. IEEE Security & Privacy Magazine 4(4), 84–87 (2006) 36. IEEE 1619.3 Technical Committee. IEEE storage standard 1619.3 (key management) (draft), https://siswg.net/ 37. International Organization for Standardization. ISO 9564-1: Banking personal identiﬁcation number (PIN) management and security, 30 pages 38. Keighren, G.: Model checking security APIs. Master’s thesis, University of Edinburgh (2007) 39. Knuth, D.: The Computer as a Master Mind. Journal of Recreational Mathematics 9, 1–6 (1976) 40. Kremer, S., Steel, G., Warinschi, B.: Security for key management interfaces. In: Proceedings of the 24th IEEE Computer Security Foundations Symposium (CSF 2011), Cernay-la-Ville, France, pp. 266–280. IEEE Computer Society Press, Los Alamitos (2011) 41. Longley, D., Rigby, S.: An automatic search for security ﬂaws in key management schemes. Computers and Security 11(1), 75–89 (1992) 42. Lowe, G.: Breaking and ﬁxing the Needham Schroeder public-key protocol using FDR. In: Margaria, T., Steﬀen, B. (eds.) TACAS 1996. LNCS, vol. 1055, pp. 147– 166. Springer, Heidelberg (1996) 43. Myers, A.C., Sabelfeld, A., Zdancewic, S.: Enforcing robust declassiﬁcation and qualiﬁed robustness. Journal of Computer Security 14(2), 157–196 (2006) 44. OASIS Key Management Interoperability Protocol (KMIP) Technical Committee. KMIP – key management interoperability protocol (February 2009), http://xml. coverpages.org/KMIP/ 45. openCryptoki, http://sourceforge.net/projects/opencryptoki/ 46. Pickover, C.A.: The Math Book: From Pythagoras to the 57th Dimension, 250 Milestones in the History of Mathematics. Sterling (2009) 47. RSA Security Inc., v2.20. PKCS #11: Cryptographic Token Interface Standard (June 2004) 48. Schneier, B.: Applied Cryptography, 2nd edn. John Wiley and Sons, Chichester (1996)

An Introduction to Security API Analysis

65

49. Steel, G.: Formal Analysis of PIN Block Attacks. Theoretical Computer Science 367(1-2), 257–270 (2006) 50. Stuckman, J., Zhang, G.: Mastermind is NP-Complete. INFOCOMP Journal of Computer Science 5, 25–28 (2006) 51. Tsalapati, E.: Analysis of PKCS#11 using AVISPA tools. Master’s thesis, University of Edinburgh (2007) 52. Youn, P., Adida, B., Bond, M., Clulow, J., Herzog, J., Lin, A., Rivest, R., Anderson, R.: Robbing the bank with a theorem prover. Technical Report UCAM-CL-TR-644, University of Cambridge (August 2005)

Cryptographic Verification by Typing for a Sample Protocol Implementation C´edric Fournet1, Karthikeyan Bhargavan2, and Andrew D. Gordon1,3 1 2 3

Microsoft Research INRIA Rocquencourt University of Edinburgh

Abstract. Type systems are effective tools for verifying the security of cryptographic protocols and implementations. They provide automation, modularity and scalability, and have been applied to large protocols. In this tutorial, we illustrate the use of types for verifying authenticity properties, first using a symbolic model of cryptography, then relying on a concrete computational assumption. (1) We introduce refinement types (that is, types carrying formulas to record invariants) for programs written in F# and verified by F7, an SMT-based type checker. (2) We describe a sample authenticated RPC protocol, we implement it in F#, and we specify its security against active adversaries. (3) We develop a sample symbolic library, we present its main cryptographic invariants, and we show that our RPC implementation is perfectly secure when linked to this symbolic library. (4) We implement the same library using concrete cryptographic primitives, we make a standard computational assumption, and we show that our RPC implementation is also secure with overwhelming probability when linked to this concrete library.

1 Verifying Protocol Implementations Cryptographic Protocols Go Wrong. Security flaws still regularly appear in widelyused protocol implementations, sometimes years after their deployment, despite the best efforts of skilled designers, developers, testers, and reviewers. We may organize these flaws into three categories: logical, cryptographic, and implementation flaws. – As a classic example of a logical flaw, consider the vulnerability in the publickey protocol of Needham and Schroeder (1978), discovered by Lowe (1996) in his seminal paper on model-checking security protocols. This man-in-the-middle attack is the staple example for protocol verification; it is well known in the formal methods research community, and many tools can discover it. Still, for instance, Cervesato et al. (2008) recently discovered that the IETF issued a public-key variant of Kerberos, shipped by multiple vendors, with essentially the same design flaw. – As an example of a cryptographic flaw, consider the padding-oracle attacks on unauthenticated ciphertexts, discovered by Bleichenbacher (1998) and Vaudenay A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 66–100, 2011. c Springer-Verlag Berlin Heidelberg 2011

Cryptographic Verification by Typing for a Sample Protocol Implementation

67

(2002). This is a well-known, practical class of side-channel attacks. Still, it involves subtle interferences between cryptographic algorithms and error handling, and remains difficult to prevent without inspecting many implementation details. For instance, Albrecht et al. (2009) recently showed that SSH implementations were still vulnerable to a similar attack. – Less interestingly, perhaps, implementation flaws due to coding errors are still commonplace. For instance, several security updates patched flaws in the processing of X.509 certificates in TLS clients, leading to secure connections with the wrong server. In contrast with functional testing, security testing does not provide much coverage against active attacks. The difficulty lies in the details of a large amount of code that might compromise the security of a protocol, for example in the parsing of untrusted data or the handling of runtime errors. We would like to reliably design, develop, and deploy protocols and similar programs that rely on cryptography. To this end, we build formal methods and automated tools to relate protocol implementations and cryptographic models. Before presenting one such method, based on refinement types, we briefly discuss cryptographic models and their practical application to production code. Symbolic and Computational Models for Cryptography. In theory, the main security properties of protocols are now well understood. Two main verification approaches have been successfully applied: (1) Symbolic, “black box” models of cryptography, first proposed by Needham and Schroeder (1978) and formalized by Dolev and Yao (1983), focus on the logical structure of the protocol, and assume that their cryptographic primitives are perfect. Cryptographic values range over abstract terms, in an algebra subject to a few rewrite rules. Adversaries range over arbitrary programs that control the network and can intercept, rewrite, and inject messages. These models naturally benefit from formalisms developed for programming languages; their security can be verified using automated tools that scale up to large systems (Abadi and Blanchet 2005). (2) Computational models of cryptography focus on algorithms and the complexity of attacks. Cryptographic values range over concrete bitstrings of bounded length. Adversaries range over probabilistic, polynomial machines. These models are more precise, but also more difficult to apply to large systems. Automated verification is more challenging, although formal tools are starting to appear. Reconciling these two approaches is an active subject of research. Under suitable restrictions and computational assumptions, one can prove that some symbolic models are computationally sound for all protocols, then conduct program verification using off-the shelf symbolic tools. On the other hand, when considering the details of concrete protocols, the existence of a sound symbolic model is far from granted, and it is often more effective to adapt verification tools to deal directly with the cryptographic patterns and assumptions of the protocol being considered. Specifications, Models, and Implementations. In practice, production code and cryptographic models evolve independently, and interact only through informal specifications.

68

C. Fournet, K. Bhargavan, and A.D. Gordon

Specifications and standards are large text documents: their main purpose is to define message formats and promote interoperability between implementations, rather than stating security goals and detailing their local enforcement. Models are short, abstract descriptions handwritten by specialists, using either formal specification languages or pseudo-code. They state what must be verified, but necessarily ignore the details of full-fledged implementations. Implementations, typically coded in C or C++, deal with low-level details. Sadly, their main security concerns still is to avoid buffer overruns and maintain memory safety. Cryptographic issues are manually reviewed from time to time, but this kind of code review can hardly be applied each time the code is modified. We propose to bridge the gap between models and implementations from both directions: we build tools that automatically verify cryptographic properties on source code, rather than simplified models; and we develop models as executable code (in this tutorial, written in F#); this code may serve as a reference implementation, aiming at simplicity rather than performance. It may additionally be used for testing other legacy implementations. Contents. This tutorial illustrates the verification of authentication properties on a sample RPC protocol coded in F#. The protocol relies on cryptographic keyed hashes, also known as MACs. It is a variant of protocols formally studied by Bhargavan et al. (2010) and Fournet et al. (2011). It is small enough to explain its code and its verification in details. We show how to verify it by typechecking against F7 interfaces, first using a symbolic model of MACs, then using a more concrete, computational assumption, namely resistance to existential forgery chosen-message attacks. The rest of the paper is organized as follows. Section 2 reviews F7, our main verification tool. Section 3 describes our sample protocol, presents its F# implementation, and discusses its security. Section 4 develops its modular, logical specification. Section 5 shows how to verify our code against active adversaries within a sample symbolic model of cryptography. Section 6 shows how to verify the same code against active, probabilistic polynomial adversaries, within a simple computational model of cryptography. Finally, Section 7 offers points for further reading. This tutorial omits most formal definitions. Another, complementary tutorial by Gordon and Fournet (2010) presents the refinement type system at the core of F7, and outlines its applications, but does not deal with cryptography. A website http://research.microsoft.com/f7 hosts additional materials, including related research papers, the latest release of the F7 tool, and the code presented in this tutorial.

2 F7: Automated Program Verification Using Refinement Types We now describe a syntax for F# programs annotated with logical specifications and refinement types and show how such programs can be verified using the F7 typechecker. This section is largely independent of the example developed in the rest of the paper. For a more detailed tutorial on the formal language and type system underlying F7, see Gordon and Fournet (2010). Here, we focus on the syntax and command-lines that are commonly used when verifying cryptographic protocols.

Cryptographic Verification by Typing for a Sample Protocol Implementation

69

2.1 Security Programming in F# We first give the core syntax of the fragment of F# we use for writing programs. This fragment is a standard, ML-like call-by-value functional language that includes recursive functions and recursive algebraic datatypes, plus two constructs for writing specifications. We can define data structures such as lists, options, trees, and XML documents in this language. Notably, however, our fragment does not support F# classes, objects, or looping constructs. Still, the language described here has proved adequate to program several thousand lines of security applications and large cryptographic protocols (Bhargavan et al. 2009a, 2010). We believe that for the security programmer, writing in this restricted fragment is a reasonable trade-off in exchange for automated verification support using F7. A program is a sequence of modules whose syntax can be defined as follows: Syntax of F# Modules (patterns, expressions, definitions) P, Q ::= (), true, false, i, “s” x (P1 , . . . , Pn ) hP e ::= (), true, false, i, “s” x (e1 , . . . , en ) fun x → e he e1 e2 let P = e1 in e2 match e with Σi (Pi → ei ) if e then e1 else e2 ) assume (p M) assert (p M) δ ::= let x = e let rec f = fun x → e type α = T type α = Σi (hi : Ti ) open M S ::= module M δ1 . . . δn

Patterns constants: unit, booleans, integers, strings variable tuple constructor Expressions constants: unit, booleans, integers, strings variable tuple function (scope of x is e) constructor application function application sequential composition (scope of variables in P is e2 ) pattern matching (scope of variables in Pi is ei ) conditional assume that p M is true require that p M must be true Definitions value definition recursive function definition (scope of f and x is e) type abbreviation recursive algebraic datatype import another module Module

We first state some syntactic conventions. Our phrases of syntax may contain two kinds of identifier. Type constructors, written α , refer to datatypes and type abbreviations, such as key or bytes. Program variables, written x, y, z refer to values and functions; by convention, we use f to refer to function names, h to refer to constructors, p to refer to predicate names, and l to refer to logical function names (explained later). Each variable has a scope and we identify phrases of syntax up to consistent renaming of bound variables. We write ψ {φ /ι } for the capture-avoiding substitution of the phrase φ for each free occurrence of identifier ι in the phrase ψ .

70

C. Fournet, K. Bhargavan, and A.D. Gordon

Each module (M) is written in a separate file (m.fs) and consists of a sequence of definitions. A definition (δ ) may introduce a new global variable (M.x, M.f) that represents a value or a (possibly recursive) function, or it may define a new type constructor symbol (M.α ) that represents a type abbreviation or a (possibly recursive) algebraic datatype with a set of term constructors (M.hi that take arguments of type Ti ). The types that appear in type definitions and abbreviations are in standard F# type syntax (not the extended F7 syntax defined later in this section). Values and type constructors in another module (N) can be referred to either by using their fully-qualified names (N.x, N.f, N.α ) or by opening the module (open N). As an illustration of the syntax, we display the module Msg from the file msg.fs Serializing messages as byte arrays: (msg.fs) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

module Msg type msg = | Request of string | Response of string let tag0 = Lib.utf8 "0" let tag1 = Lib.utf8 "1" let msg to bytes = fun m → match m with Request s → Lib.concat tag0 (Lib.utf8 s) | Response s → Lib.concat tag1 (Lib.utf8 s) let bytes to msg = fun b → let (tag,sb) = Lib.split 2 b in let s = Lib.iutf8 sb in if tag = tag0 then Request s else if tag = tag1 then Response s else failwith "incorrect tag"

The module Msg defines an algebraic datatype msg, two values tag0 and tag1 and two functions msg to bytes and bytes to msg. The type msg has two constructors Request and Response; each takes a string as argument and constructs a value of type msg. The right-hand-side of each value and function definition is an expression (e). The expressions in Msg use auxiliary functions from the module Lib; utf8 and iutf8 convert between strings and byte arrays; concat and split concatenate and split byte arrays. In addition, they use two predefined functions: = (structural equality) and failwith (to raise an error and terminate the program). An expression (e) may be a constant (2, “0”), a local or global variable (m, Msg.tag0), a tuple, a lambda expression representing a function (e.g. the definition of msg to bytes on line 10), a constructor application (Request s), a function application (Lib.utf8 s), a let-expression for sequential composition (line 17) , or a conditional expression written using either pattern matching (line 11) or if-then-else (line 18).

Cryptographic Verification by Typing for a Sample Protocol Implementation

71

Patterns (P) are used for ML-style pattern matching in match and let expressions. A pattern can be used to check the shape of an expression: whether it is a constant, a tuple (e.g. (tag,sb) on line 16), or uses a given constructor (Request s on line 12). If an expression matches a pattern, then its components are bound to the corresponding variables in the pattern. In addition, we introduce two new expression constructs (not used in Msg) to enable logical specifications within expressions. The expression assume(p M) states that at this point in the program, the logical predicate p can be assumed to hold at value M. Conversely, the expression assert(p M) states that at this point in the program, the logical predicate p is expected to hold at value M. During typechecking, these constructs are interpreted as logical annotations that serve as assumptions and verification goals (explained later in this section.) However, they have no effect at run-time—both constructs are implemented as noops in F#. Each predicate symbol p that appears in an expression must be previously declared as a datatype constructor; hence, each assume or assert is a well-formed F# expression. The programs in the rest of this tutorial conform to the syntax described here. In addition, they may use predefined values, functions, and types from the F# standard library (“Pervasives”). These include the function = that tests for the (structural) equality of two values, and the type list that has constructors for nil ([]) and cons (::). 2.2 Security Specifications in F7 We define a language for writing logical specifications for F# modules as F7 interfaces. The F7 interface language extends F# interfaces, hence every F# interface is an F7 interface, and every F7 interface may be erased to a normal F# interface. The full syntax of interfaces is given on the next page. An F7 interface for a module M is written in a separate file (m.fs7) and consists of a sequence of declarations. A declaration (σ ) may declare the F7 type of a variable or function (M.x, M.f), and this type is then expected to be the type of the corresponding value or function definition in the F# module M. A type abbreviation declaration introduces a new type constructor symbol (M.α ) as an abbreviation for an F7 type (T ). An algebraic type declaration introduces a new type constructor (M.α and a set of term constructors (hi that takes an argument of F7 type Ti ). An abstract type declaration introduces a new type constructor (M.α ) but leaves its implementation completely hidden. More generally, each type constructor may be parametrized by type arguments (e.g. ’t list) or by values (e.g. (’t; p:string) key), but we elide this detail here, and instead we treat parametrized types simply as schemas that are eliminated by inlining. Each value, function, or type declaration may have a visibility qualifier, public (default) or private: public declarations are considered part of the exported interface of the module and mark functions and types that are visible to all other F# modules (including the adversary), whereas private declarations are typically used for auxiliary functions and values that are needed for typechecking but are not exported by the module. We illustrate the syntax by displaying the interface for the Lib module used by Msg. The interface file lib.fs7 declares an abstract type bytes that represents byte arrays. It then introduces two logical functions Utf8 and Length, declares F7 types for the functions utf8, iutf8, length, concat, and split, and introduces three assumes.

72

C. Fournet, K. Bhargavan, and A.D. Gordon

Syntax of F7 Interfaces (values, formulas, types, declarations) M, N ::= (), true, false, i, “s” x (M1 , . . . , Mn ) hM lM C ::= true | false M1 = M2 | M1 = M2 pM not C | C ∧C | C ∨C C ⇒C |C ⇔C !x.C ?x.C T ::= unit, bool, int, string T ref α x : T {C} x : T1 → T2 x : T1 × T2 π ::= public | private σ ::= π val x : T π type α = T π type α = Σi (hi of Ti ) type α function val l : T1 → T2 predicate val p : T → bool assume C assert C I ::= module M σ1 . . . σn

Values constants: unit, booleans, integers, strings variable tuple constructor logical function application First-order Logic Formulas constants comparison predicate application boolean operators implication, if-and-only-if universal, first-order quantification (for all) existential, first-order quantification (exists) Types base types reference type constructor x of type T such that C (scope of x is C) dependent function type (scope of x is T2 ) dependent pair type (scope of x is T2 ) Visibility qualifier Declarations value or function declaration type abbreviation recursive algebraic datatype abstract type logical function logical predicate assume formula assert formula Interface

A logical function declaration introduces a new function symbol (M.l) that may only be used within formulas. The function symbol is initially uninterpreted, but we may add assumptions about its behaviour using an assume declaration. For example, our only assumption about Utf8 is stated in the assume on line 14, namely, that Utf8 is an injective function. When a logical function has more than one argument, they must be written in uncurried form (l(M1 , . . . , Mn )). The Lib interface uses a predefined uninterpreted logical function (|) with two arguments that represents byte array concatenation. A logical predicate declaration (not used in Lib) introduces a new predicate symbol (M.p) that may be used in formulas in interfaces and assume or assert expressions. An assume declaration states that a formula (C) is assumed to be true. Such declarations are used to define logical predicates and functions, and to state verification assumptions. For example, the assumes on lines 15 and 16 states our assumptions on

Cryptographic Verification by Typing for a Sample Protocol Implementation

73

Byte array conversions: (lib.fs7) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

module Lib type bytes function val Utf8: string → bytes function val Length: bytes → int val utf8: s:string → b:bytes {b = Utf8(s)} val iutf8: b:bytes → s:string {b = Utf8(s)} val length: b:bytes → l:int {Length(b) = l} val concat: b1:bytes → b2:bytes → b:bytes{b = b1|b2} val split: i:int{i > 0} → b:bytes → b1:bytes ∗ b2:bytes{b = b1|b2 ∧ Length(b1) = i} assume !x:string, y:string. (Utf8(x)=Utf8(y) ⇒ x = y) assume !x:bytes, y:bytes. Length(x | y) = Length(x) + Length(y) assume !b1,b2,c1,c2. (b1|b2 = c1|c2 ∧ Length(b1)=Length(c1)) ⇒ (b1=c1 ∧ b2=c2)

the relationship between Length and byte array concatenation (|); in particular, concatenating n bytes to an array increases its length by n, and if two equal byte arrays are split at the same index, the results are pointwise equal. An assert declaration requires that a formula (C) must be entailed by the formulas previously assumed. Such declarations are used to formulate (and verify) lemmas that serve as hints for subsequent steps. F7 types (T ) subsume and extend F# types. Like in F#, an F7 type may be a base type, a mutable reference containing a value of type T (T ref), or a type constructor (M.α ) representing a type abbreviation or algebraic type. A refinement type, written x : T {C}, represents a value M of type T such that C{M/x} holds. For example, the type b:bytes {b = Utf8(s)} (line 8) represents all byte arrays M that are equal to Utf8(s). A dependent pair type, written x : T1 × T2 , represents a pair of values (M, N) such that M has type T1 and N has type T2 {x/M}. In particular, we can use dependent pair types to write dependent tuples of the form x1 : T1 × . . . × xn : Tn . For example, the type b1: bytes ∗ b2:bytes{b = b1|b2 ∧ Length(b1)= i} (line 12) represents a pair of byte arrays (M,N) where the refinement type of the second component refers to the first. A dependent function type, written x : T1 → T2 , represents a function that takes an argument M of type T1 and returns a result of type T2 {M/x}. A function type is often written in the form x : T1 {C1 } → y : T2 {C2 }; here, the formula C1 is called the precondition and C2 is called the post-condition. For example, the type of split (line 12) is a function type that takes two arguments, an integer i and byte array b, and returns a pair of byte arrays (b1,b2); its precondition is that i must be greater than 0; its post-condition is that b is the concatenation of b1 and b2 and that the length of b1 is i. Formulas (C) appear in refinement types and in assume and assert declarations. Formulas are written in an untyped first-order logic with uninterpreted function and

74

C. Fournet, K. Bhargavan, and A.D. Gordon

predicate symbols. They may use standard operators for conjunction, disjunction, implication, and universal and existential quantification. For example, the assume on line 16 uses equality, conjunction, implication, and universal quantification. Each predicate or logical function has a fixed arity and must be declared before it can be used. For example, the predefined predicates for equality and inequality have arity 2. The behaviour of predicates can be defined by assuming formulas of the form ∀x.p x ⇔ C. Formulas speak about values (M). A value may be a constant such as a boolean, an integer, or a string (interpreted as a 0-ary function symbol). It may also be a constructed term (hM), where the constructor h is treated as a 1-ary function symbol that is oneto-one and invertible. It may be an n-tuple ((M1 , . . . , Mn )), which is interpreted as the application of an n-ary tuple constructor (Tuplen (M1 , . . . , Mn )). Finally, a value may be the application of a logical function symbol (l M), such as Length(b). 2.3 Modular Verification by Typechecking We consider programs of the form: L1 · · · Lm P1 · · · Pn · O where the modules L1 , . . . , Lm are trusted libraries that are assumed to behave correctly, the modules P1 , . . . , Pn are program modules that we wish to verify, and the module O represents an arbitrary adversary that does not contain any assumes or asserts. The adversary O is given access to the library and program modules through their exported (public) interfaces and hence, it may read any public variable and call any public function (any number of times). The adversary does not have to be well-typed in F7, but it must obey the F# types of the exported interface. Our verification goal is semantic safety: in every run of the program, every asserted formula logically follows from previously assumeed formulas. In particular, if the assert expressions in a program represents security goals, then these security goals should be satisfied in all runs of the program for all adversaries O. Our proof method is typechecking: we verify that each F# module in the program satisfies its typed F7 interface. Each judgment of the F7 type system is given relative to an environment, E, which is a sequence μ1 , . . . , μn , where each μi may be a value or type declaration (σ ), or subtype assumption α fsc lib.fs mac.fs format.fs rpc.fs test.fs -o rpc.exe

and the messages exchanged over TCP are: > ./rpc.exe Connecting to localhost:8080 Sending {BgAyICsgMj9mhJa7iDAcW3Rrk...} (28 bytes) Listening at ::1:8080 Received Request 2 + 2? Sending {AQA0NccjcuL/WOaYS0GGtOtPm...} (23 bytes) Received Response 4

4 Logical Refinements for the RPC Protocol We now develop more precise typed interfaces that embed logical refinements to support our verification effort. The interfaces presented is this section are used both for symbolic and computational verification. 4.1 Refined Interface for Formatting the Authenticated Digests We introduce two auxiliary predicates for the payload formats: Requested and Responded are the (typechecked) postconditions of the functions request and response. Hence, the Format module used by the protocol implementation exports the following interface. Interface to Format Modules (format.fs7): module Format open Lib predicate val Requested: bytes ∗ string → bool predicate val Responded: bytes ∗ string ∗ string → bool val request: s:string → b:bytes {Requested(b,s)} val response: s:string → t: string → b:bytes {Responded(b,s,t)} assert!v,v’,s,s’,t’. (Requested(v,s) ∧ Responded(v’,s’,t’)) ⇒ (v = v’) assert!v,v’,s,s’. (Requested(v,s) ∧ Requested(v’,s’) ∧ v = v’) ⇒ (s = s’) assert!v,s,s’,t,t’. (Responded(v,s,t) ∧ Responded(v,s’,t’)) ⇒ (s = s’ ∧ t = t’)

Cryptographic Verification by Typing for a Sample Protocol Implementation

83

Typechecking involves the automatic verification that our formatting functions are injective and have disjoint ranges, as informally explained in informal assumption (3). Verification is triggered by asserting the formulas marked with assert, so that Z3 proves them. For verifying the rest of the protocol, we can hide the definition of Requested and Responded and rely only on these three derived logical properties, which confirms that the security of our protocol depends only on these properties, rather than a specific digest format. 4.2 Refined, Ideal Interface for Cryptographic MACs To capture the intended properties of MACs, we rely on another, refined ideal interface for the Mac module, as follows: type bytespub = bytes type key type text = bytes type mac = b:bytes{Length(b) = macsize} predicate val Msg: key ∗ text → bool val GEN: unit → key val MAC: k:key → t:text{Msg(k,t)} → mac val VERIFY: k:key → t:text → m:mac → v:bool{v=true ⇒ Msg(k,t)} val LEAK: k:key{!t. Msg(k,t)} → b:bytes {Length(b) = keysize}

This refined interface is designed for protocol verification, and is similar to those used for typing symbolic models of cryptography (Bhargavan et al. 2010). It declares the type of keys as abstract, thus preventing accidental key leakage in data sent on a public network or passed to MAC. To support authentication properties, the interface introduces a logical predicate on keys and texts, Msg(k,t), to indicate that t is an authentic message MACed with key k. (In F7 syntax, predicates are declared as constructors.) This predicate occurs in the types of MAC and VERIFY, as a precondition for MACing and as a post-condition of successful verification. The interpretation of Msg(k,t) is protocol-dependent: as illustrated next for the RPC protocol, each protocol defines Msg according to the properties it wishes to authenticate using MACs, possibly giving a different interpretations to each key. Since our interface must be sound for any logical definition of Msg, this entails that, until the key is leaked, calls to VERIFY succeeds at most for texts previously passed as arguments to MAC, thus excluding the possibility of forging a MAC for any other text. The precondition of LEAK accounts for (well-typed) dynamic key compromise: the protocol may call LEAK to access the actual key bytes at any point, while other parts of the protocol still rely on the postcondition of VERIFY. Thus, the precondition !t, Msg(k,t) demands that, before leaking the key, Msg(k,t) holds for all texts. (In F7 syntax, !t is logical universal quantification over t.) The function LEAK can be used to model the potential corruption of principals by leaking their keys to the adversary; see Bhargavan et al. (2009b, 2010) for protocols verified by typing despite partial key compromise. Taking advantage of refinements, the interface also enforces the consistency of sizes for MACs and key bytes. (In F7, Length is a logical library function.) Thus, for instance,

84

C. Fournet, K. Bhargavan, and A.D. Gordon

typechecking would catch parsing errors leading to a call to VERIFY with a truncated MACs, as its argument would not match the refined type mac. Intuitively, the ideal interface is too good to be true: a simple information-theoretic argument shows that any implementation such that (1) MACs are shorter than texts and (2) MAC and VERIFY communicate only through MACs must be unsafe. (In particular, we do not have C ; I.) In the next two sections, we will show how to formally meet the properties exported by the ideal interface, first symbolically, by implementing MACs using abstracts terms instead of concrete byte arrays, then computationally, by implementing an intermediate error-correcting ideal functionality for MACs. 4.3 Instantiating the MAC Library to the RPC Protocol We are now ready to define the predicate Msg for the RPC protocol, by relating the format of the bytes that are cryptographically authenticated to their interpretation in terms of authentic requests and responses. Usage of Keys: (rpc.fs7) definition !a,b,k. KeyAB(k,a,b) ⇒ ( !m. Msg(k,m) ⇔ Corrupt(a) ∨ Corrupt(b) ∨ (?s. Requested(m,s) ∧ Request(a,b,s)) ∨ (?s,t. Responded(m,s,t) ∧ Response(a,b,s,t)))

For each key generated for the protocol, the definition gives four valid interpretations: one of the two parties is corrupted, or this is a correctly-formatted request, or this is a correctly-formatted response. This definition is not in itself trusted. It is included in the F7 interface for RPC and used to typecheck its code. If the definition was too strong, then we would get a typing error in the code that calls MAC and needs to prove Msg(k,t) as a precondition. If the definition was too weak, then we would get a typing error at the assertions, as the postcondition of VERIFY would not suffice to prove the asserted formula. Exercise 3. Explain how to model the re-use of keys in both directions, so that a key k shared between a and b may be used also for authenticating requests from b to a and responses from a to b? What is the resulting definition of predicate Msg(k,m)? Is the resulting sample protocol well-typed? Is it secure? Otherwise, describe how to fix it. Exercise 4. Explain how to model sessions that consist of a sequence of requests and responses? What is the resulting definition of predicate Msg(k,m)? 4.4 Towards Security Theorems for Our Sample Protocol Before presenting our verified cryptographic libraries, we outline the general form of our security theorems for the RPC protocol, given in the next two sections. We consider the runtime safety of programs of the form Lib · MAC · Format · RPC · Adv

Cryptographic Verification by Typing for a Sample Protocol Implementation

85

where – – – –

Lib is a standard library; MAC is our cryptographic library for MACs; RPC is our protocol code; and Adv is an active adversary, ranging over programs well-typed against the adversary interface. These programs cannot directly assume or assert logical properties, although they can cause RPC and MAC to do so by calling them through the adversary interface. Without loss of generality, we assume that Adv returns (): unit.

Our protocol is deemed secure when all runs of programs of the form above are safe, that is, all their assertions follow from their assumptions. Informally, this excludes any authentication and correlation attack against our protocol using the capabilities granted by the adversary interface. To prove that Lib · MAC · Format · RPC · Adv is safe, we will mostly rely on typing: we prove that Lib · MAC · Format · RPC · Adv : unit

(1)

then conclude by type safety. More precisely, we use F7 to perform the following tasks: (1) (2) (3) (4) (5)

typecheck Lib ; ILib (possibly assuming properties of primitive functions); typecheck ILib MAC ; IMAC ; typecheck ILib Format ; IFormat ; typecheck ILib , IMAC , IFormat RPC ; IRPC ; check the subtyping relation ILib , IMAC , IRPC f7 mac-symb.fs7 formats-symb.fs7 formats-symb.fs This verifies, in particular, that (1) our implementations of request and response comply with their specifications Request and Response, and (2) these specifications logical imply the asserted properties exported by our interface for verifying RPC. To get the details, we may insert the command-line option -scripts formats, which causes F7 to keep an SMP trace of all its calls to Z3, its underlying prover. We then observe that typechecking formats−symb.fs involves calling Z3 on 9 logical proof obligations: 2 for checking implementations against specifications (1); 4 for proving that terms are indeed logical Bytes; and 3 for proving the asserted formulas (2). 5.2 Refinement Types for the RPC Protocol Using F7, we check that our protocol code (with the Net and Crypto library interfaces, and the assumed formulas above) is a well-typed implementation of the interface below. Typed Interface for the RPC Protocol: (rpc.fs7) val keygen: a:principal → b:principal → k:key {KeyAB(k,a,b)} val corrupt: a:principal → b:principal {Corrupt(a) ∨ Corrupt(b)} → k:key {KeyAB(k,a,b)} → b:bytes //omitted symbolically: {Length(b) = keysize} val client: a:principal → b:principal → k:key {KeyAB(k,a,b)} → r:req{ Request(a,b,r) } → unit val server: a:principal → b:principal → k:key {KeyAB(k,a,b)} → ( r:req → s:rsp {Request(a,b,r) ⇒ Response(a,b,r,s)}) → unit

5.3 Modelling the Symbolic Adversary As outlined in Section 3.6, we model an active adversary as an arbitrary program welltyped against an adversary interface, Iadv . In the symbolic model, we define the interface as follows. Recall that type bytespub = t:bytes {Pub(t)}. The function GENPUB generates a new key k, assumes that Msg(k,t) holds for all t, which establishes Pub(k), and returns k. The function princ corrupt assumes the event Corrupt(a) for its argument a, and returns all the keys associated with a; we have Pub(k) for each k associated with a, because, by definition of KeyAB in Section 4.3, the predicate Msg(k,t) holds for all t.

90

C. Fournet, K. Bhargavan, and A.D. Gordon

Adversary Interface: Iadv val send: bytespub → unit val recv: (bytespub → unit) → unit val utf8: string → bytespub val iutf8: bytespub → string val tag: int → bytespub → bytespub val itag: bytespub → int ∗ bytespub val pair: bytespub → bytespub → bytespub val ipair: bytespub → bytespub ∗ bytespub val GENPUB: unit → bytespub val MAC: bytespub → bytespub → bytespub val VERIFY: bytespub → bytespub → bytespub → bool val keygen: principal → principal → unit val client: principal → principal → string val server: principal → principal → (string → string) → unit val princ corrupt: principal → bytespub list

Let a symbolic adversary Adv be a program that contains no assume or assert and that is well-typed against the adversary interface, that is, Iadv Adv : unit. 5.4 Symbolic Soundness for the RPC Protocol We are now ready to formally state our target security theorem for this protocol. We say that an expression is semantically safe when every executed assertion logically follows from previously-executed assumptions. For the formal details, see Bhargavan et al. (2010). The proof of the following is by typechecking with F7, as sketched in Section 4.4. Theorem 2 (Symbolic RPC Safety). For any symbolic adversary Adv, the module Lib · MAC · Format · RPC · Adv is semantically safe.

6 Computational Soundness We now explain how to re-use our refinement-based specification to verify the RPC protocol implementation using a more concrete model of cryptography. Formally, we need to adapt our language semantics to reason about probabilistic and polynomial-time expressions—we refer the interested reader to Fournet et al. (2011) for a detailed presentation. We also need to develop another, computationally-sound implementation of our Mac module given a standard cryptographic assumption, as explained below. On the other hand, computational soundness makes little difference as regards automated protocol verification: we still use the F7 typechecker, and we only need to slightly adapt our F# code and F7 interfaces for the RPC protocol.

Cryptographic Verification by Typing for a Sample Protocol Implementation

91

In contrast with symbolic models, our reasoning relies on the concrete representation of values of type bytes, as they are used for instance in the system implementation of Mac in Section 3.4. We let type bytes = byte array, where byte is the primitive F# type for 8-bit integers. More precisely, we often keep track of the length of byte arrays, so that we can reason about message formats and call cryptographic primitives with correct arguments. Hence, for instance, we let type mac = b:bytes {Length(b)= macsize} for concrete MAC values, so that we consistently use MACs with exactly macsize bytes. Another difference is that our computational libraries need not share a single definition for all cryptographic terms. In Section 5, the symbolic definition of bytes covers MACs, but also UTF8 encodings and concatenation of bytes; more generally, the symbolic libraries of Bhargavan et al. (2010) jointly define all message and cryptographic processing, using large algebraic data types. In contrast, our computational variants of the modules Mac and Format are independent, and more generally each cryptographic primitive can be specified and implemented in a separate module (Fournet et al. 2011). On the other hand, computationally-sound interfaces are usually more restrictive than symbolic ones. We first present our concrete implementation for authenticated message digests, as a first example of byte-level verification, then we discuss computational soundness for MACs using an intermediate, ideal implementation, and we finally give a sample computational security theorem for the RPC code. 6.1 Verifying Binary Message Formats We write (and verify) another pair of formatting functions, this time using concrete bytes and inserting 2-byte lengths for variable-length fields instead of relying on abstract pairs—these two functions are used for compiling and running the protocol. The implementation is: F# Concrete Implementation of Formatting: (format.fs) module Format open Lib let tag0 = [| 0uy |] let tag1 = [| 1uy |] let request s = concat tag0 (utf8 s) let response s t = let sb = utf8 s concat tag1 (concat (int2bytes (length sb)) (concat sb (utf8 t)))

We typecheck this implementation against the interface listed in Section 4.1 supplemented with the declarations below: val tag0: b:bytes { Length(b)=1 } val tag1: b:bytes { Length(b)=1 ∧ b = tag0 } definition !v,s. Requested(v,s) ⇔ v = tag0 | Utf8(s) definition !v,s,t. Responded(v,s,t) ⇔ v = tag1 | Int2Bytes(Length(Utf8(s ))) | Utf8(s) | Utf8(t)

92

C. Fournet, K. Bhargavan, and A.D. Gordon

The refinements on tags explicitly state that they are distinct one-byte arrays; the logical definitions give a complete specification of our digest formats. The automated verification relies on precise type annotations in our library to keep track of the lengths of concatenated byte arrays. For instance, the second asserted property of the interface logically follows by inlining the definition of Requested on both sides of v = v’, and then applying our axiom on equality of concatenations when the length of the first byte arrays are the same. 6.2 A Concrete Refined Interface for MACs We first give more precise types to the concrete MAC implementation of Section 3.4. These types reflect basic assumptions on the underlying cryptographic algorithms, which are outside the scope of this code verification effort; they are used in the proof but they are not sufficient to ensure message authentication. We use the interface type mac = b:bytes{Length(b) = macsize} predicate val GENerated: key → bool predicate val MACed: key ∗ text ∗ mac → bool val GEN: unit → k:key {GENerated(k)} val MAC: k:key → t:text → m:mac {MACed(k,t,m)} val VERIFY: k:key → t:text → m:mac → v:bool{ GENerated(k) ∧ MACed(k,t,m) ⇒ v = true } val LEAK: k:key → b:bytes{Length(b) = keysize} assume !k,t,m0,m1. GENerated(k) ∧ MACed(k,t,m0) ∧ MACed(k,t,m1) ⇒ m0 = m1 In this interface, written I C in the following, two auxiliary predicates GENerated and MACed keep track of the results returned by GEN and MAC. Relying on these predicates, the post-condition of VERIFY states a functional correctness property: verification of a MAC computed using matching keys and texts always succeeds; and the final assumption states that MAC is a deterministic function. These properties are easily met by any implementation that just recomputes the MAC for verification, as our concrete implementation does: to match IC , it suffices to add events at the end of GEN and MAC to record their result, and to rely on the functionality of MAC to establish the post-condition of VERIFY. 6.3 Resistance to Existential Chosen-Message Forgery Attacks (CMA) We now describe our computational notion of security for MACs, expressed as a game between the cryptographic functionality and a probabilistic polynomial-time adversary. We adopt a standard notion, named resistance against existential Chosen-Message forgery Attacks (CMA) and first proposed by Goldwasser et al. (1988) for signatures; we follow the presentation of Bellare et al. (1996; 2005). As usual with concrete cryptography, the MAC key is a finite bitstring, so we cannot entirely exclude that a lucky adversary may just guess the key, or some valid MACs, using for instance random sampling. Instead, our notion of security is relative to a security parameter, which informally controls the length of the key and other cryptographic

Cryptographic Verification by Typing for a Sample Protocol Implementation

93

values, and we bound the probability that an adversary succesfully forges a MAC: we say that a probabilistic, polynomial-time program is asymptotically safe when the probability that any logical assertion fails at runtime is negligible, that is, is a function of the security parameter that converges to zero faster than the inverse of any polynomial. In the game that defines CMA, an adversary attempts to forge a valid MAC; to this end, it can adaptively call two oracles for computing and verifying MACs on a single, hidden key; the adversary wins if it can produce a text and a MAC such that verification succeeds but the text has not been passed to the MAC oracle. We set up this game using an F# module, written CMA and defined as follows: let k = GEN() let log = ref [] let mac t = log := t::!log; MAC k t let verify t m = let v = VERIFY k t m in assert(not v || List.mem t !log); v This code generates a key, implements the two oracles as functions using that key, and maintains a log of all texts passed as arguments to the mac function. We intend to run this code with an adversary given access to mac and verify, but not k or log. The verification oracle also tests whether any verified pair wins the game: the assert claims that this does not happen: either v is false, or t is in the log. (In the asserted formula, || is logical disjunction.) Definition 1. A Mac implementation C is CMA-secure when, for all p.p.t. expressions A with no assume or assert such that mac: text → mac, verify: text → mac → bool A: unit, the expression C · CMA · A is asymptotically safe. We recall the two main security theorems of Fournet et al. (2011) for MACs, which establish asymptotic safety for programs that are well-typed against the ideal interface of a CMA-secure MAC, detailed in Section 4.2 and written IMAC below. Crucially, although keys are just bytes, their type in IMAC is abstract, so that protocols cannot directly generate or access their values. Theorem 3 (Asymptotic Safety for MAC). Let C be a p.p.t. implementation of the refined concrete interface: C ; I C . Let A be a p.p.t. expression such that I A : unit. If C is CMA-secure, then C · A is asymptotically safe. We will apply this theorem to show that our sample RPC code is secure against all active probabilistic polynomial-time adversaries, but first we explain how it is established by programming and typechecking. 6.4 An Ideal Functionality for MACs Our main tool is an intermediate implementation of Mac, written F and defined below, such that C · F is well-typed against the ideal interface IMAC . (We call F an ideal functionality for MACs, and call C · F an ideal MAC implementation; in terms of universal composability, C emulates F, with C as a simulator.) A second security theorem states that the concrete and ideal implementations are indistinguishable (≈ε ), that is,

94

C. Fournet, K. Bhargavan, and A.D. Gordon

the probability that a probabilistic polynomial adversary guesses which of the two implementations it is running with minus 12 (the probability of guessing at random) is negligible. Theorem 4 (Ideal Functionality for MAC). Let C be a p.p.t. CMA-secure implementation of the concrete-refined interface IC . Let A be p.p.t. and such that I A : bool. We have C · A ≈ε C · F· A. The proof relies on a standard code-based cryptographic argument to relate the code of F to the similar, but simpler code that defines CMA security (see Fournet et al. 2011). Intuitively, the ideal implementation is thus both perfectly secure and indistinguishable from the concrete one. This is achieved by intercepting all calls to the concrete implementation and correcting the result of concrete verification from true to false when the message has not been MACed and the key has not been leaked—our CMA assumption implies that such corrections occur with negligible probability. Our ideal functionality re-defines the type for keys, now represented as integer indexes, and re-defines the functions GEN, MAC, VERIFY, and LEAK using those provided by any implementation of the refined concrete interface I C . It maintains a global, private association table ks that maps each key index to some internal state, consisting of a concrete MAC key (kv), a list of previously-MACed texts (log), and a boolean flag (leaked) indicating whether the key has been leaked or not. In the code below, qualified names such as Mac.GEN refer to entries from the concrete interface I C , while names such as GEN refer to their re-definition. Ideal Functionality for MACs: (macf.fs) let ks = ref [] (* state for all keys generated so far *) let GEN () = let k = list length !ks let kv = Mac.GEN() in ks := (k,(kv,empty log k,false flag k))::!ks; k let MAC k t = let (kv,log,leaked) = assoc k !ks in log := t:: !log; Mac.MAC kv t let VERIFY k t m = let (kv,log,leaked) = assoc k !ks in Mac.VERIFY kv t m && (mem k t !log || !leaked) let LEAK k = let (kv,log,leaked) = assoc k !ks in leaked := true; Mac.LEAK kv In this code, the auxiliary functions list length, assoc, and mem are the standard ML functions List.length, List.assoc, and List.mem with more precise refinement types, and the functions empty log and false flag allocate mutable references that initially contain an empty list and false, respectively. The four main functions are as follows:

Cryptographic Verification by Typing for a Sample Protocol Implementation

95

– GEN allocates a key with index k (the number of keys allocated so far), generates a concrete key, and records its initial state, with an empty log and a flag set to false. When called with a key, the three other functions first perform a table lookup to retrieve its state (kv,log,leaked). – MAC computes the MAC using the concrete kv and adds the MACed text to the log. – VERIFY performs the concrete MAC verification, then it corrects its result from true to false when (i) the concrete verification succeeds; but (ii) the text has not been MACed, as recorded in log; and (iii) the key has not been leaked, as recorded by leaked. – LEAK returns the concrete key bytes, and sets the leaked flag to true. The state entries in the table ks are typed using the following auxiliary declarations: type (;k:key) msg = t:text{ Msg(k,t) } type (;k:key) leak = b:bool{ b=true ⇒ !t. Msg(k,t) } type (;k:key) state = Mac.key ∗ (;k) msg list ref ∗ (;k) leak ref val ks: (k:key ∗ (;k) state) list ref These types enforce runtime invariants: all texts t kept in the log for key k are such that Msg(k,t), and we have !t, Msg(k,t) whenever the flag is set to true. (In F7 syntax, (;k:key) msg declares a type whose refinements depend on the value k.) We automatically typecheck I C F ; I ; intuitively, this holds because the result of the verification function is always consistent with the log and the flag, whose refinement types record enough facts Msg(k,t) to establish the post-condition of VERIFY. We have also supposed that C ; IC , so the module C · F obtained by composing the concrete MAC implementation C with F is a well-typed implementation of the ideal interface IMAC , as F corrects any unsafe verification of C, but it is unrealistic, as F relies on global shared state. We conclude this development with an informal proof of Theorem 3 (Asymptotic Safety for MAC). For a given A such that IMAC A, we obtain C · F · A by composing our typing judgments, so C · F · A is perfectly safe, by type safety. Moreover, by Theorem 4 (Ideal Functionality for MAC). C · F · A and C · A are indistinguishable, so C · A is also safe except with a negligible probability. 6.5 Computational Soundness for the RPC Protocol We give on the next page a variant of our code of the RPC protocol. This code differs from that presented in Section 3 only in its use of the network: in the computational model of computation, we cannot rely on parallel threads to send and receive messages; instead, we let the adversary schedule the two stages of the client and the server. On the other hand, the code for generating and verifying messages is unchanged. For the second stage of the client, client2, the type interface is as follows val client2: a:principal → b:principal → k:key {KeyAB(k,a,b)} → r:req{ Request(a,b,r) } → msg → (s:rsp{ Response(a,b,r,s) ∨ Corrupt(a) ∨ Corrupt(b) }) option

96

C. Fournet, K. Bhargavan, and A.D. Gordon

F# Implementation for the Authenticated RPC Protocol: (rpc.fs) module RPC open Pi open Lib open Mac open Format type principal = string type req = string type rsp = string type msg = bytes type fact = KeyAB of key ∗ principal ∗ principal let keygen (a:principal) (b:principal) = let k = GEN() in assume(KeyAB(k,a,b)); k let corrupt (a:principal) (b:principal) k = LEAK k let client (a:principal) (b:principal) k s = let msg1 = concat (utf8 s) (MAC k (request s)) send msg1 let client2 (a:principal) (b:principal) k s msg2 = let (v,m’) = split macsize msg2 in let t = iutf8 v in if VERIFY k (response s t) m’ then Some(t) else None let server (a:principal) (b:principal) k service = recv (fun msg1 → let (v,m) = split macsize msg1 in let s = iutf8 v in if VERIFY k (request s) m then let t = service s let msg2 = concat (utf8 t) (MAC k (response s t)) send msg2)

Cryptographic Verification by Typing for a Sample Protocol Implementation

97

Let Net range over modules implementing the network and the scheduler, informally controlled by the adversary, with the following interface for the protocol:

INet = send: bytes → unit, recv: (bytes → unit)→ unit

In addition, Net may export an arbitrary interface INET for the adversary. Let CRPC be the sample protocol code above, after inlining its other (trusted) library functions, with the

following interface for the adversary: IRPC = client: string → unit, server: unit → unit We arrive at the following computational safety theorem:

Theorem 5 (Computational RPC Safety). Let C be a CMA-secure p.p.t. module with no assert such that C ; I C . Let Net be a module with no assert such that Net ;

INET , Net ; INET , and Net ·CRPC is p.p.t. Let A be a polynomial expression with no

assert such that INET , IRPC A : unit. The expression C · Net ·CRPC · A is asymptotically safe. (Technically, we state our p.p.t. assumption on Net composed with CRPC because Net

has a second-order interface.) In case INET consists of first-order functions on bytes, and for each concrete definition of Net, we obtain as a special case a computational safety theorem for an adversary A with traditional oracle access to the network and the protocol. Typability of A is then irrelevant, since A exchanges only raw bytes with the rest of the system and may implement any polynomial computation on those bitstrings. Although we now rely on computational assumptions, our protocol code still typechecks essentially for the same reasons as in Section 5. Compared with our symbolic theorem, the computational theorem involves a more complex semantics, but a simpler adversary interface: the computational adversary need not be given access to the cryptographic library or formatting functions such as concat, since it may instead include its own concrete implementation of MAC and formatting algorithms.

7 Further Reading The companion tutorial (Gordon and Fournet 2010) focuses on the type theory underlying F7, and includes pointers for further reading on operational semantics, type systems, process calculi, and refinement types. That tutorial also discusses some of the initial applications of F7, including synthesis of security protocols for multiparty sessions (Bhargavan et al. 2009a), security of multi-tier web programming (Baltopoulos and Gordon 2009), auditability by typing (Guts et al. 2009), and the use of the refined state monad to verify imperative F# code (Borgstr¨om et al. 2011), a line of work resulting in the use of F7 to verify that SQL transactions maintain database integrity (Baltopoulos et al. 2011). Recent applications of F7 to cryptographic software include verifications for F# reference implementations of the XML-based CardSpace protocol (Bhargavan et al. 2008a, 2010) and of the DKM key management library (Acar et al. 2011). The present tutorial introduces F7 as a tool for verifying cryptographic software in both the symbolic and computational models. We have illustrated an approach to computational verifications based on the work of Fournet et al. (2011); Backes et al.

98

C. Fournet, K. Bhargavan, and A.D. Gordon

(2010) develop an alternative approach, in which they obtain computationally sound verifications of F7 programs by appeal to the general CoSP framework. F7 does not support union or intersection types; Backes et al. (2011) extend the type theory of F7 with these features and show their utility for dealing with public-key cryptography and zero-knowledge proofs. In other recent work, Swamy et al. (2011) present the design and implementation of F*, which integrates the refinement types of F7 into the Fine language (Swamy et al. 2010), and hence supports cryptographic software. In earlier work on protocols in F#, Bhargavan et al. (2008b) develop FS2PV, which analyzes F# software by compilation to ProVerif (Abadi and Blanchet 2005). ThepioneersofcryptographicsoftwareverificationareGoubault-Larrecq and Parrennes (2005), whose tool Csur analyzes secrecy properties of C code via a custom abstract interpretation. Another verification tool for C is ASPIER (Chaki and Datta 2009), which relies on various abstractions and on software model-checking techniques. Dupressoir et al. (2011) recast the F7 approach to symbolic verification of protocol code in the setting of the general-purpose C verifier VCC (Cohen et al. 2009) so as to verify protocol code in C; their work is the first to prove memory safety and security properties simultaneously for C. (F7 need only prove security properties, as the underlying F# type system proves memory safety directly.) Other approaches to C cryptographic protocols include: automatic generation of protocol monitors, to detect if production protocols deviate from their specification, from abstract protocol descriptions Pironti and J¨urjens (2010), and the use of symbolic execution (Corin and Manzano 2011; Aizatulin et al. 2011) to extract potentially verifiable symbolic representations of cryptographic computations in C. In summary, research on verifying cryptographic implementations is an active area. As this tutorial illustrates, F7 shows the viability of refinement types for verifying security properties of F# code in both the symbolic and computational models. It remains a research challenge to scale up verification to large, existing codebases, and to languages more commonly used for cryptographic software, including C, Java, C#, and even JavaScript.

References Abadi, M., Blanchet, B.: Analyzing security protocols with secrecy types and logic programs. JACM 52(1), 102–146 (2005) Abadi, M., Gordon, A.D.: A calculus for cryptographic protocols: The spi calculus. Information and Computation 148, 1–70 (1999) Acar, T., Fournet, C., Shumow, D.: DKM: Design and verification of a crypto-agile distributed key manager (2011), http://research.microsoft.com/en-us/ um/people/fournet/dkm/ Aizatulin, M., Gordon, A.D., J¨urjens, J.: Extracting and verifying cryptographic models from C protocol code by symbolic execution (in submission, 2011) Albrecht, M., Paterson, K., Watson, G.: Plaintext recovery attacks against SSH. In: 2009 30th IEEE Symposium on Security and Privacy, pp. 16–26 (May 2009) Backes, M., Maffei, M., Unruh, D.: Computationally sound verification of source code. In: ACM CCS, pp. 387–398 (2010) Backes, M., Hrit¸cu, C., Maffei, M.: Union and intersection types for secure protocol implementations. In: TOSCA (2011)

Cryptographic Verification by Typing for a Sample Protocol Implementation

99

Baltopoulos, I., Gordon, A.D.: Secure compilation of a multi-tier web language. In: ACM SIGPLAN Workshop on Types in Language Design and Implementation (TLDI 2009), pp. 27–38 (2009) Baltopoulos, I., Borgstr¨om, J., Gordon, A.D.: Maintaining database integrity with refinement types. In: ECOOP 2011 (2011) Bellare, M., Rogaway, P.: Introduction to modern cryptography. In: UCSD CSE 207 Course Notes (2005) Bellare, M., Canetti, R., Krawczyk, H.: Keying hash functions for message authentication. In: Koblitz, N. (ed.) CRYPTO 1996. LNCS, vol. 1109, pp. 1–15. Springer, Heidelberg (1996) Bhargavan, K., Fournet, C., Gordon, A.D., Swamy, N.: Verified implementations of the Information Card federated identity-management protocol. In: Proceedings of the ACM Symposium on Information, Computer and Communications Security (ASIACCS 2008), pp. 123–135. ACM Press, New York (2008) Bhargavan, K., Fournet, C., Gordon, A.D., Tse, S.: Verified interoperable implementations of security protocols. ACM TOPLAS 31, 5:1–5:61 (2008) Bhargavan, K., Corin, R., Deni´elou, P.-M., Fournet, C., Leifer, J.J.: Cryptographic protocol synthesis and verification for multiparty sessions. In: CSF, pp. 124–140. IEEE Computer Society, Los Alamitos (2009) Bhargavan, K., Corin, R., Deni´elou, P.-M., Fournet, C., Leifer, J.J.: Cryptographic protocol synthesis and verification for multiparty sessions. In: 22nd IEEE Computer Security Foundations Symposium (CSF 2009), pp. 124–140. IEEE Computer Society, Los Alamitos (2009) Bhargavan, K., Fournet, C., Gordon, A.D.: Modular verification of security protocol code by typing. In: ACM Symposium on Principles of Programming Languages (POPL 2010), pp. 445–456 (2010) Bleichenbacher, D.: Chosen ciphertext attacks against protocols based on the RSA encryption standard PKCS #1. In: Krawczyk, H. (ed.) CRYPTO 1998. LNCS, vol. 1462, pp. 1–12. Springer, Heidelberg (1998) Borgstr¨om, J., Gordon, A.D., Pucella, R.: Roles, stacks, histories: A triple for Hoare. Journal of Functional Programming 21(2), 159–207 (2011) Cervesato, I., Jaggard, A.D., Scedrov, A., Tsay, J.-K., Walstad, C.: Breaking and fixing public-key Kerberos. Information and Computation 206(2-4), 402–424 (2008) Chaki, S., Datta, A.: ASPIER: An automated framework for verifying security protocol implementations. In: IEEE Computer Security Foundations Symposium, pp. 172– 185 (2009) Cohen, E., Dahlweid, M., Hillebrand, M.A., Leinenbach, D., Moskal, M., Santen, T., Schulte, W., Tobies, S.: VCC: A practical system for verifying concurrent C. In: Berghofer, S., Nipkow, T., Urban, C., Wenzel, M. (eds.) TPHOLs 2009. LNCS, vol. 5674, pp. 23–42. Springer, Heidelberg (2009) Corin, R., Manzano, F.A.: Efficient symbolic execution for analysing cryptographic pro´ Wieringa, R., Zannone, N. (eds.) ESSoS tocol implementations. In: Erlingsson, U., 2011. LNCS, vol. 6542, pp. 58–72. Springer, Heidelberg (2011) Dolev, D., Yao, A.: On the security of public key protocols. IEEE Transactions on Information Theory IT-29(2), 198–208 (1983)

100

C. Fournet, K. Bhargavan, and A.D. Gordon

Dupressoir, F., Gordon, A., J¨urjens, J., Naumann, D.: Guiding a general-purpose C verifier to prove cryptographic protocols. In: 24th IEEE Computer Security Foundations Symposium (to appear, 2011) Eastlake, D., Jones, P.: US Secure Hash Algorithm 1 (SHA1). RFC 3174 (2001) Fournet, C., Kohlweiss, M., Strub, P.-Y.: Modular code-based cryptographic verification. Technical report, sample code, and formal proofs (2011), http:// research.microsoft.com/˜fournet/comp-f7/ Goldwasser, S., Micali, S., Rivest, R.: A digital signature scheme secure against adaptive chosen-message attack. SIAM Journal on Computing 17(2), 281–308 (1988) Gordon, A.D., Fournet, C.: Principles and applications of refinement types. In: Esparza, J., Spanfelner, B., Grumberg, O. (eds.) Logics and Languages for Reliability and Security. IOS Press, Amsterdam (2010); Available as Microsoft Research Technical Report MSR–TR–2009–147 Goubault-Larrecq, J., Parrennes, F.: Cryptographic protocol analysis on real C code. In: Cousot, R. (ed.) VMCAI 2005. LNCS, vol. 3385, pp. 363–379. Springer, Heidelberg (2005) Guts, N., Fournet, C., Zappa Nardelli, F.: Reliable evidence: Auditability by typing. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 168–183. Springer, Heidelberg (2009) Krawczyk, H., Bellare, M., Canetti, R.: HMAC: Keyed-hashing for message authentication. RFC 2104 (1997) Lowe, G.: Breaking and fixing the Needham-Schroeder public-key protocol using FDR. In: Margaria, T., Steffen, B. (eds.) TACAS 1996. LNCS, vol. 1055, pp. 147–166. Springer, Heidelberg (1996) Milner, R.: Communicating and Mobile Systems: the π -Calculus. Cambridge University Press, Cambridge (1999) Needham, R., Schroeder, M.: Using encryption for authentication in large networks of computers. Commun. ACM 21(12), 993–999 (1978) Pironti, A., J¨urjens, J.: Formally-based black-box monitoring of security protocols. In: Massacci, F., Wallach, D., Zannone, N. (eds.) ESSoS 2010. LNCS, vol. 5965, pp. 79–95. Springer, Heidelberg (2010) Swamy, N., Chen, J., Chugh, R.: Enforcing stateful authorization and information flow policies in fine. In: Gordon, A.D. (ed.) ESOP 2010. LNCS, vol. 6012, pp. 529–549. Springer, Heidelberg (2010) Swamy, N., Chen, J., Fournet, C., Strub, P.-Y., Bhargavan, K., Yang, J.: Secure distributed programming with value-dependent types. In: International Conference on Functional Programming, ICFP 2011 (to appear, 2011) Vaudenay, S.: Security flaws induced by CBC padding - applications to SSL, IPSEC, WTLS... In: Knudsen, L.R. (ed.) EUROCRYPT 2002. LNCS, vol. 2332, pp. 534– 545. Springer, Heidelberg (2002)

Model-Driven Development of Security-Aware GUIs for Data-Centric Applications David Basin1 , Manuel Clavel2,3 , Marina Egea2, Miguel A. Garc´ıa de Dios2 , Carolina Dania2 , Gonzalo Ortiz2 , and Javier Valdazo2 1

ETH Z¨ urich, Switzerland [email protected] 2 IMDEA Software Institute, Madrid, Spain {manuel.clavel,marina.egea,miguelangel.garcia}@imdea.org, {carolina.dania,gonzalo.ortiz,javier.valdazo}@imdea.org 3 Universidad Complutense, Madrid, Spain [email protected]

Abstract. In this tutorial we survey a very promising instance of modeldriven security: the full generation of security-aware graphical user interfaces (GUIs) from models for data-centric applications with access control policies. We describe the modeling concepts and languages employed and how model transformation can be used to automatically lift security policies from data models to GUI models. We work through a case study where we generate a security-aware GUI for a chatroom application. We also present a toolkit that supports the construction of security, data, and GUI models and generates complete, deployable, web applications from these models.

1

Introduction

Model building is at the heart of system design. This is true in many engineering disciplines and is increasingly the case in software engineering. Model-driven engineering (MDE) [7] is a software development methodology that focuses on creating models of diﬀerent system views from which system artifacts such as code and conﬁguration data are automatically generated. Proponents of model-driven engineering have in the past been guilty of making overambitious claims: positioning it as the Holy Grail of software engineering where modeling completely replaces programming. This vision is, of course, unrealizable in its entirety for simple complexity-theoretic reasons. If the modeling languages are suﬃciently expressive then basic problems such as the consistency of the diﬀerent models or views of a system become undecidable. However, there are specialized domains where MDE can truly deliver its full potential: in our opinion, security-aware GUIs for data-centric applications is one of them. Data-centric applications are applications that manage information, typically stored in a database. In many cases, users access this information through graphical user interfaces (GUIs). Informally, a GUI consists of widgets (e.g., windows, text-ﬁelds, lists, and combo-boxes), which are visual elements that display and A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 101–124, 2011. Springer-Verlag Berlin Heidelberg 2011

102

D. Basin et al.

store information and support events (like “clicking-on” or “typing-in”). A GUI deﬁnes the layout for the widgets, as well as the actions that the widgets’ events trigger either on the application’s database (e.g., to create, delete, or update information) or upon other widgets (e.g., to open or close a window). There is an important, but little explored, link between visualization and security: When the application data is protected by an access control policy, the application GUI should be aware of and respect this policy. For example, the GUI should not display options to users for actions (e.g., to read or update information) that they are not authorized to execute on application data. This, of course, prevents the users from getting (often cryptic) security warnings or error messages directly from the database management system. It also prevents user frustration, for example from ﬁlling out a long electronic form only to have the server reject it because the user lacks a permission to execute some associated action on the application data. However, manual encoding the application’s security policy within the GUI code is cumbersome and error prone. Moreover, the resulting code is diﬃcult to maintain, since any changes in the security policy will require manual changes to the GUI code. In this tutorial we spell out our model-driven engineering approach for developing security-aware GUIs for data-centric applications. The backbone of this approach, illustrated in Figure 1, is a model transformation that automatically lifts the access control policy modeled at the level of the data to the level of the GUI [2]. More precisely, given a security model (specifying the access control policy on the application data) and a GUI model (specifying the actions triggered by the events supported by the GUI’s widgets), our model transformation generates a GUI model that is security-aware. The key idea underlying this transformation is that the link between visualization and security is ultimately deﬁned in terms of data actions, since data actions are both controlled by the security policy and triggered by the events supported by the GUI. Thus, under our approach, the process of modeling and generating security-aware GUI has the following parts: 1. 2. 3. 4.

Software engineers specify the application-data model. Security engineers specify the security-design model. GUI designers specify the application GUI model. A model transformation automatically generates a security-aware GUI model from the security model and the GUI model. 5. A code generator automatically produces a security-aware GUI from the security-aware model. The other key components of this approach are the languages that we propose for modeling the data (ComponentUML), the access control policy (SecureUML), and the GUI (ActionGUI). These languages are deﬁned by their corresponding metamodels and support the rigorous modeling of a large class of data models, security models, and GUI models. For data models, the main modeling elements are entities, along with their attributes and associations; for security models,

Model-Driven Development of Security-Aware GUIs

Software Engineer

Security Engineer

103

Data Model

Security Model

GUI Model

GUI Engineer

Model Transformation

Security-aware GUI Model

Code Generation

Security-aware GUI Application

Fig. 1. Model-driven development of security-aware GUIs

these elements are roles, permissions (possibly constrained at runtime to satisfy given properties), and the actions associated to these permissions. For GUI models, these elements are widgets, the (possibly conditional) events associated to these widgets, and the (possibly conditional) actions associated to these events. The constraint language OCL [10] is used in all of these models. For security models, OCL is used to formalize the constraints on the permissions. For GUI models, it is used to formalize the conditions on the actions, as well as to specify the information to be displayed in widgets, updated in the database, or passed from one widget to another. To support a full model-driven engineering development process, we have built a toolkit, named the ActionGUI Toolkit. This features specialized model editors for data, security, and GUI models, and implements the aforementioned model transformation to automatically generate security-aware GUI models. Moreover, our toolkit includes a code generator that, given a security-aware GUI model, automatically produces a complete web application, ready to be deployed in web containers such as Tomcat or GlassFish. A key component of this code generator is our translator from OCL to an SQL-based query language [5], which handles the OCL expressions appearing in the security-aware GUI models. More information about our ActionGUI Toolkit can be found at the URL http:// www.bm1software.com/actiongui.html.

104

D. Basin et al.

Overall, we see the full generation of security-aware GUIs from models for data-centric applications as a very promising application for model-driven engineering. By working with models and using code-generators to produce the ﬁnal products, GUI designers can focus on the GUI’s layout and behavior, instead of wrestling with the diﬀerent, often complex, technologies that are used to implement them. Moreover, by using model transformations, the problem of establishing the link between visualization and security is successfully addressed. To appreciate this last point, consider the standard alternative: the default, “ad-hoc” solution of directly hard-coding the security policy within the GUI. This is clearly disadvantageous. First, the GUI designer is often unaware of the application data security policy. Second, even if the designer is aware of it, manual hard-coding the application data security policy within the GUI code is cumbersome and error-prone. Finally, any changes in the security policy will require manual changes to the GUI code that implements this policy, which again is cumbersome and error-prone. Organization We explain in this tutorial our approach for developing security-aware GUIs for data-centric applications and present a toolkit, named ActionGUI, supporting this approach. We begin in Sections 2–4 by introducing our modeling languages for data models (ComponentUML), security models (SecureUML), and GUI models (ActionGUI). We also introduce our running example: a basic chatroom application. In Section 5, we discuss the problem of lifting the security requirements from data models to GUI models, and we present our solution: a model transformation that automatically transforms a GUI model into a security-aware GUI model with respect to the security requirements imposed on the underlying data model. We conclude this tutorial with a discussion on current and future work. All of the models we present here (and many more) are available at the ActionGUI home page. The interested reader can also evaluate there the code generated by the ActionGUI Toolkit from these models.

2

Data Models: ComponentUML

In this and the next two sections, we introduce the modeling languages that we use for the model-driven development of security-aware GUIs for data centric applications. These languages are: ComponentUML, for modeling data; SecureUML, for modeling the access control policy; and ActionGUI, for modeling the application’s GUI. To illustrate the main modeling concepts and relationships provided by these languages, we work through a running example: a simple chat application named ChitChat, which supports multiple chat rooms where users can converse with each other in diﬀerent chat rooms. We begin

Model-Driven Development of Security-Aware GUIs

105

then with ComponentUML, which is the language that we use for modeling the application data. ComponentUML also gives us a context to introduce the constraint language OCL [10], which we intensively use in our approach when modeling both access control policies and the application’s GUIs. Data models provide a data-oriented view of a system. Typically they are used to specify how data is structured, the format of data items, and their logical organization, i.e., how data items are grouped and related. ComponentUML essentially provides a subset of UML class models where entities (classes) can be related by associations and may have attributes and methods. Attributes and association-ends have types (either primitive types or entity types). As expected, the type associated to an attribute is the type of the attribute’s values, and the type associated to an association-end is the type of the objects which may be linked at this end of the association. The ComponentUML metamodel is shown in Figure 2.

Fig. 2. The ComponentUML metamodel

The ChitChat data model. In our ChitChat application, each user has a nickname, a password, an e-mail address, a mood message, and a status. The user may participate and be invited to participate in any number of chat rooms. Each chat room is created by a user, has a name, a starting and ending date, and it manages the messages sent by its participants. The table shown in Figure 3 speciﬁes the data model of the ChitChat application, using the concepts and relationships provided by ComponentUML. Each row in this table corresponds to an entity, and shows its attributes and association-ends, with their corresponding types. Notice that we also indicate, for each association-end, the association-end which corresponds to the other end of the association. The reader can ﬁnd on the ActionGUI home page the graphical representation of the ChitChat data model using the concrete syntax for ComponentUML that is supported by the ActionGUI Toolkit.

106

D. Basin et al. Entity

ChatUser

ChatRoom

body ChatMessage

AssocEnd msgSent participates owns invitedTo

Type ChatMessage ChatRoom ChatRoom ChatRoom

Other end from participants ownedBy invitees

messages participants ownedBy invitees String from chat

ChatMessage ChatUser ChatUser ChatUser ChatUser ChatRoom

chat participates owns invitedTo msgSent messages

Attribute Type nickname String password String email String moodMsg String status String name String start Date end Date

Fig. 3. The ChitChat data specification

OCL: constraints and queries The Object Constraint Language (OCL) [10] is a speciﬁcation language for expressing constraints and queries using a textual notation. As part of the UML standard, it was originally intended for modeling properties that could not be easily or naturally captured using graphical notation (e.g., class invariants in a UML class diagram). In fact, OCL expressions are always written in the context of a model, and they are evaluated on an instance of this model. This evaluation returns a value but does not change anything; OCL is a side-eﬀect free language. We summarize here the main elements of the OCL language which are used in this tutorial. OCL is a strongly type language. Expressions either have a primitive type (namely, Boolean, Integer, Real, and String), a class type, or a collection type, whose base type is either a primitive type or a class type. OCL provides the standard operators on primitive types and on collections. For example, the operator includes checks whether an object is part of a collection, and the operator isEmpty checks whether a collection is empty. More interestingly, OCL provides a dot-operator to access the values of the objects’ attributes and association-ends. For example, let u be an object of the class ChatUser. Then, the expression u.nickname refers to the value of the attribute nickname for the ChatUser u, and the expression u.participates refers to the objects linked to the ChatUser u through the association-end participates. Furthermore, OCL provides the operator allInstances to access to all the objects of a class. For example, the expression ChatRoom.allInstances() refers to all the objects of the class ChatRoom. Finally, OCL provides operators to iterate on collections. These are forAll, exists, select, reject, and collect. For example, ChatUser.allInstances()−>select(u|u.status=’on−line’) refers to the collection of objects of the class ChatUsers whose attribute status has the value “on-line”. ChitChat’s entity invariants. To illustrate the syntax (and the semantics) of the OCL language, we formalize here some entity (class) invariants for ChitChat’s

Model-Driven Development of Security-Aware GUIs

107

data model. For example, the following OCL expression formalizes that users’ nicknames must be unique: ChatUser.allInstances()−>forall(u1,u2| u1 u2 implies u1.nickname u2.nickname).

Similarly, we can formalize that the status of a ChitChat user is either “oﬀline” or “on-line” using the following OCL expression: ChatUser.allInstances() −>forall(u|u.status=’oﬀ−line’ or u.status=’on−line’).

Finally, we can formalize that each message has exactly one sender: ChatMessage.allInstances()−>forAll(m|m.from−>size()= 1).

3

Security Models: SecureUML+ComponentUML

SecureUML [3] is a modeling language for formalizing access control requirements that is based on RBAC [6]. In RBAC, permissions specify which roles are allowed to perform given operations. These roles typically represent job functions within an organization. Users are granted permissions by being assigned to the appropriate roles based on their competencies and responsibilities in the organization. RBAC additionally allows one to organize the roles in a hierarchy where roles can inherit permissions along the hierarchy. In this way, the security policy can be described in terms of the hierarchical structure of an organization. However, it is not possible in RBAC to specify policies that depend on dynamic properties of the system state, for example, to allow an operation only during weekdays. SecureUML extends RBAC with authorization constraints to overcome this limitation. SecureUML provides a language for specifying access control policies for actions on protected resources. However, it leaves open what the protected resources are and which actions they oﬀer to actors. These are speciﬁed in a so-called “dialect”. Figure 4 shows the SecureUML metamodel. Essentially, it provides a language for modeling roles (with their hierarchies), permissions, actions, resources, and authorization constraints, along with their assignments, i.e., which permissions are assigned to a role, which actions are allowed by a permission, which resource is aﬀected by the actions allowed by a permission, which constraints need to be satisﬁed for granting a permission, and, ﬁnally, which resource is aﬀected by an action. In our approach, we use a speciﬁc dialect of SecureUML, named SecureUML+ComponentUML, for modeling the access control policy on data models. The SecureUML+ComponentUML metamodel provides the connection between SecureUML and ComponentUML. Essentially, in this dialect of SecureUML, the protected resources are the entities, along with their attributes, methods, and association-ends (but not the associations as such), and the actions that they offer to the actors are those shown in Figure 5. Essentially, there are two classes of actions: atomic and composite. The atomic actions are intended to map directly

108

D. Basin et al.

Fig. 4. The SecureUML metamodel Resource Entity Attribute Method AssociationEnd

Actions create, read, update, delete, full access read, update, full access execute read, create, delete, full access

Fig. 5. The SecureUML+ComponentUML actions on protected resources

onto actual operations on the database. These actions are: create and delete for entities; read and update for attributes; create and delete for association-ends; and execute for methods. The underlined actions are the composite actions, which hierarchically group lower-level actions. Composite actions allow modelers to conveniently specify permissions for sets of actions. For example, the full access action for an attribute groups together the read and update actions for this attribute. Finally, in SecureUML+ComponentUML, authorization constraints are speciﬁed using OCL, extended by four keywords, self, caller, value, and target. These keywords have the following meanings: – self refers to the root resource upon which the action will be performed, if the permission is granted. The root resource of an attribute, an association-end, or a method is the entity to which it belongs. – caller refers to the actor that will perform the action, if the permission is granted. – value refers to the value that will be used to update an attribute, if the permission is granted. – target refers to the object that will be linked at the end of an association, if the permission is granted. The ChitChat access control policy. For the sake of our running example, consider the following (partial) access control policy for the ChitChat application:

Model-Driven Development of Security-Aware GUIs

109

– Only administrators can create or delete users; – Administrators can read any user’s nickname, email, mood message, and status. – Any user can read and update its own nickname, password, email, mood message, and status. – Any user can read other users’ nicknames, mood messages, and status. – Users can join a chat room by invitation only, but they can leave at any time. The table shown in Figure 6 speciﬁes this (partial) access control policy, using the concepts and relationships provided by SecureUML+ComponentUML. Each row in this table corresponds to a role, and shows its permissions. Moreover, for each permission it shows the actions allowed by the permission, the resource aﬀected by each of these actions, and the constraint that must be satisﬁed for the permission to be granted. For example, the permission DisjointChat authorizes any user to leave a chat room at any anytime. More precisely, it allows any user caller to delete a participates-link between a user self and a chat room target (meaning that the user self leaves the chat room target), but only if the user caller is indeed the user self (that is, the user caller is the one leaving the chat room target), and also the chat room target indeed belongs to the collection of chat rooms linked to the user caller through the association-end participates (that is, the caller is actually participating in the chat room target). The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s access control policy using the concrete syntax for SecureUML+ComponentUML that is supported by the ActionGUI Toolkit. Role

Permission Action create, delete read Admin AnyUser read read read read, update read, update User SelfUser read, update read, update read, update read OtherUser read read JointChat create DisjointChat delete

Resource ChatUser nickname email moodMsg status nickname password email moodMsg status nickname moodMsg status participates

Authorization Constraint

true

caller=self

true

self=caller and caller.invitedTo−>includes(target) participates self=caller and caller.participates−>includes(target)

Fig. 6. The ChitChat access control policy (partial)

110

4

D. Basin et al.

GUI Models: ActionGUI

ActionGUI is a modeling language for formalizing the GUIs of a rich class of data-centric applications. The ActionGUI metamodel is shown in Figure 7. In a nutshell, ActionGUI provides a language to model widgets (e.g., windows, textﬁelds, buttons, lists, and tables), events (e.g., clicking-on, typing-in), and actions, which can be on data (e.g., to update a property of an element in the database) or on other widgets, (e.g., to open a window), as well as the associations that link the widgets with the events that they support and the events with the actions that they trigger. In addition, ActionGUI provides support to formally model the following features: – Widgets can be displayed in containers, which are also widgets (e.g., a window can contain other widgets). – Widgets may own variables, which are in charge of storing information for later use. – Events may be only supported upon the satisfaction of speciﬁc conditions, whose truth value can depend on the information stored in the widgets’ variables or in the database. – Actions may be only triggered upon the satisfaction of speciﬁc conditions, whose truth value can depend on the information stored in the widgets’ variables or in the database. – Actions may take their arguments (values that instantiate their parameters) from the information stored in the widgets’ variables or in the database. The ActionGUI metamodel’s invariants specify: (i) for each type of widget, the “default” variables that widgets of this type always own; (ii) for each type of widget, the type of events that widgets of this type may support; and (iii) for each type of action, the arguments that actions of this type require, as well as the arguments (if any) that these actions may additionally take. In particular, the invariants of ActionGUI’s metamodel formalize, among others, the following constraints about the diﬀerent types of widgets: – Windows. They can contain any type of widget, except windows. Windows are not contained in any widget. – Text-field. They can be typed-in. By default, each text-ﬁeld owns a variable text of type string, which stores the last string typed-in by the user. The value of the variable text is permanently displayed in the text-ﬁeld. – Button. They can be clicked-on. – List. They contain exactly one text-ﬁeld. By default, each list owns a variable rows of type collection. A list displays as many rows as elements are in the collection stored by the variable rows, each row containing exactly one instance of the text-ﬁeld contained by the list. By default, each instance of this text-ﬁeld owns a variable row whose value is the element associated to this row from those stored by the variable rows. Finally, by default, each list owns a variable selected that holds the element associated to the last row selected by the user.

Model-Driven Development of Security-Aware GUIs

111

Fig. 7. The ActionGUI metamodel

– Combo-box. They are similar to lists, except that rows are displayed in a drop-down box. – Table. They are similar to lists, except that they can contain any number of text-ﬁelds, buttons, lists, or even tables. Also, the invariants of ActionGUI’s metamodel formalize, among others, the following invariants about the diﬀerent types of actions: – Create. It creates a data item in the database. It takes two arguments: the type of the new data item (type) and the variable that will store this element for later reference (variable). – Delete (entities). It deletes a data item in the database. It takes as argument the element to be deleted (object). – Read. It reads the value of a data item’s attribute in the database. It takes three arguments: the data item whose property is to be read (object); the property to be read (attribute); and the variable that will store, for later reference, the value read (variable). – Update. It modiﬁes the value of a data item’s attribute in the database. It takes three arguments: the data item whose attribute is to be modiﬁed (object); the attribute to be modiﬁed (attribute); and the new value (value). – Create (association-ends). It creates a new link in the database between two data items. It takes three arguments: the source data item (sourceObject); the target data item (targetObject); and the association-end (associationEnd) through which the target data item will be linked to the source data item.

112

D. Basin et al.

– Delete (association-ends). It deletes a link in the database between two data items. It takes three arguments: the source data item (sourceObject); the target data item (targetObject); and the association-end (associationEnd) from where the target data item will be removed. – Open. It opens a window. It takes as argument the window to be opened (target); additionally, for any of this window’s variables, it can take as argument a value to be assigned to this variable when opening the window. – Back. It goes back to the window from which a window was open. – Set. It assigns a new value to a widget’s variable. It takes two arguments: the variable (target) and the value to be assigned to this variable (value). Finally, actions’ conditions and arguments are speciﬁed in ActionGUI models using OCL, extended with the widget’s variables (always enclosed in square brackets). As expected, when evaluating an OCL expression that contains a widget’s variable, the value of the corresponding subexpression is the value currently stored in the variable. In case of ambiguity, a widget’s variable is denoted by its name, preﬁxed by the name of its widget (followed by a dot). Also, in case of ambiguity, the name of a widget is preﬁxed by the name of its container (followed by a dot). Notice that, within the same containers, widgets have unique names. Moreover, a widget’s variable can only be used within the window that contains its widget, either directly or indirectly. The ChitChat login window. To continue with our running example, consider the following interface for allowing a registered user to login into the ChitChat application: a window (loginWi) containing: – a writable text-ﬁeld (nicknameEn), for the user to type its nickname in; – a writable text-ﬁeld (passwordEn), for the user to type its password in; and – a clickable button (loginBu), for the user to login, using as its nickname and password the strings that it typed in the text-ﬁelds nicknameEn and passwordEn, respectively. Upon successful authentication, the user will be directed to the application’s main menu window (menuWi) as the logged-in user (caller). The table shown in Figure 8 speciﬁes this login window, using the concepts and relationships provided by ActionGUI. Each row in this table correspond to a widget, where the containment relationship is denoted by displaying the widgets using tree-like notation. For each widget, we show the variables owned by the widget and the events that it supports. Moreover, for each event, we show the actions triggered by this event, as well as its arguments, indicating the values for each of the actions’ parameters. However, we neither show in this table the “default” widget’s variables nor the events supported by the widgets when they do not trigger any action. Notice also that there are two elements that we have intentionally left undeﬁned in this table: the value to be assigned to the menuWi’s variable caller when opening the window menuWi, and the condition for this action. We now describe how both elements can be deﬁned using ActionGUI’s extension of OCL.

Model-Driven Development of Security-Aware GUIs

113

Widgets Variables Events Actions Arguments loginWi |nicknameEn |passwordEn |loginBu click-on open() window = menuWi | menuWi.caller = | authenticated user () Upon successful authentication. Fig. 8. The ChitChat login window

– The authenticated user should be the registered user in the database whose nickname and password coincide with the values of the text-variables owned, respectively, by text-ﬁelds nicknameEn and passwordEn. Using our extended OCL, we can deﬁne the authenticated user as follows: ChatUser.allInstances()−>any(u| u.nickname= [nicknameEn.text] and u.password=[passwordEn.text])

Notice that, as one of the invariants of the ChitChat data model, we speciﬁed that nicknames shall be unique. Thus, although the any-iterator will return any registered user satisfying the body of the any-iterator, there will be at most one such registered users. – The condition for opening the window menuWi should be the existence in the database of a registered user whose nickname and password coincide with the values of the text-variables owned, respectively, by text-ﬁelds nicknameEn and passwordEn. We can deﬁne this condition as follows: ChatUser.allInstances()−>exists(u| u.nickname= [nicknameEn.text] and u.password=[passwordEn.text])

The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s login window using the concrete syntax for ActionGUI that is supported by the ActionGUI Toolkit. The ChitChat menu window. Consider now the following interface for allowing a logged-in user to choose an option from ChitChat’s main menu: a window (menuWi), owning a variable caller which stores the logged-in user, and containing: – a selectable list (usersLi) with as many rows as registered users are online, each of these rows containing an unwritable text-ﬁeld (nicknameLa) showing the nickname of the registered user associated to this row; – a clickable button (editProﬁleBu ) for the caller to access the interface for editing the proﬁle (i.e., name, password, email, mood message, and status) of the user selected in the list usersLi;

114

D. Basin et al.

– a clickable button (createChatBu) for the caller to access the interface for creating a new chat room; and – a clickable button (closeChatBu) for the caller to close the window. The table shown in Figure 9 speciﬁes this menu window, using the concepts and relationships provided by ActionGUI. Notice that the collection of data items to be displayed in the list usersLi, namely, the online users, is not formally deﬁned in this table. Using ActionGUI’s extension of OCL, we can deﬁne this collection as follows: ChatUser.allInstances()−>select(u|u.status= ’on−line’)

The reader can ﬁnd in the ActionGUI home page the graphical representation of ChitChat’s menu window using the concrete syntax for ActionGUI that is supported by the ActionGUI Toolkit. Widgets Variables Events Actions menuWi caller | usersLi on-create set | | | nicknameLa on-create read || || | editProﬁleBu click-on open | | | createChatBu click-on open | closeChatBu click-on close

Arguments target = rows value = on-line users object = [UsersLi.row] attribute = nickname variable = text window = editProﬁleWi editProﬁleWi.selectedUser = [UsersLi.selected] window = createChatWi

Fig. 9. The ChitChat menu window

5

Security-Aware ActionGUI Models

In this section, we propose our solution to what we believe is the key challenge when modeling security-aware GUIs for data-centric applications: Which method should the GUI designer use for establishing the link between visualization and security? In other words, how should the GUI designer model their GUIs so as to make them aware of and respect the access control policy that protects the application data? As mentioned before, establishing this link is crucial when modeling security-aware GUIs for data-centric applications. Basically, a GUI should not display options to users for actions (e.g., to read or update information) that they are not authorized to execute on application data. This prevents the users from getting security warnings or cryptic error messages directly from the database management system. It also prevents user frustration from ﬁlling out a long electronic form only to have the server reject it because the user lacks a permission to execute some associated action on the application data.

Model-Driven Development of Security-Aware GUIs

115

As we will motivate in this section, manually modeling (and even worse, manually encoding) the application’s security policy within the GUI model is cumbersome, error prone, and scales poorly to large applications. Moreover, the resulting models are diﬃcult to maintain, since any changes in the security policy will require manual changes to the GUI models. Our solution to this problem uses a standard technique in model-driven development: model transformation. In particular, we introduce a model transformation that automatically transforms a GUI model into a security-aware GUI model with respect to the access control policy imposed on the underlying data model. One additional and substantial advantage of our solution is that, by keeping the security models and the GUI model apart, the security engineers and the GUI designers can independently model what they know best and maintain their models independently. The ChitChat edit-profile window. To motivate the problem faced by a GUI designer when modeling a security-aware GUI, let us continue with our running example. Consider the interface for allowing a logged-in user (caller) to edit the proﬁle of a previously chosen user (selectedUser), which may of course be the caller itself. More speciﬁcally, this interface shall consist of a window such that: 1. The current values of the selectedUser’s proﬁle are displayed when opening the window. 2. The caller can type in the new values (if any) for the selectedUser’s proﬁle. 3. The selectedUser’s proﬁle is updated with the new values typed in by the caller when he or she clicks on a designated button. Recall that a registered user’s proﬁle is composed of the following attributes: nickname, password, mood message, email, and status. Recall also that the access control policy for reading and updating users’ proﬁles, as speciﬁed in Figure 6, is the following: 4. A user is always allowed to read and update its own nickname, password, mood message, email, and status. 5. A user is allowed to read another user’s nickname, mood message, and status, but not the user’s password or email. 6. An administrator is always allowed to read a user’s nickname, mood message, status, and email, but not the user’s password. Now, if the GUI designer only takes into consideration the functional requirements (1–3), the ChitChat edit-proﬁle window can be modeled as shown in Figure 10. Namely, a window editProﬁleWi, which owns the variable selectedUser and caller, and contains: – A writable text-ﬁeld nicknameEn, for the caller to type in the new value (if any) with which to update the selectedUser’s nickname. Notice that when the text-ﬁeld nicknameEn is created, its “default” variable text will be assigned the current value of the selectedUser’s nickname, and therefore this value will be the string initially displayed in the text-ﬁeld nicknameEn, as requested.

116

D. Basin et al. Widgets Variables editProﬁleWi caller, selectedUser | nicknameEn | | | passwordEn | | | moodMsgEn | | | emailEn | | | statusEn | | | updateBu | |

Events Actions Arguments

on-create read

on-create read

on-create read

on-create read

on-create read

click-on

update

update | | update | | update | | update | |

object = [selectedUser] attribute = nickname variable = text object = [selectedUser] attribute = password variable = text object = [selectedUser] attribute = moodMsg variable = text object = [selectedUser] attribute = email variable = text object = [selectedUser] attribute = status variable = text object = [selectedUser] attribute = nickname value = [nicknameEn.text] object = [selectedUser] attribute = password value = [passwordEn.text] object = [selectedUser] attribute = moodMsg value = [moodMsgEn.text] object = [selectedUser] attribute = email value = [emailEn.text] object = [selectedUser] attribute = status value = [statusEn.text]

Fig. 10. The ChitChat edit-profile window (although security-unaware)

– Analogous writable text-ﬁelds for each of the other elements in a registered user’s proﬁle: password, mood message, email, and status. – A clickable button updateBu for the caller to trigger the sequence of actions that will update, as requested, the selectedUser’s nickname, password, mood message, and status, with the new values (if any) typed by the caller in the corresponding text-ﬁelds. Obviously, the edit-proﬁle window modeled in Figure 10 does not satisfy the security requirements (4–6). Any caller can read and update any value contained in the proﬁle of any selectedUser! So how can the GUI designer model the editproﬁle window to make it aware of and respect the security requirements (4–6)?

Model-Driven Development of Security-Aware GUIs

117

There are essentially two solutions available. Unfortunately both of them, if applied manually, are cumbersome and error prone and therefore impractical for large applications with complex security policies. To understand the challenge that faces a GUI designer when modeling security-aware GUIs, let us ﬁrst highlight the knowledge that he or she must acquire to accomplish this task. We decompose this into two steps. Step 1: For each action triggered by an event, and for each role considered by the access control policy, the GUI designer must determine (i) the conditions under which the given action can be securely executed by a user with the given role. This depends, of course, on the underlying access control policy, and, more speciﬁcally, on the authorization constraints assigned to the permissions which grant access to execute the given action for the given role. Also, recall that permissions are inherited along the role hierarchy. Notice however that, to obtain (i), the GUI designer can not simply compose, using disjunctions, the authorization constraints assigned to the aforementioned permissions. In particular, the variable caller, if it appears in any of these authorization constraints, must be replaced by the (widget) variable that stores the current application’s user. Furthermore, the variables self, value, and target, if they appear in any of the aforementioned authorization constraints, must also be replaced by the appropriate expressions, based on the arguments taken by the given action. For example, in the case of a readaction, the variable self should be replaced by the value of the parameter object for this action. Step 2: For each event supported by a widget, and for each role considered by the access control policy, the GUI designer must determine (ii) the conditions under which all the actions triggered by the given event can be securely executed by a user with the given role. In this case, for each given role, the GUI designer can simply compose, using conjunctions, the results to determine (i) for every action triggered by the given event. Let us now illustrate, using our running example, the two solutions that are currently available to the GUI modeler for turning a security-unaware GUI model into a security-aware one. To simplify the discussion, we assume from now on that the user currently logged-in is always stored in a (widget) variable caller, that this variable is owned by every window, and that this variable’s value is automatically passed from one window to another window when opening the latter from the former. Similarly, we assume the role of the user currently logged-in (if he or she has several roles, then the “active” role) is always stored in a (widget) variable role, that this variable is owned by every window, and that this variable’s value is automatically passed from one window to another window when opening the latter from the former. Solution A. The ﬁrst solution for the GUI designer consists in modeling as many diﬀerent edit-proﬁle windows as possible security scenarios. In our case, the

118

D. Basin et al.

GUI designer must model three diﬀerent edit-proﬁle windows (editMyProﬁleWi, editOthersProﬁle, and editUsersProﬁle), one for each of the following scenarios: 1. When the caller has the role ‘User’ and coincides with the selectedUser. 2. When the caller has the role ‘User’ but does not coincide with the selectedUser. 3. When the caller has the role ‘Admin’. In particular, the window editMyProﬁleWi associated to the security scenario (1) will contain exactly the same widgets as the window editProﬁleWi in Figure 10. In contrast, the window editOthersProﬁleWi associated to the security scenario (2), will only contain the text-ﬁelds nicknameEn, moodMsgEn , and statusEn (and not the button updateBu), since a user with the role ‘User’ can only read (but not update) other user’s nickname, mood message, and status. Similarly, the window editProﬁlesWi will only contain the text-ﬁelds nicknameEn, moodMsgEn , emailEn, and statusEn (and not the button updateBu), since a user with the role ‘Admin’ can read (but not update) any user’s proﬁle, except its password. Furthermore, for this solution to work, every event intended to give access to the interface for editing users’ proﬁles must also be aware of the security scenarios (1)–(3), in order to open the appropriate edit-proﬁle window. In particular, the GUI designer must associate the sequence of conditional actions shown in Figure 11 to each of the aforementioned events. Notice that for each open-action, the imposed condition formalizes the conditions under which all the actions triggered by all the events supported by the widgets which are contained in the window being opened can be securely executed by a user. That is, the GUI designer must gather all the knowledge corresponding to (ii), in Step 2 above, for every event supported by any widget contained in the window which the action under consideration is about to open. Actions Arguments if [role] = ’User’ and [caller] = [selected user] then open window = editMyProﬁleWi editMyProﬁleWi.selectedUser = [selected user] if [role] = ’User’ and [caller] [selected user] then open window = editOthersProﬁleWi editOthersProﬁleWi.selectedUser = [selectedUser] if [role] = ’Admin’ then open window = editUsersProﬁleWi editUsersProﬁleWi.selectedUser = [selectedUser] Fig. 11. Solution A: Conditions for opening the edit-profile windows

Solution B. Another solution for the GUI designer consists of specifying, for each of event supported by a widget, the conditions under which all the actions triggered by the given event can be securely executed by a user. In our case, the GUI design must convert the edit-proﬁle window model shown in Figure 10 into the one shown in Figure 12.

Model-Driven Development of Security-Aware GUIs

119

Widgets Events Actions Arguments nicknameEn on-create if true then read object = [selectedUser] attribute = nickname variable = text passwordEn on-create if [role] = ’User’ and [caller] = [selectedUser] then read object = [selectedUser] attribute = password variable = text moodMsgEn on-create if true then read object = [selectedUser] attribute = moodMsg variable = text emailEn on-create if (([role] = ’User’ and [caller] = [selectedUser]) or [role] = ’Admin’) then read object = [selectedUser] attribute = email variable = text statusEn on-create if true then read object = [selectedUser] attribute = status variable = text updateBu click-on if [role] = ’User’ and [caller] = [selectedUser] then update object = [selectedUser] attribute = nickname value = [nicknameEn.text] update object = [selectedUser] attribute = password value = [passwordEn.text] update object = [selectedUser] attribute = moodMsg value = [moodMsgEn.text] update object = [selectedUser] attribute = email value = [emailEn.text] update object = [selectedUser] attribute = status value = [statusEn.text] Fig. 12. Solution B: The ChitChat edit-profile window (now security-aware)

This solution is certainly simpler and less intrusive than Solution A with respect to the original, security-unaware GUI model (since, for example, no new windows need to be added to the original design). However, in order to implement this solution, the GUI designer must gather all the knowledge corresponding to (ii), in Step 2 above, for every event supported by every widget in the model.

120

D. Basin et al.

A model-transformation approach. Clearly, manually modeling the application’s security policy within the GUI model is problematic. It is cumbersome, error prone, and scales poorly to large applications. Moreover, it requires the GUI designer to have complete knowledge of the access control policy on the application data. Finally, the resulting GUI models are diﬃcult to maintain. To address the problem of establishing the link between visualization and security, we employ a standard technique from model-driven development: model transformation. A model transformation takes as input a model (or several models) conforming to given metamodel (respectively, several metamodels) and produces as output a model conforming to a given metamodel. More speciﬁcally, the ActionGUI Toolkit implements a model transformation that takes as input an ActionGUI model and a SecureUML+ComponentUML model, and automatically produces as output an ActionGUI model. This output model is identical to the input ActionGUI model except that it is now security-aware with respect to the access control policy speciﬁed in the input SecureUML+ComponentUML model. In particular, our model transformation follows the ideas behind Solution B: it speciﬁes for each event supported by a widget, the conditions under which all the actions triggered by this event can be securely executed by a user. As expected, at the core of this model transformation, we have implemented a function (on the input models) that automatically gathers all the knowledge corresponding to (i) and (ii), in Step 1 and Step 2 above, for each action triggered by an event and for each event supported by a widget, respectively. Continuing with our running example, let us consider how models can be made security-aware and maintained over time. First, how can the GUI designer model the edit-proﬁle window to make it aware of and respect the security requirements (4–6)? Using our ActionGUI Toolkit, the designer simply calls the aforementioned model transformation on ChitChat’s access control policy shown in Figure 6 and the (security-unaware) ChitChat edit-proﬁle window model shown in Figure 10. Our model transformation then automatically generates (in practically no time) the security-aware ChitChat edit-proﬁle window model shown in Figure 12. Finally, what must the GUI designer do, with respect to the edit-proﬁle window model, if ChitChat’s security policy happens to change? For example, suppose that any user is allowed to read the email of other users when the former participates in a chat room where the latter is also participating. The designer must simply call again the model transformation, this time taking as inputs the modiﬁed model of ChitChat’s access control policy and, as before, ChitChat’s (security-unaware) edit-proﬁle window model. shown in Figure 10.

6

Related Work and Conclusions

The ever-growing development and use of information and communication technology is a constant source of security and reliability problems. Clearly we need better ways of developing software systems and approaching software engineering as a well-founded discipline. In many engineering disciplines, model building is at the heart of system design. This is increasingly the case in software engineering

Model-Driven Development of Security-Aware GUIs

121

since model-driven software engineering was ﬁrst proposed over a decade ago. In our opinion, the late adoption of this methodology is due to the diﬃculty in deﬁning eﬀective domain-speciﬁc modeling languages and also the eﬀort required for developers to learn modeling languages and the art of model building. Deﬁning a good domain-speciﬁc modeling language requires ﬁnding the right abstractions and degree of precision to capture relevant aspects of the structure and the logic of a software system. In addition, for a modeling language to be really usable and useful for software developers, appropriate tools must be provided to build models, analyze them, and keep them synchronized with end products. We wish to emphasize in this regard the need to focus on concrete domains like access controls, GUIs, data models, and the like. In our experience, only by limiting the domain is it possible to build suﬃciently precise modeling languages that support the automatic generation of fully functional applications. Automatic code generation brings with it important advantages. Once a code generator is implemented for a platform (a one-oﬀ cost), modelers can use it like a compiler for a very high-level language, dramatically increasing their productivity. But even when the model-driven development process is not completely automatic, there are experience reports (see, for instance, [9,11]), in which the productivity of a developer in industry is said to increase by a factor of 2 or 3. There is an additional argument for using model-driven development to develop security-critical systems, i.e., for model-driven security [1]. Namely, security is often built redundantly into systems. For example, in a web-application, access control may be enforced at all tiers: at the web server, in the back-end databases, and even in the GUI. There are good reasons for this. Redundant security controls is an example of defense in depth and is also necessary to prevent data access in unanticipated ways, for example, directly from the database thereby circumventing the web application server. Note that access control on the client is also important, but more from the usability rather than the security perspective. Namely, although client-side access control may be easy to circumvent, it enhances usability by presenting honest users an appropriate view of their options: unauthorized options can be suppressed and users can be prevented from entering states where they are unauthorized to perform any action, e.g., where their actions will result in security exceptions thrown by the application server or database. The above raises the following question: must one specify security policies separately for each of these tiers? The answer is “no” for many applications. Security can often be understood in terms of the criticality of data and an access control policy on data can be speciﬁed at the level of component (class) models, as discussed in Section 3. Afterwards, an access control policy modeled at the level of components may be lifted to other tiers. When the tiers are also modeled, this lifting can be accomplished using model transformation techniques and in a precise and meaningful way as we have illustrated in Section 5. In general, model transformations support problem decomposition during development where design aspects can be separated into diﬀerent models which are later composed. As a methodology for designing security-aware GUIs, this approach supports

122

D. Basin et al.

the consistent propagation of a security policy from component models to GUI models and, via code generation, to GUI implementations. This decomposition also means that security engineers and GUI designers can independently model what they know best and maintain their models independently. Related Work There are also other tools like WebRatio [12], Olivanova [4], and Lightswitch [8] that support development methodologies for building data-centric applications that are similar to the model-driven methodology presented in this work. In these tools, application development starts by building a data model that reﬂects the data structure required for the database. The development process continues by applying diﬀerent UI generation patterns to the data model. These patterns enable data retrieval, data editing, data creation, and database search. A detailed comparison of the expressiveness provided by the languages supported by these tools with the languages supported by the ActionGUI Toolkit is interesting; however it falls out of the scope of this paper. Nevertheless, we note that, in contrast to what these tools can provide, the ActionGUI Toolkit oﬀers developers the full ﬂexibility to create designs without burdening them with the restrictions imposed by the obligatory use of a ﬁxed number of given patterns. The above tools also impose a major restriction at the level of data management, i.e., at the level of data access and visualization: The information that can be referenced and therefore that can be accessed and visualized within one screen can only come from one table of the database or, at most, from two tables that are reachable from each other within one navigation step. The three tools, WebRatio, Olivanova, and Lightswitch, support the deﬁnition and generation of RBAC policies at diﬀerent granularity levels. Lightswitch supports granting or denying permissions (whose actual behavior must be manually programmed) to execute create, read, update, or delete actions on entities for users in diﬀerent roles. WebRatio and Olivanova also support granting or denying permissions to execute a similar set of actions on entity’s properties, individually, for users in diﬀerent roles. In WebRatio and Olivanova, the role of the authorization constraints could be played by preconditions restraining the invocation of actions through a concrete UI. Note, however, that none of these tools implement an algorithm capable of lifting to the UIs the security policy that governs the access to data. Future Work The toolkit we presented supports the construction of security, data, and GUI models and generates complete, deployable, security-aware web applications from these models. However, there is still much work ahead to turn this toolkit into a full, robust commercial application. In particular, we plan to add to the language (and to its code-generator) other actions which do not act upon the database or the GUI elements. Examples are sending an email to a contact selected in a list or printing a table.

Model-Driven Development of Security-Aware GUIs

123

Our work here, as well as our past work in model-driven security, has focused primarily on access control. However, many systems have security requirements that go beyond access control, for example, obligations on how data must or must not be used once access is granted. We are currently working on handling usage control policies in the context of model-driven security. The challenge here is to deﬁne modeling languages that are expressive enough to capture these policies, support their formal analysis, and provide a basis for generating infrastructures to enforce or, at least, monitor these policies. There are many challenging questions concerning model analysis. Here, our goal is to be able to analyze the consistency of diﬀerent system views. For example, suppose that access control is implemented at multiple tiers (or levels) of a system, e.g., at the middle tier implementing a controller for a web-based application and at the back-end persistence tier. If the policies for both of these tiers are formally modeled, we would like to answer questions like “will the controller ever enter a state in which the persistence tier throws a security exception?” Note that with advances in model transformations, perhaps such questions will some day not even need to be asked, as we can uniformly map a security policy across models of all tiers. Ultimately we see model-driven security playing an important role in the construction and certiﬁcation of critical systems. For example, certiﬁcation under the Common Criteria requires models for the higher Evaluation Assurance Levels. Model-driven security provides many of the ingredients needed: models with a well-deﬁned semantics, which can be rigorously analyzed and have a clear link to code. As the acceptance of model-driven development techniques spread, and as they become better integrated with well-established formal methods that support a detailed behavioral analysis, such applications should become a reality. Acknowledgements. This work is partially supported by the EU FP7-ICT Project “NESSoS: Network of Excellence on Engineering Secure Future Internet Software Services and Systems” (256980) by the Spanish Ministry of Science and Innovation Project “DESAFIOS-10” (TIN2009-14599-C03-01), and by Comunidad de Madrid Program “PROMETIDOS-CM” (S2009TIC-1465).

References 1. Basin, D., Clavel, M., Egea, M.: A decade of model driven security. In: Proceedings of the 16th ACM Symposium on Access Control Models and Technologies (SACMAT 2011). ACM Press, New York (2011) (invited paper, in press) 2. Basin, D., Clavel, M., Egea, M., Schl¨ apfer, M.: Automatic generation of smart, security-aware GUI models. In: Massacci, F., Wallach, D., Zannone, N. (eds.) ESSoS 2010. LNCS, vol. 5965, pp. 201–217. Springer, Heidelberg (2010) 3. Basin, D., Doser, J., Lodderstedt, T.: Model driven security: From UML models to access control infrastructures. ACM Transactions on Software Engineering and Methodology 15(1), 39–91 (2006) 4. Care Technologies. Olivanova – the programming machine (2011), http://www.care-t.com

124

D. Basin et al.

5. Egea, M., Dania, C., Clavel, M.: MySQL4OCL: A stored procedure-based MySQL code generator for OCL. Electronic Communications of the EASST 36 (2010) 6. Ferraiolo, D.F., Sandhu, R.S., Gavrila, S., Kuhn, D.R., Chandramouli, R.: Proposed NIST standard for role-based access control. ACM Transactions on Information and System Security 4(3), 224–274 (2001) 7. Kleppe, A., Bast, W., Warmer, J.B., Watson, A.: MDA Explained: The Model Driven Architecture–Practice and Promise. Addison-Wesley, Reading (2003) 8. Microsoft. Visual studio lightswitch (2010), http://www.microsoft.com/ visualstudio/en-us/lightswitch 9. Mohan, R., Kulkarni, V.: Model driven development of graphical user interfaces for enterprise business applications - experience, lessons learnt and a way forward. In: Sch¨ urr, A., Selic, B. (eds.) MODELS 2009. LNCS, vol. 5795, pp. 307–321. Springer, Heidelberg (2009) 10. Object Management Group. Object Constraint Language specification Version 2.2 (February 2010), OMG document, http://www.omg.org/spec/OCL/2.2 11. Schramm, A., Preußner, A., Heinrich, M., Vogel, L.: Rapid UI development for enterprise applications: Combining manual and model-driven techniques. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) MODELS 2010. LNCS, vol. 6394, pp. 271– 285. Springer, Heidelberg (2010) 12. Web Models Company. Web ratio – you think, you get (2010), http://www. webratio.com

On Intransitive Non-interference in Some Models of Concurrency Roberto Gorrieri and Matteo Vernali Dipartimento di Scienze dell’Informazione, Universit` a di Bologna, Mura A. Zamboni, 7, 40127 Bologna, Italy [email protected]

Abstract. Intransitive non-interference (INI for short) is a behavioural property extensively studied by Rushby over deterministic automata with outputs associated to transitions (Mealy machines) in order to discuss the security of systems where declassiﬁcation of secret information is allowed. In this paper, we ﬁrst propose a natural transposition of Rushby’s deﬁnition on deterministic labelled transition systems, we call INI as well, and then an alternative, yet more easily checkable, formulation of INI, called NI with downgraders (NID for short). We show how NID can be naturally extended to the case of nondeterministic automata by using a variation of it based on bisimulation equivalence (BNID). The most novel contribution of this paper is the extension of this theory on the class of Petri nets called elementary net systems: we propose a semistatic technique, called PBNID and based on the inspection of the net structure, that is shown to be equivalent to BNID.

1

Introduction

Non-interference has been deﬁned in the literature as an extensional property based on some observational semantics: the high part (usually considered the secret part) of a system does not interfere with the low part (i.e., the public one) if whatever is done at the high level produces no visible eﬀect on the low part of the system. The original notion of non-interference in [14] was deﬁned, using trace semantics, for deterministic automata with outputs. Generalized notions of noninterference were then designed to include (nondeterministic) labeled transition systems and ﬁner notions of observational semantics such as bisimulation (see, e.g., [24,7,25,9]). The security properties in this class are based on the dynamics of systems; they are deﬁned by means of one (or more) equivalence check(s); hence, non-interference checking is as diﬃcult as equivalence checking, a wellstudied hard problem in automata theory and concurrency theory. When it is necessary to declassify information (e.g., when a secret plan has to be made public for realization), the two-level approach (H/secret – L/public) is usually extended with one intermediate level of downgrading (D), so that when an action in that class is performed, the previously performed high actions become observable to low users. This security policy is known under the name of A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 125–151, 2011. c Springer-Verlag Berlin Heidelberg 2011

126

R. Gorrieri and M. Vernali

intransitive noninterference (INI for short) because the information ﬂow relation is considered not transitive: even if information ﬂows from level H to D and from D to L are allowed, direct ﬂows from H to L are forbidden. Rusby [23] deﬁnes INI only for deterministic ﬁnite automata with output associated to transitions (Mealy machines), based on earlier work of Haigh and Young [13], in turn inﬂuenced by [15]. The basic intuition is that a machine in this class is INI if for any trace σ, the states reached after σ and ipurge(σ) (where all the action in H not justiﬁed by subsequent actions in D are removed) oﬀer the same output. Here we redeﬁne it on deterministic ﬁnite labeled transition systems, by interpreting output equivalence as low language equivalence of the reached states. This extensional deﬁnition is, however, rather cumbersome as it requires an equivalence check for any possible trace, hence inﬁnitely many checks, despite the fact that ﬁnite automata have only ﬁnitely many states and transitions. Therefore, we propose a diﬀerent property, called NID (non-interference with downgraders) that essentially requires that for any high transition the source state and the target one are equivalent for low observers. NID is reminescent of the so-called unwinding condition in [23], even if ours better reveals the role of downgraders. We prove that NID is an equivalent, yet more easily checkable, characterization of INI. When extending the approach to nondeterministic systems, we observe that NID is inadequate in some cases, as trace semantics is too weak to discriminate situations of possible danger. Nonetheless, by considering bisimulation semantics in place of trace semantics, we get BNID which seems to be fully satisfactory. Intuitively, the many deﬁnitions of (transitive or intransitive) noninterference that have been proposed in the literature try to capture the essence of information ﬂow as an extensional property. On the contrary, one may think that there are clear physical reasons for the occurrence of an information ﬂow, that can be better understood if one exploits a computational model where causality of actions and conﬂict among actions can be modelled directly. Indeed, this is not the case of labeled transitions systems, a typical example of an interleaving model, where parallelism is not primitive. For this reason, in [2,3,4] Busi and Gorrieri have shown that these extensional noninterference properties can be naturally deﬁned also on Petri Nets, in particular on Elementary Nets [6], a well-known model of computation where causality and conﬂict are primitive concepts. More interestingly, they address the problem of deﬁning statically non-interference for Elementary Nets, by looking at the structure of the net systems under investigation: • in order to better understand the relationship between a ﬂow of information and the causality (or conﬂict) relation between the activities originating such a ﬂow, hence grounding more ﬁrmly the intuition about what is an interference, and • in order to ﬁnd more eﬃciently checkable noninterference properties that are suﬃcient (sometimes also necessary) conditions for those that have already received some support in the literature.

On Intransitive Non-interference in Some Models of Concurrency

127

Structural noninterference was ﬁrst proposed in [3,4] on the basis of the absence of particular places in the net. Two special classes of places are of interest: causal places, i.e., places for which there are an incoming high transition and an outgoing low transition; and, conﬂict places, i.e. places for which there are both low and high outgoing transitions. Intuitively, causal places represent potential source of interference because the occurrence of the high transition is a prerequisite for the execution of the low transition. Similarly, conﬂict places represent potential source of interference because if the low event is not executable, then we can derive that a certain high transition has occurred. The absence of causal and conﬂict places is clearly a static property that can be easily checked (linear in the size of the net) by a simple inspection of the (ﬁnite) net structure. Interestingly enough, we show that absence of such places is a suﬃcient condition to ensure BNID. In order to characterize more precisely BNID, the notion of causal place and conﬂict place is slightly reﬁned, yielding the so-called active causal place and active conﬂict place. These new deﬁnitions are based also on a limited exploration of the state-space of the net (i.e. of its marking graph), hence, the absence of such places is not a purely structural property, rather a hybrid, semi-static property. When active causal and active conﬂict places are absent, we get a property, called Positive Place–Based Non–Interference with Downgraders (PBNID for short), which turns out to be equivalent to BNID. 1.1

Contribution of This Paper

The ﬁrst, minor contribution of the paper is a reformulation of Rushby’s deﬁnition [23] of (basic) noninterference (NI for short) in the setting of deterministic labeled transition systems (LTSs for short). Apparently, this reformulation, called NI with abuse of notation, is new because it is the only one requiring low equivalence of the reached states. Then, an alternative characterization of NI called SNDC, is given in terms of a ﬁnite number (equal to the number of high transitions in the LTS) of equivalence checks. One further contribution is its extension to nondetermistic systems, yielding the bisimulation-based SNDC, called SBNDC for short. Both SNDC and SBNDC are not original [7,9,5], but new is the result that proves that NI is equivalent to SNDC for deterministic LTSs. The same game is played for intransitive noninterference. First, we provide a reformulation of Rushby’s deﬁnition [23] in the setting of deterministic LTSs, which is new. Then, the alternative characterization of INI, called NID, is given in terms of a ﬁnite number (equal to the number of high transitions in the LTS) of equivalence checks. Then, one further contribution is its extension to nondetermistic systems, yielding the bisimulation-based NID, called BNID for short. Also in this case, NID and BNID were deﬁned already in [5] under the name of DSN DC and DSBN DC, but new is the result of equivalence between INI and NID.

128

R. Gorrieri and M. Vernali

The main result of the paper is the extension of the structural approach to noninterference of [2,3,4] to the richer setting of intransitive noninterference. We prove that an elementary net is BNID iﬀ it is PBNID. Hence, we essentially prove that there is an illegal information ﬂow directly from H to L iﬀ there is a direct causality (or conﬂict) relation between a high transition and a low one. From a complexity point of view, BNID over LTSs has complexity O(n3 ) (see [5] for a discussion on the complexity of DSBN DC) where n is the number of states. By following the same arguments reported in [10], we can conclude that BNID on elementary nets has complexity O(pn23p ), where p is the number of places and n the number of transitions; hence, since the marking graph has O(2p ) states, BNID is cubic in the number of the states also for elementary net systems. The complexity of checking for the absence of potential causal/conﬂict places is O(f + p), where f is the number of arcs in the net. The complexity of PBNID is O(pn22p ) in the worst case, hence PBNID is quadratic in the number of states. These algorithms, for the case without downgrading actions (i.e., classic noninterference), have been already implemented in a software tool, called the Petri Net Security Checker (PNSC for short), which provides functionalities for creating, editing and executing Petri nets, as well as automatically detecting places that are potentially/actively causal/conﬂict [10]. 1.2

Related Work

Deterministic Systems In [20] Pinsky studies intransitive noninterference (INI) over deterministic Mealy machine; his deﬁnition is very close to Rushby’s original one; his main contribution is an algorithm for detecting this property, whose complexity is not examined, but seems exponential. This work has been extended in [21] to cover the case of nondeterministic machines. In [11,12] an algorithmic approach is proposed to solve the problem of veriﬁcation of the property of INI, using tools and concepts of the theory of supervisory control of discrete event systems (DES). The algorithm is exponential in the number of states of the deterministic automaton. In [17] Ron van der Meyden discusses some alternative variations of INI in the area of deterministic Moore machines. He suggests that Rushby’s deﬁnition of INI is somehow inadequate in some practical cases and he proposes some stronger variations that better ﬁt with Rushby’s unwinding conditions. Nondeterministic Systems In [22] Roscoe and Goldsmith propose a rather discriminating deﬁnition of INI for nondeterministic systems which is based on strong assumptions about determinacy of low-level view. Their view is somehow related to the debate about what nondeterminism can be actually observed.

On Intransitive Non-interference in Some Models of Concurrency

129

Mullins in [19] study the problem of intransitive non-interference for a variant of CCS [18], hence on (nondeterministic) LTSs. His proposed property, called Admissible Interference (AI for short), is based on trace semantics: E is AI if for all reachable state E , (E \ D)/H (where downgrading actions are forbidden and high level actions are invisible) is (weak) trace equivalent to E \ H ∪ D (where both downgrading actions and high level actions are forbidden). It is not diﬃcult to prove that AI is strictly weaker than BNID. For instance, the system E = h.l1 .0 + l1 .0 + l2 .0 is AI, while it is not even NID and indeed, E is not secure because a low level user that cannot perform l2 is sure that h has been performed. Lafrance and Mullins in [16] propose a (weak) bisimulation-based version of AI, called BN AI (reminiscent of Focardi and Gorrieri’s SBSN N I), which turns out to be strictly weaker than BNID. For instance, system F = h.l1 .0 + τ.l1 .0 + l2 .0, which is a slight variant of E above, is BN AI, but it is not even NID and indeed, F is not secure for the same reason as above. In their thorough study [5] Bossi et al. consider various deﬁnitions of unwinding-based INI properties for a CCS-like language, with no precise indication of which of them is the right one. Among these properties, DSBN DC is actually the same deﬁnition as our BNID. No attempt is made to compare their many variants with Rushby’s original deﬁnition. They prove general composability properties (w.r.t. some operators of CCS) and provide conditions under which horizontal and vertical reﬁnements are preserved. Petri Nets To the best of our knowledge, this is the ﬁrst paper approaching INI over Petri nets. It builds over the results of [3,4] for the case of (basic) two-level noninterference, where SBNDC (the property obtained from BNID in absence of downgrading actions) is proved to be equivalent to PBNI+(the property obtained from PBNID when downgrading actions are absent). The paper is organised as follows. In Section 2 we recall the basic background deﬁnitions about Labeled Transition Systems and Elementary Net systems. In Section 3 we study basic noninterference, NI for short, starting from the deﬁnition proposed by Rushby for deterministic automata with outputs. We reformulate it over deterministic LTS and then propose our alternative, unwinding-like deﬁnition (SNDC) and prove the equivalence of the two. We discuss the extension to nondeterministic LTSs, yielding SBNDC. In Section 4 we present Rushby’s deﬁnition of Intransitive Noninterference, INI for short, then we reformulate it over deterministic LTSs and provide an alternative characterization in terms of a local property, called NID. We then consider LTSs in general, and the property BNID, based on bisimulation. In Section 5 we deﬁne BNID over elementary net systems. We then formulate the semi-static property PBNID by looking at the presence of (active) causal/conﬂict places, and provide the proof that BNID is the same as PBNID. In Section 6 we draw some conclusions.

130

2

R. Gorrieri and M. Vernali

Background

2.1

Labeled Transition Systems

Here we recall some basic deﬁnitions over LTSs. Definition 1. A labeled transition system is a triple T S = (St, E, →) where • St is the set of states • E is the set of events • →⊆ St × E × St is the transition relation. e

e

In the following we use s → s to denote (s, e, s ) ∈→. Given a transition s → s , s is called the source, s the target and e the label of the transition. A rooted transition system is a pair (T S, s0 ) where T S = (St, E, →) is a transition system and s0 ∈ St is the initial state. A transition system T S = (St, E, →) is ﬁnitestate if both St and E are ﬁnite sets. Definition 2. A labeled transition system T S = (St, E, →) is deterministic iﬀ e e the following holds: ∀s ∈ St, ∀e ∈ E if s → s1 and s → s2 then s1 = s2 . e

This means that ∀s ∈ St, ∀e ∈ E there is at most one s such that s → s , but such an s may also not exist. Definition 3. Given a labeled rooted transition system T S = (St, E, →, s0 ), a e1 en s2 . . . sn → sn+1 . We say path from s1 to sn+1 is a sequence of transitions s1 → that s is reachable from s if there exists a path from s to s . T S is reduced when the set of states reachable from s0 is exactly St, i.e., all the states are reachable from the initial state. In the following, we will usually consider only reduced labeled rooted transition systems. Definition 4. Let T S = (St, E, →, s0 ) be a rooted transition system. A trace of TS is a (possibly empty) sequence of events e1 . . . en such that there exists a path e1 en . . . sn → sn+1 with s1 = s0 . The set of traces of TS is denoted by T r(s0 ). s1 → Let T S = (St, E, →) be a transition system and let s1 , s2 ∈ St. We say that s1 and s2 are trace equivalent (denoted with s1 ∼ s2 ) iﬀ T r(s1 ) = T r(s2 ). Definition 5. A bisimulation between T S1 and T S2 is a relation R ⊆ (St1 × St2 ) such that if (s1 , s2 ) ∈ R then for all e ∈ (E1 ∪ E2 ) e

e

• ∀s1 such that s1 → s1 , then ∃s2 such that s2 → s2 and (s1 , s2 ) ∈ R e e • ∀s2 such that s2 → s2 , then ∃s1 such that s1 → s1 and (s1 , s2 ) ∈ R. If T S1 = T S2 we say that R is a bisimulation on T S1 . It is well-known that trace semantics and bisimulation semantics do coincide over deterministic LTSs, while in general bisimulation semantics is more discriminating (see, e.g., [18]).

On Intransitive Non-interference in Some Models of Concurrency

2.2

131

Elementary Net Systems

Here we introduce basic deﬁnitions about the class of Petri Nets we use. Some familiarity with Petri net terminology is assumed. More details can be found in [6,3]. Definition 6. An elementary net is a tuple N = (S, T, F ), where • S and T are the (ﬁnite) sets of places and transitions, such that S ∩ T = ∅ • F ⊆ (S × T ) ∪ (T × S) is the ﬂow relation, usually represented as a set of directed arcs connecting places and transitions. A ﬁnite subset of S is called a marking. Given a marking m and a place s, if s ∈ m then we say that the place s contains a token, otherwise we say that s is empty. Let x ∈ S ∪ T . The preset of x is the set • x = {y | F (y, x)}. The postset of x is the set x• = {y | F (x, y)}. The preset and postset functions are generalized in the obvious way to sets of elements: if X ⊆ S ∪ T then • X = x∈X • x and X • = x∈X x• . A transition t is enabled at marking m if • t ⊆ m and t• ∩ m = ∅. The ﬁring (i.e., execution) of a transition t enabled at m produces the marking m = (m \ • t) ∪ t• . This is usually written as m[tm . With the notation m[t we mean that there exists m such that m[tm . An elementary net system is a pair (N, m0 ), where N is an elementary net and m0 is a marking of N , called initial marking. With abuse of notation, we use (S, T, F, m0 ) to denote the net system ((S, T, F ), m0 ). The set of markings reachable from m, denoted by [m, is deﬁned as the least set of markings such that • m ∈ [m • if m ∈ [m and there exists a transition t such that m [tm then m ∈ [m. The set of ﬁring sequences is deﬁned inductively as follows: • m0 is a ﬁring sequence; • if m0 [t1 m1 . . . [tn mn is a ﬁring sequence and mn [tn+1 mn+1 then also m0 [t1 m1 . . . [tn mn [tn+1 mn+1 is a ﬁring sequence. Given a ﬁring sequence m0 [t1 m1 . . . [tn mn , we call t1 . . . tn a transition sequence. We use σ to range over transition sequences. The marking graph of a net system N is the transition system M G(N ) = ([m0 , T, {(m, t, m ) | m ∈ [m0 ∧ t ∈ T ∧ m[tm }). A net is transition simple if the following condition holds for all x, y ∈ T : if x = • y and x• = y • then x = y. A marking m contains a contact if there exists a transition t ∈ T such that • t ⊆ m and t• ∩ m = ∅. A net system is contact–free if no marking in [m0 contains a contact. A net system is reduced if each transition can occur at least one time: for all t ∈ T there exists m ∈ [m0 such that m[t. In the following we consider contact-free elementary net systems that are transition simple and reduced. •

132

3

R. Gorrieri and M. Vernali

Basic Noninterference on LTSs

3.1

Rushby’s Definition

The deﬁnition of Rushby [23], that elaborates over the original one of Goguen and Meseguer [14], is given over ﬁnite-state deterministic automata with outputs associated to transitions: these are essentially Mealy machines, where every action is allowed from each state. Definition 7. A system M is a tuple (S, A, O, step, output) where: • • • • •

S is a ﬁnite set of states, with a distinguished initial state s0 ; A is a ﬁnite set of actions (or inputs); O is a ﬁnite set of outputs; step : S × A → S is the transition function, and output : S × A → O is the function that returns the output associated to the transition.

Function step can be extended to sequences of actions by means of function run : S × A∗ → S as follows: run(s, ) = s run(s, aα) = run(step(s, a), α) where denotes the empty sequence and α a sequence of actions. Rushby’s deﬁnition is given in general for a set of security domains onto which a particular security policy is described. For simplicity’s sake, we restrict our attention to the simple case of two levels only, usually called high – H (for classiﬁed or secret information) and low – L (for public information). Hence, we reformulate his deﬁnition in this simpliﬁed setting.1 Some auxiliary deﬁnitions are necessary. Given a set D = {L, H} of security domains we consider a function dom : A → D which associates a security domain to each action. A security policy (or interference relation) is a reﬂexive relation →i ⊆ D × D, which, in our simpliﬁed setting, is given by the relation {(L, L), (L, H)(H, H)}, stating that information can ﬂow from low to high. With i we denote the complementary relation of noninterference, that in our case is just the relation {(H, L)}, meaning that information ﬂows from high to low are forbidden. A security policy is transitive if its interference relation→i is so, which is true in our case. There is an information ﬂow from domain u to domain v when the actions performed by domain u make the system, as observed by domain v, diﬀerent from the case when such actions are not performed. 1

This simpliﬁcation is not against generality: given a n-level system, this can be veriﬁed by means of a set of two level checks, for any partitioning of the n levels in two groups.

On Intransitive Non-interference in Some Models of Concurrency

133

Definition 8. Given v ∈ D and α ∈ A∗ , we deﬁne purge(α, v) as the subsequence of α obtained by removing all the actions associated with the domains u such that u i v: purge(, v) = a purge(α, v) if dom(a) →i v purge(aα, v) = purge(α, v) otherwise. In our specialized setting, as the forbidden information ﬂow is only the one from H to L, purge(α, v) returns α when v = H, while it simply removes the high actions from α when v = L. Definition 9. A system M is NI (i.e., non-interferent) iﬀ the following holds: ∀α ∈ A∗ ∀a ∈ A output(run(s0 , α), a) = output(run(s0 , purge(α, dom(a))), a). This essentially amounts to say that, whenever a low action a is to be performed, the state s1 , reached by performing α, and the state s2 , reached by performing purge(α, L), oﬀer the same output when performing a. Hence, NI holds if low outputs (i.e., the outputs performed in correspondence of low inputs) do not depend on high inputs. 3.2

How to Define NI on Deterministic LTSs?

Our aim is to analyse systems that can perform two kinds of actions: high level actions, representing the interaction of the system with high level users, and low level actions, representing the interaction with low level users. We want to verify if the interplay between the high user and the high part of the system can aﬀect the view of the system as observed by a low user. We assume that the low user knows the structure of the system, and we check if, in spite of this, he is not able to infer the behavior of the high user by observing the low view of the execution of the system. Hence, we consider LTSs whose set of events E is partitioned into two subsets: the set EH of high level events and the set EL of low level events. To emphasize this partition we use the following notation: with (St, EL , EH , →) we denote the LTS (St, E, →) where E = EL ∪ EH and EL ∩ EH = ∅. We would like to redeﬁne property NI over deterministic labeled transition systems in a way to preserve as much as possible the original intuition of the deﬁnition given by Rushby. We have to cope with some issues. First of all, the label of a transition in an LTS is either an input (high or low) or an output (high or low). Hence, the rigid synchrony of inputs and outputs is invalid in this more general model. Moreover, the equivalence on outputs required by the deﬁnition of NI above makes little sense on deterministic LTSs, where usually equivalence is expressed in terms of language equality (i.e., equality of the set of traces). We ﬁrst deﬁne the variant purge function in this context, that we call hide. Definition 10. Given a deterministic LTS (St, EL , EH , →), function hide (or low view) of a sequence of events α ∈ E ∗ is deﬁned as follows:

134

R. Gorrieri and M. Vernali

• hide() = a hide(α) if a ∈ EL • hide(aα) = hide(α) otherwise Also function run should be adapted as follows, because deterministic LTSs do not ensure that in any state a transition is present for any event: • run(s, ) = s a run(s , α) if s → s • run(s, aα) = undeﬁned otherwise So the deﬁnition of NI in this setting should be something like this: ∀α ∈ E ∗ run(s0 , α) ∼L run(s0 , hide(α)). where ∼L is some notion of low-equivalence we have not yet identiﬁed and the equivalence is meant to hold whenever the left hand side of the equation is deﬁned, i.e. when trace α is executable. Now let us try to identify what is a sensible candidate for ∼L . The idea is to have that the two reached states are indistinguishable for a low observer, at least until some high action takes place. Hence, we ﬁrst deﬁne the initial low view Λ(α) of a trace α ∈ E ∗ as follows: • Λ() = a Λ(α) if a ∈ EL • Λ(aα) = otherwise. Λ is similar to function hide, however it diﬀers because Λ truncates α to its ﬁrst high level action. Definition 11. Given a deterministic LTS, we say that two states s1 and s2 are initial low-view equivalent, denoted with s1 ∼L s2 , iﬀ Λ(T r(s1 )) = Λ(T r(s2 )). It is immediate to observe that ∼L is an equivalence relation. Moreover, it is decidable for ﬁnite-state LTS’s (actually PSPACE-complete), as it can be reduced to the trace equivalence problem. One may wonder why we need function Λ in the deﬁnition above and do not use instead function hide. The following example explains this point. Example 1. The deterministic LTS in Figure 1 is clearly insecure because if a low user observes action l2 then (s)he is sure that the high action h has been performed an odd number of times. Indeed, this system is not NI, but it would satisfy the variant deﬁnition where function hide is used in place of Λ. So, we are now ready to deﬁne our notion of noninterference, we call NI with abuse of notation, as from now on this deﬁnition surpasses the previous one. Definition 12. Given a deterministic labeled rooted transition system T S = (St, EL , EH , →, s0 ), we say that T S satisﬁes NI if the following holds: ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then so is run(s0 , hide(α)) and run(s0 , α) ∼L run(s0 , hide(α))

On Intransitive Non-interference in Some Models of Concurrency h s1

s0

h

l1

135

s2 l2

s3 Fig. 1. An LTS not satisfying N I

3.3

Unwinding – SNDC

The deﬁnition of NI is rather cumbersome and diﬃcult to check, because of the universal quantiﬁcation over all possible traces in E ∗ . As a matter of fact, a direct, brute force algorithm checking NI would fail miserably because the number of equivalence checks is inﬁnite in principle, even for ﬁnite-state LTS’s. However, in [15,23] are reported some local conditions ensuring NI that allows for a better algorithmic veriﬁcation of NI. These are called unwinding conditions in the jargon of information ﬂow security. In this section we want to propose one local property which is necessary and suﬃcient to prove NI. This property is called SNDC [7,9,5]. Definition 13. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SNDC h

iﬀ ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ∼L s . Hence, checking SNDC is decidable for ﬁnite-state LTS’s, as we have to make as many equivalence checks as are the high transitions in the LTS, which are ﬁnitely many, and each equivalence check s ∼L s is decidable. We can then state the main theorem for this part of the paper, whose proof is postponed to Appendix A. Theorem 1. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SNDC iﬀ it satisﬁes NI. 3.4

Extending the Approach to Nondeterminism

The deﬁnition of SNDC is rather satisfactory for deterministic LTSs, but when we move to general (i.e., possibly nondeterministic) LTSs, it is somehow inadequate, as the following example shows. Example 2. The system in Figure 2 is SNDC because the states s0 and s1 are low-view equivalent. However, such a system is insecure, because a low level user willing to perform trace ll may be unable to do so and in such a case (s)he is sure that h has been performed. This example clariﬁes that we need a more discriminating notion of initial lowview equivalence. A natural way out could be to reﬁne the deﬁnition by using the ﬁner bisimulation equivalence in place of trace equivalence.

136

R. Gorrieri and M. Vernali s1

s0 l l

h

l

l

l

Fig. 2. An LTS that satisﬁes SNDC but that is not secure.

Definition 14. Let T S = (St, EL , EH , →) be and LTS. An initial low–view bisimulation on T S is a relation R ⊆ St × St such that if (s1 , s2 ) ∈ R then for all a ∈ EL : a

a

• if s1 → s1 then there exists s2 such that s2 → s2 , with (s1 , s2 ) ∈ R a a • if s2 → s2 then there exists s1 such that s1 → s1 , with (s1 , s2 ) ∈ R We say that two states s1 and s2 are initial low-view bisimulation equivalent, denoted with s1 ≈L s2 , if there exists an initial low-view bisimulation R with (s1 , s2 ) ∈ R. It is immediate to observe that ≈L is an equivalence relation. Moreover, for ﬁnite-state LTS’s, ≈L is decidable in polynomial time, because so is strong bisimulation. We are now ready to deﬁne the improved information ﬂow property. Definition 15. A reduced LTS (St, EL , EH , →, s0 ) satisﬁes SBNDC if and only h if ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ≈L s . It is easy to observe that the system in Example 2 is not SBNDC. Moreover, SBNDC is decidable for ﬁnite-state LTS’s because ﬁnite is the number of decidable equivalence checks s ≈L s . Finally, SBNDC is a conservative extension of SNDC as the following proposition states. Theorem 2. A reduced deterministic LTS (St, EL , EH , →, s0 ) satisﬁes SBNDC iﬀ it satisﬁes SNDC. By our experience, we conjecture that SBNDC is the right property in the setting of nondeterministic LTSs.

4

Intransitive Noninterference on LTSs

Basic (or transitive) noninterference is, from a practical point of view, too draconian: one often wants to make public some data that were secret previously. This operation is known as declassiﬁcation and can be modeled naturally with an intransitive version of noninterference. Typically, we have three levels: H for secret actions, L for public actions and D for downgrading actions; we admit ﬂows from H to L only if mediated by an action in D. Hence, in this three-level approach, the interference relation is {(L, L), (L, H)(H, H), (D, D), (L, D), (D, L), (H, D), (D, H)} (where only (H, L) is forbidden), which is clearly not transitive.

On Intransitive Non-interference in Some Models of Concurrency

4.1

137

Rushby’s Definition

In Section 3.1 we have presented Rushby’s model, based on Mealy machines, for NI. Now we report his extension to intransitive non-interference, called INI, as described in [23]. We ﬁrst need to identify which actions in a trace α are not to be deleted with the intransitive version of the purge function. Definition 16. Function sources : A∗ × D → P(D) is deﬁned as follows: sources(, u) = {u} ⎧ ⎨ sources(α, u) ∪ {dom(a)} if ∃v : v ∈ sources(α, u) ∧dom(a) →i v sources(aα, u) = ⎩ sources(α, u) otherwise. Essentially, v ∈ sources(α, u) means either that v = u or that there exists a subsequence of α composed of actions in the domains w1 , w2 , . . . , wn such that w1 →i w2 →i · · · →i wn , v = w1 and u = wn . Hence, when considering if a, performed before α, is allowed to inﬂuence domain u, we ask if there is any v ∈ sources(α, u) such that dom(a) →i v. Definition 17. Function ipurge : A∗ × D → A∗ (that is intransitive-purge) is deﬁned as follows: ipurge(, u) = a ipurge(α, u) if dom(a) ∈ sources(aα, u) ipurge(aα, u) = ipurge(α, u) otherwise. Essentially, ipurge(α, u) is the subsequence of α where all the actions that cannot interfere with u are removed. It is interesting to observe that, in the simpliﬁed case of three levels only we consider in this paper, ipurge(α, H) = α, ipurge(α, D) = α, while ipurge(α, L) returns the subsequence of α where all the high level actions occurring before a low level action are removed, unless a downgrading action does occur in between the two. Definition 18. A system M is INI (i.e., intransitive non-interferent) iﬀ the following holds: ∀α ∈ A∗ ∀a ∈ A output(run(s0 , α), a) = output(run(s0 , ipurge(α, dom(a))), a). This essentially amounts to say that, whenever a low action a is to be performed, the state s1 , reached by performing α, and the state s2 , reached by performing ipurge(α, L), oﬀer the same output when performing a. Hence, INI holds if low outputs (i.e., the outputs performed in correspondence of low inputs) do not depend on non-downgraded high inputs. 4.2

Defining INI for Deterministic LTSs

As we have three levels of actions, the deﬁnition of LTS should reﬂect this partitioning, hence T S = (St, EL , ED , EH , →). We start by deﬁning an intransitive variant of function hide.

138

R. Gorrieri and M. Vernali

Definition 19. Given a deterministic LTS (St, EL , ED , EH , →), the function ihide (or intransitive low view) of a sequence of events α ∈ E ∗ is deﬁned as follows: • ihide() = ⎧ ⎨ ihide(α)a if a ∈ EL • ihide(αa) = ihide(α) if a ∈ EH ⎩ αa if a ∈ ED Function Λ extends naturally to the case of the presence of downgrading actions: it truncates traces at the point the ﬁrst high action or downgrading action is met. Even if the deﬁnition of Λ works as expected also in the three level setting, we prefer to change its name to Δ to emphasize that we are working on this richer scenario. Hence, also the deﬁnition of initial low-view equivalence is syntactically slightly changed: s1 ∼D s2 iﬀ Δ(T r(s1 )) = Δ(T r(s2 )). It is immediate to observe that ∼D is an equivalence relation. Moreover, it is decidable for ﬁnite-state LTS’s. Definition 20. Given a deterministic labelled rooted transition system T S = (St, EL , ED , EH , →, s0 ), we say that T S satisﬁes INI iﬀ the following holds: ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then so is run(s0 , ihide(α)) and then run(s0 , α) ∼D run(s0 , ihide(α)). The deﬁnition above is indeed a conservative extension of NI because, if we consider a deterministic labelled rooted transition system with empty set of downgrading actions, we get that INI and NI do coincide. This is due to the fact that ihide(α) = hide(α) if α ∈ (EL ∪ EH )∗ . Theorem 3. Given a deterministic labelled rooted transition system T S = (St, EL , ∅, EH , →, s0 ), it holds that T S satisﬁes NI iﬀ T S satisﬁes INI. 4.3

Unwinding – NID

For basic (i.e., two-level) noninterference, we have shown that SNDC is a local property characterizing NI. Here we play the same game, by providing an obvious generalization of SNDC, we call NID (NonInterference with Downgraders) and then by showing that NID and INI are the same property (proof postponed to Appendix B). Definition 21. A reduced deterministic LTS (St, EL , ED , EH , →) satisﬁes NID h

iﬀ ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ∼D s . Theorem 4. A reduced deterministic LTS (St, EL , ED , EH , →, s0 ) satisﬁes NID iﬀ it satisﬁes INI. NID is decidable for ﬁnite-state LTS’s: we have to perform as many equivalence checks as are the transitions labeled with an high level action; moreover, each equivalence check of the form s ∼D s is decidable, as it can be reduced to the trace equivalence problem.

On Intransitive Non-interference in Some Models of Concurrency

4.4

139

Extending the Approach to Nondeterminism

The deﬁnition of NID seems rather satisfactory for deterministic LTSs, but when we move to nondeterministic systems, it may be inadequate, as the system in Figure 2 shows: the states s0 and s1 are low-view equivalent, but a low level user willing to perform trace ll may be unable to do so and in such a case (s)he is sure that h has been performed. A similar example is reported in Figure 3. s1

s0 h d l

h s3

s2 d

h l

l

l

l

l

s4

Fig. 3. An insecure nondeterministic system satisfying N ID

As discussed in Section 3.4, we should replace the trace based notion of initial low-view equivalence with a ﬁner one based on bisimulation equivalence. The deﬁnition of low-view observational equivalence of Deﬁnition 14 is unchanged for this richer scenario, but to emphasize the diﬀerence (presence also of the intermediate level D), we denote it with ≈D instead of ≈L . We are now ready to deﬁne the improved information ﬂow property, we call BNID (Bisimulation-based NID). Definition 22. A reduced LTS (St, EL , ED , EH , →) satisﬁes BNID if and only h

if ∀s ∈ St, ∀h ∈ EH whenever s → s we have that s ≈D s . It is easy to observe that the system in Figure 3 is not BNID. Moreover, BNID is decidable for ﬁnite-state LTS’s because ﬁnite is the number of decidable equivalence checks s ≈D s . Finally, based on our experience, we conjecture that BNID is the right property in the setting of nondeterministic LTSs.

5

Noninterference on Elementary Nets

We ﬁrst brieﬂy survey the work in [3,4], presenting the adaptation of the property SBNDC [7,9] in the case of elementary net systems, and then the idea of structural noninterference. Then, Section 5.3 reports the original extension to the case of intransitive noninterference. We remind that we consider elementary net systems that are contact-free, reduced and transition simple.

140

5.1

R. Gorrieri and M. Vernali

A Dynamic Non-interference Property: SBNDC

We consider nets whose set of transitions is partitioned into two subsets: the set H of high level transitions and the set L of low level transitions. To emphasize this partition we use the following notation. Let L and H be two disjoint sets: with (S, L, H, F, m0 ) we denote the net system (S, L ∪ H, F, m0 ). Among the many non-interference properties deﬁned by Focardi and Gorrieri in [7,8,9], here we consider SBNDC (Strong Bisimulation Non-Deducibility on Composition). To properly deﬁne it over Petri nets, we need the auxiliary deﬁnition of initial low-view bisimulation over elementary net systems. Definition 23. Let N = (S, L, H, F, m0 ) be a net system. An initial low–view bisimulation is a relation R ⊆ Pf in (S) × Pf in (S) such that if (m1 , m2 ) ∈ R then for all t ∈ L: • if m1 [tm1 then there exists m2 such that m2 [tm2 , and (m1 , m2 ) ∈ R; • if m2 [tm2 then there exists m1 such that m1 [tm1 , and (m1 , m2 ) ∈ R. We say that m1 is initial low–view bisimilar to m2 , denoted by m1 ≈L m2 , if there exists an initial low–view bisimulation R such that (m1 , m2 ) ∈ R. Now we are ready to deﬁne SBNDC. Definition 24. Let N = (S, L, H, F, m0 ) be a net system. N is SBNDC iﬀ for all markings m ∈ [m0 and for all h ∈ H the following holds: m[hm implies m ≈L m . The intuition behind SBNDC is that, whenever a high transition h is performed, the markings before h and after h are observationally indistinguishable for a low observer. Note that SBNDC is clearly decidable for ﬁnite elementary net systems because the number of reachable markings is ﬁnite, as well as the set H of high transitions; hence, the number of equivalence checks is ﬁnite; moreover, each equivalence check m ≈L m is decidable, because so is bisimulation equivalence over ﬁnite elementary net systems. Example 3. As a simple case study, consider the net in Figure 4, which represents a mutually exclusive access to a shared resource (represented by the token in s) by a high-user (left part of the net) and a low-user (right part of the net). Even if it might appear, at ﬁrst sight, that the system is secure (and indeed, it is BSNNI (Bisimulation Strong Nondeterministic Non-Interference) [7,9]), actually it is not SBNDC: consider the reachable marking m = {p1,2 , s, p2,2 }; it is easy to observe that m[h2 m = {p1,3 , s, p2,2 } and m[l2 , but l2 is not ﬁreable from m . Indeed, the system is not secure, because an unsuccesful attempt of a low level user to perform l2 just after having performed l1 will give him the information that the high user has performed h2 but not yet h3 .

On Intransitive Non-interference in Some Models of Concurrency p1,2

s

p2,2

h2

h1

l2

p1,3

p2,3

h3 p1,1

141

l1

l3 s

p2,1

Fig. 4. The net system for a mutually exclusive access to a shared resource

5.2

Structural Non-interference

Consider a net system N = (S, L, H, F, m0 ). Consider a low level transition l of the net: if l can ﬁre, then we know that the places in the preset of l are marked before the ﬁring of l; moreover, we know that such places become unmarked after the ﬁring of l. If there exists a high level transition h that produces a token in a place s in the preset of l (see the system N1 in Figure 5), then the low level user can infer that h has occurred if he can perform the low level transition l. We note that there exists a causal dependency between the transitions h and l, because the ﬁring of h produces a token that is consumed by l. In this case we will say that s is a potential causal place. Consider now the situation illustrated in the system N2 of Figure 5: in this case, place s is in the preset of both l and h, i.e., l and h are competing for the use of the resource represented by the token in s. Aware of the existence of such a place, a low user knows that the high-level action h has been performed, if he is not able to perform the low-level action l. Place s represents a conﬂict

h s

s

l

N1

h

l

N2

Fig. 5. Examples of net systems containing causal and conﬂict places

142

R. Gorrieri and M. Vernali

between transitions l and h, because the ﬁring of h prevents l from ﬁring. In this case we will call s a potential conﬂict place. In order to avoid the deﬁnition of a security notion that is too strong, and that rules out systems that do not reveal information on the high-level actions that have been performed, we need to reﬁne the concepts illustrated above. In particular the potential causal place is an active causal place if there is an execution where the token produced by the high level transition is eventually consumed by the low level transition. Similarly, a potential conﬂict place is active if the token that could be consumed immediately by a high level transition can be later on also consumed by a low level transition. The formal deﬁnitions follow. Definition 25. Let N = (S, L, H, F, m0 ) be an elementary net system. Let s be a place of N such that s• ∩ L = ∅ (i.e., a token in s can be consumed by a low transition). The place s ∈ S is a potentially causal place if • s ∩ H = ∅ (i.e., a token in s can be produced by a high transition). A potentially causal place s is an active causal place if the following condition holds: there exist l ∈ s• ∩ L, h ∈ • s ∩ H, m ∈ [m0 and a transition sequence σ such that m[hσl and s ∈ t• for all t ∈ σ. The place s ∈ S is a potentially conﬂict place if s• ∩ H = ∅ (i.e., the token in s can be consumed also by a high transition). A potentially conﬂict place is an active conﬂict place if the following condition holds: there exist l ∈ s• ∩ L, h ∈ s• ∩ H, m ∈ [m0 and a transition sequence σ such that m[h, m[σl and s ∈ t• for all t ∈ σ. Definition 26. Let N = (S, L, H, F, m0 ) be an elementary net system. We say that N is PBNI+ (positive Place Based Non-Interference) if, for all s ∈ S, s is neither an active causal place nor an active conﬂict place. The following non-trivial result, proved in [4], states that the behavioural noninterference property SBNDC is equivalent to the semi-static, structural property PBNI+. Theorem 5. Let N = (S, L, H, F, m0 ) be an elementary net system. Then N is PBNI+ iﬀ N is SBNDC. An obvious consequence is that if N has no potentially causal and potentially conﬂict places, then N is SBNDC. Hence, a simple strategy to check if N is SBNDC is to ﬁrst identify potential causal/conﬂict places, a procedure that has complexity O(f + p) in the size of the net (p is the number of places and f of arcs). If no place of these sorts is found, then N is PBNI+, hence SBNDC. Otherwise, any such a candidate place should be better studied to check if it is actually an active causal/conﬂict place, a procedure that requires a limited exploration of the marking graph. The complexity of PBNI+ in the worst case is O(pn22p ) (see [10] for details). Observe that the net in Figure 4 of our running example is not PBNI+ because place s is an active conﬂict (and also active causal) place.

On Intransitive Non-interference in Some Models of Concurrency

5.3

143

Extending the Approach for Intransitive Noninterference

Not too surprisingly, the theory presented in the two previous subsections can be adapted to the more general scenario when also downgrading actions are present. First of all, observe that the notion of initial low view bisimulation equivalence ≈L of Deﬁnition 23 works as expected also for the three level scenario; however, to better reﬂect the fact that also actions in D are possible, we rename it with ≈D . Hence, the deﬁnition of BNID over elementary net systems is essentially the same as SBNDC. Definition 27. Let N = (S, L, D, H, F, m0 ) be a net system. N is BNID iﬀ for all markings m ∈ [m0 and for all h ∈ H the following holds: m[hm implies m ≈D m . Then, we adapt the structural noninterference approach to the three level scenario. Definition 28. Let N = (S, L, D, H, F, m0 ) be an elementary net system. A place s ∈ S is a potentially causal place if • s ∩ H = ∅ and s• ∩ L = ∅. Place s ∈ S is an active causal place if it is potentially causal and there exist h ∈ • s∩H, l ∈ s• ∩ L, m ∈ [m0 and a transition sequence σ ∈ (H ∪ L)∗ such that m[hσl and for all t ∈ σ, s ∈ / t• . Observe that the only diﬀerence w.r.t. Deﬁnition 25 is that the transition sequence σ is constrained not to contain downgrading actions. The idea is that if a downgrading action d is performed inevitably in between h and l, then the ﬂow is mediated by d and so the information ﬂow becomes legal. Definition 29. Let N = (S, L, D, H, F, m0 ) be an elementary net system. A place s ∈ S is a potentially conﬂict place if s• ∩ H = ∅ and s• ∩ L = ∅. Place s ∈ S is an active conﬂict place if it is potentially conﬂict and there exist h ∈ s• ∩ H, l ∈ s• ∩ L, m ∈ [m0 and a transition sequence σ ∈ (H ∪ L)∗ such that m[h, m[σl and for all t ∈ σ, s ∈ / t• . Also for active conﬂict places, the only diﬀerence is the constraint on σ about downgrading actions. Indeed, if a downgrading action d is to be performed before l, then h and l are not really conﬂicting, because that conﬂict is made public by action d. Definition 30. Let N = (S, L, D, H, F, m0 ) be an elementary net system. We say that N is PBNID ( Positive Place Based Non Interference with Downgraders) if for all s ∈ S, s is neither a active causal place, nor an active conﬂict place. The main result is that the dynamic property BNID and the semi-static property PBNID are actually equivalent on elementary net systems. The proof of the following theorem, based on the analogous proof in [4] of Theorem 5, is reported in Appendix C.

144

R. Gorrieri and M. Vernali

Theorem 6. Let N = (S, L, D, H, F, m0 ) be an elementary net system. N is BNID if and only if N is PBNID. The complexity of checking BNID is the same as checking SBNDC and, as reported in [10], it is O(pn23p ) where p is the number of places and n the number of transitions. Similarly, the complexity of checking PBNID is the same as checking PBNI+ and, as reported in [10], varies from a minimum of O(f + p) (where f is the number of arcs) to a maximum of O(pn22p ). Example 4. Consider again the insecure net in Figure 4. A possible way to make the net secure is to include additional downgrading transitions as reported in Figure 6. d1 p1,2

s

p2,2

h2

h1

l2

p1,3

p2,3 l3

h3 p1,1

l1

s

p2,1 d2

Fig. 6. Secure mutually exclusive access to a shared resource

6

Conclusion

In this paper we presented a study about the noninterference properties in the basic case of two levels of conﬁdentiality (H and L) and in the richer case of three levels, where the level D of downgrading actions is considered additionally. In both settings, we start by ﬁrst recalling the deﬁnitions by Rushby in [23] for deterministic ﬁnite state automata with outputs (Mealy machines); we try to provide similar deﬁnitions for the more popular model of deterministic labeled transition system, following as much as possible the original intuition, yielding our deﬁnition of NI and INI. Then, in both scenarios, we investigate local information ﬂow properties, called SNDC and NID, respectively, that are necessary and suﬃcient conditions to ensure NI and INI, respectively. In both cases, we discuss the need of bisimulation-based deﬁnition of information ﬂow security properties as soon as one considers nondeterministic LTSs. Finally, we extend the work in [3,4] over the model of elementary net systems to the case of intransitive noninterference.

On Intransitive Non-interference in Some Models of Concurrency

145

Future work will be devoted to study systematic techniques for adapting an insecure system to make it secure. This is a rather challenging problem because there is a large range of possible modiﬁcations of a system that can be considered; e.g., enlarging the behaviour of the low part of the system, inserting suitable declassiﬁers whenever possible (as we have done in Example 4) or even cutting some possible high level behaviour. Moreover, we plan to study more selective forms on intransitive noninterference where the occurrence of a downgrading action does not reveal all the previously performed high level actions, as it is now prescribed in Rushby’s deﬁnition (cf. Deﬁnition 19). Furthermore, it is interesting to see if the approach presented here can be extended to the richer setting of P/T Petri nets. Some results are reported in a recent paper [1] (actually written after this), where it is shown that SBNDC and BNID are decidable for ﬁnite P/T nets (hence for a class of inﬁnite-state systems), even if it is left open the problem of understanding if (transitive as well as intransitive) noninterference can be equivalently characterized as a relation of causality or conﬂict among particular high actions and low ones. Acknowledgements. The authors would like to thank the anonymous referees for helpful comments.

References 1. Best, E., Darondeau, P., Gorrieri, R.: On the Decidability of Non Interference over Unbounded Petri Nets. EPTCS 51, 16–33 (2010); Procs. SecCo 2010 2. Busi, N., Gorrieri, R.: A Survey on Non-Interference with Petri Nets. In: Desel, J., Reisig, W., Rozenberg, G. (eds.) Lectures on Concurrency and Petri Nets. LNCS, vol. 3098, pp. 328–344. Springer, Heidelberg (2004) 3. Busi, N., Gorrieri, R.: Positive Non-Interference in Elementary and Trace Nets. In: Cortadella, J., Reisig, W. (eds.) ICATPN 2004. LNCS, vol. 3099, pp. 1–16. Springer, Heidelberg (2004) 4. Busi, N., Gorrieri, R.: Structural Non-Interference in Elementary and Trace Nets. Mathematical Structures in Computer Science 19(6), 1065–1090 (2009) 5. Bossi, A., Piazza, C., Rossi, S.: Modelling Downgrading in Information Flow Security. In: Procs. 17th IEEE Computer Security Foundations Workshop (CSFW 2004), pp. 187–201. IEEE Press, Los Alamitos (2004) 6. Engelfriet, J., Rozenberg, G.: Elementary Net Systems. In: Reisig, W., Rozenberg, G. (eds.) APN 1998. LNCS, vol. 1491, pp. 12–121. Springer, Heidelberg (1998) 7. Focardi, R., Gorrieri, R.: A Classiﬁcation of Security Properties. Journal of Computer Security 3(1), 5–33 (1995) 8. Focardi, R., Gorrieri, R.: The Compositional Security Checker: A Tool for the Veriﬁcation of Information Flow Security Properties. IEEE Transactions on Software Engineering 23(9), 550–571 (1997) 9. Focardi, R., Gorrieri, R.: Classiﬁcation of Security Properties (Part I: Information Flow). In: Focardi, R., Gorrieri, R. (eds.) FOSAD 2000. LNCS, vol. 2171, pp. 331–396. Springer, Heidelberg (2001) 10. Frau, S., Gorrieri, R., Ferigato, C.: Structural Noninterference at Work: the Petri Net Security Checker. In: Degano, P., Guttman, J., Martinelli, F. (eds.) FAST 2008. LNCS, vol. 5491, pp. 210–225. Springer, Heidelberg (2009)

146

R. Gorrieri and M. Vernali

11. Hadj-Alouane, N.B., Lafrance, S., Lin, F., Mullins, J., Yeddes, M.M.: Characterizing Intransitive Noninterference for 3-Domain Security Policies With Observability. IEEE Transactions on Automatic Control 50(6), 920–925 (2005) 12. Hadj-Alouane, N.B., Lafrance, S., Lin, F., Mullins, J., Yeddes, M.M.: On the veriﬁcation of intransitive noninterference in multilevel security. IEEE Transactions on Systems, Man, and Cybernetics, Part B 35(5), 948–958 (2005) 13. Haigh, J.T., Young, W.D.: Extending the noninterference version of MLS for SAT. IEEE Trans. on Software Engineering SE-13(2), 141–150 (1987) 14. Goguen, J.A., Meseguer, J.: Security Policy and Security Models. In: Proc. of Symposium on Security and Privacy (SSP 1982), pp. 11–20. IEEE CS Press, Los Alamitos (1982) 15. Goguen, J.A., Meseguer, J.: Unwinding and Inference Control. In: Proc. of Symposium on Security and Privacy (SSP 1984), pp. 75–86. IEEE CS Press, Los Alamitos (1984) 16. Lafrance, S., Mullins, J.: Bisimulation-based Nondeterministic Admissible Interference and ita Applications to the Analysis of Cryptographic Protocols. Information and Software Technology 45(11), 779–790 (2003); Preliminary version in Electronic Notes in Theoretical Computer Science 61, 1–24 (2002) 17. van der Meyden, R.: What, Indeed, Is Intransitive Noninterference? In: Biskup, J., L´ opez, J. (eds.) ESORICS 2007. LNCS, vol. 4734, pp. 235–250. Springer, Heidelberg (2007) 18. Milner, R.: Communication and Concurrency. Prentice-Hall, Englewood Cliﬀs (1989) 19. Mullins, J.: Nondeterministic Admissible Interference. Journal of Universal Computer Science 11, 1054–1070 (2000) 20. Pinsky, S.: Absorbing Covers and Intransitive Noninterference. In: Proc. of IEEE Symposium on Security and Privacy (SSP 1995), pp. 102–113. IEEE CS press, Los Alamitos (1995) 21. Pinsky, S., Zieglar, E.: Noninterference Equations for Nondeterministic Systems. In: Proc. of IEEE Computer Security Foundations Workshop (CSFW 2001), pp. 3–14. IEEE CS press, Los Alamitos (2001) 22. Roscoe, A.W., Goldsmith, M.-H.: What is Intransitive Noninterference? In: Proc. of 12th Computer Security Foundations Workshop (CSFW), pp. 228–238. IEEE CS Press, Los Alamitos (1999) 23. Rushby, J.: Noninterference, Transitivity, and Channel-control Security Policies. Technical Report CSL-92-02, SRI International (1992) 24. Ryan, P.Y.A.: Mathematical Models of Computer Security. In: Focardi, R., Gorrieri, R. (eds.) FOSAD 2000. LNCS, vol. 2171, pp. 1–62. Springer, Heidelberg (2001) 25. Ryan, P.Y.A., Schneider, S.: Process Algebra and Noninterference. In: Proc. of 12th Computer Security Foundations Workshop (CSFW), pp. 214–227. IEEE CS Press, Los Alamitos (1999)

On Intransitive Non-interference in Some Models of Concurrency

147

Appendix A

Proofs of Section 3

Proposition 1. If a deterministic reduced LTS satisﬁes NI, then it satisﬁes SNDC. Proof. By hypothesis, we have that for all α ∈ E ∗ such that run(s0 , α) is deﬁned, run(s0 , α) ∼L run(s0 , hide(α)). Let s = run(s0 , α ) and t = run(s0 , hide(α )); b hence, s ∼L t. Take α = α b. If b ∈ EH then s → s and run(s0 , hide(α b)) = t, hence s ∼L t. Hence, by transitivity of ∼L , we conclude that s ∼L s , i.e., SNDC holds. In order to prove the reverse implication (i.e., SNDC implies NI ), it is useful to prove ﬁrst the following lemma. Lemma 1. If a deterministic reduced LTS satisﬁes SNDC then, for all s, t ∈ St a a and for all a ∈ EL , s ∼L t and s → s imply t → t and s ∼L t . Proof. As s ∼L t, for all α = aβ we have that α ∈ Λ(T r(s)) iﬀ α ∈ Λ(T r(t)). If a aβ ∈ Λ(T r(s)), there must exist a state s such that s → s with β ∈ Λ(T r(s )). a Similarly, If aβ ∈ Λ(T r(t)) there must exist a state t such that t → t with β ∈ Λ(T r(t )). Hence for all β we have that β ∈ Λ(T r(s )) iﬀ β ∈ Λ(T r(t )), i.e., s ∼ L t . Proposition 2. If a deterministic reduced LTS satisﬁes SNDC, then it satisﬁes NI. Proof. We prove by induction on the length of traces α that s ∼L t ⇒ run(s, α) ∼L run(t, hide(α))

(1)

for all α such that run(s, α) is deﬁned. When in the equation (1) above we set s = t = s0 , we get the thesis. The base case when α = is trivial. For the inductive case, assume that the thesis holds for all α of length n for which run(s, α) is deﬁned, and consider trace aα. If run(s, a) is undeﬁned, then trace aα is to be ignored; otherwise let s = run(s, a). We have that run(s, aα) = run(s , α). On the other hand for run(t, hide(aα)) we have to distinguish two cases. (1. a ∈ EL ) In this case, the deﬁnition of hide justiﬁes that run(t, hide(aα)) = run(t, a hide(α)). By hypothesis that s ∼L t and since s = run(s, a), there must exist a t = run(t, a). Hence, run(t, a hide(α)) = run(t , hide(α)). As s ∼L t and SNDC holds, we get by Lemma 1 that s ∼L t . By applying the inductive hypothesis, we get run(s , α) ∼L run(t , hide(α)), and so the implication in (1) holds for aα. (2. a ∈ EH ) In this case, the deﬁnition of hide justiﬁes that run(t, hide(aα)) = run(t, hide(α)). On the other hand, as a ∈ EH and SNDC holds, we get s ∼L s , hence (since by hypothesis s ∼L t) also s ∼L t. By applying the inductive hypothesis, we get run(s , α) ∼L run(t, hide(α)), and so the implication in (1) holds for aα.

148

R. Gorrieri and M. Vernali

Corollary 1. A deterministic reduced LTS is NI if and only if it is SNDC. Proof. It follows directly from Proposition 1 and Proposition 2.

B

Proofs of Section 4

Proposition 3. If a deterministic reduced LTS satisﬁes INI, then it satisﬁes NID. Proof. By hypothesis, we have that for all α ∈ E ∗ such that run(s0 , α) is deﬁned, run(s0 , α) ∼D run(s0 , ihide(α)). Let s = run(s0 , α ) and t = run(s0 , hide(α )); b

hence we have that s ∼D t. Take α = α b. If b ∈ EH then s → s and run(s0 , ihide(α b)) = t, hence s ∼D t. Hence, by transitivity of ∼D , we conclude that s ∼D s , i.e., NID holds. In order to prove the reverse implication (i.e., NID implies INI), it is useful to prove ﬁrst the following lemma. Lemma 2. If a deterministic reduced LTS satisﬁes NID then, for all s, t ∈ St a a and for all a ∈ EL , s ∼D t and s → s imply t → t and s ∼D t . Proof. As s ∼D t, for all α = aβ we have that α ∈ Δ(T r(s)) iﬀ α ∈ Δ(T r(t)). If a aβ ∈ Δ(T r(s)), there must exist a state s such that s → s with β ∈ Δ(T r(s )). a Similarly, If aβ ∈ Δ(T r(t)) there must exist a state t such that t → t with β ∈ Δ(T r(t )). Hence for all β we have that β ∈ Δ(T r(s )) iﬀ β ∈ Δ(T r(t )), i.e., s ∼L t . Proposition 4. If a deterministic reduced LTS satisﬁes NID, then it satisﬁes INI. Proof. We prove, by induction on the length of α, that ∀α ∈ E ∗ if run(s0 , α) is deﬁned, then run(s0 , α) ∼D run(s0 , ihide(α))

(2)

The base case when α = is trivial. For the inductive case, assume that the thesis holds for all α of length n for which run(s0 , α) = s is deﬁned, and consider trace αa. If run(s, a) is undeﬁned, then trace αa is to be ignored; otherwise let s = run(s, a). We have that run(s0 , αa) = run(s, a) = s . On the other hand for run(s0 , ihide(αa)) we have to distinguish three cases. (1. a ∈ EL ) In this case, the deﬁnition of ihide justiﬁes that run(s0 , ihide(αa)) = run(s0 , ihide(α)a) = run(t, a). The inductive hypothesis ensures that s ∼D t; hence, by Lemma 2 we have that s ∼D t , where t = run(t, a), and so the implication in (2) holds for aα. (2. a ∈ EH ) In this case, the deﬁnition of ihide justiﬁes that run(s0 , ihide(αa)) = run(s0 , ihide(α)) = t. On the other hand, run(s0 , α) = s and run(s, a) = s . By inductive hypothesis, s ∼D t. On the other hand, as a ∈ EH and NID holds, we get s ∼D s , hence also s ∼D t, and so the implication in (2) holds for aα. (3. a ∈ ED ) In this case, by deﬁnition of ihide, we get run(s0 , ihide(αa)) = run(s0 , αa). Since ∼D is reﬂexive, the thesis follows.

On Intransitive Non-interference in Some Models of Concurrency

149

Corollary 2. A deterministic reduced LTS is INI if and only if it is NID. Proof. It follows directly from Proposition 3 and Proposition 4.

C

Proofs of Section 5

Theorem 7. Let N = (S, L, D, H, F, m0 ) be an elementary net system. If N satisﬁes P BN ID, then N satisﬁes BN ID. Proof. Let N be PBNID. We will show that N is BNID. Take m ∈ [m0 such that m[hm for h ∈ H. We have to prove that there exists an initial low-view bisimulation R on N such that (m, m ) ∈ R. Let R = {(m1 , m2 ) | ∀l ∈ L ∀s ∈ • l : m1 (s) = m2 (s) ⇒ (∀σ∀i ∈ {1, 2} : mi [σl ⇒ ∃l1 ∈ σ : s ∈ l1 • )} be the candidate relation. 1. We show that R is an initial low-view bisimulation on N . Let (m1 , m2 ) ∈ R. Suppose m1 [lm1 . We show that also m2 [l. Suppose that there exists s ∈ • l such that m2 (s) = 0, hence m1 (s) = m2 (s). As (m1 , m2 ) ∈ R and m1 [l, by deﬁnition of R (with σ = ), there must exists t ∈ , reaching a contradiction. Hence ∀s ∈ • l m2 (s) ≥ 1, and so there exists m2 such that m2 [lm2 (by contact-freeness). Now we show that (m1 , m2 ) ∈ R. Suppose that (m1 , m2 ) ∈ / R. Then there exist l , s ∈ • l such that m1 (s ) = m2 (s ) and there exist σ and i such that mi [σl and s ∈ l1 • for no l1 ∈ σ. As mi [lmi for i = 1, 2, m1 (s ) = m2 (s ) and there exists j ∈ {1, 2} such that mj [lσl and s ∈ l1 • for no l1 ∈ σ. Seeing that m1 (s ) = m2 (s ) necessarily s ∈ / l • , hence s ∈ l1 • for no l1 ∈ lσ. Thus we obtain (m1 , m2 ) ∈ / R, reaching a contradiction. Hence, we have that (m1 , m2 ) ∈ R. The symmetric case can be proved in the same way, hence we obtain that R is an initial low-view bisimulation on N . 2. We show that (m, m ) ∈ R. If there does not exist s and l ∈ s• such that m(s) = m (s), we are done: (m, m ) ∈ R. Hence, suppose that there exist s and l ∈ s• such that m(s) = m (s). We show that ∀σ : m[σl ⇒ ∃t ∈ σ : s ∈ t• and ∀σ : m [σl ⇒ ∃t ∈ σ : s ∈ t• . As m[hm , from m(s) = m (s) we deduce that one of the following holds: • s ∈ h• . Hence s is a potentially causal place. Take a sequence σ such m[σl. We show that there exists t ∈ σ such that s ∈ t• . Two subcases can happen: - σ = hσ . As PBNID holds, s is not an active causal place. Hence, for all m ¯ ∈ [m0 and for all σ ¯ : if m[h¯ ¯ σ l then there exists t ∈ σ ¯ such that s ∈ t• . As m[σl and σ = hσ , then there exists t ∈ σ ¯ such that s ∈ t• . - σ = or σ = t σ with t = h. As s ∈ h• we obtain m(s) = 0. As m[σl and s ∈ • l, there must exist a transition t ∈ σ that produces one token in s, i.e., such that s ∈ t• . In particular, σ = .

150

R. Gorrieri and M. Vernali

Consider now a sequence σ such that m [σl. We show that there exists t ∈ σ such that s ∈ t• . As m[hm , we have that m[hσl, hence, because PBNID holds, there exists t ∈ σ such that s ∈ t• . • s ∈ • h. Hence, s is a potentially conﬂict place. Take a sequence σ such that m[σl. We show that there exists t ∈ σ such that s ∈ t• . As PBNID holds s cannot be an active conﬂict place. Hence, for all m ¯ ∈ [m0 and for all σ ¯ : if m[h ¯ and m[¯ ¯ σ l then there exists t ∈ σ ¯ such that s ∈ t• . As m[hm and m[σl, there exists t ∈ σ such that s ∈ t• . Take now a sequence σ such that m [σl. As s ∈ • h, we have m (s) = 0. As s ∈ • l from m [σl we obtain that there must exists a transition t ∈ σ producing one token in s, i.e., s ∈ t• . Theorem 8. Let N = (S, L, D, H, F, m0 ) be an elementary net system. If N is BN ID then N is P BN ID. Proof. Suppose that N is BN ID. We show that no place in N can be an active causal place or an active conﬂict place. • Suppose that s is an active causal place. Then, there exist h ∈ • s, l ∈ s• , m ∈ [m0 and σ ∈ (H ∪ L)∗ such that m[hσl and ∀t ∈ σ, s ∈ / t• . Among the markings and the transition sequences that satisfy the conditions above, take m and σ such that σ contains the minimun number of transitions in H. Two cases can happen: 1. All transitions in σ belong to L. We have that m[hm . By BNID there exists an initial low-view bisimulation on N containing the pair (m, m ). As m [σl, also m[σl. But from h ∈ • s and m[h we deduce that s ∈ / m; we also know that ∀t ∈ σ, s ∈ / t• ; hence, after the ﬁring of σ place s is still empty, contradicting the fact that m[σl. 2. There exists a high level transition in σ. Let h be the last high transition in σ. Hence, there exist σ1 , σ2 , such that σ = σ1 h σ2 and all transitions in σ2 belong to L. Thus, there exist m1 , m2 such that m[hσ1 m1 [h m2 [σ2 l. From m1 [h m2 , by BNID there exists an initial low-view bisimulation on N containing the pair (m1 , m2 ). From m2 [σ2 l, we obtain that also m1 [σ2 l, thus obtaining the ﬁring sequence m[hσ1 σ2 l, contradicting the fact that the chosen transition sequence was one with the least number of high transitions. • Suppose that s is an active conﬂict place. There there exist h ∈ s• , l ∈ s• , m ∈ [m0 and σ ∈ (H ∪ L)∗ such that m[h, m[σl and ∀t ∈ σ, s ∈ / t• . Among the markings and the transition sequences that satisfy the conditions above, take m and σ such that σ contains the minimun number of transitions in H.

On Intransitive Non-interference in Some Models of Concurrency

151

Two cases can happen: 1. All transitions in σ belong to L. We have that m[hm . By BNID there exists an initial low-view bisimulation on N containing the pair (m, m ). As m[σl, then m [σl. But from h ∈ s• and m[h we deduce that s ∈ / m , • we also know that ∀t ∈ σ, s ∈ / t ; hence, after the ﬁring of σ place s is still empty, contradicting the fact that m [σl. 2. There exists a high level transition in σ. Let h be the last high transition in σ. Hence, there exist σ1 , σ2 , such that σ = σ1 h σ2 and all transitions in σ2 belong to L. Thus, there exist m1 , m2 such that m[σ1 m1 [h m2 [σ2 l. From m1 [h m2 , by BNID there exists an initial low-view bisimulation on N containing the pair (m1 , m2 ). From m2 [σ2 l, we obtain that also m1 [σ2 l, thus obtaining the ﬁring sequence m[σ1 σ2 l, contradicting the fact that the chosen transition sequence was one with the least number of high transitions.

A Method for Security Governance, Risk, and Compliance (GRC): A Goal-Process Approach Yudistira Asnar and Fabio Massacci Department of Information Engineering and Computer Science, University of Trento, Italy [email protected]

Abstract. The Governance, Risk, and Compliance (GRC) management process for Information Security is a necessity for any software systems where important information is collected, processed, and used. To this extent, many standards for security managements at operational level exists (e.g., ITIL, ISO27K family etc). What is often missing is a process to govern security at organizational level. In this tutorial, we present a method to analyze and design security controls that capture the organizational setting of the system and where business goals and processes are the main citizen. The SI*-GRC method is a comprehensive method that is composed of i) a modeling framework based on a requirement engineering framework, with some extensions related to security & GRC concerns, such as: trust, permission, risk, and treatment, 2) a analysis process defining systematical steps in analyzing and design security controls, 3) analytical techniques to verify that certain security properties are satisfied and the risk level is acceptable, and at last 4) a CASE tool, namely the SI* Tool to support analysts in using the method. To illustrate this method, we use a running example on e-Health adapted from a real-life process in an hospital partner.

1 Introduction The last decade has seen two parallel and conflicting demands on businesses services: business processes are increasing in complexity and unpredictability, while demands for accountability, regulatory compliance and security are becoming mandatory. Thus, a structured approach to Governance, Risk and Compliance (GRC) of Information Security has become a high priority goal [1]: – Governance is the set of policies, laws, culture, and institutions that define how an organization should be managed (as opposed to the way it is actually managed); – Risk Management is coordinate activities that direct and control an organization forecasting and managing events/risks that might have a negative impact on the business; – Compliance is the act of adhering to regulations as well as corporate policies and procedures. GRC products support the achievement of compliance by automatic monitoring of controls across different applications, offers audit facilities, the prioritization of corrective actions according risk based profiles [2]. Traditional audit methods involve the review A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 152–184, 2011. c Springer-Verlag Berlin Heidelberg 2011

A Method for Security Governance, Risk, and Compliance (GRC)

153

of transactions at a given time (i.e., sampling). However, this approach does not give sufficient assurance over the level of security and compliance of business processes over an entire period, and more advanced GRC products (e.g. SAP and Oracle Financials) offer Continuous Controls Monitoring techniques. Tools such as Audit Command Language [3] are becoming increasingly popular as cheap and flexible options for the implementation of continuous controls. GRC solutions for technical systems are well understood (even if not thoroughly deployed) and a number of textbooks clearly identify the desired security properties. For example in [4] we find confidentiality, availability, and integrity of computer-related assets, and it also includes possession/ownership, utility, authenticity, non-repudiation, and privacy of information. In addition to those qualities, compliance with specific regulations might require some additional qualities, such as accountability (SoX, Basel II), assurance (SAS 70, CMMI, CommonCriteria), etc. However, it is becoming increasingly evident that managing security and GRC of a system is not only a matter of technical solutions. Most of the state of practices (e.g., ISACA RiskIT[5], COBIT [6]) define risks as the obstruction originated from the availability of resources (e.g., documents, servers, production machines). However, the availability of resources is not sufficient to guarantee the security or GRC of a system, because sometime a disruption is originated from the process or even the objective level of the system. In other words, security violations might be originated from any layer of the organization (i.e., strategic, process, infrastructure, information). The focus of this work is to consider as a “system”, a more complex construction and namely a socio-technical system [7] where both social and technical aspects, and its relations are considered as an integrated whole. The method presented in this tutorial distills a number of research articles and empirical studies of the University of Trento and a number of colleagues from industry and academia (See the acknowledgments). It analyzes security and GRC aspects of a socio-technical system and focuses on the organization’s goals and processes of the system. The method as a whole [8] assumes that a security initiative is a long-life program that an organization needs to carry on (i.e., not a project-based initiative) based on the Deming management cycle of plan-docheck-act [9]. In other words, one needs to consider how to monitor and continuously improve the implemented controls in the system. Here we put the emphasis on the Plan phase where analysts model the target system and analyze the security GRC issues and design security mechanisms to manage the excess risk. Table1 shows the basic checklist for the method. In the following sections, we demonstrate how the SI*-GRC method is used to analyze and design security and GRC concerns of the information system used in a rich scenario from e-Health (§2). We then we provide an overview of the method describing who the participating roles are, what the steps are, and what is contained in the models of the method (§3). We explain in details how the method support the Security and GRC of the system during the Plan phase. We start by describing how to capture a target of analysis the “SI*-GRC” way (§4), then how to analyze security and GRC concerns (§5), and finally how to define the required controls (§6). We discuss briefly the technique for in-vivo monitoring and update of the controls (§7) and conclude.

154

Y. Asnar and F. Massacci Table 1. SI*-GRC Method Checklist – The Role Identification and Organizational Set-up specifies the roles and responsibilities for the SI*-GRC team, the business process owners and the IT management. – The Specification of the Target of Analysis captures at high level the socio-technical system with the key relevant information. The analyst should quickly identify the following aspects: • Actor Models capture relevant actors (i.e., social and technical ones) their organizational structure. • Goal Models capture the objectives of the organizations by spelling out the intentions and capability of identified actors and the functional interdependency between actors and goals. • Process and Services Maps identify the processes which achieve the organization’s goals. In this model, one can see the relation between how a goal is being carried out by a business process, and how the business process is supported by series of business services and/or resources. An organization typically has formalized its business process in some formalization (e.g., BPMN, Flow-chart, UML Activity Diagram). This formalization should be re-used as much as possible. – The Security and Risk Analysis captures which aspects of the system are not protected • Business Continuity Critical Points Identification specifies the business processes and actors whose commitment is essential to achieve the high-level goals of the organization. • Unauthorized Processing Identification identifies existing permissions (if known) and how they are delegated to other actors. Provide as output potential unlawful processing and over-entitlement scenarios. • Trusted Computing Base Identification identifies the trust relationships between actors in possibly making references to specific goals. Provided as output the boundaries of the TCB that can be sources of potential failures of reliability and misuses of permission in terms of scenarios for failures of the organizational goals. • Unwanted Scenario Identification supports the analysts in identifying threat, events and gives guidance how to structure them at resource, process or strategic level. • Risk Assessment estimates the risk level of identified risks and specifies which risks will be treated or accepted – Control Analysis identifies the control mechanisms put in place to address the risks. • Control Goal Model specifies and elaborate the control goals in order to cover most (if not all) risks, with appropriate and precise measures. • The Control Processes and Services Map identifies the control processes that achieve the control goals.

2 The Running Example Scenario The scenarios used in this article are adopted from the drug reimbursement process in a major hospital in Italy [10]. This process is only applied for a specific set of drugs (called File F drugs) to be delivered to outpatients, patients in “day hospital” regimen, and patients during the last day of hospitalization. This list includes innovative and expensive drugs, caring chronic diseases with a high social impact (tumors, multiple sclerosis, HIV, etc.).

A Method for Security Governance, Risk, and Compliance (GRC)

155

The economic aspect of this business process is particularly relevant as the regional wide expenditure for these drugs is raising year by year (in +12% in 2006, +10% in 2008), bringing to an augmented attention of the Health Authority toward the existence of an effective system of controls over the dispensations of these drugs1. As a consequence, this business process is subject to significant security and compliance requirements: all regulations2 of the File F Regulatory Framework have to be respected by the hospital in order to obtain the reimbursement from the regional health authority. Moreover, since the drug reimbursement process includes the processing of personal and sensitive data (the recipients of the drugs), it must also satisfy the regulations on privacy from the Legislative Decree no. 196 of 30.06.2003 called ”personal data protection code”, in addition to well-established security requirements (e.g., nonrepudiation of a drug dispensation, availability of the prescription data, integrity of the drug dispensation process) In nutshell, the Drug Reimbursement process is a mechanism that allows the hospitals to refund the costs of the dispensed File F drugs. The process is composed of three macro-phases: Drugs Prescription, Drugs Dispensation, Generation & Sending File F reports to the Health Authority. In the Drugs Prescription phase, a prescription is done by a doctor to a patient using an IT system. The doctor selects File F drugs to administer to the patient and of their posology and quantity. The selection of the drugs can be done by copying the past prescriptions (in case of a chronic diseases) or also choosing them from a complete list of File F drugs. Finally, a prescription sheet is produced containing the patient’s name and other personal data (e.g. the Fiscal Code), and also sensitive data (e.g., the patient disease, the prescribed drugs and their quantities/posologies). In the Drug Dispensation phase, dispensers (doctors or nurses) dispense File F drugs following a prescription given to a patient. At this phase the dispensers need to indicate which drugs, from the prescription, are being dispensed, since ones might dispense a part of the prescription for some reasons (e.g., the drug is currently unavailable). In some occasion, some drugs appear to be unavailable in the computer system though in reality they are available in the hospital ward drug stock. In this case, the dispensers must dispense the drugs because it is critical for the patient’s safety. Finally, a dispensation sheet is printed and needs to be signed by the dispensers and the patients. This sheet contains similar information as the prescription except the patient disease. The Report Generation phase is constituted two parts: 1) the generation of the File F reports to be sent to the Health Authority; and 2) the sending of the File F reports to the Health Authority. In the first part, the Accounting Office retrieves all the data about the dispensed drugs in a month, checks the data, produces the File F reports, and sends them to the Hospital Medical Director. In the second part, an operator from the 1

2

The Regional Directive n. 5743 - 31.10.2007 provides indications to optimize and improve the process design about prescription/dispensation/accounting of File F drugs, and the Regional Directive VIII/1375 - 14.12.2005 stresses the priority to implement actions towards the verification of the appropriateness of the use of File F drugs. To have an idea, without mentioning privacy requirements, the File F mechanism was instituted by the regional circular 17/SAN 3.4.1997, and successively has been emended by the Circular No.5/SAN 30.1.2004, by the Circular No.45/SAN 23.12.2004, by the Note 30.11.2007 H1.2007.0050480, by the Note 27.3.2008 H1.2008.0012810 and by the Note 04.12.2008 H1.2008.0044229.

156

Y. Asnar and F. Massacci

Medical Director verifies the File F reports and sends them to the Health Authority supervised by the head of department. All the operations are assisted by the IT system and the File F reports have to be sent to the Health Authority following a special guideline, namely the “debito informativo” IT channel, that permits to send and receive information through common e-mail clients with additional security mechanisms that guarantee the confidentiality of the exchanged data (encryption of the message and of the attachments, digital signature, etc.). It is another security &compliance requirement posed in the information system. For technical details, we assume the drug dispensation process is supported by an information system implemented in terms of services, in SOA [11] sense. Still, we believe this method is applicable for other technical architecture (e.g., object-oriented, web-based)

3 Overview of the Method This section illustrate the SI*-GRC method at high-level, starting from the organizational context to be prepared before using the method, the conceptual modeling used in the SI*-GRC, namely the SI* modeling framework, and finally illustrate the SI*-GRC process that will be detailed in the following sections. 3.1 Roles and Organizational Set-Up In order to manage security and GRC, the preliminary organizational steps are essential to ensure that security & compliance is achieved effectively across the organization and in a timely manner. More importantly, they aim to obtain management buy-in, to clarify the ownership of a process & the responsibility of controls, to align the organization’s strategic objectives with regulatory requirements, and above all to maintain security across system evolution. Often such aspect is usually neglected in research papers on information system engineering (e.g., [11,12]) which directly focuses on the functional design of the system. The preparation steps are the following ones: 1. The Identification or set-up a GRC council ensures the periodic review of the organization’s security policies and procedures; the level of compliance with these policies and procedures; the level of exposure of key information assets during implementation; the ongoing improvement of existing control implementations (i.e., control processes and indicators); and the availability of sufficient resources for SI*-GRC. In other words, these actors are essential staff members during the Check and Act phase. 2. The Definition of roles and responsibilities for the SI*-GRC team. Actors that make up the analysis-design Council: (i) management (e.g., members of the Board of Directors, the Chief Executive Officer (CEO), and Shareholders); (ii) business process owners (key personnel that are responsible for the daily operations of the concerned business process); and (iii) IT management (those responsible for the governance, daily operations and maintenance of the IT infrastructure).

A Method for Security Governance, Risk, and Compliance (GRC)

157

The followings are actors involved in the design of SI*-GRC controls: Business Analysts are those responsible for the analysis of the organization’s business goals & processes; Risk Analysts are responsible to assess the level of risk of some threats to the business, and decide whether they are acceptable or not; Security Analysts are responsible for the design and implementation of actions aimed at providing a reasonable level of assurance concerning compliance with security policies. Security analysts generally make part of IT management; SI*-GRC Analysts are proficient with the method that can coordinate the analysis and design process of control processes and indicators. Moreover, the SI*-GRC analysts are responsible for ensuring that the SI*-GRC controls and indicators are implemented according to the designs and provides an appropriate/improved level of assurance concerning the achievement of strategic objectives.3 Note the performance of these actors is critical for the success of security & GRC initiatives. 3. The creation of a timeline for the SI*-GRC deployment. indicating the key goals and milestones of the SI*-GRC application, as well as those responsible for each milestone. The timeline should list all tasks to be carried out during the SI*-GRC application including decision points. 3.2 Process Overview The SI*-GRC method includes a description of systematic steps to support the PlanDo-Check-Act (PDCA) of the Deming Cycle [9] to ensure the continuous monitoring and improvement of the secure system as depicted in Figure 1. The Plan phase aims at capturing and analyzing the problems in the target system and at the end results in design of necessary controls. The Do phase concerns on the implementation up to the execution of such controls. The Check phase aims at reviewing the implemented controls and the existing organization context. Finally, analysts need to re-act upon the review results to improve the security and GRC of the system at the Act phase. These reactions must be planned before they are implemented in the system. The general steps of the Plan phase are depicted in Figure 2, where includes 1. Target System Definition where an information system is formalized in the context of an organization; 2. Security and Risk Analysis where and it captures and analyzes security and GRC aspects (i.e., including trust and permission) and then assess the possible risks; 3. Control Analysis where some necessary controls are formalized including their quantitative indicators indicating their effectiveness and performance. The Target System Modeling step covers three modeling activities. Firstly, the Actor Modeling captures actors (i.e., human or technical) involve in the system including their structure in the organization. Analysts then capture and analyze the strategic interests in 3

In this work, we assume an improvement at the information security, governance, compliance, and risk management will lead to the better assurance of the business.

158

Y. Asnar and F. Massacci

• ! * ! • *+ &!* • *+&!

• '! • !*! • )&!!

SI-GRC Method

• ') "! • +) " ( ! • !& (!

• '( /"' ! & ! • '(&!& $ &!*/- .

Fig. 1. The Plan-Do-Check-Act Cycle of the SI*-GRC Method

the Goal Modeling. Moreover, analysts need to capture strategic rationales and dependencies of an actor [13] and each capability. To achieve those interests, an organization defines systematic steps (i.e., a business process). In our method, we use BPMN [14]) to capture the business process though we believe other process models (e.g., YAWL [15]) will also be applicable. The Processes and Services Mapping indicates which process satisfies which goal. Moreover, in the business process one must indicate which resources (i.e., data object in BPMN or underlying business services [11]) involve in the execution of business process. At Security Analysis step, it composed of four activities: 1. Unwanted Scenario Identification which identifies potential events 2. the Unauthorized Processing Identification models actors’ entitlements and the delegation permission to another actor, the 3. Trusted Computing Base Identification where capture the trust relationship between actors involved in the system, 4. the Risk Assessment needs to measure the level of risk for the high-level goals of the target system after considering its trust and permission models, and finally decides which risks that are unacceptable therefore require some controls to mitigate them. The Control Analysis aims at analyzing further the control mechanisms. It starts from the Control Goal Establishment describing the objectives of the controls. The Control Goal Analysis specifies and elaborate the established control goal so that they cover most (if not all) risks, accurate, and have a clear-cut definition of their fulfillment. At last, the analysts need Control Process and Services Map which details the means to achieve a control goal. This map is very similar to the one used for business process and

A Method for Security Governance, Risk, and Compliance (GRC)

• • •

• •

• • •

159

• •

•

Fig. 2. The SI*-GRC Process Overview for the Plan Phase

business goals excepts that it aims at protecting the business process from the excess risks by achieving the control goals. The final outcomes of the SI*-GRC for the plan phase are a list of detailed control mechanisms composed of control goals describing the state-of-affairs to be achieved/protected, a process description of how those goals are implemented in term of control processes, and a series of indicators indicating their effectiveness and performance, and the organization’s trust and permission model. Example 1. Based on the scenario given in Section 2. – Control Goal - File F reports must respect the privacy preference defined by a patient; – Control Process - Anonymize the name of patient that requests to, after File F reports are generated; – Indicators • The number of privacy litigation coming from patients – effectiveness; • The number of File F records not being anonymized where the patient requests to be anonymized – performance; • How often the anonymization process is executed – performance; • How many records have been anonymized – performance. – Trust Model - Accounting Officers trust that dispensers will be honest in inputting dispensation data; – Permission Model - Doctors have permissions to access a patient’s medical info. More details will follow in the incoming sections (i.e., Section 4-6); and for other phases (i.e., Do, Check, and Act, we only illustrate the usages of the method in-vivo of the information system (Section 7). 3.3 SI* Modeling Framework The SI*-GRC method uses the SI* modeling framework [16] for most of its modeling except the modeling of business process where uses BPMN. The SI* framework is a

160

Y. Asnar and F. Massacci

contribution

delegatum/ trustum

decomposition

Event

contribution

Delegation/ Trust

has

Impact

delegatee/ trustee

delegator/trustor impacting

Business Object

Actor

alleviation

Goal

Process mean-end

Resource mean-end decomposition

decomposition

Control Goal

Control Process

Fig. 3. The SI* Conceptual Modeling

modeling framework extending the i* framework [13] to support security requirement analysis. In Figure 3, the conceptual model of this modeling framework is based on basic concepts representing requirement analysis Actor is an autonomous entity that has its own intentions (human and software), capabilities, and entitlements. This concept is then realized as; Goal is a state-of-affair that an actor intends to achieve; Process is a means to fulfill a goal, to furnish a resource; Resource is an artifact that is consumed/produced by a process; Event is an uncertain circumstance that affects a goal satisfaction (in/directly); Trust captures a believe of the capability/honesty of one actor (trustor) to another actor (trustee) in fulfilling/using a business object (trustum); Delegation depicts a transfer of the responsibility/right from one actor (delegator) to another actor (delegatee) in fulfilling/using a business object (delegatum). The Appendix A lists all constructs (i.e., concepts and relation) in the SI* Framework including the graphical diagrams and DLV predicates. Details relations between constructs will be explained and illustrated along the analysis and design process. To avoid confusion, we use an italic text to indicate a SI*-GRC basic concepts and a sans-serif text for the one related to the scenario.

4 Target System Definition The first step of the SI*-GRC process is modeling the target system where one needs to analysts its security and GRC concerns. In some literature in the organization behavior [17], management information system [18], and enterprise architecture [19], one can organize an information system in an enterprise into three levels of abstraction as depicted in Figure 4. Essentially, an information system always serves the objectives of the stakeholders (i.e., including the organization’s shareholders). These objectives are implemented by series of business processes that are inter-related one to another. To

Objective

A Method for Security Governance, Risk, and Compliance (GRC)

Stakeholder Goals

161

Company Vision-Mission

Business Objective BO 1

BO 2

Patient

Go for Medical ExaminationGo for Patient Medical Personal Examination Information Go for

Receive Drugs

Process

Patient

+

+

Medical Personal Examination Information Go for Medical Personal Examination Information

Write Prescription

Doctor

+

Receive Drugs Dispensation Receive Sheet Drugs + Dispensation Receive Sheet Drugs

Patient

Doctor

+

Dispensation Sheet

Write

Personal Information Prescription Drugs Write Prescription Prescription + Drugs Write

DoctorPrescription +

Dispensation Sheet

Doctor Paramedics

Prescription Dispense + Prescription Drugs Drugs

Paramedics Receive

Prescription

+

Receive Prescription

Dispense Prescription Drugs Drugs

Paramedics

+

Paramedics Receive

Prescription Receive Prescription

Dispense Drugs +

Dispense Drugs

Resource

+

Phsical & Logical Resource

Software

Application Documents Server

Fig. 4. Three Layer Model of an Information System in an Enterprise

execute such processes, participating actor might need some resources either physical or logical ones. To capture such settings, analysts need to define the Target System in terms of the structure of actors in the organization, their interests and capabilities, and interdependencies between them. Finally, we need to map which process is associated to which stakeholders’ goal. The overall process of this phase is described in Figure 5. Note that in the SI*-GRC method analysts might choose any approach for a Process Modeling in the organization, such as Event-Driven [20], Workflow (e.g., BPMN [14], YAWL [15]), UML-based [21], Declarative BPs [22], or Case-base [23]. 4.1 Actor Modeling First task towards modeling the target system is the identification of the key social actors whose intentions need to be fulfilled by the system or are mostly involved in the system. Depending on the level of analysis one might consider and analyze strategic interests of social actors[24] or technical actors (e.g., email service, data management agent) and analyze the social actors’ goals that are delegated to the technical actors. For a comprehensive analysis we need to consider most actors involve in all phases of the system existence (if relevant) and in particular

162

Y. Asnar and F. Massacci

Actor Modeling

Process Modeling

Goal Modeling

Process Mapping

Fig. 5. The Process Model of the Target System Definition Step

– subject actors, whose information will be stored/processed by the information system - e.g., Patient; – usage actors will use/consume the information produced/managed by the information system - e.g.,: Pharmacy, the Health Authority, the Auditor, the Accreditation Office (e.g., Joint Commission); – system actors who are part of the information system to use and maintain it - e.g., the IT departments, the Risk & Compliance Office, business analysts, and all actors in Figure 6; Analysts need to be not confusing between role or agent. A role is an abstract characterization of intentional entity’s behaviors, and an agent is an intentional entity with a concrete manifestation. In other words, one can distinguish an agent as an object level entity that has one-to-one correspondence with an entity in the system world; while role as the class actor needs to be distinguished into role. Example 2. In our scenario, the Hospital “H”, the healthcare authority, Dr. Dave, Alice, and the Head of Hospital are considered as agent; while doctor, nurse, dispenser, and accounting officer as role. Given a series of actors, analysts need to structure them using the following relations: 1. play indicates an agent that plays a particular role; 2. is part of indicates an actor that is part/member/composed of another actor; 3. is a captures a specialization relation between two actors (i.e., role-to-role or agentto-agent); 4. supervise captures a supervision relation between two actors (i.e., role-role or agentagent). Example 3. In Figure 6, we model the Hospital “H” that is composed of a series of operational unit (e.g., Cardiology, Gynecology, etc.), an Accounting Office and a Medical Director; leads by a head of hospital Mr. Henri. Dr Dave and Dr Dubois play a role of doctor, while Alice plays a role as nurse and dispenser. All these agents are member of (is part of ) the Cardiology Department. Since Doctor and Nurse are (is a) dispenser, all these can act as a dispenser in the Cardiology Department. These actors are essential to set up the scope of the following modeling activities. Note in Example 3, we only consider actors that are part of the drug reimbursement system.

A Method for Security Governance, Risk, and Compliance (GRC)

163

Play

Doctor Dr. Dave

Hospital H

Head of Operational Unit

Is Part Of

Is Part Of Is Part Of

Dispenser

Operational Unit

Accounting Office

Henri

Is Part Of

Is Part Of

Play

Nurse

Head of Med. Director

Is Part Of

Is A

Play

Medical Director

Is Part Of Is Part Of

Alice

Is A

Is Part Of

Is A

Dr. Dubois

Head of Hospital

Is Part Of

Play Play

Is Part Of

Accounting Officer

Medical Director Officer

Play

Is A

Play

Is Part Of

Harry

Is Part Of

Is Part Of

Cardiology Dept.

Andrea

Mary

Fig. 6. The Actor Model in the Hospital “H”

4.2 Goal Modeling For a given actor model, analysts need to identify the strategic interests (goals or objectives) that the actors intend to achieve and model using a Request relation depicted in Figure 7. These cover responsibilities, motivations, and intentions of actors in the organization and business objectives of the organization. Example 4. The patient needs to get medical service and the hospital interest to obtain the reimbursement of dispensed drugs. In particular to the context of security and GRC, one considers also the high level norms imposed by the organization information policy or by the standardization body or by the regulator. Example 5. The hospital “H” management intends to keep the all medical records in two modalities (e.g., paper and digital) for the next 3 years; The Joint Commission (a standardization bodies) requires an hospital to initiate and maintain all clinical records of every patient assessed or treated (MCI.19) - completeness, sufficiency, integrity, and accountability; while MCI.10 & MCI.11 require information privacy & security are maintained; – The health authority, in Regional Circular No.5/SAN 30-1-2004, obliges an hospital to follow a strict guideline to send a drug reimbursement report in to a defined format (e.g., three files - personal data, drug info, drug cost) delivered via a special channel, namely the “debito informativo” channel. Often an actor cannot achieve its goal by itself therefore one might 1) appoint another actor to full the goal using a delegation of execution, or 2) decompose the goal further into more detail/precise subgoals using an AND decomposition relation and assign

164

Y. Asnar and F. Massacci

Obtain the JCI Accrediation

R

Patient

De

Joint Comission

The Healthcare Auhtority

De

Hospital H

De R

Obtain the drug reimbursement

R

De

Get medical service

Get medical service

Provide medical service

Treat Patient

Operational Unit

Record all reimbursed drugs

Deliver the report

Generate the drug reimbursement report

AND De

Treat Patient

Send the report via the debito informativo

Deliver the report in person

AND

Register patient

Examine patient

De

AND

De De

Dispense drugs

De

Prescribe therapy/drugs

De

Accounting Office

Doctor

Generate the drug reimbursement report AND

Dispenser

Medical Director

De

Review the report

Produce the report

Retrieve the dispensed drug data

Fig. 7. The Goal Model of the Drug Reimbursement System

parts of them to other actors. AND-decomposition indicates that all subgoals must be fulfilled by the system in order to achieve the main goal. Sometimes it is useful to specify alternatives to fulfill a goal using several means-end relations indicating alternatives to fulfill the goal. This decomposition is rarely used in practice during an audit or assessment step of existing security mechanisms. However, it is mostly used during the plan phase of a new security mechanism. Example 6. To obtain a drug reimbursement, the hospital “H” needs to do all these (AND-decomposition): record all reimbursed drugs, produce the drug reimbursement report, and deliver the report. To achieve the goal of deliver the report, the hospital can send the report via the “debito informativo” channel managed by the Healthcare Authority or deliver the report in person This analysis is an iterative process, where sub goals need to be refined further with AND-decomposition/means-end relations or being delegated its fulfillment to another actor. This decomposition stops for an actor when all leaf goals (i.e., the lowest subgoals) are have being delegated to other actors via delegation execution or because the actor can fulfill the subgoal by itself.

A Method for Security Governance, Risk, and Compliance (GRC)

165

Later in the analysis of the target system we might refine further this capability by specifying a business process that achieves that. Otherwise, we do not care how the goal is actually achieved (but we know it will be achieved) we can simply mark this assumption in the model. Notice that this is an important assumption that would need to be validated at some point as it enters in the trust model of the system. Example 7. For achieving the goal of generate the drug reimbursement report, the hospital delegates it to the accounting office. However, the accounting office is only capable to retrieve the dispensed drug data and produce the report, but it still needs to delegate further a part of the goal, namely the subgoal review the report to the medical director. The final outcome of this modeling is a goal model, as in Figure 7, which captures a network of actors’ dependencies on fulfilling their goals and the rationales of such goals. 4.3 Process and Services Mapping As mentioned earlier, SI*-GRC allows analysts to use an existing formalization for the process modeling The aim of this modeling to indicate which process, a part of the business process, is a means to fulfill actors’ goals. In the goal modeling, analysts have identified the leaf-goals that actors are able to fulfill; and in the process mapping, analysts map those leaf-goals to the processes using means-end relations in which their executions will fulfill the leaf-goals as depicted in Figure 8.

GOAL MODEL

Doctor

Prescribe therapy/drugs AND

Obtain patient medical record

Prepare prescription

Finalize the prescription sheet

BPMN MODEL Doctor

Select Drugs Medical Info

+

Nurse/ Doctor

Case Details

Retrieve Patient Medical & Personal Info

Identify Patient

Print and Sign Prescription Sheet

Old Prescription

Archive Prescription Sheet

Prescription

Specify Privacy Preferences

Fig. 8. The Process Mapping in the Prescription Phase of the Drug Reimbursement Process

166

Y. Asnar and F. Massacci

Example 8. To achieve the goal prescribe therapy/drugs, a doctor needs to fulfill three subgoals as depicted in Figure 8. To fulfill obtain patient medical record, a doctor/nurse needs to perform two activities retrieve patient medical & personal info and identify patient consecutively. Moreover, those two activities can also be performed by a nurse. By means in this mapping, analysts indirectly define which actors have capabilities to perform particular processes/activities (including sub-process or task in BPMN) to fulfill a goal. Moreover, it can also infer which actors can provide/require particular resources (i.e., data object in BPMN) from which activity produces/consumes some resources. In a SI* model, these capabilities are captured with Provide relations. Example 9. Based on Figure 8, we can infer the capabilities of a doctor and a nurse that can perform the activity of retrieve patient medical & personal info and provide the resource of medical info. As mentioned before, sometime analysts might prefer to indicate actors’ capabilities at the high-level (i.e., goal) without caring how it is actually achieved. As we shall see later this is an important trust assumption. Example 10. A doctor can provide the fulfillment of the goal examine patient. From Figure 8, the complete capabilities of each actor from are listed in Table 2. In SI*-GRC resource is not solely data-related artifacts, but it expands into other physical and logical resource, such as: prescription & dispensation sheet, underlying business web services that supports the execution of some activity, softwares used to process some data while performing some activity, and infrastructures used to collect, store, and transmit the data during the process execution. At the end of this mapping, analysts have completed a three-layer model of the enterprise information system as depicted Figure 4. Table 2. Actors’ Capabilities on the Drug Reimbursement System Capability - Provide relation Performing an Activity/Process • Patient Medical & Personal Info • Identify Patient • Specify Privacy Preferences • Select Drugs • Print & Sign Prescription Sheet • Archive Prescription Sheet Providing a Resource • Medical Info • Case Details • Old Prescription • Prescription

Doctor Nurse X X X X X X

X X X -

X X X

X -

A Method for Security Governance, Risk, and Compliance (GRC)

167

5 Security and Risk Analysis After defining the target system, analysts need to analyze the security and GRC concerns of the system. In SI*-GRC we consider a whole system as an asset, something valuable, therefore we need to protect it from unacceptable events that might harm the security and compliance requirements of the system. This step aims to analyze security and GRC concerns of the system; it starts by identifying critical points that can disrupt the business continuity. Analysts then identify the ownership of processes and resources in the system, and delegation its permission to other actors, and defining the trust relations between actors exist within the organization. Given such settings, we might identify several unwanted events (i.e., threats, failures, errors) that can compromise the security & GRC of the system. To decide which unwanted events need to be controlled, analysts need to estimate the level of risk of such events, and finally take into account the organization policies on controlling them (e.g., risk tolerance, ethics, IT architecture policy). 5.1 Identification of Business Continuity Critical Points The outcome of the target analysis phase is a complex web of relations between actors and business objects. The preliminary analysis of this model is to ensure that all critical points for business continuity have been identified. Intuitively, a business continuity disruption happens when an intention to fulfill a goal (or a part of goal) is passed across actors using delegations of execution relations and it ends to an actor who does not have explicitly the capability to fulfill the goal (or part of it) in term of performing a process or providing a resource. This property can be easily visualized on the model as a path that does not end with either a means-end relation into a business process or with an atomic provide relation between the last actor in-charge and the goal. Example 11. In Figure 7, the Hospital “H” need to delegates the goal register patient to the Operation Unit. Unfortunately, that actor, including any actor part of it, does not have necessary capabilities to fulfill the goal, and not even pass the goal to another actor. This potential disruption can be resolved by reorganizing the delegation of execution to end with a specific business process or by allocating to actor that are explicitly declared to be capable of achieving the desired goal (i.e., outsourcing might be the least thing that an organization can do, when none in the system is capable). In the former case we have specified how the goal will be actually achieved in the system and the business process itself is a critical point for business continuity. In the latter case we have only specified that an actor is in charge of this goal which therefore makes the behavior of this actor critical for the overall achievement. This case can be later analyzed by considering possible unwanted scenarios that lead to the disruption. Analytical techniques are available with the SI* tool. The papers [16,25] describe the formal properties in more details.

168

Y. Asnar and F. Massacci

Dp

O

Hospital H

Patient O

Medical Info

O Prescription Dp

Dp

Doctor Old Prescription

O

Prescription Process +

Dp

Nurse

Fig. 9. The Permission Model in the Prescription Phase of the Drug Reimbursement Process

5.2 Unauthorized Processing Identification So far we have modeled the transfer of responsibility via delegation execution due to actors’ capabilities. However, this concept does not capture the notion of ownership of a particular process or resource. Sometime to execute a process or to access a resource, an actor needs to have a permission from the owner of process/resource. In this modeling phase, we capture the notion of ownership of process/resource and the notion transfer of rights on a business object from one actor to another using a delegation permission relation. The Owner relation depicts a relation between an actor to a process/resource that indicates the actor has full authority concerning the execution of a process or the access of a resource. 4 Example 12. The patient owns the resource of its medical info; and the Hospital “H” has a full authority for all activities in the prescription process depicted in Figure 8. Typically, defining the owner of a resource is simple because a resource is tangible and often some policies/regulations already define the notion of ownership (e.g., EU Data Protection Directive-Directive 95/46/EC or the company property policy). However, it is less trivial for the process ownership. However, in many situation the owner of such process/resource needs to delegate the permission to another actor because the owner does not have capability to execute the process or the other actor needs to access the resource/to execute the process to fulfill its responsibilities. For such situation, the SI* models the action of giving a permission using a delegation permission relation. Example 13. The Hospital “H” delegates the permission to doctor on executing activities in the prescription process (as in Figure 8) because the hospital “H” needs the doctor to fulfill the goal prescribe therapy/drugs that at the end fulfills the top level goal provide medical service. The final result of this modeling activity is represented in term of a SI* model as depicted in Figure 9. This model can then be used to identify more precisely two possible threats which are very important for compliance: 4

Process Owner is defined as an actor that is held accountable and responsible on the performance and improvement of a defined process (including its subprocesses) in http://www. gao.gov/special.pubs/bprag/bprgloss.htm

A Method for Security Governance, Risk, and Compliance (GRC)

169

– lack of Authorization (or unlawful processing) is present when an actor has been assigned a process or managing a resource without a proper authorization path stemming from the owner. This aspect is particularly critical when demonstrating compliance with privacy legislation; Example 14. In Figure 9, the doctor does not have authorization to access the medical info that is required to perform the prescription process – Over-entitlement when an actor has been delegated the permission to access a resource or to execute a process but the latter is not required to achieve the goals assigned to the actor. At minimum this might be a violation of the minimal disclosure principle for the compliance with privacy legislation or might be the source of more serious troubles if the actor can potentially misbehave (e.g., fraud, internaltrading). Example 15. In Figure 9, the nurse have permissions to execute all activities in the prescription process, though in practice the nurse only responsible to perform the retrieve patient medical & personal info activity and the identify patient activity. These properties can be easily visualized on the model as paths. Intuitively, a unlawful processing for a key business objectives is present when there is a path across decompositions and delegations of execution that arrives to an actor. Yet this processing actor, in order to achieve its delegated subgoal make use of some process or some resource whose owner has not delegated the authorization directly or indirectly to the processing actor. This latter properties is simply the absence of some path from the owner to the processor. Also in this case analytical techniques are available with the SI* tool vulnerabilities and threats of a SI* model formalized in Answer-Set Programming (ASP) [16,25]. These potential threats should then be considered in the final mitigation reports because both threats introduce various type risks to the system’s security, governance, and compliance. 5.3 Trusted Computing Base Identification Trust, in SI*-GRC method, captures a social relationship that indicates the belief of one actor (trustor) on another actor (trustee) capability or honesty. The SI*-GRC method distinguishes two types of trust relation: – Trust of execution represents the trustor’s expectations concerning the ability of the trustee accomplishes the trustum (i.e., achieving a goal, executing a process, delivering a resource); Example 16. The operational unit trusts the doctor on fulfilling the goal of prescribe therapy/drugs – Trust of permission models the trustor’s expectations that the trustee is honest/does not misuse the permission on the trustum (i.e., fulfillment of a goal, execution of a process, access to a resource). In other words, by trusting in permission, the trustor believes that the trustee will not use the given permission for accomplishing a purpose different from the one for which the permission has been granted.

170

Y. Asnar and F. Massacci

Te

Prescribe therapy/drugs

Te

Operational Unit

Doctor Tp Retrive patient medical & perosnal info

Play

Tp

Nurse

Play Dave

De

Retrive patient medical & perosnal info

De Alice

Se Se

Fig. 10. Trust Model in the Prescription Phase

Example 17. The doctor believes that the nurse will not misuse the permission to retrieve patient medical and personal info A frequent error in this analysis phase is to assume that each delegation of execution step has to be “completed” by adding the corresponding trust relationship. However, such situation is not always be the appropriate because we might need to delegate actions to people we don’t trust. Example 18. Dr. Dave might not trust Alice because of some personal experience. However, he needs to delegate the goal of retrieve patient medical record because he needs to do other urgent responsibilities and only Alice is available to help. A more frequent example at organizational level is that Example 19. company “X” is known to be unreliable from the point of view of Henri (Head of the Hospital “H”), the hospital “H” needs to archive their prescriptions with the company “X” because they have won a county-wide procurement contracts for all hospitals on the region. Once the model has been specified it can be used for a precise analysis of boundary of the (un)trusted computing base at organizational level: – Potential Unreliability might happen when an actor has been assigned a goal but is not trusted to achieve it. This might generate a potential cascading failure for some of the key goals of the actor who delegated this responsibility. Example 20. Since in Figure 7 the patient might perceived at risk because it does not trust the Hospital “H” to fulfill its goal get medical service – Potential Misuse when an actor has been delegated an authorization but is not trusted not to misuse it.

A Method for Security Governance, Risk, and Compliance (GRC)

171

Example 21. From Figure 9 the Hospital “H” believes a potential misuse on the permission on executing the prescription process given to the Nurse Also in this case analytical techniques are available with the SI* tool vulnerabilities and threats of a SI* model formalized in Answer-Set Programming (ASP) [16,25]. These potential failures indicate some critical points in the trusted computing base. The trusting computing base might be too narrow with respect to our expectations in terms of successfully achieving the goals of the target systems and preventing misuses. At this point we have two alternatives: – identify possible scenarios on a breach of trust for the critical actor and their critical goals (following the steps detailed in the next section and identify possible countermeasures); – or extend our trust models to incorporate the critical actors in our trusted computing base. This last step should not be interpreted as “we just add the trust link in the model” (albeit this is the most common errors that students do). Rather this means that we put in place alternatives techniques for trust building such as those based on legal measures as identified in [26]. Example 22. To extend the trusted computing base in Example 19, Henri includes in the service contract a clause about compensation for punitive damages due to disclosure of the prescription information by the Company “X”. 5.4 Unwanted Scenarios Identification The next step is to identify unwanted events that might disrupt the system in term of its security & GRC relevant properties (e.g., confidentiality, integrity, availability, authenticity, privacy, etc.). In this tutorial, we emphasize on events with negative impact, and a more comprehensive classification can be found in [27]. Looking at the architecture of an enterprise information system in Figure 4 one might consider events that might risk the system into three classes: – Resource level - events that occur because some disruptions at the resource level; Example 23. The Archive service is not functioning so that it cannot archive the prescription sheet that surely disrupts the continuity of the prescription process. – Process level - events that occur because the business process has been performed different from its design; Example 24. The Dr. Dave is too busy to print & sign the prescription sheet therefore he asks Alice to do so. This scenario might compromise the goal maintain audit trail of prescription that is critical to get an accreditation from the Joint Commission. – Objective level - events that prevent the state-of-affairs that the system intends to fulfill.

172

Y. Asnar and F. Massacci

Objective Level Protect patient's privacy

Hospital H Ensure accountability of the medical info

Doctor

AND

Remove senstive patient's privacy

AND

Finalize the prescription sheet

Maintain Audit Trail of Prescription

--

Privacy Leakage from Alliance Partner

...

Process Level Specify Privacy Preference

...

Print and Sign Prescription Sheet

Archive Prescription Sheet

Remove Patient Name in the Report

...

--

Resource Level Prescription

Unauthorized Prescription Signature

Old Prescription

--

Archive Service Disruption

Archive Service

Fig. 11. Unwanted Scenarios in the Drug Reimbursement System

Example 25. A new business policy of making a Hospital Alliance with competitor hospitals require the hospital “H” to give an access to its medical info that might cause a privacy leakage. Analysts starts from eliciting events that might compromise a security & GRC property of the system. We recommend analysts to start the event identification process from the resource level then move up to the process, and finally the objective level. In this manner, we prevent the spurious identification of an event’s impact. If an event disrupts a resource, then certainly it will also produce a disruption effect to the process that uses the resource, and consequently this will affect the goals that the process aim to satisfy. Conversely, in the case of the negative event at the process level (e.g., employee strike), the event cannot be mis-captured as an event originated from a resource disruption. All these is depicted in a SI* model as in Figure 11. Finally, analysts can identifying what other security & GRC properties that might also be compromised. 5.5 Risk Assessment This assessment includes risk analysis (identification and estimation) and risk evaluation [28]. Based on the unwanted scenarios identified in the previous section the

A Method for Security Governance, Risk, and Compliance (GRC)

173

analysts estimate their likelihood and severity [28]. Note that risk estimation can be either qualitative or quantitative depending on the availability of evidence for the occurrence of unwanted scenarios. The SI*-GRC method allows analysts using any risk models to estimate the risk level of the system, such as Probablistic Risk Analysis [29], Bayes Network [30], CORAS [31], Fault-Tree [32], GR-Framework [27]. In the paper [33] we explain how to interplay the goal methodology with other industrial methodologies. For practical reasons our SI* Tool, so far, only implements the automated reasoning from the GR framework. Risk evaluation compares the result of risk the estimation with the risk criteria (e.g., costs, benefits, priorities, acceptable loss) defined by the GRC council (see Section 3.1) . In case the risk level is unacceptable (i.e., it is beyond the risk tolerance that an actor can sustain), the analyst needs to identify appropriate treatments to mitigate the risk either by reducing their likelihood or their severity. Mitigation by control is explained in the next section.

6 Control Analysis Once the risk level of an event is unacceptable, analysts need to formalize a statement of the desired result/state-of-affair or purpose to be achieved by implementing controls. In SI*-GRC we call such statement as control goal. The establishment of control goals and their corresponding implementation by control processes is what ensures the security of business objectives and processes. 6.1 Control Goal Analysis During this step, the analysts must describe the control goals together with the actors responsible to manage them (i.e., the owner). These goals can be refined in a process that mirrors the construction of the target goal model in Section 4. For the refinement of business goals the process can be stopped as soon as we can identify a business process that can fulfill the goal. For control goals we should stop the process of refinement until we can fulfill the following qualities: Complete - all unwanted scenarios leading to unacceptable risks are addressed by at least one control goals ; Appropriate - there must be sufficient evidence that the achievement of a control goal will actually avoid the unwanted scenario or mitigate its effects; Precise - control goals and actors in charge of their achievement must be clearly specified, enabling unambiguous interpretation of the level of compliance or failure of a business process with regard to the control goal. These three qualities are complementary; that is, a set of control goals might be complete, but not appropriate. For example, it covers all relevant business needs, but wrong security assumptions might lead to an unacceptable risk level. The analysis might be appropriate (and determine the right effect in terms of impacts and likelihood of harmful events), but the description of the control is not precise enough to allow for the

174

Y. Asnar and F. Massacci Table 3. Establishing Control Goals

Business Goal: Prescribe therapy/drugs Compliance Requirement: Quality-Attribute: • Ensure accountability of the medical info Q1 Integrity Q2 Accountability Risk: Control Goal: • Compromised Prescription (Risk of Q1 ) • Ensure all prescriptions are correct • Unauthorized modification of the medical • Detect unauthorized modification of the medinfo (Risk of Q1 ) ical info • Unidentifiable clinical case entry in the • Trace any un-accountable entry to the medical medical info (Risk of Q1 ) info

correct implementation or the automation of the solutions, for example because it does not specify who is in charge of achieving the control goals. The control goals specified in Table 3 might be clear and easy to understand by the stakeholders. These control goals are still not precise enough to be machine implemented and monitored in terms of their effectiveness and performance. By taking advantage of repetitive patterns in control design, we can reduce the modeling effort and provide compliance experts with reusable process knowledge through a set of control patterns. Each pattern acts as a generalized description of actions that are frequently used in mitigating similar risks. See [34] for further details and examples of control patterns. At the level of control goals the following patterns are the ones most frequently found in the literature: – avoidance controls try to select an alternative path where the unwanted scenario does not materialize; Example 26. To avoid such unwanted incident in Example 11, the hospital “H” can decide to delegate the goal register patient to another actor that has necessary capabilities (e.g., the Patient Admin Division) – a prevention mechanism aims at preventing unwanted events to occur in the system (or at least reducing their likelihood); Example 27. In Example 25, such initiative might cause a breach on patients’ privacy. However, the sharing is necessary because it brings numerous business opportunities. Therefore, every sharing of medical information to another hospital must be anonymized except the data subject is the patient at the other hospital. – an attenuation mechanism tackle an unwanted events that cannot be prevented and tries to attenuate its severity. Example 28. In Example 25, analysts might decide to obtain an insurance in case the hospital “H” get a privacy-related law-suit.

A Method for Security Governance, Risk, and Compliance (GRC)

175

Table 4. Control Goal Template <modality + verb> – Hospital – should preserve – should ensure – must prevent – need to protect

– privacy – integrity – confidentiality – continuity – reliability

<artefact> – patient personal information – drug reimbursement records – the register patient process – the medical service to patients – all prescriptions

Qualitative-Attribute Reliability

Business Objective

Artefact Prescription & Prescription Archive

Prescribe Therapy/Drugs

More complete

Control Goal

Risk

Ensure all prescriptions are correct

Prevent any modifications to old prescriptions

Compromised prescriptions

Review a new prescription by a Doctor Pharmacy reviews new prescriptions

Protect the archive service from unauthorized accesses

Issue the auth token as the personnel badge

Incorrect prescription -Unauthorized prescription signature More precise

Log all activities done in the archive service

Prevent of having several active sessions for a single operator

control

Attacker obtains an access to the archive service

Attacher hijacks the active session

Unauthorized modification of prescription archive

Server admin modifies the archive entry

Operator misplaces the auth. token

More appropriate

Fig. 12. Control Goal Refinement

Moreover, one might follow the control goal templates (Table 4) in describing control goals in the context of the drug reimbursement system at the hospital “H”. A control goal describing the activity above is at a high-level of abstraction. Therefore, it needs to be further refined into specific definitions. The refinement is performed

176

Y. Asnar and F. Massacci Table 5. KAIs of Control Goals Control Goal

KAI

• Issue the authorization token as the personnel badge • Prevent of having several active sessions for a single operator • Protect the archive service from unauthorized access • Log all activities done in the archive • Pharmacy reviews new prescriptions

The number of system access done by reported missing personnel badges The number of two activities done from two different locations with in less than 30 minutes The number of suspected modification

• Review a new prescription by a doctor • Ensure all prescriptions are correct

The number of identified suspicious access The number of possible adverse drug reactions The drug values from excessive prescriptions The number of near-miss event 5 The number of reported medical error

iteratively, resulting in a “tree” of control goals, similar with the tree in the goal modeling (Section 4.2). Further down the tree, the “leaf” control goals become more precise and specific, and they are designed to provide specific risk mitigation function. This refinement of control goals can be continued in parallel to a refinement of the unwanted scenarios. Each refinement and review iteration of the models leads to an increase in precision, while the broadening of controls increase completeness as shown in Figure 12; where the starred constructs are newly identified during the refinement. Example 29. In Figure 12, we analyze the control goal ensure all prescriptions are correct so that complete, appropriate, and precise. Along the refinement of the risk model (i.e., unwanted event), analysts refine the control goal. into finer (sub) control goals. The event of unauthorized prescription signature, as in Example 24, is newly identified after the 1st iteration because it might affect one of the (sub)control goals (i.e., review a new prescription by a doctor). This situation requires analysts to enrich the control goal model by introducing a new control goal pharmacy reviews new prescriptions. More detailed risk analysis improves the appropriateness of the control and the corresponding mitigation effects. This refinement process is “sufficient” when further refinement of risks do not give new control goals or make the existing control goals more precise. In other words the analyst stops the refinement process because there is no advantage in producing a richer or more detailed risk model. At the end of the refinement process it should be possible to identify Key Assurance Indicators (KAIs for short) which assess the achievement of these objectives. These indicators should not depend on the way in which the control goals are actually implemented by a control process but only on the final outcome in terms of desired and achieved security properties. Intuitively the KAIs play the same role of Key Goal Indicators (KGIs) in COBIT [6]. Some examples from identified control goals, in Figure 12, are illustrated in Table 5.

A Method for Security Governance, Risk, and Compliance (GRC)

177

6.2 Control Process and Services Map A control process is a realization of a detailed control goal (the leaf nodes of Figure 12) and in a SOA environment can be implemented as a wrapper to the business components (as depicted in Figure 136 ) to preserve their quality attributes. Example 30. To implement the control goal prevent of having several active sessions for a single operator, the control process intercepts the message of case-details and consequently blocks the prescription process proceeding to the select drugs activity. The control process triggers an error event of multi-session where there has already an active session (via SessionChace service) for that particular operator, otherwise will release the prescription process. The error event will lead to the current existing process to be aborted and all operations are being rollback.

multi-session

Medical Info

Retrieve Patient Medical & Personal Info

Identify Patient

rollback

Specify Privacy Preferences Prescription

Case Details

Case-details

Select Drugs

Print and Sign Prescription Sheet

+

Archive Prescription Sheet

Old Prescription Prescription

Case-details

Check Active Session Receive Case-details

Prescription Archive

Review Prescription every Friday

Validate a new entry on the Medical Info

New Prescription suspected entry

multi-session

Enterprise Service Bus Legend: Business Process Control Process

Session Cache

Medical review service

Medical entry validator service Log Service

Archive Service

Fig. 13. Interwoven Control Process and Business Process 6

Grayed constructs are the parts that are introduced for implementing a control goal. Moreover, a construct with “mechanical-gear” indicates an underlying technical services supporting the execution of a process (i.e., business or control process).

178

Y. Asnar and F. Massacci Table 6. ESB-related policies – Block every result from the underlying service of Archive Prescription Sheet before storing the prescription at ArchiveService, and forward to MedicalEntryValidatorServices to be validated. MedicalEntryValidatorServices emits a release event to the ESB when the presription is not suspected entry – Release the results of ArchivePrescriptionSheetService after receiving the release event from MedicalEntryValidatorServices; and store the prescription at the ArchiveService. – Trigger the log event each when there is any request to ArchiveService and forward the event log to be saved at LogService – Trigger a suspected entry event when the validate a new entry on the medical info activity, using MedicalEntryValidatorService, suspects the prescription – Trigger a start event, every Friday, for Pharmacy to Review Prescription all prescriptions indicated as suspected entries. This activity is also checked a sample of new prescriptions.

To implement control processes one needs to explicitly specify which service events need to be controlled. So a human mapping is necessary, eventually aggregating conditions on services for achieving a control goal on the specific services on which it is required. The particular forms in which a control process is implemented might vary a lot depending on the architecture that is currently used. However, we can use also in this case a number of security patterns [35]. For a SOA scenario, a possible architecture is based on the Enterprise Service Bus (ESB) [36]. The ESB has the capability to detect when a message (e.g., request, response, notification) arrives and to perform some actions (e.g., block, delete, delay, release modify, forward, trigger). The basic principles for interweaving control and business services are the following: – If a control service is executed before the business service is invoked – (i.e., filter in/out), the ESB will block the request message to the business service and forward the request to the control service. The control service will notify the messaging service whether to remove the blocked request if it is considered to be an inappropriate request, or to release it; as in Example 30. – If a control service is executed after the business service invoked (i.e., verify), the ESB will block the result of the business service invocation before dispatching it to the subsequent service in the business process, and release the results after performing some operations (e.g., modify/add/remove some data items, attach signature) or even remove the result if it violates some policy (e.g., not sending confidential data). Table 6 shows some examples of security control policies for the ESB scenario in implementing the control goal of log all activities done in the archive and the control goal of pharmacy reviews new prescriptions.

A Method for Security Governance, Risk, and Compliance (GRC)

179

Table 7. KSIs of Control Process Control Process Name

KSI-correctness/coverage

• Abort the prescription process Cor: The ration between the occurrences of multiwhen there is already an active session event and the rollback events session Cov: Percentage of the number of check active session execution over the specify privacy preference execution • Log all requests to Cor&Cov: The difference between the number log enArchiveService tries of archiving a prescription and the number of prescription sheet • Friday review done by Pharmacy Cov: The ratio between the number of reviews done by Pharmacy on the suspected prescription entry and the occurrences of suspected entry event Cov: The ratio between the number of prescription reviews done by Pharmacy and the number of prescription sheet • Validate new prescriptions by Cor: The number of prescription which triggers susMedicalEntryValidatorpected entry event but at the end it is benign Service Cor: The number of archived prescription in ArchiveService that is suspected Cov: The ratio between the number of execution of MedicalEntryValidatorService in supporting the Validate a new entery in the medical info and the number of prescription sheet

At the end, each control process must be defined its Key Security Indicators (KSIs for short) which assess the level of correctness and coverage of such controls. By KSI-correctness, these indicators must indicate whether the result of the control is correct according to its design specification; while KSI coverage indicates the level coverage of the control in protecting the business process execution. Note KSI is closely dependent on the technical details how the control is implemented in the system. Intuitively the KSIs play the same role of Key Performance Indicators (KPIs) in COBIT [6]. Some examples from identified control presses, in Figure 13, are illustrated in Table 7. Note that often it is hard to define KSIs of the correctness of (semi-)manual control, because the correctness criteria are in the human knowledge and hardly encode-able for the knowledge of technical system. Besides implementing control processes, designers need to define the events (e.g., a service start/ finish/suspend or messages exchanged among services) that will be used to compute the KAIs and KSIs. To process these events, the business activity monitoring (BAM) [37] can be used, since it allows one to analyze real-time events from the business transaction and, furthermore, to compute KAIs/KSIs following the mathematical formula defined by the designers. BAM-related policies for computing KAI/KSIs related to the control goal of prevent of having several active session for a single operator are illustrated in Table 8.

180

Y. Asnar and F. Massacci Table 8. KAIs/KAIs Policies in BAM

•

Control Goal Prevent of having several active sessions for a single operator KAI The number of two activities done by the same operator from two different locations within less than 30 minutes BAM stores the last location and access’/activity’s timestamp from each operator taken. This information is obtained from the message request to SessionChace. It increments the value of KAI when the most recent activity is within 30 minutes form the previous one and done from different location • Control Process Abort the prescription process when there is already an active session KSIcor The ration between the occurrences of multi-session event and the rollback events Nmulti-session /Nrollback ; where multi-session denotes that Check Active Session founds an active session in SessionCache for the same operator, and rollback is triggered when the prescription process is aborted and some rollback procedures is executed KSIcor Cov: Percentage of the number of check active session execution over the specify privacy preference execution Ncheck-session /Nspecify-privacy × 100; where check-session is an event triggered when Check Active Session is executed, and specify-privacy is triggered when the operator has specified the patient’s privacy preference

7 SI*-GRC In-Vivo Once these controls are in place, we must still assess and monitor them at runtime for ensuring the level of security & compliance of the system; and improving the implemented controls. 7.1 The Check Phase: Monitor and Review Indicators For each control goal and process, analysts need to identify indicators that measure its correctness and effectiveness. For these purposes, we use key assurance indicators (KAIs) and key security indicators (KSIs) as illustrated in Table 5 and Table 7 respectively. Typically, KAIs are the focus of the business analysts, because business analysts are more concerned with the level of compliance rather than how the control is implemented. KSIs, on the other hand, are of interest to risk/security analysts as they measure how well controls are implemented. Both KAIs and KSIs are critical for monitoring, evaluating and improving the GRC implementation. The indicators are computed independently to distinguish between cases in which the KAI of a control goal is “low” but the KSI’s associated control processes are “high”. In the former case, analysts might conclude that there are some risks that have not been mitigated. In the latter, it might be that the compliance of a business process is achieved through external factors (from luck to organizational procedures), rather than deployed controls. The technological infrastructure implementing the controls and indicators identified by the method should then support Continuous Controls Monitoring techniques. For

A Method for Security Governance, Risk, and Compliance (GRC)

181

example, the tools developed in the MASTER project7 makes this possible: monitoring and assessment infrastructures provide one main source of indication and feedback for the review. The monitoring infrastructure gathers and correlates events emitted from monitored services based on monitoring policies defined by control processes. The assessment infrastructure evaluates the relevant KAI/KSI values based on the events triggered by the monitoring infrastructure. The events may be stored in an audit trail or event log database for further processing and assessment as necessary.The KAI/KSI values arethen presented to a (human) control process supervisor through a dashboard or a reporting tool. 7.2 Act Phase: React and Improve In a SOA-based business environment, orchestration of services associated to business processes could change regularly. Each change may violate existing control goals or impact the effectiveness of the relevant controls. It is, therefore, crucial to have a continuous review of the compliance level of existing business processes and the effectiveness of implemented controls. Following the result of the review different types of actions can be performed: augmentative, corrective and preventive actions, can be taken based on the review findings discussed earlier to ensure that actual control processes always reflect the latest business compliance requirements. The methodological steps are detailed below: 1. Identification of changes. Generally, augmentative actions are required when there is an update to existing: (i) legal requirements; (ii) business requirements; or (iii) control activities as the result of a review performed on the associated control processes. Based on these updates, we first identify what potential changes to existing control objectives/activities that need to be made. 2. Planning and implementation of changes. This step repeats the tasks involved in the P LAN and D O phases so that new controls can be put in place or modification to existing controls can be implemented. 3. Documentation and review. All information gathered from performing the above steps should be documented for future references.

8 Final Remarks What we have seen in this tutorial is a summary overview of SI*-GRC method to analyze and design security controls. The gist of the method is to capture the organizational setting of the system making sure that business goals and processes are the main citizen. The basic building blocks of the methods are 1. a modeling framework based on a requirement engineering framework, with some extensions related to security & GRC concerns, such as: trust, permission, risk, and treatment, 2. a analysis process defining systematical steps in analyzing and design security controls, 7

http://www.master-fp7.eu

182

Y. Asnar and F. Massacci

3. a number of analytical techniques to verify that certain security properties are satisfied and the risk level is acceptable, and at last 4. a CASE tool, namely the SI* tool to support analysts in using the method. Many details of the research methods that constitute the building blocks of this methods can be found in our research papers and in the research s of the our co-authors that we list in the acknowledgement section. Acknowledgments. We would like to thank Paolo Giorgini, John Mylopoulos, Nicola Zannone from Trento for many useful discussion on the large and unruly family of security requirements engineering. Volha Bryl, Fabiano Dalpiaz, Federica Paci, Ayda Saidane, Angelo Susi, Minh Sang Tran and many others contributed to the discussion on what really is a good security requirements methodology. Daniela Marino and Andrea Micheletti from Hospital San Raffaele, Valentino Meduri and Alessandra Tedeschi from DeepBlue and Carlo Ricucci and Domenico Presenza from Engineering proved to be an essential soundboard for the applicability of our ideas. Claire Worledge and Bernadette Louat were instrumental to link our methodology to COBIT and other ISACA methodologies. Any mistake is of course ours. This work has been partially funded by EU-FP7-ICT-IP-ANIKETOS, EU-FP7-ICTIP-SecureChange, and EU-FP7-ICT-NoE-NESSoS project.

References 1. Rasmussen, M., Kark, K., Penn, J., McClean, C., Bernhardt, S.: Trends 2007: Governance, risk and compliance: Organizations are motivated to formalize a federated GRC process. Technical report, Forrester Research (April 2007) 2. McClean, C., Whiteley, R., Kark, K., Dill, A.: The Forrester Wave: Enterprise governance, risk, and compliance platforms, Q3 2009. Technical report, Forrester Research (July 2009) 3. ACL: Audit command language, http://www.acl.com/ (last check July 15, 2010 (3020)) 4. Pfleeger, C.P., Pfleeger, S.L.: Security in Computing, 4th edn. Prentice-Hall, Englewood Cliffs (2006) 5. ISACA: The Risk IT Framework. ISACA (November 2009) 6. Institute, T.I.G.: CoBIT - Framework Control Objectives Management Guidelines Maturity Models. 4.1 edn. ISACA.org (2007) 7. Trist, E.: The evolution of Socio-Technical systems. Occasional Paper 2 (1981) 8. Asnar, Y., Lim, H.W., Massacci, F., Worledge, C.: Realizing trustworthy business services through a new GRC approach. ISACA Journal - JOnline 2 (2010) 9. Deming, W.E.: Out of the Crisis. MIT Press, Cambridge (2000) 10. Marino, D., Potral, J.J., Hall, M., Rodriguez, C.B., Rodriguez, P.S., Sobota, J., Jiri, M., Asnar, Y.: Master scenarios. Project Deliverable D1.2.1, MASTER Consortium (2009); This case study has been provided by Hospital San Raffaele Foundation, Its complete description is available at http://www.masterfp7.eu/index.php?option=com_docman& task=doc_details&gid=53&Itemid=60 11. Erl, T.: SOA Principles of Service Design. Prentice-Hall, Englewood Cliffs (2007) 12. Casteleyn, S., Daniel, F., Dolog, P., Matera, M.: Engineering Web Applications. SpringerVerlag New York Inc., New York (2009) 13. Yu, E.: Modelling Strategic Relationships for Process Engineering. PhD thesis, University of Toronto, Department of Computer Science (1995)

A Method for Security Governance, Risk, and Compliance (GRC)

183

14. OMG: Business process modeling notation (January 2009) 15. Hofstede, A.H.M., Aalst, W.M.P., Adams, M., Russell, N. (eds.): Modern Business Process Automation-YAWL and its Support Environment. Springer, Heidelberg (2010) 16. Giorgini, P., Massacci, F., Mylopoulos, J., Zannone, N.: Requirements engineering for trust management: model, methodology, and reasoning. International Journal of Information Security 5(4), 257–274 (2006) 17. Robbins, S.P.: Organizational Behavior, Concepts, Controversies, Applications, 7th edn. Prentice-Hall, Englewood Cliffs (1996) 18. OCC: Management information systems. the comptroller’s handbook, Office of the Comptroller of the Currency (May 1995) 19. Zachman, J.A.: A framework for information systems architecture. IBM Systems Journal 26(3), 276–292 (1987) 20. van der Aalst, W.M.P.: Formalization and verification of event-driven process chains. Information and Software Technology 41(10), 639–650 (1999) 21. Eriksson, H.E., Penker, M.: Business modeling with UML: Business Patterns at Work. John Wiley & Sons, Chichester (2000) 22. van der Aalst, W.M.P., Pesic, M., Schonenberg, H.: Declarative workflows: Balancing between flexibility and support. Computer Science - Research and Development (March 2009) 23. van der Aalst, W.M., Weske, M., Gr¨unbauer, D.: Case handling: a new paradigm for business process support. Data & Knowledge Engineering 53(2), 129–162 (2005) 24. Bresciani, P., Perini, A., Giorgini, P., Giunchiglia, F., Mylopoulos, J.: Tropos: An AgentOriented software development methodology. Autonomous Agents and Multi-Agent Systems 8(3), 203–236 (2004) 25. Massacci, F., Mylopoulos, J., Zannone, N.: Computer-aided support for secure tropos. Automated Software Engineering 14(3), 341–364 (2007) 26. Compagna, L., Khoury, P.E., Krausov´a, A., Massacci, F., Zannone, N.: How to integrate legal requirements into a requirements engineering methodology for the development of security and privacy patterns. Artificial Intelligence and Law 17(1), 1–30 (2008) 27. Asnar, Y., Giorgini, P., Mylopoulos, J.: Goal-driven risk assessment in requirements engineering. Requirements Engineering 16(2), 101–116 (2011), 10.1007/s00766-010-0112-x 28. ISO/IEC: Risk Management-Vocabulary-Guidelines for Use in Standards, Published: ISO/IEC Guide 73 (2002) 29. Vose, D.: Risk Analysis: A Quantitative Guide. Wiley, Chichester (2000) 30. Mosleh, A., Hilton, E.R., Browne, P.S.: Bayesian probabilistic risk analysis. SIGMETRICS Perform. Eval. Rev. 13(1), 5–12 (1985) 31. Lund, M.S., Solhaug, B., Stl?en, K.: Model-Driven Risk Analysis - The CORAS Approach. Springer, Heidelberg (2011) 32. Vesely, W., Goldberg, F., Roberts, N., Haasl, D.: Fault Tree Handbook. U.S Nuclear Regulatory Commission (1981) 33. Delande, O., Felix, E., Massacci, F., Paci, F.: Managing changes with legacy security engineering processes. In: Zeng, D., Yang, C.C., Collberg, C. (eds.) Proc. of IEEE Internat. Conf. on Intelligence and Security Informatics (ISI 2011). IEEE Press, Los Alamitos (2011) 34. Namiri, K.: Model-Driven Management of Internal Controls for Business Process Compliance. PhD thesis, Universit¨at Fridericiana zu Karlsruhe (2008) 35. Schumacher, M., Fernandez-Buglioni, E., Hybertson, D., Buschmann, F., Sommerlad, P.: Security Patterns: Integrating Security and Systems Engineering, 1st edn. Wiley, Chichester (2006) 36. Chappell, D.: Enterprise Service Bus. O’Reilly Media, USA (2004) 37. Kochar, H.: Business Activity Monitoring and Business Intelligence, (December 2005), http://www.ebizq.net/topics/bam/features/6596.html (last access at July 03, 2010)

184

Y. Asnar and F. Massacci

Appendix A - SI* Syntax SI* Diagram Actor

DLV Predicate

Role

Agent

actor(Actor) role(Role) agent(Agent) Goal

goal(Goal) Task

Process +

process(Process) task(Task)

Resource

Service Message

Data Store

resource (Resource)

Unwanted Event

event(UnwatedEvent)

Goal

and Subgoal 1

Subgoal 2

and decomposition(Goal, SubGoal1, SubGoal2) A

De

Goal

De

B

del exec(A,B,Goal) A

Te

Goal

Te

B

trust exec(A,B,Goal) A

Dp

Dp

Process

B

+

del perm(A,B,Process) A

Tp

Tp

Resource

B

trust perm(A,B,Resource)

Resource

R

Goal

`P

O

Actor

Process +

request(Actor,Goal) owner(A,Resource)

provide(Actor,Process)

Note: See BPMN [14] for complete constructs for the Process Modeling.

The Geometry of Lattice Cryptography Daniele Micciancio UC San Diego [email protected]

Abstract. Lattice cryptography is one of the hottest and fastest moving areas in mathematical cryptography today. Interest in lattice cryptography is due to several concurring factors. On the theoretical side, lattice cryptography is supported by strong worst-case/average-case security guarantees. On the practical side, lattice cryptography has been shown to be very versatile, leading to an unprecedented variety of applications, from simple (and eﬃcient) hash functions, to complex and powerful public key cryptographic primitives, culminating with the celebrated recent development of fully homomorphic encryption. Still, one important feature of lattice cryptography is simplicity: most cryptographic operations can be implemented using basic arithmetic on small numbers, and many cryptographic constructions hide an intuitive and appealing geometric interpretation in terms of point lattices. So, unlike other areas of mathematical cryptology even a novice can acquire, with modest eﬀort, a good understanding of not only the potential applications, but also the underlying mathematics of lattice cryptography. In these notes, we give an introduction to the mathematical theory of lattices, describe the main tools and techniques used in lattice cryptography, and present an overview of the wide range of cryptographic applications. This material should be accessible to anybody with a minimal background in linear algebra and some familiarity with the computational framework of modern cryptography, but no prior knowledge about point lattices.

1

Introduction

Lattice cryptography is one of the most fascinating areas of mathematical cryptography. Supported by a solid theoretical foundation, lattice cryptography can also be very attractive in practice, as an alternative to more traditional solutions based on number theory. One of the features that makes lattice cryptography potentially attractive in practice is its simplicity: the most fundamental operation of lattice cryptography is just a modular integer matrix-vector multiplication Ax (mod q). Typically, q is a small integer, which comfortably ﬁts in a machine level register, so that all the basic operations can be eﬃciently performed (and easily implemented) without the need of a “big-num” library for arbitrary precision arithmetic. While many cryptographic functions based on lattices can be easily described using matrix notation (and this is how they are typically described

Supported in part by the National Science Foundation under Grant CNS-0831536.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 185–210, 2011. c Springer-Verlag Berlin Heidelberg 2011

186

D. Micciancio

in most research papers), the operations they perform and the computational problems they hide are fundamentally geometric. The matrix formulation may be all that is needed to specify and implement lattice cryptography. But in order to truly understand it and develop an intuition of what makes lattice cryptography secure, one needs to look under the hood, and explore the geometric structure of the underlying lattices. As we will see, while quite diﬀerent from the mathematics used in more conventional cryptography, the mathematics of lattices (or at least most of the mathematics of lattices used in cryptography) is not that hard to understand. In fact, once you learn the most basic concepts, it is pretty simple! After all, geometry is one of the most intuitive areas of mathematics, well grounded in our everyday experiences in the physical world, and many of the concepts we will introduce can be easily illustrated by pictures.1 If you can draw a picture that captures the intuition of why factoring the product of two large prime numbers is a computationally hard problem, I would like to see your picture! So, let us leave our fears behind, and move on with lattices. In these notes, we will ﬁrst learn about lattices, their geometric structure, and the computational problems they deﬁne. Then we will look at how these simple geometric object can be used for the construction of cryptographic functions. Beside the potential for eﬃcient implementation, lattice cryptography is very attractive because of its versatility: using lattices, in the last few years, researchers were able to develop solutions for an incredibly rich set of security problems, from simple (but eﬃcient) hash functions [57,68,48,51], to hierarchical identity based public key encryption [34,22,1,2], and much more. As of this writing, the latest and greatest discovery in lattice cryptography is the development of fully homomorphic encryption, pioneered by Gentry in [28], and still a very fast moving research target [82,29,65,79,80,32,25,19,31,30]. In these notes, you will not learn about the most complex applications of lattices, including fully homomorphic encryption, but you will learn enough about lattices and lattice cryptography to proceed on your own and read research papers in the area. Beside an introduction to the fundamentals of lattice cryptography, this paper includes a extensive bibliography, that can be used as a source of pointers for further study. Notational conventions. We use Z for the set of integers, and R for the set of real numbers. Elements of these sets are denoted by lowercase letters. Sets are denoted with uppercase letters. We use bold lower case letters x for vectors and bold upper case letters A for matrices. All operations and functions are extended to sets in the usual way, e.g., for any number x ∈ R and sets A, B ⊆ R, we have x · A = {x · a | a ∈ A} and A + B = {a + b | a ∈ A, b ∈ B}. We use standard asymptotic notation O(·), o(·), ω(·), Ω(·) to describe the order of 1

To be clear, all pictures you will ever see (e.g., in the slides that accompany these lecture notes, available from the author’s web page at http://cseweb.ucsd.edu/~daniele) describe 2-dimensional, or in some rare cases 3-dimensional lattices, while for lattice cryptography to be secure one needs to work with lattices in high dimensional space. Still, 2-dimensional lattices often provide good examples to illustrate the geometry of the problems.

The Geometry of Lattice Cryptography

187

growth of functions. Namely, f = O(g) or g = Ω(f ) means that f (n) ≤ c · g(n) for some c and all suﬃciently large n, and f = ω(g) or g = o(f ) means that limn→∞ g(n)/f (n) = 0.

2

Point Lattices: A Primer

Lattices are regular arrangements of points in n-dimensional Euclidean space. The simplest example of a lattice is Zn , the set of all n-dimensional vectors with integer entries. In general, a lattice L ⊂ Rn is a set of points L = BZk = {Bx: x ∈ Zk } ⊂ Rn obtained by applying a non-singular linear transformation B ∈ Rn×k to the integer lattice Zk . (See Figure 1 for a 2-dimensional example.)

b1

b2

Fig. 1. A 2-dimensional lattice B · Z2 generated by the basis B = [b1 , b2 ]. The shaded area is the parallelepiped P(B) = B · [0, 1)2 spanned by the basis vectors. The determinant of the lattice is the volume (area) of the parallelepiped.

The matrix B ∈ Rn×k is called a lattice basis, and non-singular means that its k columns should be linearly independent. For simplicity, in these notes, we focus on full dimensional lattices, i.e., lattices where n = k, and the lattice BZn is generated by a square matrix B ∈ Rn×n with nonzero determinant. n The lattice n BZ generated by a basis B can be equivalently written as the set L = { i=1 bi · xi : ∀i.xi ∈ Z} of all integer linear combinations of the basis vectors B = [b1 , . . . , bn ]. Notice the diﬀerence between a lattice and a vector space spanned by a basis B: the vector space spanned by B is the set of all real linear combinations i bi · xi (with xi ∈ R) of the basis vectors, while in the case of lattices we only take combinations with integer coeﬃcients. A direct consequence of the deﬁnition (very easy to visualize, and only slightly harder to prove) is that lattices are discrete sets: the points in a lattice cannot

188

D. Micciancio

be arbitrarily close to each other, i.e., for every lattice L there is an > 0 such that the distance between any two distinct lattice points is at least . Due to this discrete structure, problems on lattices cannot generally be solved by simple linear algebra. For example, a basic fact from linear algebra is that any vector space has an orthogonal basis, and orthogonal bases are useful to solve a host of other problems. This is not true for lattices: not every lattice admits an orthogonal basis (e.g., see the lattice in Figure 1,) and this is what makes many lattice problems computationally hard to solve, and useful for cryptography. The discrete structure of lattices yields also a more abstract (basis independent) characterization of lattices: a lattice is a discrete additive subgroup of Rn .2 We recall that a subgroup is a set which is closed under addition and subtraction.3 Not every subgroup of Rn is a lattice, i.e., Rn itself is not a lattice because it is not discrete. In computer science and cryptography, it is common to focus on integer lattices, i.e., lattices L ⊆ Zn whose vectors have all integer coordinates. Any set of integer vectors is certainly discrete, so integer lattices can be characterized simply as additive subgroups Zn . 2.1

Fundamental Parameters

As n-dimensional objects, lattices can be hard to visualize, but there are several geometric quantities naturally associated to any lattice that succinctly describe some of its most important properties. These include – The determinant of the lattice, which is the volume of the n-dimensional parallelepiped P(B) = B[0, 1)n spanned by the basis vectors det(BZn ) = vol(P(B)). (See Figure 1.) Intuitively, the determinant of a lattice describes the (inverse) density of lattice points in space, i.e., the number of lattice points in a suﬃciently large and regular region of space S ⊆ Rn , is approximately equal to |S ∩ L| ≈ vol(S)/ det(L). To see this, notice that the space Rn can be tiled with copies of the fundamental parallelepiped P(B), with one copy x+P(B) for every lattice point x ∈ L = BZn . So, there is precisely one lattice point in every region of space x+P(B) of volume det(L) = vol(P(B)). For full dimensional lattices, the lattice determinant simply equals the absolute value of the matrix determinant det(BZn ) = | det(B)|. In general, when B ∈ Rn×k for √ k ≤ n, the determinant can be computed using the formula det(BZk ) = BT B. It is sometime convenient to consider the root determinant of a lattice, which is just the nth root of the determinant det(BZn )1/n . – The minimum distance of a lattice, i.e., the smallest distance between any two lattice points λ(L) = inf{ x − y | x, y ∈ L, x = y}. The minimum distance is always attained by some pair of lattice points, so the infimum in the previous formula is in fact a minimum. 2

3

The fact that any such discrete subgroup admits a basis, and therefore is a lattice, is not at all obvious. But for the lattices considered in this survey, the proof is relatively easy and it is left to the reader as an exercise. Notice that requiring closure under subtraction is enough because x + y = x − ((y − y) − y), and closure under addition is implied.

The Geometry of Lattice Cryptography

189

– More generally, one can deﬁne the successive minima of a lattice λ1 , . . . , λn , where for every i = 1, . . . , n, λi (L) is the smallest positive real r such that the ball B(0, r) = {x: x ≤ r} of radius r centered at the origin contains at least i linearly independent vectors. As a special case, the ﬁrst minimum λ1 (L) is the length of the shortest nonzero lattice vector and equals the minimum distance λ of the lattice. – The covering radius of a lattice ρ(L) is the smallest real number r such that spheres of radius r centered around all lattice points cover the entire space (spanned by the lattice). a useful way to visualize the minimum distance of a lattice is in terms of sphere packings. Think of putting equal spheres (of radius r) around every point in a lattice. When the radius r is very small, the spheres are disjoint. Now make the radius r as large as possible, but still subject to the constraint that the spheres centered around lattice points do not intersect. The maximum value of r is called the packing radius of the lattice, and it clearly equals half the minimum distance λ(L)/2. If we keep increasing the radius of the spheres beyond the packing radius λ/2, the spheres will start intersecting with each other, but they may not cover the entire space, leaving holes to be ﬁlled. But at some point, when r reaches the covering radius of the lattice ρ, all holes will be ﬁlled and the spheres will cover the entire space. All these quantities are lattice invariant, i.e., they depend only on the set of lattice points BZn , and not on the speciﬁc basis B used to represent them. All these quantities are also invariant under rotations, i.e., for any orthogonal matrix Q ∈ Rn we have λ(L) = λ(QL), and similarly for the successive minima, covering radius, determinant and root determinant. All these quantities, with the exception of the determinant, are also linear in the sense that λ(c · L) = |c| · λ(L) for any real scaling factor c, and similarly for λi , ρ and the root determinant. Taken together, all these parameters provide a lot of useful information about the geometry of the lattice. The diligent reader can reﬁne his geometric understanding of lattices by proving the following simple statements. Exercise 1. Show that for any lattice L and target point t (in the linear span of the lattice) there is at most one lattice point within distance (strictly less than) λ(L)/2 from t. Exercise 2. Show that for any lattice L and target point t (in the linear span of the lattice) there is at least one lattice point within distance ρ(L) from t. Exercise 3. Show that for any n-dimensional lattice, the parameters λ1 , . . . , λn , ρ are related by the chain of inequalities4 √ λ1 ≤ λ2 ≤ · · · ≤ λn ≤ 2ρ ≤ n · λn . Far less trivial to prove (but still not too hard, e.g., see [60] for a simple proof) is the following upper bound on the successive minima in terms of the (root)determinant of the lattice. 4

The last two inequalities are not quite trivial. As a starter, the reader may want ﬁrst to prove the weaker inequalities λn ≤ 2nρ ≤ n2 λn .

190

D. Micciancio

Theorem 1 (Minkowski). For any lattice Λ, λ(L) ≤

n

1/n λi (L)

≤

√

γn · det(L)1/n

i=1

where γn ≤ n is a function of the dimension only, and does not depend on the specific lattice. For every n, the optimal value of γn in Minkowski’s theorem is called Hermite’s constant in dimension n. The exact value of γn is known only for n = 1, 2, . . . , 8 and n = 24, but the asymptotic estimate γn = Θ(n) is all that is needed to understand lattice cryptography. Minkowski’s bound provides useful information about the minimum distance of a lattice and it is pretty close to optimal when the lattice is chosen at random according to many natural probability distributions on lattices. (At this point the reader may want to try to come √ up with an explicit construction of an n-dimensional lattice with λ(L) = Ω( n det(L)1/n ). It is not an easy task, and as we will see these are good examples of “cryptographically hard” lattices. See Exercise 13.) On the other hand, the actual minimum distance λ (or the mean value ( i λi )1/n ) can be arbitrarily smaller than Minkowski’s upper bound. Exercise 4. Build a lattice basis B such that λ(L(B)) is smaller than Minkowski’s upper bound by at least a factor c > 1. [Hint: it is enough to consider lattices in dimension n = 2.] On the other √ hand, for the mean value ( i λi )1/n , Minkowski’s bound is loose by at most a n factor. Exercise 5. Show that for any n-dimensional lattice L, ( i λi )1/n ≥ det(L)1/n . [Hint: find a set of linearly independent vectors of length λ1 , . . . , λn , and bound the volume of the parallelepiped they span using Hadamard’s inequality.] Another very useful relation between the fundamental lattice parameters λ and ρ is given by the so-called transference theorems, but in order to present them we need ﬁrst to deﬁne the dual lattice. This is probably one of the more technical and less intuitive concepts used in the study of lattice cryptography, and it takes some time and eﬀort to start developing some geometric intuition about it. But the notion of dual lattice turns out to be so useful that the eﬀort is certainly justiﬁed. Let’s put geometry aside for the moment. The reader is probably familiar with the deﬁnition of the dual of a vector space: the dual of a vector space V ⊆ Rn is the set V † of all linear functions φ from V to R. These functions can be naturally represented as vectors x ∈ V , where φx (v) = v · x, and using this representation V † is isomorphic to the original vector space V . The deﬁnition of dual lattice is analogous, but this time it is natural to consider linear functions φ: L ⊆ Rn → Z that take integer values when evaluated on lattice points. As before, φ can be represented by a vector x in the linear span of L such that φ(v) = v · x. The representation of linear functions as vectors leads to the following deﬁnition: the dual lattice L† is the set of all vectors x (in the linear span of L) such that

The Geometry of Lattice Cryptography

191

v · x ∈ Z for every lattice point v ∈ L. As you may expect, the dual lattice L† is a lattice, and the dual of the dual is the original lattice (L† )† = L. An initial understanding of the geometry of the dual lattice is provided by the following simple observations. Exercise 6. If B ∈ Rn×n is a lattice basis, then a basis of the dual lattice (BZn )† is given by the inverse transpose matrix B−T , i.e., (BZn )† = B−T Zn . More generally, the dual basis of B ∈ Rn×k , is given by B† = B(BT B)−1 . It easily follows that for any lattice L and scaling factor c > 0, the dual lattice of c · L is (c · L)† = (1/c) · L† , i.e., expanding a lattice by a factor c shrinks the dual lattice by the same factor. Also, we have that det(L† ) = 1/ det(L). So, in an informal sense, the dual lattice is the “inverse” of the original lattice. The transference theorems show that the fundamental parameters λ and ρ are also connected by an inverse relation through the dual lattice, although only an approximate one. Theorem 2. For any n-dimensional lattice L, we have 1 ≤ λi (L)·λn−i+1 (L† ) ≤ n and 1 ≤ 2ρ(L) · λ(L† ) ≤ n. So, in an approximate sense (up to a factor n), λ1 (L† ), . . . , λn (L† ) are the inverses of λn (L), . . . , λ1 (L), and λ(L† ) is the inverse of 2ρ(L). This reinforces the geometric intuition behind the informal statement that the dual lattice L† is the inverse of L. To get a better geometric grasp of this statement, consider the following illustrative example. Let L is a lattice such that L† contains a very short vector x of length x = λ† . By deﬁnition, all lattice points in L have integer scalar product with x. So, x can be used to partition L into layers Li = {v ∈ L | x · v = i}. Clearly, the covering radius ρ(L) must be at least as large as half the distance between these layers, but the distance between the layers is precisely 1/ x = 1/λ† . Therefore 1/(2λ† ) ≥ ρ, yielding the lower bound 2λ† · ρ ≥ 1. The upper bound 2ρ · λ† ≤ n is far less trivial to prove, and requires the use of harmonic analysis techniques [11]. 2.2

Computational Problems

We have seen how the determinant of a lattice B can be eﬃciently computed, e.g., by the formula det(L) = det(BT · B). However, the same is not true for the minimum distance λ(L), any of the successive minima λi or the covering radius ρ. In fact, all these parameters give raise to well known hard computational problems on lattices. The best polynomial time approximation algorithms to solve any of the problems described below only achieve approximation factors γ(n) = 2O(n log log n/ log n) almost exponential in the dimension of the lattice. Lattice cryptography is based on the assumption that no eﬃcient algorithm can achieve polynomial approximation factors γ(n) = nO(1) , at least in the worstcase, i.e., no eﬃcient algorithm can achieve γ(n) = nO(1) approximation factor for every possible input lattice. Worst-case hardness assumptions are supported by many NP-hardness results for lattice problems [4,21,15,10,55,26,27,42,43,38],

192

D. Micciancio

although these results only hold for much smaller (sub-polynomial) approximation factors. The main problem underlying lattice cryptography is called the Shortest Independent Vectors Problem, and it is the natural computational problem associated to the parameter λn . Cryptography is based on the hardness of solving this problem even approximately, as deﬁned below. Definition 1. The Shortest Independent Vector Problem (SIVPc ) is the following: given an n-dimensional lattice L (usually represented by a basis B), find n linearly independent lattice vectors b1 , . . . , bn ∈ L such that maxi bi ≤ c · λn (L). In the above deﬁnition, c ≥ 1 is the approximation factor, and it is usually a function of the dimension n. The case c = 1 corresponds to solving the problem exactly, i.e., ﬁnding n linearly independent lattice vectors such that the quantity maxi bi is as small as possible. We remark that n linearly independent lattice vectors are not necessarily a basis for the given lattice. For example, the column vectors 2B are linearly independent vectors in BZn , but they are not a basis for BZn . This is because they only generate the sublattice 2 · L rather than the entire lattice L = BZn . It is usually surprising to the novice to learn that there are lattices L such that any basis must necessarily contain vectors much longer than λn . In particular, there are lattices such that any solution to the SIVP problem will not be a basis for the lattice. Exercise 7. Let L ⊂ Zn be the set of all integer vectors such that in each vector all coordinates have the same parity. (I.e., the coordinates are either all even, or all odd integers.) Assume n ≥ 5. Prove that λ1 = λ2 = · · · = λn = 2. Show that L = BZn has no basis such that maxi bi ≤ λn . If you think this is surprising and counter intuitive, it is not for no reason. In dimension (up to) n = 3, a set of linearly independent vectors of length

bi = λi (for i = 1, 2, 3) is always a lattice basis. Only in dimension 4, they may or may not generate the entire lattice, and as the previous exercise shows, starting in dimension 5, minimizing the length of the vectors may necessarily yield only a basis for a sublattice. So, in order to “see” all this, you need to “look” beyond our 3-dimensional physical world. Fortunately, in most applications (and in particular, in cryptography), a set of short linearly independent vectors is just as good as a basis, and the reader can disregard the subtle distinction between a short basis, and a set of short linearly independent vectors. Moreover, if a basis is really needed, one can always convert a set of linearly independent vectors into a basis of slightly longer vectors. Exercise 8. Show that for any set of n linearly independent lattice √ vectors v1 , . . . , vn ∈ L there is a basis for L = BZn such that maxi bi ≤ n maxi vi . Moreover, such B can be eﬃciently computed from v1 , . . . , vn and an arbitrary basis for L. Another fundamental computational problem on point lattices is the Closest Vector Problem, which may be formulated as follows.

The Geometry of Lattice Cryptography

193

Definition 2. Given a lattice L ⊂ Rn , a target point t ∈ Rn , and a distance bound d, the Closest Vector Problem (CVP) asks for a lattice point v ∈ L at distance t − v ≤ d from the target, provided such a lattice point exists. In the exact version of CVP, the distance bound is taken to be the distance d = μ(t, L) = minv∈L t − v between the target and the lattice. In the approximate problem CVPγ one sets d = γ · μ. Two special versions of CVP that play a prominent role in cryptography are: 1. the Bounded Distance Decoding problem (BDD), where d < λ/2, and 2. the Absolute Distance Decoding problem (ADD), where d ≥ ρ. The importance of these speciﬁc parameter settings is that when d < λ/2, if a solution exists, then it is unique. (See Exercise 1.) On the other hand, when d ≥ ρ, a solution is always guaranteed to exist (for any target t), and it is generally not unique. (See Exercise 2.) Easier variants of BDD and ADD are obtained by introducing a slackness factor γ ≥ 1, and strengthening the constrains on d to d < λ/(2γ) (for BDDγ ) and d ≥ γρ (for ADDγ ). Informally, BDDγ is the problem of ﬁnding the lattice vector closest to a target when the target is very close to the lattice, while ADDγ is the problem of ﬁnding a lattice vector not too far from the target, where far is measured with respect to the absolute bound ρ within which a solution is always guaranteed to exist. As usual, the approximation factor γ can be a function of the dimension. The problems SIVP, ADD and BDD are connected by some very interesting relations, which also play an important role in cryptography. To start with, observe that if the lattice is L = Zn , then CVP can be solved exactly and eﬃciently by rounding each coordinate of the target √ t to the closest integer, and this always results in a lattice vector within distance n/2 from the target. More generally, for an arbitrary lattice L, we can attempt to solve ADD or BDD as follows. Let B be a lattice basis and think of L = BZn as the integer lattice Zn distorted by the linear transformation deﬁned by B. In order to ﬁnd a lattice point close to a target t we may 1. ﬁrst apply the inverse transformation B−1 to get B−1 t, 2. round B−1 t to the closest integer vector B−1 t ∈ Zn , 3. map the the resulting integer vector to the lattice point v = BB−1 t. The quality of this rounding procedure can be analyzed in terms of the two quantities smin (B) = minn Bx / x

x∈R

smax (B) = maxn Bx / x

x∈R

which express by how much the transformation B can shrink or expand the length of a vector. Exercise 9. Show that√the above rounding procedure always returns a lattice point within distance n · smax (B)/2 from t. Moreover, if t is within distance

194

D. Micciancio

smin (B)/2 from the lattice, then the rounding procedure ﬁnds the (necessarily unique) lattice point within distance smin (B)/2 from t. So, in order to solve ADD (resp. BDD) we need to ﬁnd a basis B such that smax (B) is small (resp. smin (B) is large). This can be done by solving SIVPγ , either in the lattice L or in its dual L† . Let’s look Using an SIVPγ √ ﬁrst at ADD. √ oracle we can ﬁnd a basis B with maxi bi ≤ √nγλn ≤ 2 nγρ. (See Exercises 3 and 8.) It is also easy to check that smax (B) ≤ n maxi bi ≤ 2γnρ. This yields a solution to ADDn1.5 γ . In fact, by using a set of linearly independent vectors rather than a basis, in conjunction with a better rounding procedure based on orthogonalized projections, this can be improved to ADDγ √n . Theorem 3. For any γ ≥ 1, there is a polynomial time (dimension preserving) reduction from ADDγ √n to SIVPγ . Let’s now turn to BDD. This time we want smin (B) to be large. Since smin (B) = 1/smax(B† ), this can be achieved by computing a short dual basis B† using SIVPγ . As before, the basis satisﬁes smax (B† ) ≤ 2γnρ(L†), and therefore by the transference theorem smin (B) ≥ 1/(2γnρ† ) ≥ λ/(γn2 ). This yields a solution to BDDγn2 . As before, this can be improved to BDDγn using a better rounding method, and further improvements are probably possible. Theorem 4. For any γ ≥ 1, there is a polynomial time (dimension preserving) reduction from BDDγn to SIVPγ . In summary, Theorems 3 and 4 show that SIVP is the hardest of the three problems, in the sense that both ADD and BDD can be eﬃciently solved given an SIVP oracle, at least approximately, up to polynomial approximation factors. In fact, it is not hard to show that SIVPγ √n reduces to ADDγ . (This is implicit in the proof of the last two inequalities in Exercise 3.) So, ADD and SIVP are equivalent problems, up to small polynomial factors in the quality of the solution. Interestingly, BDD seems an easier problem than ADD and SIVP: BDD reduces to ADD and SIVP, but in the opposite direction reductions are known only under quantum algorithms! Giving a (classical, possibly randomized, but not quantum) reduction from ADD or SIVP to BDD is an important open problem in the complexity of lattice problems, and a problem that, as we will see later, has special cryptographic signiﬁcance. We conclude this section with a brief mention of the standard CVPγ and the Shortest Vector Problem SVPγ , which is the problem of ﬁnding a nonzero lattice vector of length at most cλ1 . These are the two most famous computational problems in the algorithmic study of point lattices. They are also the hardest: they are equivalent to each other (up to polynomial approximation factors), and SIVP reduces to them. However, no reduction (even quantum) is known in the opposite direction. In particular, it is not known how to build cryptographic function based on the conjectured (worst-case) intractability of SVP or CVP. Therefore, another very important open problem in the complexity of lattice problems is to give a reduction from SVP or CVP to SIVP.

The Geometry of Lattice Cryptography

3

195

Random Lattices and Lattice Cryptography

Lattice cryptography is based on the conjectured hardness of SIVP, i.e., the problem of computing (a set of n linearly independent) short vectors in a lattice. Intuitively, one can think of building a cryptosystem where public keys are lattices (e.g., represented by a basis consisting of long vectors), with the corresponding secret key given by the short lattice vectors. If ﬁnding short vectors is hard, then the secret key cannot be easily computed from the public description of the lattice. Moreover, the public/secret key pair can be used to encrypt/decrypt messages as follows.5 – Let L = BZn be a lattice and S a set of linearly independent lattice vectors, serving respectively as the public and secret key of the cryptosystem. – A message m is encrypted by encoding it as a short random vector x, selecting a “random” point v ∈ B† Zn in the dual lattice, and perturbing v by x, to yield ciphertext c = v + x. Notice that the ciphertext is a point within distance x from the dual lattice L† . The reason we use a dual lattice vector here is that a good basis S of L allows to solve BDD in L† . – If x < λ(L† )/2, then recovering x gives raise to a BDD problem. As seen in Theorem 4, the secret key S can be used to ﬁnd the lattice point v closest to the target ciphertext c, provided the error vector x is short enough. The message is then recovered as x = c − v. In summary, recovering the secret key S from the public description of the lattice B corresponds to a SIVP instance, while decrypting without the knowledge of the secret key is an instance of BDD. Of course, this is very informal, and leaves many important questions unanswered. What basis B should be used as the public key? How should the “random” lattice point v ∈ L† be selected? What probability distribution should be used/assumed on the error vector x? In what sense is the above scheme secure? For many of these questions, provably optimal answers are known [54]: both integer lattices L† ⊆ Zn and their cosets x + L† admit easily computable normal forms, or standard representatives.6 Since these standard representatives can be easily computed from any other representation of the lattice and its cosets, they provide the minimum possible amount of information to the adversary, and therefore the highest level of security. But what distribution x should be used to encode the messages, and, more importantly, 5

6

Here we treat encryption somehow informally, like textbook RSA, where the message is assumed to be somehow random, and we identify cryptosystems with the theoretical notion of trapdoor function families. The reader familiar with modern cryptography will certainly know that for cryptosystems to provide a reasonable level of security, they need to be randomized. The example here is only meant to provide the intuitive geometric intuition of how lattice cryptography works, without getting into technical details. For example, given a lattice basis B, a standard representative of the coset x + BZn is given by the unique point of x + BZn inside the parallelepiped P(B). For the basis B one may use the Hermite Normal Form [54].

196

D. Micciancio

what distribution should be used for the lattices L, L† to ensure that SIVP (in L) and BDD (in L† ) are computationally hard? The conjectured (worst-case) intractability of SIVP, ADD and BDD is not directly relevant here. In cryptography one needs computational problems that are hard to solve on the average: when you pick the secret key of a cryptographic function at random (according to an appropriate probability distribution), it is not enough to know that some key is computationally hard to break. You want to be conﬁdent that your randomly chosen key is hard to break with high probability. So, in order to formalize lattice cryptography, we need to introduce a notion of random lattice, so that when L is chosen according to this random distribution, SIVP on L is hard. 3.1

Random Lattices

The two most common distributions over n-dimensional lattices encountered in cryptography are deﬁned as follows. Fix positive integers k ≤ n ≤ q, where k serves as the main security parameter. Typically n is a small multiple of k (e.g., n = O(k) or n = O(k log k)) and q is a small prime with O(log k) bits. Notice that q is very small, not at all like the large primes (with O(k) bits) used in deﬁne number theoretic cryptography. For any matrix A ∈ Zk×n q n Λ⊥ q (A) = {x ∈ Z : Ax = 0 mod q}

Λq (A) = {x ∈ Zn : x = AT s mod q for some s ∈ Zkq }. Intuitively, Λq (A) is the lattice generated by the rows of A modulo q, while Λ⊥ q (A) is the set of solutions of the system of k linear equations modulo q deﬁned by the rows of A. It is easy to check that Λq (A) and Λ⊥ q (A) are subgroups of Zn , and therefore they are lattices. It is also apparent from the deﬁnition that these lattices are q-ary, i.e., they are periodic modulo q: one can take the ﬁnite set Q of lattice points with coordinates in {0, . . . , q − 1}, and recover the whole lattice by tiling the space with copies Q + qZn . The matrix A used to represent them is not a lattice basis. A lattice basis B for the corresponding lattices can be eﬃciently computed from A using linear algebra, but it is typically not needed: cryptographic operations are usually expressed and implemented directly in terms of A. A random lattice is obtained by picking A ∈ Zk×n q uniformly at random. The corresponding distributions are denoted Λq (n, k) and Λ⊥ q (n, k). Regarding the error vector distribution, one possible way to choose x may be to select it uniformly at random among all integer vectors of bounded norm, but for technical reasons a diﬀerent distribution is often more convenient. In lattice cryptography, perturbation vectors are typically chosen according to the Gaussian distribution Dα which picks each x ∈ Zn with probability (roughly) proportional to exp(−π x/α 2 ). The Gaussian distribution has the analytical advantage that the probability of a point x depends only on its norm x , and still the coordinates of x can be chosen independently (namely, each with probability proportional to exp(−π|xi /α|2 )). It can be shown that when x is chosen

The Geometry of Lattice Cryptography

197

√ according to this distribution (over Zn ), Pr{ x > nα} is exponentially small. So, by truncating a negligibly small tail, Dα can be regarded √ as a probability distribution over the integer vectors of norm bounded by α n. Before providing a theoretical justiﬁcation for using these distributions in cryptography, let us try to get a better understanding of the lattices. We begin by observing that Λq (A) and Λ⊥ q (A) are dual to each other, up to a scaling factor q. † ⊥ † Exercise 10. Show that Λq (A) = q · Λ⊥ q (A) and Λq (A) = q · Λq (A) . In par⊥ n k×n ticular, det(Λq (A)) · det(Λq (A)) = q . Moreover, for any A ∈ Zq , we have k n−k det(Λ⊥ . q (A)) ≤ q and det(Λq (A)) ≥ q

To better understand the relation between Λq (n, k) and Λ⊥ q (n, k), it is convenient ⊥ ˜ to deﬁne two auxiliary distributions. Let Λq (n, k) be the conditional distribution of a lattice chosen according to distributions Λ⊥ q (n, k), given that the lattice has determinant exactly q k . Similarly, let Λ˜q (n, k) be the conditional distribution of a lattice chosen according to Λq (n, n − k), given that the determinant of the lattice is q n−k . In both cases, when q is a prime, the condition is equivalent to requiring that the rows of A are linearly independent modulo q. How much do these conditional distributions diﬀer from the original ones? Not much. Exercise 11. Show that if A ∈ Zk×n is chosen uniformly at random, then q k n−k Pr{det(Λ⊥ } ≥ 1 − 1/q n−k . q (A)) = q } = Pr{det(Λq (A)) = q

˜ Moreover, the conditional distributions Λ˜⊥ q (n, k) = Λq (n, n − k) are identical. So, for typical settings of the parameters (e.g., n = 2k), lattices chosen according k to Λ⊥ q (n, k) or Λq (n, n−k) have determinant q except with negligible probability ≤ q −k , and the distributions Λ⊥ q (n, k) and Λq (n, n − k) are almost identical ˜ because they are both statistically close to Λ˜⊥ q (n, k) = Λq (n, n − k). We now move to estimating the parameters of lattices chosen √ according to these distributions. Clearly, we always have λ1 ≤ λn ≤ q and ρ ≤ nq because qI gives a set of n linearly independent vectors of length q. In fact, from Theorem 1 √ and Exercise 10, we know that λ(L) ≤ nq k/n for any L ∈ Λ⊥ q (n, k). This upper bound is essentially tight. Exercise 12. There is a constant that if L is chosen according to √ k/nδ > 0 such Λ⊥ } ≤ 1/2n. [Hint: consider all integer vectors q (n, k), then Pr{λ(L) < δ nq √ of norm at most δ nq k/n and use a union bound.] What about the other parameters λn , ρ? Also these parameters are very close to Minkowski’s upper bound with high probability. Exercise 13. If L is chosen according to Λ⊥ q (n, k), then Pr{λ(L) ≤ 2ρ(L) < 1 √ k/n n · n · q } ≤ 1/2 .[Hint: Prove the bound for Λq (n, n − k) ≈ Λ⊥ q (n, k) instead, δ and use duality and the transference theorems.]

198

D. Micciancio

˜⊥ In summary, when a lattice is chosen according to Λ⊥ q (n, k) ≈ Λq (n, k) = Λ˜q (n, n − k) ≈ Λq (n, n − k), all the parameters λ1 , . . . , λn , ρ are within a con√ stant factor from Minkowski’s bound nq k/n with overwhelming probability. This provides very useful information about what it means to solve ADD or BDD on these random lattices. Let L be a lattice chosen according to Λ⊥ q (n, k), and let t = v + x be a lattice point v ∈ L perturbed by a noise vector x cho√ sen according to distribution Dcqk/n over B(c nq k/n ). Finding a lattice point √ within distance c nq k/n from t is an ADD problem when c > δ and it is a BDD problem when c < 1/δ. 3.2

One Way Functions

We now show that solving ADD and BDD on random lattices can be formulated as the problem of inverting the function fA (x) = Ax

(mod q).

(1)

when the matrix A ∈ Zk×n is chosen uniformly at random and the input is q chosen according to distribution Dcqk/n , for some c > 0. Of course, given a matrix A and a value b = fA (x), recovering a possible preimage of y under fA is just a matter of performing some linear algebra, and it can be eﬃciently accomplished in a variety of ways. In order to get a hard-to-invert function √ from (1), one needs to regard Dcqk/n as a probability distribution over B(c nq k/n ), and consider fA as a function with this domain. The relation between inverting fA (x) = b (i.e., ﬁnding small solutions to the inhomogeneous system Ax = b (mod q)) and lattice problems is easily explained. Using linear algebra, one can eﬃciently ﬁnd an arbitrary solution t ∈ Znq to the system, but this solution will generally have large entries and not belong to the domain of fA . Linear algebra gives us a little more than an arbitrary solution. It tells us that any solution to the inhomogeneous system Ax = b can be expressed as the sum of any ﬁxed speciﬁc solution t to At = b (mod q) and a solution z to the homogeneous system Az = 0 (mod q). But the set of solutions to the homogeneous system is precisely the lattice Λ⊥ q (A). So, ﬁnding a small x = t + z is equivalent to ﬁnding a lattice point −z ∈ Λ⊥ q (A) within distance t − (−z) = x from the target t. In summary, inverting fA is√equivalent to the problem of ﬁnding lattice k/n vectors in Λ⊥ from the target. Depending of the q (A) within distance c nq value of c, this is the ADDc problem (for c > δ ) or the BDD1/c problem (for c < 1/δ). These two diﬀerent ranges for c also correspond to very diﬀerent statistical properties of the function fA . Namely, when c < 1/δ , the function fA is injective with high probability (which corresponds to BDD having at most one solution). √ Exercise 14. Show that for c ≤ δ the function fA with domain B(c nq k/n ) is injective with overwhelming probability over the choice of A. Similarly, when c > δ, the function fA is surjective with high probability. In fact, all the output values in Znq have almost the same probability under fA (Dcqk/n ).

The Geometry of Lattice Cryptography

199

√ Theorem 5 ([34]). Let q be a prime, n ≥ 2k log q and c ≥ ω( log n). Then, with overwhelming probability over the choice of A ∈ Zk×m , if x is chosen q according to distribution Dc·qk/n , then f (x) is statistically close to uniform over Zkq . Now, let us go back to the question: are lattice problems hard on average when the lattice is chosen according to Λ⊥ q (n, k)? A ﬁrst answer to this question was given in a seminal paper of Ajtai [6] which marks the beginning of modern day lattice based cryptography. Ajtai’s result has been subsequently improved and simpliﬁed in a sequence of papers [20,56,63]. The strongest known result is the one due to [63] with some additional reﬁnements in [34]. Theorem 6 (Implicit in [63,34]). For any polynomially bounded n(k) and c(k) > δ, and any ω(k log k) ≤ q(k) ≤ k O(n/k) , the following holds. Assume SIVPγ is hard in the √ worst case (over k-dimensional lattices) for approximation factors γ(k) = ω(c n · k log k√· q k/n ). Then fA is a one-way function with input distribution Dcqk/n (over B(c nq k/n )). Typically, q(k) = k O(1) is a small polynomial, n = k log q = O(k log k), and c > δ a constant. With this parameters, Theorem 6 gives a provably secure oneway function under the assumption that SIVPγ is hard to approximate (in the worst-case, over k-dimensional lattices) within factors γ(k) = ω(k log k) almost linear in the dimension of the lattice. Recall that the one-way function inversion problem from Theorem 6 is an average case version of ADD for lattices distributed according to Λ⊥ q (n, k). Also, in the previous section we have seen that SIVP is equivalent to ADD, up to small polynomial approximation factors. So, Theorem 6 can be interpreted as a connection between the worst-case and average-case complexity of ADD, and justiﬁes using distribution Λ⊥ q (n, k) to select the lattice. If ADD is not hard on average for this distribution, then it is not hard at all! What about BDD, i.e., inverting fA (x) when the input x is distributed according to Dcqk/m for some small c < 1/δ? For these inputs the behavior of the function fA is quite diﬀerent: we have seen that fA is injective (with high probability) and therefore it is trivially collision resistant: there exist no x = y such that fA (x) = fA (y). But for injective functions, collision resistance is a trivial property and does not imply any form of one-wayness. Still, it is possible to prove that fA is a one-way function, using the stronger complexity assumption that SIVP is hard to solve in the worst case even by quantum algorithms. Since we are interested in solving BDD in the dual lattice, we state this result in terms † ⊥ of the (scaled) dual lattice distribution q(Λ⊥ q (n, k)) = Λq (n, k) ≈ Λq (n, n − k). Theorem 7 (Implicit√in [73]). For all c < 1/δ, some polynomial n(k) = k O(1) , and prime q > (2 k/c)n/(n−k) , the following holds true. Assume SIVPγ is hard in the worst case (over k-dimensional lattices) even for quantum algorithms, √ for approximation factor γ(k) = ω( k)·k·q k/n /c. Then fA is a one-way function √ with input distribution Dc·q1−k/n and domain B(c n · q 1−k/n ). Notice that Dc·q1−k/n is the natural distribution to use when the lattice is chosen 1−k/n according to Λ⊥ q (n, n − k) because these lattices have root determinant q

200

D. Micciancio

rather than q k/n . Theorem 7 asserts that fA is a one-way function only when n(k) is some polynomially bounded, but unspeciﬁed, function of k. This is because the BDD problem in Λq (n, k) lattices in [73] is described as a learning problem (the LWE problem, for Learning With Errors), where each coordinate of the lattice corresponds to a noisy sample, and the adversary is assumed to have access to an arbitrary (but polynomially bounded) number of samples. In principle, it should be possible to tighten the proof in [73] to show that a relatively small value of n(k) is suﬃcient for the reduction to go through. But, anyway, this is not the main diﬀerence if we compare the result with Theorem 6. A more fundamental diﬀerence between Theorem 6 and Theorem 7 is that the latter is based on a quantum reduction. The reason the power of quantum computers is needed for establishing the one-wayness of fA in the BDD setting is currently one of the mysteries of lattice cryptography, and it reﬂects our poor understanding of the relation between ADD and BDD problems. Even in the classical setting of worstcase reductions, we know how to approximately reduce BDD to ADD, but no reduction is known in the opposite direction, at least classically. So, BDD seems an easier problem than ADD also in the worst-case. In the average case setting, Theorems 6 and 7 can be summarized as saying that ADD on random q-ary lattices is as hard as approximating SIVP with classical algorithms, while BDD on random q-ary lattices is as hard as approximating SIVP but with quantum algorithms, under which SIVP may be easier to solve. We remark that for certain choices of parameters (e.g., when q = 2O(n) is exponentially large,) it is possible to show that the function fA of Theorem 7 is one-way, assuming the classical worst-case hardness of some lattice problems [67]. The worst-case problems can be BDD, a special version of SVP with “unique solution”, or the problem or approximating the value of λ without necessarily ﬁnding a short lattice vector. All these problems are essentially equivalent to each other [50] and conceivably easier than SIVP. So, in some settings, Theorem 7 can be interpreted as a connection between the worst-case and average-case complexity of BDD under classical algorithms. In summary, assuming that SIVP is hard to solve in the worst case with classic algorithms, is enough to build simple cryptographic functions, like one-way functions, and as will will see in the next section, also collision resistant hash functions, and commitment schemes. The construction of public key encryption and more complex cryptographic primitives requires the stronger assumption that either BDD is hard in the worst-case (if an exponentially large value of q can be used) or SIVP and ADD are hard even under quantum algorithms. Providing a classical reduction from SIVP and ADD to BDD is an important open problem as it would unify the complexity assumptions used for the construction of all lattice cryptography primitives.

4

Applications

In this section we give some representative examples of cryptographic applications based on lattices. The presentation is informal, emphasizing the geometric

The Geometry of Lattice Cryptography

201

ideas behind the constructions, rather than the technical details of the cryptographic deﬁnitions and security proofs. Here we aim primarily at illustrating the wide range of applications enabled by lattice cryptography, and highlight how cryptographic constructions, which are often described simply in terms of matrices and indistinguishability properties, often have a natural geometric interpretation. However, we remark that all the constructions discussed here can be properly analyzed and proved secure based on the conjectured one-wayness of the functions fA discussed in Section 3, or, using Theorems 6 and 7, the worst-case hardness of SIVP under classical or quantum algorithms. For precise statements, security deﬁnitions and formal proofs, the reader is referred to the original papers, cited throughout this section. 4.1

Hash Functions

Hash functions are (keyed) functions that compress long input strings, into shorter digests, and still have the property that they are collision resistant: it is computationally hard to ﬁnd two distinct inputs x = y such that f (x) = f (y). For injective functions, this property holds trivially, so we may regard collision resistance as a computational relaxation of injectivity, and say that fA is pseudo-injective. Now, consider the function fA for A ∈ Zk×m and input x ← Dcqk/n . It q turns out that (when A is chosen at random) the proof of Theorem 6 shows that the function fA is not only one-way, but also collision resistant. By Exercise 14, when c < 1, the function fA is injective with high probability, and collisions fA (x) = fA (y) do not exist. When c grows beyond 1, the function fA is no longer injective over the domain associated to the input distribution. Still, as proved in [63,34], the function fA is collision resistant for the parameters of Theorem 6. The intuitive explanation is that inverting fA for these parameters corresponds to the ADD problem, while√ﬁnding collisions cork/m responds to ﬁnding short vectors Λ⊥ ) ≈ λn . Since q (A) of length O( nq ADD and SIVP are equivalent problems, we can expect that these two problems have roughly the same complexity. In this informal argument, we are hiding the fact that the equivalence between ADD and SIVP is proved for the worst-case formulation of the problems, and that SIVP requires to ﬁnd not just one, but many short vectors. Still, the intuition holds, and fA can be formally shown to be collision resistant, on the average, when A is chosen at random. Hash functions are usually designed using ad-hoc methods, without supporting proofs of security. Lattice based functions like fA are not competitive with such ad-hoc constructions. Still, using special classes of algebraic lattices it is possible to get much more eﬃcient constructions [57,48,68,51] which sill are provably hard to break under worst-case assumptions (albeit over a restricted class of lattices) and at the same time are comparable to ad-hoc constructions in terms of eﬃciency.

202

4.2

D. Micciancio

Pseudorandom Generators

When c > 1 is large enough, the output of fA (x) with x ← Dcqk/n is statistically close to random, and the function compresses its input. When c < 1 is small, the output of fA (x) is certainly not close to uniform, just because the function stretches its input to a longer string. Still, the output of fA (x) can be shown computationally indistinguishable from uniform, i.e., it is pseudo-random. In other words, fA is a pseudo-random generator family, indexed by A. The pseudo-randomness of fA (x) was proved in a loose sense already in [73]. The proof in [73] was loose in the sense that in order to prove the pseudorandomness of the output of fA for certain values of the parameters n, k, q, it required to assume that fA was one-way for a (polynomially related, but) much bigger value of n. Recently [61] it has been shown that such increase is not necessary, and the pseudo-randomness of the output can be proved for the same values of the parameters as in Theorem 7. We recall that inverting fA for this range of parameters corresponds to a BDD problem. On the other hand, distinguishing the output of fA from random corresponds to the decisional version of BDD: the input to BDD is a point close to the lattice, while the uniform distribution in space can be shown to be far from the lattice with high probability [36,37]. So, proving that the output of fA is pseudo-random roughly corresponds to showing that if you can tell if a target is either very close, or very far from a lattice, then given a point very close to the lattice one can ﬁnd the closest lattice point. For a reduction of this kind in the worst-case setting, see [45]. 4.3

Commitment Schemes

A (non-interactive) commitment scheme is a function that allows one party to commit to a value x while keeping it hidden. At a later time the same party can open the commitment, and reveal the value x. The commitment should be binding, i.e., it should be computationally infeasible to commit to a value x and then open the commitment to a diﬀerent value x . The construction of commitment schemes from lattices is very simple: in order to commit to a value represented by a short vector x, choose a short random vector r, and output fA (x, r), where matrix A is wide enough that it can be multiplied by the concatenation of the two vectors (x; r). This commitment scheme was proposed in [41], where it is also used for the construction of a concurrently secure identiﬁcation scheme based on lattices. We remark that, depending on the value of the parameters, this construction gives both commitments that are statistically hiding and computationally binding, and also commitments that are computationally hiding, but statistically binding. The fact that this construction gives a secure commitment scheme is closely related to the fact that fA is both (pseudo-)random and (pseudo-) injective. (Pseudo-)randomness of the output gives the hiding property of the commitment, while the (pseudo-)injective property implies that the commitment

The Geometry of Lattice Cryptography

203

is binding. Depending on the setting of the parameter c in the input distribution Dcqk/n , either the hiding or binding property (but not both) may hold in a strong statistical sense. 4.4

Public Key Encryption

A public key encryption scheme can be built along the lines described at the beginning of Section 3: – The public key is a random lattice Λq (A). The secret key is given by a short basis or set of short vectors S. – Ciphertexts are obtained by encoding a message into a short vector x, and computing a standard representative for the coset x + (Λq (A))† . Such representative can be computed, for example, as fA (x). – Decryption is a BDD problem in the dual lattice (Λq (A))† , and it can be performed using the short basis S. We already know that recovering the secret key from the public one corresponds to an SIVP problem in Λq (A), which by Theorem 6 is hard on average. Similarly, decryption is an average-case BDD problem, and by Theorem 7 it is also hard. All we are missing is a method to generate a random lattice Λq (A) together with a short set of lattice vectors S. In fact, we don’t need a full set of n-linearly independent short lattice vectors in order to build a public key encryption scheme. Assume for simplicity we want to encrypt a single bit message. Then the message 0 may be encoded as a point close to the (dual) lattice, while 1 may be encoded as a random point, which will be far from the lattice with high probability. Then, we can decrypt using a single short (dual) vector s as follows: recall that a lattice vector s ∈ L can be used to partition the dual lattice into layers L†i = {v ∈ L† | v · s = i}. One can check if a point x is either close or far from any of these layers by computing the product x · s and checking if the result is close to an integer. This idea was already present in the cryptosystem of Ajtai and Dwork [7], which didn’t use q-ary lattices. So, how can we obtain this short vector s when the lattice (Λq (A)) is chosen at random? Remember that ﬁnding short vectors in these random lattices is a hard problem, and it has to be as it corresponds to the key recovery problem of our cryptosystem. Fortunately we can plant s in the lattice during key generation process. We recall that Λq (A) is the lattice generated by the columns of AT modulo q. So, we can choose the ﬁrst k − 1 columns of AT = [AT0 |p] truly at random, and then we set the last column to our secret short vector s. Of course, including s in the public key would reveal the key used for decryption. So, we mask s with a random lattice point, and set p = AT0 w + s, where w is chosen at random. Geometrically, instead of revealing s, we reveal its coset s + Λq (A0 ), from which recovering s (or an equally short vector) is a computationally hard task. Clearly, the lattice Λq ([AT0 , p]T ) still contains s, which can be used for decryption.

204

D. Micciancio

Encrypting one bit at a time is not very practical, but a much more eﬃcient system can be obtained along similar lines by embedding not just one, but many short vectors, as ﬁrst proposed in [71], for a syntactically diﬀerent formulation of the encryption scheme. The geometric description presented in these notes can be instantiated in several ways, depending on the length of s. If s is long enough, then p will be distributed almost uniformly at random, and A will follow the distribution Λq (n, k). If s is shorter, then the distribution of A will be far from Λq (n, k), but it will be indistinguishable from it because p is pseudorandom. Either setting of parameters is reasonable, as long as the length of the vector x used during the encryption process is chosen accordingly. If s is longer, then x should be shorter, while if s is shorter, then x can be longer. The corresponding proof of security will invoke a statistical or computational argument when analyzing either the public key or encrypted ciphertext. These two instantiations were originally presented as two diﬀerent cryptosystems in [73] and [34], where they are described using quite diﬀerent matrix formulations. Looking at these two cryptosystems through the lens of geometry and duality, [59] showed that they are in fact the same cryptosystem instantiated for diﬀerent values of the parameters. [59] also suggested that the eﬃciency of the cryptosystem could be improved using intermediate values of the parameters and using a computational argument both during the analysis of the public key and encrypted ciphertext. Signiﬁcant eﬃciency improvements were later demonstrated in [44] which proposed and analyzed a concrete instantiation of the general scheme of [59]. This is the currently most eﬃcient example of a lattice based cryptosystem based on general lattices. 4.5

Identity Based Encryption

We have seen how generating a random lattice together with a single short vector is enough to build a public key encryption scheme. But some applications do require a full basis. For example, this is the case for the identity based encryption (in the random oracle model) of [34]. We recall that an identity based encryption is an encryption scheme where public keys are simply given by the parties’ names or public identiﬁers. The corresponding secret keys are computed by a trusted entity using a master secret key. In the scheme of [34] A is generated together with a full trapdoor basis S, which is used as a master key. Then the (identity based) public key encryption scheme is built as described in the previous subsection, with one twist: instead of choosing the short decryption key s at random, and embedding it in a new independently generated lattice, the short vector s is extracted (sampled) from the lattice Λq (A) using the master key S to solve an instance of the ADD problem. More speciﬁcally, given a target point t that represents a party’s identity, a corresponding secret key is computed using S to ﬁnd a lattice point not too far from t. The problem of generating a random lattice together with a full short basis is studied in [5,8]. A new, much simpler and more eﬃcient method to generate such lattices has been recently discovered in [62].

The Geometry of Lattice Cryptography

205

Lattice based identity based encryption has been further studied in [22,1,2] yielding both solutions in the standard model, and solutions to the more general hierarchical identity based encryption problem, where identities form a hierarchy, and each node can delegate limited decryption capabilities to its children.

5

Conclusion

Lattice cryptography is still a very young and fast developing area, and still it has produced a substantial body of work. We conclude these notes by providing some pointers to the literature that can be used to get a deeper understanding of the concepts touched upon in this tutorial. Other surveys presenting additional perspectives on lattice cryptography include [58,64,74,75]. Work in lattice cryptography can be roughly categorized into two groups: foundational work aimed at establishing that fA is a one way function, and cryptographic applications that use the properties of fA to solve more complex cryptographic tasks. We start with the ﬁrst category. Works in this area tend to be mathematically more involved, but that’s where the real “magic” happens: connecting the average-case and worst-case complexity of lattice problems. The reader interested in getting a feeling of how the proof of Theorem 6 works, is referred to the survey [58]. From there, the reader may proceed to [63,73]. Both papers make use of harmonic analysis and high dimensional Fourier transform, which is very useful both to yield stronger and more concise proofs. Still on the foundational side, is a line of research initiated in [57] aimed at substantially improving the eﬃciency of lattice cryptography, while maintaining a form of provable security, based on the use of lattices with special algebraic structure. We expect these lattices will play a major role in bringing lattice cryptography closer to practice. The original paper [57] is still a good starting point to learn about these lattices. For more recent developments in this direction see [48,69,68,51,53,61]. On the application side, there is a huge variety of cryptographic problems that have been recently solved using lattices. But in the end, all solutions ultimately rely on the results and techniques developed in a small number of foundational papers. In fact, many applications can be described and analyzed simply using as a starting point the one-wayness of the function fA , without even any explicit reference to lattices. But we believe that the geometric perspective advocated in these notes can be of great value to understand those constructions, and interpreting the variety of solutions in the literature in terms of a small number of basic geometric concepts. For the reader interested in applications, here we give some pointers to relevant papers, organized by category. For public key encryption see [73,72,52,81,67,40,44]. For digital signatures see [34,49,47,18]. For group and ring signatures see [35,77,24]. For identity based cryptography see [2,22,1,78]. For (fully) homomorphic encryption see [33,28,80,65,29,3,82,79,32,25,19,31,30]. For zero-knowledge proofs and identiﬁcation protocols see [23,46,76,83,41,70]. For still more cryptographic primitives and protocols see [66,14,13,17,12,39,9,71,16].

206

D. Micciancio

References 1. Agrawal, S., Boneh, D., Boyen, X.: Eﬃcient lattice (H)IBE in the standard model. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 553–572. Springer, Heidelberg (2010) 2. Agrawal, S., Boneh, D., Boyen, X.: Lattice basis delegation in ﬁxed dimension and shorter-ciphertext hierarchical IBE. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 98–115. Springer, Heidelberg (2010) 3. Melchor, C.A., Gaborit, P., Herranz, J.: Additively homomorphic encryption with d-operand multiplications. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 138–154. Springer, Heidelberg (2010) 4. Ajtai, M.: The shortest vector problem in L2 is NP-hard for randomized reductions (extended abstract). In: Proceedings of STOC 1998, pp. 10–19. ACM, New York (1998) 5. Ajtai, M.: Generating hard instances of the short basis problem. In: Wiedermann, J., Van Emde Boas, P., Nielsen, M. (eds.) ICALP 1999. LNCS, vol. 1644, pp. 1–9. Springer, Heidelberg (1999) 6. Ajtai, M.: Generating hard instances of lattice problems. Complexity of Computations and Proofs, Quaderni di Matematica 13, 1–32 (2004); Preliminary version in STOC 1996 7. Ajtai, M., Dwork, C.: A public-key cryptosystem with worst-case/average-case equivalence. In: Proceedings of STOC 1997, pp. 284–293. ACM, New York (1997) 8. Alwen, J., Peiker, C.: Generating shorter bases for hard random lattices. In: Proceedints of STACS, pp. 75–86 (2009) Invited to Theory of Computing Systems, 48(3), 535–553. Prelim. Version in STACS 2009 9. Applebaum, B., Cash, D., Peikert, C., Sahai, A.: Fast cryptographic primitives and circular-secure encryption based on hard learning problems. In: Halevi, S. (ed.) CRYPTO 2009. LNCS, vol. 5677, pp. 595–618. Springer, Heidelberg (2009) 10. Arora, S., Babai, L., Stern, J., Sweedyk, E.Z.: The hardness of approximate optima in lattices, codes, and systems of linear equations. Journal of Computer and System Sciences 54(2), 317–331 (1997); Preliminary version in FOCS 1993 11. Banaszczyk, W.: New bounds in some transference theorems in the geometry of numbers. Mathematische Annalen 296, 625–635 (1993) 12. Barak, B., Haitner, I., Hofheinz, D., Ishai, Y.: Bounded key-dependent message security. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 423–444. Springer, Heidelberg (2010) 13. Bendlin, R., Damg˚ ard, I.: Threshold decryption and zero-knowledge proofs for lattice-based cryptosystems. In: Micciancio, D. (ed.) TCC 2010. LNCS, vol. 5978, pp. 201–218. Springer, Heidelberg (2010) 14. Bendlin, R., Damg˚ ard, I., Orlandi, C., Zakarias, S.: Semi-homomorphic encryption and multiparty computation. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 169–188. Springer, Heidelberg (2011) 15. Bl¨ omer, J., Seifert, J.-P.: On the complexity of computing short linearly independent vectors and short bases in a lattice. In: Proceedings of STOC 1999, pp. 711–720. ACM, New York (1999) 16. Boneh, D., Freeman, D.: Homomorphic signatures for polynomial functions. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 149–168. Springer, Heidelberg (2011)

The Geometry of Lattice Cryptography

207

17. Boneh, D., Freeman, D.: Linearly homomorphic signatures over binary ﬁelds and new tools for lattice-based signatures. In: Catalano, D., Fazio, N., Gennaro, R., Nicolosi, A. (eds.) PKC 2011. LNCS, vol. 6571, pp. 1–16. Springer, Heidelberg (2011) 18. Boyen, X.: Lattice mixing and vanishing trapdoors: A framework for fully secure short signatures and more. In: Nguyen, P.Q., Pointcheval, D. (eds.) PKC 2010. LNCS, vol. 6056, pp. 499–517. Springer, Heidelberg (2010) 19. Brakerski, Z., Vaikuntanathan, V.: Fully homomorphic encryption from ring-LWE and security for key dependent messages. In: CRYPTO (to appear, 2011) 20. Cai, J.-Y., Nerurkar, A.P.: An improved worst-case to average-case connection for lattice problems (extended abstract). In: Proceedings of FOCS 1997, pp. 468–477. IEEE, Los Alamitos (1997) 21. Cai, J.-Y., Nerurkar, A.P.: Approximating the SVP to within a factor (1 + 1/dim ) is NP-hard under randomized reductions. Journal of Computer and System Sciences 59(2), 221–239 (1999); Preliminary version in CCC 1998 22. Cash, D., Hofheinz, D., Kiltz, E., Peikert, C.: Bonsai trees, or how to delegate a lattice basis. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 523–552. Springer, Heidelberg (2010) 23. Cayrel, P.-L., Lindner, R., R¨ uckert, M., Silva, R.: Improved zero-knowledge identiﬁcation with lattices. In: Heng, S.-H., Kurosawa, K. (eds.) ProvSec 2010. LNCS, vol. 6402, pp. 1–17. Springer, Heidelberg (2010) 24. Cayrel, P.-L., Lindner, R., R¨ uckert, M., Silva, R.: A Lattice-Based Threshold Ring Signature Scheme. In: Abdalla, M., Barreto, P.S.L.M. (eds.) LATINCRYPT 2010. LNCS, vol. 6212, pp. 255–272. Springer, Heidelberg (2010) 25. Coron, J.-S., Mandal, A., Naccache, D., Tibouchi, M.: Fully-homomorphic encryption over the integers with shorter public-keys. In: CRYPTO (to appear, 2011) 26. Dinur, I.: Approximating SV P ∞ to within almost-polynomial factors is NP-hard. Theoretical Computer Science 285(1), 55–71 (2002); Preliminary version in CIAC 2000 27. Dinur, I., Kindler, G., Raz, R., Safra, S.: Approximating CVP to within almostpolynomial factors is NP-hard. Combinatorica 23(2), 205–243 (2003); Preliminary version in FOCS 1998 28. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Proceedings of STOC, pp. 169–178. ACM, New York (2009) 29. Gentry, C.: Toward basing fully homomorphic encryption on worst-case hardness. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 116–137. Springer, Heidelberg (2010) 30. Gentry, C.: Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, Report 2011/277 (2011) 31. Gentry, C., Halevi, S.: Fully homomorphic encryption without squashing using depth-3 arithmetic circuits. Cryptology ePrint Archive, Report 2011/279 (2011) 32. Gentry, C., Halevi, S.: Implementing gentrys fully-homomorphic encryption scheme. In: Paterson, K.G. (ed.) EUROCRYPT 2011. LNCS, vol. 6632, pp. 129– 148. Springer, Heidelberg (2011) 33. Gentry, C., Halevi, S., Vaikuntanathan, V.: A simple BGN-type cryptosystem from lwe. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 506– 522. Springer, Heidelberg (2010) 34. Gentry, C., Peikert, C., Vaikuntanathan, V.: Trapdoors for hard lattices and new cryptographic constructions. In: Proceedings of STOC, pp. 197–206. ACM, New York (2008)

208

D. Micciancio

35. Gordon, D., Katz, J., Vaikuntanathan, V.: A group signature scheme from lattice assumptions. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 395–412. Springer, Heidelberg (2010) 36. Guruswami, V., Micciancio, D., Regev, O.: The complexity of the covering radius problem. Computational Complexity 14(2), 90–121 (2005); Preliminary version in CCC 2004 37. Haviv, I., Lyubashevsky, V., Regev, O.: A note on the distribution of the distance from a lattice. Discrete and Computational Geometry 41(1), 162–176 (2009) 38. Haviv, I., Regev, O.: Tensor-based hardness of the shortest vector problem to within almost polynomial factors. In: Proceedings of STOC, pp. 469–477. ACM, New York (2007) 39. Katz, J., Vaikuntanathan, V.: Smooth projective hashing and password-based authenticated key exchange based on lattices. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 636–652. Springer, Heidelberg (2009) 40. Kawachi, A., Tanaka, K., Xagawa, K.: Multi-bit cryptosystems based on lattice problems. In: Okamoto, T., Wang, X. (eds.) PKC 2007. LNCS, vol. 4450, pp. 315– 329. Springer, Heidelberg (2007) 41. Kawachi, A., Tanaka, K., Xagawa, K.: Concurrently secure identiﬁcation schemes and ad hoc anonymous identiﬁcation schemes based on the worst-case hardness of lattice problems. In: Pieprzyk, J. (ed.) ASIACRYPT 2008. LNCS, vol. 5350, pp. 372–389. Springer, Heidelberg (2008) 42. Khot, S.: Hardness of approximating the shortest vector problem in lattices. Journal of the ACM 52(5), 789–808 (2005); Preliminary version in FOCS 2004 43. Khot, S.: Hardness of approximating the shortest vector problem in high Lp norms. J. of Computer Systems Sciences 72(2), 206–219 (2006); Preliminary version in FOCS 2003 44. Lindner, R., Peikert, C.: Better key sizes (and attacks) for LWE-based encryption. In: Kiayias, A. (ed.) CT-RSA 2011. LNCS, vol. 6558, pp. 319–339. Springer, Heidelberg (2011) 45. Liu, Y.-K., Lyubashevsky, V., Micciancio, D.: On bounded distance decoding for general lattices. In: D´ıaz, J., Jansen, K., Rolim, J.D.P., Zwick, U. (eds.) APPROX 2006 and RANDOM 2006. LNCS, vol. 4110, pp. 450–461. Springer, Heidelberg (2006) 46. Lyubashevsky, V.: Lattice-based identiﬁcation schemes secure under active attacks. In: Cramer, R. (ed.) PKC 2008. LNCS, vol. 4939, pp. 162–179. Springer, Heidelberg (2008) 47. Lyubashevsky, V.: Fiat-Shamir with aborts: Applications to lattice and factoringbased signatures. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 598–616. Springer, Heidelberg (2009) 48. Lyubashevsky, V., Micciancio, D.: Generalized compact knapsacks are collision resistant. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 144–155. Springer, Heidelberg (2006) 49. Lyubashevsky, V., Micciancio, D.: Asymptotically eﬃcient lattice-based digital signatures. In: Canetti, R. (ed.) TCC 2008. LNCS, vol. 4948, pp. 37–54. Springer, Heidelberg (2008) 50. Lyubashevsky, V., Micciancio, D.: On bounded distance decoding, unique shortest vectors, and the minimum distance problem. In: Halevi, S. (ed.) CRYPTO 2009. LNCS, vol. 5677, pp. 577–594. Springer, Heidelberg (2009) 51. Lyubashevsky, V., Micciancio, D., Peikert, C., Rosen, A.: SWIFFT: a modest proposal for FFT hashing. In: Nyberg, K. (ed.) FSE 2008. LNCS, vol. 5086, pp. 54–72. Springer, Heidelberg (2008)

The Geometry of Lattice Cryptography

209

52. Lyubashevsky, V., Palacio, A., Segev, G.: Public-key cryptographic primitives provably as secure as subset sum. In: Micciancio, D. (ed.) TCC 2010. LNCS, vol. 5978, pp. 382–400. Springer, Heidelberg (2010) 53. Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 1–23. Springer, Heidelberg (2010) 54. Micciancio, D.: Improving lattice based cryptosystems using the Hermite normal form. In: Silverman, J.H. (ed.) CaLC 2001. LNCS, vol. 2146, pp. 126–145. Springer, Heidelberg (2001) 55. Micciancio, D.: The shortest vector problem is NP-hard to approximate to within some constant. SIAM Journal on Computing 30(6), 2008–2035 (2001); Preliminary version in FOCS 1998 56. Micciancio, D.: Almost perfect lattices, the covering radius problem, and applications to Ajtai’s connection factor. SIAM Journal on Computing 34(1), 118–169 (2004); Preliminary version in STOC 2002 57. Micciancio, D.: Generalized compact knapsacks, cyclic lattices, and eﬃcient oneway functions. Computational Complexity 16(4), 365–411 (2007); Preliminary version in FOCS 2002 58. Micciancio, D.: Cryptographic functions from worst-case complexity assumptions. In: The LLL Algorithm: Survey and Applcations, Springer, Heidelberg (2009) 59. Micciancio, D.: Duality in lattice cryptography. In: Proceedings of PKC. LNCS. IACR, Springer (May 2010) (invited talk); Slides available from author’s web page 60. Micciancio, D., Goldwasser, S.: Complexity of Lattice Problems: a cryptographic perspective. The Kluwer International Series in Engineering and Computer Science, vol. 671. Kluwer Academic Publishers, Boston (2002) 61. Micciancio, D., Mol, P.: Pseudorandom knapsacks and the sample complexity of LEW search-to-decision reductions. In: CRYPTO (to appear, 2011) 62. Micciancio, D., Peikert, C.: Trapdoor for lattices: Simpler, tighter, faster, smaller (manuscript, 2011) 63. Micciancio, D., Regev, O.: Worst-case to average-case reductions based on Gaussian measure. SIAM Journal on Computing 37(1), 267–302 (2007); Preliminary version in FOCS 2004 64. Micciancio, D., Regev, O.: Lattice-based cryptography. In: Post-quantum Cryptography. Springer, Heidelberg (2008) 65. Ogura, N., Yamamoto, G., Kobayashi, T., Uchiyama, S.: An improvement of key generation algorithm for Gentry homomorphic encryption scheme. In: Echizen, I., Kunihiro, N., Sasaki, R. (eds.) IWSEC 2010. LNCS, vol. 6434, pp. 70–83. Springer, Heidelberg (2010) 66. O’Neill, A., Peikert, C., Waters, B.: Bi-deniable public-key encryption. In: CRYPTO (to appear, 2011) 67. Peikert, C.: Public-key cryptosystems from the worst-case shortest vector problem. In: Proceedings of STOC, pp. 333–342. ACM, New York (2009) 68. Peikert, C., Rosen, A.: Eﬃcient collision-resistant hashing from worst-case assumptions on cyclic lattices. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 145–166. Springer, Heidelberg (2006) 69. Peikert, C., Rosen, A.: Lattices that admit logarithmic worst-case to average-case connection factors. In: Proceedings of STOC, pp. 478–487. ACM, New York (2007) 70. Peikert, C., Vaikuntanathan, V.: Noninteractive statistical zero-knowledge proofs for lattice problems. In: Wagner, D. (ed.) CRYPTO 2008. LNCS, vol. 5157, pp. 536–553. Springer, Heidelberg (2008)

210

D. Micciancio

71. Peikert, C., Vaikuntanathan, V., Waters, B.: A framework for eﬃcient and composable oblivious transfer. In: Wagner, D. (ed.) CRYPTO 2008. LNCS, vol. 5157, pp. 554–571. Springer, Heidelberg (2008) 72. Peikert, C., Waters, B.: Lossy trapdoor functions and their applications. In: Proceedings of STOC, pp. 187–196. ACM, New York (2008) 73. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. Journal of ACM 56(6), 34 (2009); Preliminary version in STOC 2005 74. Regev, O.: Learning with errors over rings. In: Hanrot, G., Morain, F., Thom´e, E. (eds.) ANTS-IX. LNCS, vol. 6197, pp. 3–3. Springer, Heidelberg (2010) 75. Regev, O.: The learning with errors problem (invited survey). In: CCC, pp. 191–204 (2010) 76. R¨ uckert, M.: Adaptively secure identity-based identiﬁcation from lattices without random oracles. In: Garay, J.A., De Prisco, R. (eds.) SCN 2010. LNCS, vol. 6280, pp. 345–362. Springer, Heidelberg (2010) 77. R¨ uckert, M.: Lattice-based blind signatures. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 413–430. Springer, Heidelberg (2010) 78. R¨ uckert, M.: Strongly unforgeable signatures and hierarchical identity-based signatures from lattices without random oracles. In: Sendrier, N. (ed.) PQCrypto 2010. LNCS, vol. 6061, pp. 182–200. Springer, Heidelberg (2010) 79. Smart, N., Vercauteren, F.: Fully homomorphic encryption with relatively small key and ciphertext sizes. In: Nguyen, P.Q., Pointcheval, D. (eds.) PKC 2010. LNCS, vol. 6056, pp. 420–443. Springer, Heidelberg (2010) 80. Stehl´e, D., Steinfeld, R.: Faster fully homomorphic encryption. In: Abe, M. (ed.) ASIACRYPT 2010. LNCS, vol. 6477, pp. 377–394. Springer, Heidelberg (2010) 81. Stehl´e, D., Steinfeld, R., Tanaka, K., Xagawa, K.: Eﬃcient public key encryption based on ideal lattices. In: Matsui, M. (ed.) ASIACRYPT 2009. LNCS, vol. 5912, pp. 617–635. Springer, Heidelberg (2009) 82. van Dijk, M., Gentry, C., Halevi, S., Vaikuntanathan, V.: Fully homomorphic encryption over the integers. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 24–43. Springer, Heidelberg (2010) 83. Xagawa, K., Tanaka, K.: Zero-knowledge protocols for NTRU: Application to identiﬁcation and proof of plaintext knowledge. In: Pieprzyk, J., Zhang, F. (eds.) ProvSec 2009. LNCS, vol. 5848, pp. 198–213. Springer, Heidelberg (2009)

Quantitative Information Flow and Applications to Diﬀerential Privacy M´ ario S. Alvim, Miguel E. Andr´es, Konstantinos Chatzikokolakis, and Catuscia Palamidessi INRIA and LIX, Ecole Polytechnique, France

Abstract. Secure information ﬂow is the problem of ensuring that the information made publicly available by a computational system does not leak information that should be kept secret. Since it is practically impossible to avoid leakage entirely, in recent years there has been a growing interest in considering the quantitative aspects of information ﬂow, in order to measure and compare the amount of leakage. Information theory is widely regarded as a natural framework to provide ﬁrm foundations to quantitive information ﬂow. In this notes we review the two main information-theoretic approaches that have been investigated: the one based on Shannon entropy, and the one based on R´enyi min-entropy. Furthermore, we discuss some applications in the area of privacy. In particular, we consider statistical databases and the recently-proposed notion of diﬀerential privacy. Using the information-theoretic view, we discuss the bound that diﬀerential privacy induces on leakage, and the trade-oﬀ between utility and privacy.

1

Introduction

In the last few decades the amount of information ﬂowing through computational systems has increased dramatically. Never before in history has a society been so dependent on such a huge amount of information being generated, transmitted and processed. It is expected that this vertiginous trend of increase will continue in the near future, reinforcing the need for eﬃcient and safe ways to cope with this reality. One of the concerns in the use of computational systems is to avoid the leakage of secret information through public observables. If some information is supposed to be conﬁdential, then unauthorized users should not be allowed to infer such information from the output or the behavior of the system. Ideally we would like systems to be completely secure, i.e. protect the secret information entirely, but in practice this goal is usually impossible to achieve. The following example illustrates some of the issues.

This work has been partially supported by the project ANR-09-BLAN-0169-01 PANDA and by the INRIA DRI Equipe Associ´ee PRINTEMPS. The work of Miguel E. Andr´es has been supported by the LIX-Qualcomm postdoc fellowship.

A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 211–230, 2011. c Springer-Verlag Berlin Heidelberg 2011

212

M.S. Alvim et al.

Example 1. Consider the password-checker algorithm Table 1. Passworddeﬁned in Table 1, where K1 K2 . . . KN is the passchecker pseudocode word, and x1 x2 . . . xN is the string entered by the user. An attacker may obtain the password by simout := OK ply trying a string, if she is so lucky to guess the for i = 1, ..., N do string that matches the password. if xi = Ki then Furthermore, even if the attacker makes the wrong out := FAIL guess, she still obtains some (small) amount of inforexit() mation: the information that the password is not the end if string she just entered, thus restricting the guessing end for range of potential further attempts. Worse yet, the algorithm is subject to timing attacks: the attacker may be able to determine from the duration of the execution how many iterations are performed, thus inferring a preﬁx of the password even in case of failure. Thus it is important to express the amount of leakage in quantitative terms, so to be able to assess whether a system is better than another, although they may both be insecure. Another reason to consider the quantitative aspects is that the system may have a probabilistic behavior. This is the case, for instance, of anonymity protocols, which use randomization to obfuscate the link between the identity of the culprit and the observable outcome. Example 2. DC-Net is a protocol for anonymous broadcasting based on the paradigm of the dining 1 0 1 cryptographers [7]. 1 The participants in the protocol are assumed to 1 1 be the vertexes of a graph, whose edges are associ1 0 culprit 0 ated to binary coins which are visible to the adjab=1 0 1 1 cent vertexes. The protocol works as follows: sup0 0 0 pose that a participant x wants to broadcast one bit 1 b. Then, all coins get ﬂipped, and each participant 0 1 0 reads the value of the coins which are visible to her and computes the binary sum of these values. Then Fig. 1. Example of a all participants but x declare the result, while x (bidc-net narily) adds b to her result, and then declares it. Fig. 1 illustrates the situation. It is easy to see that, since each coin is counted twice, the global contribution of all coins is 0. Hence the binary sum of all the declarations gives b, thus achieving the goal of making b public. Furthermore thanks to the “noise” created by the coins, it is generally impossible for an adversary to determine with certainty the culprit, i.e. the vertex who is broadcasting b. We will see that a stronger property actually holds: if the coins are fair, and the graph is connected, then for an external adversary the probability of each vertex to be the culprit does not change after she observes the declarations.

Quantitative Information Flow and Applications to Diﬀerential Privacy

213

Several authors have proposed to use concepts from information theory to model information ﬂow and to deﬁne the leakage in a quantitative way. So far, most of the approaches were based on Shannon entropy [25,24,30,8,21,22,5]. This is probably due to the fact that Shannon entropy is the most established and useful notion of entropy in information theory, because of its mathematical properties and its correspondence with the channel transmission rate. Nevertheless, other notions have been considered, and argued to be more appropriate for security in certain scenarios. These include: R´enyi min-entropy [27,29], guessing entropy [23], and marginal guesswork [26]. The common idea in these information-theoretic approaches is that a system can be seen as a channel in the information-theoretic sense, where the secret is the input and the observables are the output. The entropy of the input provides a measure of its vulnerability, i.e. how easy is for an attacher to discover the secret. Therefore, as argued in [19], the notion of entropy should be chosen according to the model of attacker, and to the way we extimate the success of the attack. Normally, the entropy of a random variable represents its uncertainty, hence the vulnerability is anti-monotonic on the entropy. Independently from the intended model of attacker, the notion of leakage can be expressed in a uniform way as the diﬀerence between the secret’s initial uncertainty i.e. the uncertainty before we run the system, and the secret’s remaining uncertainty, i.e. the uncertainty after we run the system and observe its outcome: information leakage = initial uncertainty − remaining uncertainty

(1)

In general the observation of the outcome should increase the probabilistic knowledge, and consequently decrease the uncertainty of the secret. Therefore we expect the result of (1) to be non-negative. This is indeed the case, for all the notions of entropy mentioned above, when the channel is deterministic. In the more general case of a probabilistic channel, however, the non-negativeness is ensured only for Shannon entropy and for R´enyi min-entropy. In these notes we aim at encompassing also the probabilistic case, hence we will restrict to the latter two notions of entropy. Recently, the above notions have been applied also to the problem of statistical disclosure control in the area of databases. The goal is to learn properties of the population as a whole, while maintaining the privacy of individuals. For example, in medical research it is desirable to collect the personal medical information of a large number of individuals. Researchers or public authorities can calculate a series of statistics from the sample and decide, for instance, how much money the health care system should spend next year in the treatment of a speciﬁc disease. It desirable, however, that the participation in the sample will not harm the privacy of any individual: usually people do not want to have disclosed their speciﬁc status with respect to a given disease, or other personal matters. Some studies show indeed that when individuals are guaranteed anonymity and privacy they tend to be more cooperative in giving in personal information [18]. The fact that the answer is publicly available, however, constitutes a threat for the privacy of the individuals. For instance, assume that we are interested

214

M.S. Alvim et al.

in the query “what is the percentage of individuals with a given disease?”. The addition of an individual to the database will modify the percentage, and reveal whether the individual has the disease or not. A common solution to the above problem is to introduce some output perturbation mechanism based on randomization: instead of the exact answer to the query, we report a “noisy” answer. Namely, we use some randomized function K which produces values according to some probability distribution. Of course, depending oin the distribution, it may still be possible to guess the value of an individual with a high probability of success, i.e. there may still be a risk of violating privacy. The notion of diﬀerential privacy, due to Dwork [12,15,13,14], is a proposal to control such risk. The idea is to say that K satisﬁes -diﬀerential privacy (for some > 0) if the ratio between the probabilities that two adjacent databases give the same answer is bound by e , where by “adjacent” we mean that the databases diﬀer for only one individual. Often we will abbreviate “-diﬀerential privacy” as -d.p. Obviously, the smaller is , the greater is the privacy protection. In particular, when is close to 0 the output of K is nearly independent from the input (all distributions are almost equal). Unfortunately, such K is practically useless. The utility, i.e. the capability to retrieve accurate answers from the reported ones, is the other important characteristic of K, and it is clear that there is a trade-oﬀ between utility and privacy. On the other hand, these two notions are not the complete opposite of each other, because utility concerns the relation between the reported answer and the real answer, while privacy is concerns the relation between the reported answer and the information in the database. This asymmetry makes more interesting the problem of ﬁnding a good compromise between the two. At this point, we would like to remark an intriguing analogy between the area of diﬀerential privacy and that of quantitative information ﬂow (QIF), both in the motivations and in the conceptual framework. At the motivational level, the concern about privacy is akin the concern about information leakage. At the conceptual level, the randomized function K can be seen as an informationtheoretic channel, and the limit case of = 0, for which the privacy protection is total, corresponds to the case in which the answer to the query does not add any information about the input distribution, and therefore the leakage is 0. In this notes we recall the notion of diﬀerential privacy and its implications, in light of the min-entropy framework developed for QIF. In particular, we explore the relation between -d.p., leakage, and utility. The rest of the paper is organized as follows: in the next section we discuss and compare Shannon entropy and R´enyi min-entropy, and their interpretation in terms of attack model. In Section 3 we illustrate the interpretation of systems as channels, from the point of view of leakage, and we review the main results concerning the information-theoretic approaches based on these two notions of entropy. In Section 4 we show that -diﬀerential privacy induces a bound on the leakage of statistical databases. In Section 5 we show that -diﬀerential privacy induces a bound also on their utility, and we present a method to deﬁne a

Quantitative Information Flow and Applications to Diﬀerential Privacy

215

randomization mechanism that gives the maximum utility while providing diﬀerential privacy. In Section 6 we discuss some related work on the relation between diﬀerential privacy and QIF.

2

Adversary Models and Entropy Notions

In this section we review the two main notions of entropy used in the literature on quantitative information ﬂow, and we discuss the relation with the model of attacker. In general we consider the kind of threats that in the model of [19] are called brute-force guessing attacks, which can be summarized as follows: The goal of the adversary is to determine the value of a random variable. He can make a series of queries to an oracle. Each query must have a yes/no answer. In general the adversary is adaptive, i.e. he can choose the next query depending on the answer to the previous ones. We assume that the adversary knows the probability distribution. In the following, X, Y denote two discrete random variables with carriers X = {x1 , . . . , xn }, Y = {y1 , . . . , ym }, and probability distributions pX (·), pY (·), respectively. We will use X ∧ Y to represent the random variable with carrier X × Y and joint probability distribution pX∧Y (x, y) = pX (x) · pY |X (y | x), where pY |X (y | x) is the conditional probability that Y = y when X = x. The notation X ·Y will denote the random variable with carrier X ×Y and probability distribution deﬁned as product, i.e. pX·Y (x, y) = pX (x) · pY (y). Clearly if X and Y are independent we have X ∧ Y = X · Y . We shall omit the subscripts on the probabilities when they are clear from the context. In general, pX (·) is called a priori distribution of X, and the conditional probability pX|Y (· | ·) is called a posteriori distribution. We will also refer to the individual probabilities p(x) and p(x|y) as the a priori an the a posteriori probabilities of x. 2.1

Shannon Entropy

In this section we illustrate a model of adversary that corresponds to Shannon entropy. The ideas discussed here are based on the works [26,20]. Deﬁnition 1. A Shannon adversary is characterized by questions are of the form “is X ∈ X ?” where X ⊆ X . Her goal is to determine exactly the value of X, and to do it as fast as possible, i.e. to ask as few questions as possible. Clearly the strategy of the adversary in choosing the sets X will have an inﬂuence on the number of queries necessary to determine the value of X. Intuitively the best strategy is to choose X so that its mass probability is as close as possible to that of X \ X , where X is the set of values that are currently determined as possible for X. The next example illustrates the situation. Example 3. Consider the set X = {a, b, c, d, e, f, g, h}, and assume that the probability is distributed as follows: 1 1 1 p(a) = p(b) = p(c) = p(d) = p(e) = p(f ) = p(g) = p(h) = 4 8 16

216

M.S. Alvim et al.

Fig. 2 illustrates a possible strategy of the adversary. The leafs represent the secrets, i.e. the elements of X . Each internal node is labeled with a set X ⊆ X , representing the query “is X ∈ X ?”. Depending on the answer to the query, the adversary will continue searching in the left or in the right subtree. In this particular case, at each step the left and Fig. 2. Search space the right subtrees have the same probability mass, and as we will see later this means that the strategy is optimal. For a given strategy σ of the adversary, let nσ (x) be the number of questions that are needed to determine the value of X when X = x. For instance, in the example above, we have nσ (a) = 2, and nσ (e) = 4. The expected value of nσ is: 1 1 1 11 Enσ = p(a) nσ (a) + . . . + p(f ) nσ (f ) = 2 × 2 + 2 × 3 + 4 × 4 = 4 8 16 4 Note that if we replace nσ (·) by log2 p(·) in the above deﬁnition of Enσ , we obtain the formula for Shannon entropy (cf. Deﬁnition 2). Coming back to Fig. 2, we wish remark that each element x ∈ X is uniquely associated to the path from the root to the leaf associated to x. If we label the arcs of the tree with 0 (left) and 1 (right), then each x ∈ X can be represented by the binary string relative to the path. The problem of minimizing the expected value of nσ corresponds therefore to that of ﬁnding the optimal coding, i.e. the coding that minimize the expected number of bits needed to represent the elements of X . Shannon entropy is also related to the optimal coding, and more precisely it corresponds to the expected length of the code. Deﬁnition 2 ([28]). The Shannon entropy of a random variable X is deﬁned as H(X) = − p(x) log2 p(x) x∈X

The minimum value H(X) = 0 is obtained when p(·) is concentrated on a single value (i.e. when p(·) is a Dirac measure). The maximum value H(X) = log2 |X | is obtained when p(·) is the uniform distribution. Usually the base of the logarithm is set to be 2 and, correspondingly, the entropy is measured in bits. Proposition 1. H(X) is a lower bound for the expected value of nσ (·), with respect to all possible strategies σ. Note that the lover bound mentioned in the above proposition (i.e. the entropy) is actually achieved in Example 3. Indeed, a simple calculation shows that H(X) = 11 4 . In general the lower bound H(X) can be achieved when for each x ∈ X we have that p(x) is a power of 2, and we can organize the questions as a tree, with the characteristic that each node (question) splits the probability mass in two. We end this section with the deﬁnition of conditional entropy.

Quantitative Information Flow and Applications to Diﬀerential Privacy

Deﬁnition 3. The conditional entropy of X given Y is H(X | Y ) = p(y) H(X | Y = y) y∈Y

where H(X | Y = y) = −

217

(2)

p(x|y) log2 p(x|y)

x∈X

It is possible to prove that 0 ≤ H(X | Y ) ≤ H(X). The minimum value, 0, is obtained when X is completely determined by Y . The maximum value, H(X), is obtained when Y reveals no information about X, i.e. when X and Y are independent. 2.2

R´ enyi Min-Entropy

In this section we illustrate a model of adversary that corresponds to R´enyi min-entropy. This material is based on [29]. Deﬁnition 4. A one-try adversary is a model of adversary in which the adversary can ask exactly one question of the form: “is X = x?”. Her goal is to maximize the probability of guessing the right element in just one single try. Of course, the best strategy for the adversary consists in choosing the x with the maximum probability. Therefore the measure of success of this kind of adversary is maxx∈X p(x). We discuss now how the probability of success is related to the R´enyi minentropy. Let us ﬁrst give a quick overview of the context in which this concept originated. In [27], R´enyi introduced a one-parameter family of entropy measures, intended as a generalization of Shannon entropy. The R´enyi entropy of order α (α > 0, α = 1) of a random variable X is deﬁned as 1 Hα (X) = log2 p(x)α 1−α x∈X

R´enyi’s motivations were of axiomatic nature: Shannon entropy satisﬁes four axioms, namely symmetry, continuity, value 1 on the Bernoulli uniform distribution, and the chain rule1 : H(X ∧ Y ) = H(X | Y ) + H(Y )

(3)

Shannon entropy is also the only function that satisﬁes those axioms. However, if we replace (3) with a weaker property representing the additivity of entropy for independent distributions: H(X · Y ) = H(X) + H(Y ) 1

The original axiom, called the grouping axiom, does not mention the conditional entropy. However it corresponds to the chain rule if the conditional entropy is deﬁned as in (2).

218

M.S. Alvim et al.

then there are more functions satisfying the axioms, among which all those of the R´enyi’s family. Shannon entropy is obtained by taking the limit of Hα as α approaches 1. In fact we can easily prove, using l’Hˆ opital’s rule, that def H1 (X) = lim Hα (X) = − p(x) log2 p(x) α→1

x∈X

We are particularly interested in the limit of Hα as α approaches ∞. This is called min-entropy. It can be proven that def

H∞ (X) =

lim Hα (X) = − log2 max p(x)

α→∞

x∈X

which gives the connection with the model of adversary described in Deﬁnition 4. As for the α-generalization of the conditional entropy, R´enyi did not deﬁne it, and there is no agreement on what it should be. Various researchers, including Cachin [4], have considered the following deﬁnition, based on (2): p(y) Hα (X | Y = y) HαCachin (X | Y ) = y∈Y

which, as α → ∞, becomes Cachin (X | Y ) = − H∞

y∈Y

p(y) log2 max p(x | y) x∈X

An alternative proposal for H∞ (· | ·) came from [11,10], and was again advocated by Smith in his seminal paper [29]: Smith H∞ (X | Y ) = − log2 y∈Y maxx∈X p(x, y) (4) The most interesting version of H∞ (X | Y ), in terms of security, seems to be that of Smith: Indeed, y∈Y maxx∈X p(x, y) represents the expected value of the a posteriori probability of the adversary’s success. The complement of this expected value is also known as probability of error or Bayes risk, and has been used as a measure of information ﬂow also by other authors [6]. Observing Y decreases the probability of error, and consequently we can prove formally Smith that H∞ (X | Y ) ≤ H∞ (X), with equality if X and Y are independent. This inequality will ensure that the leakage is always nonnegative, and it is therefore Smith Cachin another reason to choose H∞ (· | ·): the alternative H∞ (· | ·) does not have this feature. For the sake of simplicity, in the following we will omit the superscript “Smith” and simply write H∞ (· | ·).

3

Information Leakage and Channels

In this section we consider the interpretation of the leakage in computational systems in terms of channels, and we review some results concerning the Shannon and the min-entropy approaches.

Quantitative Information Flow and Applications to Diﬀerential Privacy

219

The main idea is that if we focus on the problem of the leakage of secret information trough public observables, then we can regard a computational system as an information-theoretic channel, where the secrets are the inputs, and the observables are the outputs. The following examples illustrate this precept revisiting the examples in the introduction. Example 4. Consider again the password-checker algorithm deﬁned in Example 1. Assume that the password is 110. Depending on whether the adversary is supposed to be able to detect the number of iterations or not, we have two diﬀerent channels, as illustrated in the ﬁgure below. Note that the program is deterministic, and this is reﬂected by the fact that, in both channels, each input corresponds to only one output.

000 001 010 011 100 101 110 111

Fail

OK

(a)

No iteration count

000 001 010 011 100 101 110 111

(b)

(Fail,1) (Fail,2) (Fail,3) OK

Counting the iterations

Fig. 3. Channels associated with the program in Example 1. The password is 110.

Example 5. Consider the dc-nets of Example 2, and consider a particularly simple case: a net with two nodes only, connected by an arc, as illustrated in Fig. 4(a). The channel, illustrated in Fig. 4(b), takes as input the identity of the agent and the bit that she wants to broadcast2 . On the right-hand side, the binary strings represent the declarations of the nodes. Note that this program is not deterministic. The input (a, 1), for instance, can produce two diﬀerent outcomes: the declaration 10, produced when the coin is 0 (assuming that the ﬁrst bit from left to right is the declaration of a). The other possible declaration, 01, is produced when the coin is 1. Deﬁnition 5. An information-theoretic channel is a triple (X, Y, M ) where X and Y are two random variables representing the input and the output, and M is a matrix (called the channel matrix) which contains the conditional probabilities p(y|x) for each x ∈ X and each y ∈ Y. Example 6. The matrices associated to the channels in Figures 3(a) and 3(b) are given in Tables 2(a) and 2(b), respectively. Note that they contain only 0’s and 1’s. This is characteristic of deterministic channels. 2

One may argue that the bit to be broadcasted should not be considered as a secret, and that having it as input will alter the computation of the leakage. This is a valid observation, but there is no harm in representing this bit as input, because we can still remove its contribution to the leakage when we make the calculation.

220

M.S. Alvim et al.

a (a)

b

coin

(a, 1)

10

(b, 1)

01

(a, 0)

00

(b, 0)

11

(b)

A dc-net with

Channel associated with

the two-nodes dc-net

two nodes

Fig. 4. A simple dc-net and its channel Table 2. Channel matrices for the password-checker algorithm (a) 000 001 010 011 100 101 110 111

(b)

No iter. c.

Fail 1 1 1 1 1 1 0 1

OK 0 0 0 0 0 0 1 0

Counting the iterations

(Fail, 1) (Fail, 2) (Fail, 3) OK 000 001 010 011 100 101 110 111

1 1 1 1 0 0 0 0

0 0 0 0 1 1 0 0

0 0 0 0 0 0 0 1

0 0 0 0 0 0 1 0

Example 7. Consider again Example 2. The channel matrix depends on the coins: whether they are biased, and how much biased they are. Here we consider the simple two-nodes net of Example 5 and two particular situations: the case in which the coins are unbiased (cf. Table 3(a)) and the case in which they are all biased in favor of 0, and more precisely, they give 0 with probability 23 (cf. Table 3(b)). Table 3. Channel matrices for the two-node dc-net of Example 5 (b) (a)

Biased

(p(0) =

Fair coin

10 01 00 11

2 3,

coin

p(1) =

1 3)

10 01 00 11

(a, 1)

1 2

1 2

0 0

(a, 1)

2 3

1 3

0 0

(b, 1)

1 2

1 2

0 0

(b, 1)

1 3

2 3

0 0

(a, 0) 0 0

1 2

1 2

(a, 0) 0 0

2 3

1 3

(b, 0) 0 0

1 2

1 2

(b, 0) 0 0

2 3

1 3

We consider now the deﬁnition of leakage for the two approaches.

Quantitative Information Flow and Applications to Diﬀerential Privacy

3.1

221

Leakage in the Shannon Approach

As explained in the introduction, we regard the leakage as the diﬀerence between the initial uncertainty and the remaining uncertainty after the observation. Since the uncertainty is represented by the entropy, the leakage can be represented by the well-known (information-theoretic) notion of mutual information, which is the diﬀerence between the entropy of the input and the conditional entropy of the input given the output: Deﬁnition 6. The mutual information between A and B is deﬁned as I(X; Y ) = H(X) − H(X | Y ) and it measures the amount of information about X that we gain by observing Y. It can be shown that I(X; Y ) = I(Y ; X) and 0 ≤ I(X; Y ) ≤ H(X). The notion of capacity, deﬁned as C = max I(X; Y ) pX (·)

is often used to represent the worst-case leakage. We end this section with some relevant properties of the Shannon capacity: Proposition 2. The capacity of a channel is 0 if and only if all the rows of the matrix are identical. The capacity is maximum (i.e. equal to H(X)) if and only if each column has only one positive value. 3.2

Leakage in the Min-entropy Approach

Also in the case of min-entropy the leakage can be expressed using a concept analogous to mutual information3 : I∞ (X; Y ) = H∞ (X) − H∞ (X | Y ) and analogously for the capacity: C∞ = max I∞ (X; Y ) pX (·)

This min-entropy variant of the capacity is called min-capacity. Only a weak form of Proposition 2 holds for C∞ , namely, only the if part. Proposition 3. The min-capacity of a channel is 0 if all the rows of the matrix are identical. The capacity is maximum (i.e. equal to H∞ (X)) if each column has only one positive value. 3

Some authors, including Smith, prefer avoiding the notation I∞ and referring to it as “mutual information” because some of the properties of Shannon mutual information are not satisﬁed in the min-entropy case. In particular, symmetry is not satisﬁed, i.e. I∞ (X; Y ) = I∞ (Y ; X) in general.

222

4

M.S. Alvim et al.

Diﬀerential Privacy: Leakage

In this section we discuss the notion of diﬀerential privacy and we show the relation with the notion of leakage. Let us start with some deﬁnitions. Let Ind be a ﬁnite set of individuals that may participate in a database and Val a ﬁnite set of possible values for the attribute of interest of these individuals. In order to capture in a uniform way the presence/absence of an individual in the database, as well as its value, we enrich the set of possible values with an element a representing the absence of the individual. Thus the set of all possible databases is the set X = V Ind , where V = Val ∪ {a}. We will use u and v to denote the cardinalities of Ind and V , |Ind | and |V |, respectively. Hence we have that |X | = v u . A database x can be represented as a u-tuple v0 v1 . . . vu−1 where each vi ∈ V is the value of the corresponding individual. Two databases x, x are adjacent (or neighbors), written x ∼ x , if they diﬀer for the value of exactly one individual. For instance, for u = 3, v0 v1 v2 and v0 w1 v2 , with w1 = v1 , are adjacent. The structure (X , ∼) forms an undirected graph. Intuitively, diﬀerential privacy is based on the idea that a randomized query function provides suﬃcient protection if the ratio between the probabilities of two adjacent databases to give a certain answer is bound by e , for some given > 0. Formally: Deﬁnition 7 ([14]). A randomized function K from X to Z satisﬁes diﬀerential privacy if for all pairs x, x ∈ X , with x ∼ x , and all S ⊆ Z, we have that: Pr [K(x) ∈ S] ≤ e × Pr [K(x ) ∈ S] The above deﬁnition takes into account the possibility that Z is a continuous domain. In our case, since Z is ﬁnite, the probability distribution is discrete, and we can rewrite the property of -d.p. more simply as (using the notation of conditional probabilities, and considering both quotients): 1 Pr [Z = z|X = x] ≤ ≤ e e Pr [Z = z|X = x ]

for all x, x ∈ X with x ∼ x , and all z ∈ Z

where X and Z represent the random variables associated to X and Z, respectively. 4.1

Graph Symmetries

In this section we explore some classes of graphs that allow us to derive a strict correspondence between -d.p. and the a posteriori entropy of the input. Let us ﬁrst recall some basic notions. Given a graph G = (V, ∼), the distance d(v, w) between two vertices v, w ∈ V is the number of edges in a shortest path connecting them. The diameter of G is the maximum distance between any two vertices in V. The degree of a vertex is the number of edges incident to it. G is

Quantitative Information Flow and Applications to Diﬀerential Privacy

223

called regular if every vertex has the same degree. A regular graph with vertices of degree k is called a k-regular graph. An automorphism of G is a permutation σ of the vertex set X , such that for any pair of vertices x, x , if x ∼ x , then σ(x) ∼ σ(x ). If σ is an automorphism, and v a vertex, the orbit of v under σ is the set {v, σ(v), . . . , σ k−1 (v)} where k is the smallest positive integer such that σ k (v) = v. Clearly, the orbits of the vertices under σ deﬁne a partition of V. The following two deﬁnition introduce the classes of graphs that we are interested in. The ﬁrst class is well known in literature. Deﬁnition 8. Given a graph G = (V, ∼), we say that G is distance-regular if there exist integers bi , ci , i = 0, ..., d such that for any two vertices v, w in V with distance i = d(v, w), there are exactly ci neighbors of w in Gi−1 (x) and bi neighbors of v in Gi+1 (x), where Gi (x) is the set of vertices y of G with d(x, y) = i. The next class is a variant of the VT (vertex-transitive) class: Deﬁnition 9. A graph G = (V, ∼) is VT+ (vertex-transitive +) if there are n automorphisms σ0 , σ1 , . . . σn−1 , where n = |V|, such that, for every vertex v ∈ V, we have that {σi (v) | 0 ≤ i ≤ n − 1} = V. In particular, the graphs for which there exists an automorphism σ which induces only one orbit are VT+ : in fact it is suﬃcient to deﬁne σi = σ i for all i from 0 to n − 1. From graph theory we know that neither of the two classes subsumes the other. They have however a non-empty intersection, which contains in particular all the structures of the form (V Ind , ∼), i.e. the database domains. Proposition 4. The structure (X , ∼) = (V Ind , ∼) is both a distance-regular graph and a VT+ graph. The two symmetry classes deﬁned above, distance-regular and VT+ , will be used in the next section to transform a generic channel matrix into a matrix with a symmetric structure, while preserving the a posteriori min entropy and the -d.p.. This is the core of our technique to establish the relation between diﬀerential privacy and quantitive information ﬂow, depending on the structure induced by the database adjacency relation. 4.2

Deriving the Relation between Diﬀerential Privacy and QIF on the Basis of the Graph Structure

This section discusses a general technique for determining the relation between -diﬀerential privacy and leakage, and between -diﬀerential privacy and utility, depending on the graph structure induced by ∼ and f . The idea is to use the symmetries of the graph structure to transform the channel matrix into an equivalent matrix with certain regularities, which allow to establish the link between -diﬀerential privacy and the a posteriori min entropy.

224

M.S. Alvim et al.

Let us illustrate brieﬂy this transformation. Consider a channel whose matrix M has at least as many columns as rows. First, we transform M into a matrix M in which each of the ﬁrst n columns has a maximum in the diagonal, and the remaining columns are all 0’s. Second, under the assumption that the input domain is distance-regular or VT+ , we transform M into a matrix M whose diagonal elements are all the same, and coincide with the maximum element of M , which we denote here by maxM . We are now going to present formally our the technique. Let us ﬁrst ﬁx some notation: In the rest of this section we consider channels with input A and output B, with carriers A and B respectively, and we assume that the probability distribution of A is uniform. Furthermore, we assume that |A| = n ≤ |B| = m. We also assume an adjacency relation ∼ on A, i.e. that (A, ∼) is an undirected graph structure. With a slight abuse of notation, we will also write i ∼ h when i and h are associated to adjacent elements of A, and we will write d(i, h) to denote the distance between the elements of A associated to i and h. We note that a channel matrix M satisﬁes -d.p. if for each column j and for each pair of rows i and h such that i ∼ h we have that: 1 Mi,j ≤ ≤ e . e Mh,j M The a posteriori entropy of a channel with matrix M will be denoted by H∞ (A|B).

Theorem 1. Consider a matrix M , and let r be a row of M . Assume that (A, ∼) is either distance-regular or VT+ , and that M satisﬁes -d.p. For each distance d from 0 to the diameter of (A, ∼), let nd be the number of nodes j at distance d from r. Then we have that: 1 M H∞ (A|B) ≥ − log2 nd e d

(5)

d

Note that this bound is tight, in the sense that we can build a matrix for which (5) holds with equality. In the next section, we will see how to use this theorem for establishing a bound on the leakage and on the utility. 4.3

Application to Leakage

As already hinted in the introduction, we can regard K as a channel with input X and output Z. From Proposition 4 we know that (X , ∼) is both distance-regular and VT+ , we can therefore apply Theorem 1. Let us ﬁx a particular database x ∈ X . The number of databases at distance d from x is u nd = (v − 1)d (6) d

Quantitative Information Flow and Applications to Diﬀerential Privacy

225

where u = |Ind | and v = V . In fact, recall that x can be represented as a u-tuple with values in V . We need to select d individuals in the u-tuple and then change their values, and each of them can be changed in v − 1 diﬀerent ways. Using the nd from (6) in Theorem 1 we obtain a binomial expansion in the denominator, namely: M H∞ (X|Z) ≥ − log2

u d=0

u d

1 (v − 1)d

e

(u−d)

= −u log2

e v − 1 + e

e u

which gives the following result: Theorem 2. If K satisﬁes -d.p., then for the uniform input distribution the information leakage is bound from above as follows: I∞ (X; Z) ≤ u log2

5

v e v − 1 + e

Diﬀerential Privacy: Utility

We turn now our attention to the issue of utility. We focus on the case in which K is oblivious, which means that it depends only on the (exact) answer to the query, i.e. on the value of f (x), and not on x. An oblivious function can be decomposed in the concatenation of two channels, one representing the function f , and the other representing the randomization mechanism H added as output perturbation. The situation is illustrated in Figure 5. The standard way to deﬁne utility is by means of guess and gain functions. The functionality of the ﬁrst is guess : Z → Y, and it represents the user’s strategy to retrieve the correct answer form the reported one. The functionality Utility

X dataset

Y

f

real answer

query

H randomization mechanism

Z reported answer

K (-diff. priv. randomized function) Leakage Fig. 5. Schema of an oblivious randomized function

226

M.S. Alvim et al.

of the latter is gain : Y × Y → R. the value gain(y, y ) represents the reward for guessing the answer y when the correct answer is y . The utility U can then be deﬁned as the expected gain: U(Y, Z) = p(y, z) gain(guess(z), y) y,z

We focus here on the so-called binary gain function, which is deﬁned as 1 if y = y gain(y, y ) = 0 otherwise This kind of function represents the case in which there is no reason to prefer an answer over the other, except if it is the right answer. More precisely, we get a gain if and only if we guess the right answer. If the gain function is binary, and the guess function represents the user’s best strategy, i.e. it is chosen to optimize utility, then there is a well-known correspondence between U and the Bayes risk / the a posteriori min entropy. Such correspondence is expressed by the following proposition: Proposition 5. Assume that gain is binary and guess is optimal. Then: U(Y, Z) = max(p(z|y) p(y)) = 2−H∞ (Y |Z) z

y

In order to analyze the implications of the -d.p. requirement on the utility, we need to consider the structure that the adjacency relation induces on Y. Let us deﬁne ∼ on Y as follows: y ∼ y if there are x, x ∈ X such that y = f (x), y = f (x ), and x ∼ x,. Note that K satisﬁes -d.p. if and only if H satisﬁes -d.p. If (Y, ∼) is distance-regular or VT+ , then we can apply Theorem 1 to ﬁnd a bound on the utility. In the following, we assume that the distribution of Y is uniform. Theorem 3. Consider a randomized mechanism H, and let y be an element of Y. Assume that (Y, ∼) is either distance-regular or VT+ and that H satisﬁes -d.p. For each distance d from 0 to the diameter of (Y, ∼), let nd be the number of nodes y at distance d from y. Then we have that: 1 U(Y, Z) ≤ nd d

(7)

e d

The above bound is tight, in the sense that (provided (Y, ∼) is distance-regular or VT+ ) we can construct a mechanism H which satisﬁes (7) with equality. More precisely, deﬁne 1 c = nd e d d

Quantitative Information Flow and Applications to Diﬀerential Privacy

227

Then deﬁne H (here identiﬁed with its channel matrix for simplicity) as follows: Hi,j =

c e

d(i,j)

(8)

Theorem 4. Assume (Y, ∼) is distance-regular or VT+ . Then the matrix H deﬁned in (8) satisﬁes -d.p. and has maximal utility: 1 U(Y, Z) = nd e d d

Note that we can always deﬁne H as in (8): the matrix so deﬁned will be a legal channel matrix, and it will satisfy -d.p.. However, if (Y, ∼) is neither distanceregular nor VT+ , then the utility of such H is not necessarily optimal. We end this section with an example (borrowed from [1]) to illustrate our technique. Example 8. Consider a database with electoral information where each row corresponds to a voter and contains the following three ﬁelds: – Id : a unique (anonymized) identiﬁer assigned to each voter; – City: the name of the city where the user voted; – Candidate: the name of the candidate the user voted for. Consider the query “What is the city with the greatest number of votes for a given candidate cand ?”. For such a query the binary utility function is the natural choice: only the right city gives some gain, and all wrong answers are equally bad. It is easy to see that every two answers are neighbors, i.e. the graph structure of the answers is a clique. Let us consider the scenario where City = {A, B, C, D, E, F } and assume for simplicity that there is a unique answer for the query, i.e., there are no two cities with exactly the same number of individuals voting for candidate cand . Table 4 shows two alternative mechanisms providing -diﬀerential privacy (with = log 2). The ﬁrst one, M1 , is based on the truncated geometric mechanism method used in [16] for counting queries (here extended to the case where every pair of answers is neighbor). The second mechanism, M2 , is obtained by applying the deﬁnition (8). From Theorem 4 we know that for the uniform input distribution M2 gives optimal utility. For the uniform input distribution, it is easy to see that U(M1 ) = 0.2242 < 0.2857 = U(M2 ). Even for non-uniform distributions, our mechanism still provides better utility. For instance, for p(A) = p(F ) = 1/10 and p(B) = p(C) = p(D) = P (E) = 1/5, we have U(M1 ) = 0.2412 < 0.2857 = U(M2 ). This is not too surprising: the geometric mechanism, as well as the Laplacian mechanism proposed by Dwork, perform very well when the domain of answers is provided with a metric and the utility function is not binary4 . It also works well when 4

In the metric case the gain function can take into account the proximity of the reported answer to the real one, the idea being that a close answer, even if wrong, is better than a distant one.

228

M.S. Alvim et al.

Table 4. Mechanisms for the city with higher number of votes for candidate cand (a) M1 : truncated geometric mechanism In/Out A B C D E F

A 0.535 0.465 0.405 0.353 0.307 0.267

B 0.060 0.069 0.060 0.053 0.046 0.040

C 0.052 0.060 0.069 0.060 0.053 0.046

D 0.046 0.053 0.060 0.069 0.060 0.052

E 0.040 0.046 0.053 0.060 0.069 0.060

F 0.267 0.307 0.353 0.405 0.465 0.535

(b) M2 : our mechanism In/Out A B C D E F

A 2/7 1/7 1/7 1/7 1/7 1/7

B 1/7 2/7 1/7 1/7 1/7 1/7

C 1/7 1/7 2/7 1/7 1/7 1/7

D 1/7 1/7 1/7 2/7 1/7 1/7

E 1/7 1/7 1/7 1/7 2/7 1/7

F 1/7 1/7 1/7 1/7 1/7 2/7

(Y, ∼) has low connectivity, in particular in the cases of a ring and of a line. But in this example, we are not in these cases, because we are considering binary gain functions and high connectivity.

6

Related Work

As far as we know, the ﬁrst work to investigate the relation between diﬀerential privacy and information-theoretic leakage for an individual was [2]. In this work, a channel is relative to a given database x, and the channel inputs are all possible databases adjacent to x. Two bounds on leakage were presented, one for teh R´enyi min entropy, and one for Shannon entropy. Barthe and K¨opf [3] were the ﬁrst to investigates the (more challenging) connection between diﬀerential privacy and the R´enyi min-entropy leakage for the entire universe of possible databases. They consider the “end-to-end diﬀerentially private mechanisms”, which correspond to what we call K in our paper, and propose, like we do, to interpret them as information-theoretic channels. They provide a bound for the leakage, but point out that it is not tight in general, and show that there cannot be a domain-independent bound, by proving that for any number of individual u the optimal bound must be at least a certain expression f (u, ). Finally, they show that the question of providing optimal upper bounds for the leakage of -diﬀerentially private randomized functions in terms of rational functions of is decidable, and leave the actual function as an open question. In our work we used rather diﬀerent techniques and found (independently) the same function f (u, ) (the bound in Theorem 1), but we actually proved that f (u, ) is the optimal bound5 . Another diﬀerence is that [3] captures the case in which the focus of diﬀerential privacy is on hiding participation of individuals in a database. In our work, we consider both the participation and the values of the participants. Clarkson and Schneider also considered diﬀerential privacy as a case study of their proposal for quantiﬁcation of integrity [9]. There, the authors analyze 5

When discussing our result with Barthe and K¨ opf, they said that they also conjectured that f (u, ) is the optimal bound.

Quantitative Information Flow and Applications to Diﬀerential Privacy

229

database privacy conditions from the literature (such as diﬀerential privacy, kanonymity, and l-diversity) using their framework for utility quantiﬁcation. In particular, they study the relationship between diﬀerential privacy and a notion of leakage (which is diﬀerent from ours - in particular their deﬁnition is based on Shannon entropy) and they provide a tight bound on leakage. Heusser and Malacaria [17] were among the ﬁrst to explore the application of information-theoretic concepts to databases queries. They proposed to model database queries as programs, which allows for statical analysis of the information leaked by the query. However [17] did not attempt to relate information leakage to diﬀerential privacy. In [16] the authors aim at obtaining optimal-utility randomization mechanisms while preserving diﬀerential privacy. The authors propose adding noise to the output of the query according to the geometric mechanism. Their framework is very interesting in the sense it provides a general deﬁnition of utility for a mechanism M that captures any possible side information and preference (deﬁned as a loss function) the users of M may have. They prove that the geometric mechanism is optimal in the particular case of counting queries. Our results do not restrict to counting queries, but on the other hand we only consider the case of binary loss function.

References 1. Alvim, M.S., Andr´es, M.E., Chatzikokolakis, K., Degano, P., Palamidessi, C.: Differential privacy: on the trade-oﬀ between utility and information leakage. Technical report (2011), http://hal.inria.fr/inria-00580122/en/ 2. Alvim, M.S., Chatzikokolakis, K., Degano, P., Palamidessi, C.: Diﬀerential privacy versus quantitative information ﬂow. Technical report (2010) 3. Barthe, G., K¨ opf, B.: Information-theoretic bounds for diﬀerentially private mechanisms. In: Proc. of CSF (to appear, 2011) 4. Cachin, C.: Entropy Measures and Unconditional Security in Cryptography. PhD thesis (1997) 5. Chatzikokolakis, K., Palamidessi, C., Panangaden, P.: Anonymity protocols as noisy channels. Inf. and Comp. 206(2-4), 378–401 (2008) 6. Chatzikokolakis, K., Palamidessi, C., Panangaden, P.: On the Bayes risk in information-hiding protocols. J. of Comp. Security 16(5), 531–571 (2008) 7. Chaum, D.: The dining cryptographers problem: Unconditional sender and recipient untraceability. Journal of Cryptology 1, 65–75 (1988) 8. Clark, D., Hunt, S., Malacaria, P.: Quantitative information ﬂow, relations and polymorphic types. J. of Logic and Computation 18(2), 181–199 (2005) 9. Clarkson, M.R., Schneider, F.B.: Quantiﬁcation of integrity, Tech. Rep. (2011), http://hdl.handle.net/1813/22012 10. Dodis, Y., Ostrovsky, R., Reyzin, L., Smith, A.: Fuzzy extractors: How to generate strong keys from biometrics and other noisy data. SIAM Journal of Computing 38(1), 97–139 (2008) 11. Dodis, Y., Reyzin, L., Smith, A.: Fuzzy extractors: How to generate strong keys from biometrics and other noisy data. In: Cachin, C., Camenisch, J.L. (eds.) EUROCRYPT 2004. LNCS, vol. 3027, pp. 523–540. Springer, Heidelberg (2004)

230

M.S. Alvim et al.

12. Dwork, C.: Diﬀerential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006) 13. Dwork, C.: Diﬀerential privacy in new settings. In: Proc. of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2010, Austin, Texas, USA, January 17-19, pp. 174–183. SIAM, Philadelphia (2010) 14. Dwork, C.: A ﬁrm foundation for private data analysis. Communications of the ACM 54(1), 86–96 (2011) 15. Dwork, C., Lei, J.: Diﬀerential privacy and robust statistics. In: Proc. of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31-June 2, pp. 371–380. ACM, New York (2009) 16. Ghosh, A., Roughgarden, T., Sundararajan, M.: Universally utility-maximizing privacy mechanisms. In: Proc. of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, pp. 351–360. ACM, New York (2009) 17. Heusser, J., Malacaria, P.: Applied quantitative information ﬂow and statistical databases. In: Degano, P., Guttman, J.D. (eds.) FAST 2009. LNCS, vol. 5983, pp. 96–110. Springer, Heidelberg (2010) 18. Joinson, A.N.: Self-disclosure in computer-mediated communication: The role of self-awareness and visual anonymity. Eur. J. Soc. Psychol. 31(2), 177–192 (2001) 19. K¨ opf, B., Basin, D.A.: An information-theoretic model for adaptive side-channel attacks. In: Proc. of CCS, pp. 286–296. ACM, New York (2007) 20. K¨ opf, B., Basin, D.A.: An information-theoretic model for adaptive side-channel attacks. In: Proc. of CCS, pp. 286–296. ACM, New York (2007) 21. Malacaria, P.: Assessing security threats of looping constructs. In: Proc. of POPL, pp. 225–235. ACM, New York (2007) 22. Malacaria, P., Chen, H.: Lagrange multipliers and maximum information leakage in diﬀerent observational models. In: Proc. of PLAS, pp. 135–146. ACM, New York (2008) 23. Massey: Guessing and entropy. In: Proc. of ISIT, p. 204. IEEE, Los Alamitos (1994) 24. Moskowitz, I.S., Newman, R.E., Crepeau, D.P., Miller, A.R.: Covert channels and anonymizing networks. In: Proc. of PES, pp. 79–88. ACM, New York (2003) 25. Moskowitz, I.S., Newman, R.E., Syverson, P.F.: Quasi-anonymous channels. In: Proc. of CNIS, pp. 126–131. IASTED (2003) 26. Pliam, J.O.: On the incomparability of entropy and marginal guesswork in bruteforce attacks. In: Roy, B., Okamoto, E. (eds.) INDOCRYPT 2000. LNCS, vol. 1977, pp. 67–79. Springer, Heidelberg (2000) 27. R´enyi, A.: On Measures of Entropy and Information. In: Proc. of the 4th Berkeley Symposium on Mathematics, Statistics, and Probability, pp. 547–561 (1961) 28. Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423, 625–656 (1948) 29. Smith, G.: On the foundations of quantitative information ﬂow. In: de Alfaro, L. (ed.) FOSSACS 2009. LNCS, vol. 5504, pp. 288–302. Springer, Heidelberg (2009) 30. Zhu, Y., Bettati, R.: Anonymity vs. information leakage in anonymity systems. In: Proc. of ICDCS, pp. 514–524. IEEE, Los Alamitos (2005)

Risk Analysis of Changing and Evolving Systems Using CORAS Mass Soldal Lund1 , Bjørnar Solhaug1 , and Ketil Stølen1,2 1

SINTEF ICT, P.O. box 124 Blindern, 0314 Oslo, Norway Department of Informatics, University of Oslo, Norway {mass.s.lund,bjornar.solhaug,ketil.stolen}@sintef.no 2

Abstract. Risk analysis is the identiﬁcation and documentation of risks with respect to an organisation or a target system. Established risk analysis methods and guidelines typically focus on a particular system conﬁguration at a particular point in time. The resulting risk picture is then valid only at that point in time and under the assumptions made when it was derived. However, systems and their environments tend to change and evolve over time. In order to appropriately handle change, risk analysis must be supported with specialised techniques and guidelines for modelling, analysing and reasoning about changing risks. In this paper we introduce general techniques and guidelines for managing risk in changing systems, and then instantiate these in the CORAS approach to model-driven risk analysis. The approach is demonstrated by a practical example based on a case study from the Air Traﬃc Management (ATM) domain. Keywords: Risk management, risk analysis, change management, Air Traﬃc Management, security.

1

Introduction

Risk management is coordinated activities to direct and control an organisation with regard to risk [15]. Risk analysis is a core part of risk management, and involves the identiﬁcation and documentation of risks with respect to the organisation or the target system in question. When deriving the risk picture for the target of analysis, established risk analysis methods and guidelines typically focus on a particular system conﬁguration at a particular point in time. The resulting risk picture is then valid only at that point in time and under the assumptions made when it was derived. However, systems and their environments tend to change and evolve over time. In order to appropriately handle change, each of the risk management activities must be supported with specialised techniques and guidelines. Figure 1 is adapted from the ISO 31000 risk management standard [15] and illustrates the seven activities of the risk management process. The ﬁve activities in the middle constitute the core activities of a risk analysis, and are described as follows: A. Aldini and R. Gorrieri (Eds.): FOSAD VI, LNCS 6858, pp. 231–274, 2011. c Springer-Verlag Berlin Heidelberg 2011

232

M.S. Lund, B. Solhaug, and K. Stølen

Estimate risks

Monitor and review

Identify risks Risk assessment

Communicate and consult

Establish the context

Evaluate risks

Treat risks

Fig. 1. Risk management process

– Establish the context is to deﬁne the external and internal parameters to be accounted for when managing risk, and to set the scope and risk criteria for the risk management policy. – Risk identification is to ﬁnd, recognise and describe risks. – Risk estimation is to comprehend the nature of risk and to determine the risk level. – Risk evaluation is to compare the risk estimation results with the risk criteria to determine whether the risk and its magnitude are acceptable or tolerable. – Risk treatment is the process of modifying the risk. The remaining two activities are continuous activities of the overall risk management process, and are described as follows: – Communicate and consult are the continual and iterative processes an organisation conducts to provide, share or obtain information, and to engage in dialogue with stakeholders about risk management. – Monitoring involves the continuous checking, supervising and critically observing the risk status in order to identify changes from the performance level required or expected, whereas review focuses on the activity undertaken to determine the suitability, adequacy and eﬀectiveness of the subject matter necessary to achieve established objectives. When targeting changing and evolving systems the main challenge is to ensure that the analysis results are kept valid under change. A straightforward way to

Risk Analysis of Changing and Evolving Systems Using CORAS

233

ensure this is to conduct a full risk analysis from scratch when faced with a potentially risk relevant change. Needless to say, such a strategy is not to prefer as it is time and resource consuming, and as it often implies conducting exactly the same analysis again to the extent that the risk picture is persistent. Instead, a customised analysis method for changing systems should provide guidelines and techniques for how to systematically trace the relevant changes from the target to the risks, and thereby for how to update only the part of the risk picture that is aﬀected by the changes. How to adequately deal with changes in a risk analysis depends, however, on the nature of the changes in each speciﬁc case. We have for this reason identiﬁed three diﬀerent perspectives on change, each with its speciﬁc methodological needs. These are the maintenance perspective, the before-after perspective, and the continuous evolution perspective. The approach of this paper is to take as starting point the risk management principles and guidelines of the ISO 31000 standard and generalise these to the setting of changing and evolving systems. This standard is quite general, and most of the established risk management methodologies can be understood as instantiations of the activities it prescribes. The objective of such an approach is to understand and explain how to deal with changing risks without restricting to a speciﬁc methodology. ISO 31000 comes with no guidance on risk analysis techniques. While a risk analysis method provides methodological advice on how to carry out the various activities of risk management, a risk analysis technique is more narrow in the sense that it addresses only some aspects of the risk analysis process. A risk analysis method typically makes use of one or more risk analysis techniques. Risk modelling refers to techniques that are used to support the activities of risk identiﬁcation and risk estimation. In this paper we take the risk graphs of [3], which can be understood as a common abstraction of diﬀerent risk modelling techniques, as the starting point for generalising risk modelling to the setting of changing systems. The idea is, as is also demonstrated in the paper, that the extensions we make to risk graphs carry over to other risk modelling techniques. In order to demonstrate and validate our approach, we present a case study from the domain of Air Traﬃc Management (ATM). The speciﬁc risk analysis method applied in this case study is CORAS [20] used as an instantiation of the general approach to risk analysis of changing systems. CORAS is a modeldriven approach to risk analysis that consists of a method, a language and a tool to support the risk analysis process. The ATM case study thus demonstrates a concrete instantiation of the approach in this paper in both the CORAS method and the CORAS language. The structure of the paper is as follows. In Sect. 2 we make a classiﬁcation of change by introducing the perspectives of maintenance, before-after and continuous evolution. As the methodological needs are strongly situation dependent, we discuss these in relation to the mentioned perspectives before we present our approach to risk analysis of changing and evolving systems in Sect. 3. Section 4 introduces the formal foundation for our approach by presenting the syntax and

234

M.S. Lund, B. Solhaug, and K. Stølen

semantics of risks graphs and their generalisation to the setting of changing and evolving risks. The section moreover presents a calculus with rules for reasoning about risk graphs, and a set of guidelines for how to do consistency checking of risk graphs. In Sect. 5 we instantiate our approach with CORAS, thus generalising CORAS to the setting of changing and evolving systems. In Sect. 6 we demonstrate our approach by applying this generalisation of CORAS for changing and evolving systems to the practical example of the ATM case study. Finally, we present related work in Sect. 7 before we conclude in Sect. 8.

2

Classification of Change

The appropriate way of handling change when managing, modelling, analysing and documenting risks depends greatly on the context and on the kind of changes we face [19]. For example, do the changes result from maintenance or from bigger, planned changes? Do the changes comprise a transition from one stable state of the target of analysis to another stable state, or do they reﬂect the continuous evolution of a target designed to change over time? Do the changes occur in the target system or in the environment of the target? The answers to such questions have implications on how to methodologically approach the problem. In the following we present three main classes of change by introducing three respective perspectives on change: maintenance, before-after, and continuous evolution. After this presentation, the remainder of the paper focuses on the before-after perspective. This is the perspective of the case study from which the practical example is provided, and is perhaps the perspective that most clearly illustrates the need for specialised methods when dealing with risk analysis of changing and evolving systems. 2.1

The Maintenance Perspective

The changes we address from the maintenance perspective are those that accumulate over time and gradually make any previous risk analysis results invalid. The challenge for the risk analysts can be described by the following example scenario: The risk analysts conducted a risk analysis three years ago and are now requested by the same client to analyse the same target system anew so as to update the risk picture and reﬂect any changes to the target or its environment, thus restoring the validity of the analysis. Typical changes in this setting can be bug ﬁxes and security patches, an increase in network traﬃc, and increase in the number of attacks. For such changes, the risk picture remains more or less the same, but risk values may have changes such that previously acceptable risks are now unacceptable, or vice versa. The objective is then to maintain the documentation of the previous risk analysis by conducting an update.

Risk Analysis of Changing and Evolving Systems Using CORAS

235

Updates Old target

Current target

Old risks

Current risks

Old risk picture

Current risk picture Risk analyst

Fig. 2. The maintenance perspective

The illustration in Fig. 2 shows the principles by which the risk analysts conduct the analysis from the maintenance perspective. Assuming that we have available the documentation of the previous risk analysis, as well as the description of the old and current target of analysis, we start by identifying the changes that have occurred. We then use the relevant changes as input when deriving the current risk picture based on the previous risk analysis results. Methodologically, the main challenge is how to reuse the old risk analysis results and not conducting a full risk analysis from scratch. This requires a systematic way of identifying the updates of the target and which parts of the risk picture that are aﬀected such that the risk picture can be updated without addressing and spending eﬀort on the unaﬀected parts. 2.2

The Before-After Perspective

The changes we address from the before-after perspective are those that are planned or anticipated, and perhaps of a radical kind. The challenge for the risk analysts can be described by the following scenario: The risk analysts are asked to predict the eﬀect on the risk picture of implementing the changes to the target of analysis. Examples of changes in the before-after perspective are the rolling out of a new system, or making major organisational changes such as implementing a merger agreement between two companies. We must thus understand the current risk picture and how this will change to the future risk picture as a consequence of implementing the changes to the target of analysis. As the planned or anticipated process of change may involve risks in itself, the before-after perspective may require a risk analysis of this process. The illustration in Fig. 3 shows the principles by which the risk analysts conduct the analysis from the before-after perspective. Assuming that we have available the description of the current target of analysis as well as the change

236

M.S. Lund, B. Solhaug, and K. Stølen Planned changes Current target

Current risks

Future target

Risks due to change process

Future risks

Risk picture

Risk analyst

Fig. 3. The before-after perspective

process to bring the target from the current to the future state, we can work out a coherent risk picture for the future target and for the change process. Methodologically, the main challenges are how to obtain and present a risk picture that describes the current and the future risks, as well as the impact of the change process itself. This requires an approach to making a description of the target “as-is” and a description of the target “to-be”, describing the process of change in suﬃcient detail, identifying and documenting the current and future risks without doing double work, and identifying risks due to the change process. 2.3

The Continuous Evolution Perspective

The changes we address from the continuous evolution perspective are predictable and gradual evolutions that can be described as functions of time. Such predictions can be based on well-founded forecasts or planned developments. The challenge for the risk analysts can be described by the following scenario: The risk analysts are asked to predict the future evolutions of risk, which mandates that they conduct a risk analysis that establish a dynamic risk picture that reﬂects the expected evolution of the target. Examples of predictable changes are the slow increase in the number of components working in parallel or gradually including more sites in a system. Examples of well-founded forecasts are the expected steady increase of end-users, adversary attacks, and annual turnover.

Risk Analysis of Changing and Evolving Systems Using CORAS

Target at time t0

Evolution

Risks at time t0

Target at time t1

Evolution

…

Risks at time t1

Target at time tn

237

…

Risks at time tn

Risk picture

Risk analyst

Fig. 4. The continuous evolution perspective

The illustration in Fig. 4 shows the principles by which the risk analysts conduct the analysis from the continuous evolution perspective. Assuming that we have available a description of the target of analysis as a function of time, such that we can derive the target at any given point in time, we use this as input to the risk analysis. Understanding how the target and its environment evolve, we seek to work out a risk picture as a function of time that shows how risks evolve. Methodologically, the main challenges are how to identify evolving risks and present them in a dynamic risk picture. This requires us to generalise the target description to capture evolution and to identify and likewise generalise the risks aﬀected by evolution.

3

Our Approach

As explained in Sect. 2 this paper focuses on the before-after perspective. Conﬁning to this perspective, we present in this section our approach to risk analysis of changing systems by adequately generalising the risk management process of ISO 31000. Our main concern is the ﬁve activities in the middle of Fig. 1 as this comprise the actual risk analysis activities. We moreover focus on what is needed for handling change, referring to the ISO 31000 standard [15] for the principles and guidelines for risk management in the traditional setting. The reader is also referred to the practical example in Sect. 6 for further details about the various activities and tasks.

238

M.S. Lund, B. Solhaug, and K. Stølen

In the following we go through each of the ﬁve activities of the risk analysis process. Subsequently, in Sect. 4, we introduce language support for this process by a generalisation of risk graphs to a risk graph notation with the expressiveness to explicitly model changing risks in the before-after perspective. 3.1

Context Establishment

The activity of establishing the context involves making the premises for the subsequent risk analysis. Establishing the context should result in a target description, which is the documentation of all the information that serves as the input to and basis for the risk analysis, and a set of risk evaluation criteria. Any information about the target of analysis that is relevant for the risk analysis and its outcome needs to be included in the target description. This means that any risk relevant change in the target of analysis must be reﬂected by changes to the target description. Establishing the context of the analysis also includes articulating the goals and objectives of the analysis and deciding its focus and scope. In particular, we need to determine precisely what the target of analysis is and what the assets that need to be protected are. The risk analysis is conducted with respect to the identiﬁed assets, and it is only by precisely understanding what the assets are that we can conduct a risk analysis that meets the overall goals and objectives of the analysis. Establishing the Target Description. The target description includes the documentation of the target of analysis, the focus and scope of the analysis, the environment of the target, the assumptions of the analysis, the parties and assets of the analysis, and the context of the analysis. The UML [22] class diagram of Fig. 5 gives an overview of the elements of a target description. In a risk analysis, the notions of party, asset and risk are closely related. A party is an organisation, company, person, group or other body on whose behalf

Assumption *

Context *

1 1

1 Environment

Party 1..*

1

1

1 Target of analysis

1 1..*

* 1..*

*

Asset within

1 Scope

1 part of

Focus

Fig. 5. Target description

Risk Analysis of Changing and Evolving Systems Using CORAS

239

the risk analysis is conducted. An asset is something to which a party assigns value and hence for which the party requires protection. A risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. This means that if there is no party, it makes no sense to speak of assets. And without assets there can moreover be no risks. The target of the analysis is the system, organisation, enterprise, or the like that is the subject of a risk analysis. The focus of the analysis is the main issue or central area of attention. The focus is within the scope of the analysis, which the extent or range of the analysis. The scope deﬁnes the border of the analysis, i.e. what is held inside and what is held outside of the analysis. The environment of the target is the surrounding things of relevance that may aﬀect or interact with the target; in the most general case the environment is the rest of the world. The assumptions are something we take as granted or accept as true, although it may not be so; the results of a risk analysis are valid only under the assumptions. The context of the analysis is the premises for and the background of the analysis. This includes the purposes of the analysis and to whom the analysis is addressed. For a given target of analysis we assume that we can use a traditional risk analysis method to conduct the context establishment while not taking into account changes. The additional task of establishing the changes in the context includes making a description of the target of analysis when the changes have been taken into account. This extended target description should include both a description of the changes and the result of implementing them, although it should be possible to deduce the latter from the current target description and the description of the changes. Changes that concern the target of analysis can be new or diﬀerent work processes, the introduction of new services or applications, changes in users or roles, etc. These may imply changes the risk picture, and therefore imply the need for a new risk analysis of parts of the target. There may, however, also be changes in parties, changes in assets or asset priorities, changes in the environment or in the assumptions, changes in the focus or scope, and so on, that must be documented in the description of the changing target of analysis. Establishing the Risk Evaluation Criteria. The risk evaluation criteria are a speciﬁcation of the risk levels that the parties of the risk analysis are willing to accept. The criteria will later be used to evaluate the signiﬁcance of risk, and should reﬂect the values, objectives and resources of the parties in question. When we are deciding the risk evaluation criteria we need to take into account not only the views of the parties, but also the nature of the assets, the types of consequences and how they should me measured and described. Speciﬁcally, we need for each asset to deﬁne a consequence scale where each consequence value describes a level of impact of an unwanted incident on an asset in terms of harm or reduced asset value. We furthermore need to deﬁne a likelihood scale, the values of which will be used to describe the frequency or probability of unwanted incidents and threat scenarios to occur. Recall that a risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. The risk level is the level or value of a risk as derived from

240

M.S. Lund, B. Solhaug, and K. Stølen

its likelihood and consequence. The risk level of each combination of a likelihood and a consequence is calculated by a risk function. Since it is only the party of a given asset that can determine the severity of a risk, it is the party that must determine an adequate risk function. Essentially, the risk evaluation criteria are a speciﬁcation of the level at which risks become unacceptable. Some changes are of a kind that does not aﬀect the assets or other values, objectives or resources of the parties. In this case, there is no need to reconsider the risk evaluation criteria. For other changes, the value or priorities of assets may change, new assets may arise, the parties may become more or less risk averse, and so forth. In that case we need a new iteration on establishing and documenting the risk evaluation criteria. 3.2

Risk Identification

The risk identiﬁcation involves identifying and documenting unwanted incidents with respect to the identiﬁed assets, as well as identifying vulnerabilities and sources of risk. The risk identiﬁcation should involve people with appropriate expert knowledge about the target of analysis. The activity of extracting the relevant information relies on techniques and tools for identifying risk relevant information, for structuring the information in a sensible way, and for adequately documenting the information. While the documentation of the risks that are identiﬁed should serve as a means for reporting the ﬁndings to the relevant stakeholders, it should at the same time facilitate the subsequent estimation and evaluation. In this section we focus on the methodological guidelines for risk identiﬁcation of changing and evolving systems. In Sect. 4 we discuss more closely the required risk modelling techniques. As mentioned above, it is the target description that serves as the input to and basis for the subsequent risk analysis. The objective of the risk identiﬁcation is to identify and document the changing risks given the description of the changing target. A main principle is that to the extent that we have identiﬁed and documented the risks for the target of analysis before changes, we only address the parts of the target that are aﬀected by the change when identifying the changing risks. This means that when considering the target description without the changes, the risk identiﬁcation and the risk documentation are conducted according to traditional risk analysis methods. When this is completed we need to update the resulting risk documentation according to the changes. This is conducted by making a walkthrough of the current target description and risk documentation. The risks that are persistent under change can immediately be included in the new documentation with no further investigation. The risks that may be aﬀected by change need to be considered again: Previous scenarios, incidents, etc. may change, new may arise, and others may disappear.

Risk Analysis of Changing and Evolving Systems Using CORAS

241

The methodological guidelines for risk identiﬁcation of a changing target of analysis are summarised as follows: 1. Identify and document risks by using as input the target description before changes have been taken into account. 2. Establish and document the traceability between the target description before change and the risk documentation resulting from the previous step. 3. Based on the traceability and the description of the changed target, identify the parts of the risk documentation that are persistent under change. 4. Conduct the risk identiﬁcation of the changed target only with respect to the parts of the target and the risks that are aﬀected by the change. In order to conduct the activity of risk identiﬁcation of a changing target of analysis, there is thus a need for techniques for identiﬁcation and modelling of risks that change, as well as techniques for establishing and modelling traceability between target description and risk models, both which are topics of Sect. 4. 3.3

Risk Estimation

The objective of the risk estimation is to establish an understanding of the severity of identiﬁed risks, and to provide the basis for the subsequent risk evaluation and risk treatment. By considering the causes and sources of risk, the risk estimation amounts to estimating and documenting the likelihoods and consequences of the identiﬁed unwanted incidents. It is the likelihoods of unwanted incidents and their consequences for assets that constitute risks, and by making estimates of the likelihoods and consequences we can understand which risks are the most in need of treatment and which risks are less relevant. Given the documentation of identiﬁed risks from the previous activity, including the documentation of the changing risks, the risk estimation of a changing and evolving target is quite similar to traditional risk analysis: The estimation is conducted by a walkthrough of the risk documentation addressing each of the relevant elements in turn. To the extent that risks are persistent under the changes, the estimation is not repeated. The estimates need to be continuously documented, which means that there must be adequate support for including the estimates in the risk models. In order to conduct the activity of risk analysis and the documentation of the results, there is hence a need for techniques for making estimates of likelihoods and consequences of changing risks, as well as modelling support for documenting the results. As we shall see, such techniques and support is provided by the approach to the modelling of changing risk presented in Sect. 4. 3.4

Risk Evaluation

The objective of the risk evaluation is to determine which of the identiﬁed risks that need treatment, and to make a basis for prioritising the treatment options. Basically, the risk evaluation amounts to estimating the risk levels based on the

242

M.S. Lund, B. Solhaug, and K. Stølen

likelihood and consequence estimates, and to compare the results with the risk evaluation criteria. The risk evaluation of a changing and evolving target is conducted in the same way as risk evaluation of a traditional risk analysis. Given the risk documentation of the changing risks with the risk estimates, the risk evaluation is conducted by calculating the risk level of each pair of an unwanted incident and asset harmed by the incident. The calculation is straightforwardly done by using the risk function deﬁned during the context establishment. For changing systems, the criteria may of course be diﬀerent before and after the changes in question. 3.5

Risk Treatment

The risk treatment succeeds the risk identiﬁcation, estimation and evaluation activities, and the objective is to identify and select a set of treatment options for the risks that are not acceptable according to the risk evaluation criteria. A treatment option is thus an appropriate measure to reduce risk level, and the implementation of the selected treatments should bring the risk level down to an acceptable level. For changes that are planned or predicted, it may be that we are only concerned about the future risks and in ensuring that implementation of the changes results in a system with an acceptable risk level. For changes that are planned to be implemented over a longer time span it may, however, be that we also need to identify treatments for current unacceptable risks and treatments that are consecutively implemented in the future in order to maintain an acceptable risk level under the period of change.

4

Foundations

Risk analysis involves the process of understanding the nature of risks and determining the level of risk [15]. Risk modelling refers to techniques that are used to aid the process of identifying, documenting and estimating likelihoods and consequences of unwanted incidents. In this section we present an approach to risk modelling referred to as risk graphs [3]. Once we have introduced the syntax and semantics of risk graphs, we generalise these to enable modelling and reasoning about changing risks. We then provide a calculus for analysing likelihoods in risk graphs and a means for relating risk graphs to target models. 4.1

Risk Graphs

A risk model is a structured way of representing an unwanted incident and its causes and consequences by means of graphs, trees or block diagrams [24]. We introduce risk graphs as an aid for structuring events and scenarios leading to incidents and estimating likelihoods of incidents. There exist several modelling techniques that can be used for such structuring of scenarios and incidents, and for the reasoning about likelihoods of incidents, for example fault trees [13],

Risk Analysis of Changing and Evolving Systems Using CORAS

v1 [P1]

243

Pa

v2 [P2]

Pc

v3 [P3]

Pb

v4 [P4]

Pd

Pf Pe

v5 [P5]

v7 [P7]

v6 [P6]

Fig. 6. Risk graph

event trees [14], attack trees [25], cause-consequence diagrams [21], Bayesian networks [2] and CORAS threat diagrams (see Sect. 5). Risk graphs can be understood as a common abstraction of these modelling techniques [3]. By giving formal semantics to risk graphs, we thereby also provide a risk model semantics that can be used to explain and reason about several established approaches to risk modelling. A risk graph consists of vertices (representing threat scenarios) and a ﬁnite set of directed relations (representing the “leads to” relationship) between them. An example risk graph is shown in Fig. 6. Each vertex in a risk graph is assigned a set of likelihood values representing the estimated likelihood for the scenario to occur. The assignment of several likelihood values, typically a likelihood interval, represents underspeciﬁcation of the likelihood estimate. A relation from vertex v to vertex v means that v may lead to v . Also the relations can be assigned likelihood sets. These are conditional likelihoods that specify the likelihood for a scenario leading to another scenario when the former occurs. One threat scenario may lead to several other threat scenarios, so the probabilities on the relations leading from a threat scenario may add up to more than 1. A risk graph is furthermore allowed to be incomplete in the sense that a given threat scenario may lead to more scenarios than what is accounted for in the risk graph. The probabilities of the relations leading from a threat scenario may for this reason also add up to less than 1. The Syntax of Risk Graphs. Formally a risk graph is a set D of elements e. An element is a vertex v or a relation v − → v . Let P ⊆ [0, 1] denote a probability set. We then write v(P ) to indicate that the probability set P is assigned to v. P Similarly, we write v − → v to indicate that the probability set P is assigned to the relation from v to v . If no probability set is explicitly assigned, we assume the probability set assigned to the element to be [0, 1]. Using this textual notation, the risk graph shown in Fig. 6 can be represented by D = {v1 (P1 ), v2 (P2 ), v3 (P3 ), v4 (P4 ), v5 (P5 ), v6 (P6 ), v7 (P7 ), P

P

P

P

P

Pf

a b c d e v1 −−→ v3 , v2 −→ v3 , v3 −→ v4 , v4 − −→ v7 , v5 −→ v6 , v6 −−→ v7 }

244

M.S. Lund, B. Solhaug, and K. Stølen

The Semantics of Risk Graphs. Risk graphs are used for the purpose of documenting and reasoning about risks, particularly the documentation and analysis of threat scenarios and unwanted incidents and their likelihoods. The approach of [3] assumes that scenarios and their probabilities are represented by a probability space [4] on traces of events. We let H denote the set of all traces (both ﬁnite and inﬁnite) and HN the set of all ﬁnite traces. A probability space is a triple (H, F, µ). H is the sample space, i.e. the set of possible outcomes, which in our case is the set of all traces. F is the set of measurable subsets of the sample space, and µ is a measure that assigns a probability to each element in F . The semantics of a risk graph is statements about the probabilities of the trace sets that represent vertices or the composition of vertices. In other words, the semantics is a set of statements about the measure µ. For composition of vertices, v v denotes the occurrence of both v and v where the former occurs before the latter. We let v v denote the occurrence of at least one of v and v . A vertex is atomic if it is not of the form v v or v v . We use lower case v as the naming convention for arbitrary vertices, and upper case V as the naming convention for the set of ﬁnite traces representing the vertex v. When deﬁning the semantics of risk graphs we use the auxiliary function tr( ) that yields a set of ﬁnite traces from an atomic or combined vertex. Intuitively, tr(v) is the set of all possible traces leading up to and through the vertex v, without continuing further. The function is deﬁned by def

tr(v) = HN V when v is an atomic vertex def tr(v v ) = tr(v) tr(v ) def tr(v v ) = tr(v) ∪ tr(v )

where is the operator for sequential composition of trace sets, for example weak sequencing in UML sequence diagrams [10]. Notice that the deﬁnition of the composition v v does not require v to occur immediately before v . The deﬁnition implies that tr(v v ) includes traces from v to v via ﬁnite detours. A probability interval P assigned to v, denoted v(P ), means that the likelihood of going through v is a value p ∈ P , independent of what happens before or after v. The semantics of a vertex is deﬁned by def

[[v(P )]] = µc (tr(v)) ∈ P where the expression µc (S) denotes the probability of any continuation of the trace set S ⊆ H, and is deﬁned as def

µc = µ(S H)

A probability interval P assigned to a relation v − → v means that the likeli hood of v occurring after an occurrence of v is a value in P . This likelihood is referred to as the conditional likelihood. The semantics of a relation is deﬁned by def P [[v − → v ]] = µc (tr(v v )) ∈ µc (tr(v)) · P

Risk Analysis of Changing and Evolving Systems Using CORAS

245

def

[pi , pj ] · [pk , pl ] = [pi · pk , pj · pl ] def

[pi , pj ] · p = [pi · p, pj · p] def

[pi , pj ] + [pk , pl ] = [pi + pk , pj + pl ] def

[pi , pj ] + p = [pi + p, pj + p] def

[pi , pj ] − [pk , pl ] = [pi − pk , pj − pl ] def

[pi , pj ] − p = [pi − p, pj − p] def

p − [pk , pl ] = [p − pk , p − pl ] Fig. 7. Interval arithmetic

Our deﬁnitions of interval arithmetic in the setting of risk graphs are given in Fig. 7. The semantics [[D]] of a risk graph is the conjunction of the expressions deﬁned by the elements in D, formally deﬁned as def

[[D]] =

e∈D [[e]]

A risk graph is said to be correct (with respect to the world or a speciﬁcation of the relevant part of the world) if each of the conjuncts of [[D]] is true. We say that D is inconsistent if it is possible to deduce False from [[D]]. 4.2

Risk Graphs for Changing Risks

In order to support the modelling of changing risks we need to generalise risk graphs to allow the simultaneous modelling of risks both before and after the implementation of some given changes. For this purpose we extend the risk graph notation to three kinds of vertices and three kinds of relations, namely before, after and before-after. When an element (vertex or relation) is of kind before it represents risk information before the changes, when it is of kind after it represents risk information after the changes, and when it is of kind before-after it represents risk information that holds both before and after the changes. The Syntax of Risk Graphs with Change. A risk graph with change is represented by a pair (Db , Da ) of sets of elements, the former consisting of the vertices and relations of kind before and the latter consisting of vertices and relations of kind after. Table 1 gives an overview of the language constructs and the naming conventions we use for referring to them. The symbols written in sans serif and the arrows denote speciﬁc language constructs, whereas v denotes an arbitrary vertex of any kind. Table 2 gives an overview of the various ways of specifying likelihoods. Recall that any of the likelihoods can be undeﬁned, in which case they are completely underspeciﬁed.

246

M.S. Lund, B. Solhaug, and K. Stølen Table 1. Naming conventions Variable Diagram construct v vb va v v− → v v− →b v v− →a v

Vertex before-after Vertex before Vertex after Vertex Relation before-after Relation before Relation after

Table 2. Denoting likelihoods Likelihood spec. Interpretation v(P P )

v occurs with likelihood P before, and v occurs with likelihood P after vb occurs with likelihood P before va occurs with likelihood P after

vb(P ) va(P )

P P

v −−−→ v

v leads to v with conditional likelihood P before, and v leads to v with conditional likelihood P after

P

v− →b v P

v− →a v

v leads to v with conditional likelihood P before

v leads to v with conditional likelihood P after

Figure 8 shows an example of the visual representation of a risk graph with change. Solid lines, like on the vertex v1 and the relation from v1 to v3 , indicates elements that only exists before, while dashed lines indicates elements that exists after. The vertices with a white shadow, like v2 , are those that exist both before and after, while those with black shadows, like v5 , exist only after. The dashed relations with a single probability set, like the relation from v5 to v6 , exist only after, while those with double probability sets, like the relation from v3 to v4 , exist both before and after. v1 [P1]

v2 [P2]/[P2’]

Pa Pb/Pb’

v3 [P3]/[P3’]

Pc/Pc’

v4 [P4]/[P4’]

Pd/Pd’

Pf v5 [P5]

Pe

v6 [P6]

Fig. 8. Risk graph for changing risks

v7 [P7]/[P7’]

Risk Analysis of Changing and Evolving Systems Using CORAS

v1 [P1]

v2 [P2]

v2 [P2’]

Pa Pb

Pc

v3 [P3]

v4 [P4]

Pd v7 [P7]

Pb’

Pc’

v3 [P3’]

v4 [P4’]

Pd’

Pf v5 [P5]

247

Pe

v7 [P7’]

v6 [P6]

Fig. 9. Two views on risk graphs for changing risk

Since we are operating with vertices and relations of kind before-after as language element of their own, we also allow the representation of risk graphs with change as a single set D of vertices and relations, where each element is of one of the kinds before, after or before-after. This single set of elements is then syntactic sugar for the equivalent representation of a pair of sets of elements. For such a combined representation D we use the functions before( ) and after( ) to ﬁlter the combined risk graph with respect to the elements of kind before and after, respectively. The following deﬁne the function before( ) for singleton sets of elements. def

before({v(P P )}) = {vb(P )} def before({vb(P )}) = {vb(P )} def before({va(P )}) = ∅ P P

def

P

P

def

P

P

def

before({v −−−→ v }) = {v − →b v } before({v − →b v }) = {v − →b v } before({v − →a v }) = ∅ The ﬁltering of a risk graph with change D with respect to the before elements is then deﬁned as def before(D) = e∈D before({e}) The deﬁnition of the function after( ) is symmetric. For a risk graph with change D of elements of the three diﬀerent kinds, the representation as a pair of elements of kind before and elements of kind after is then given by (before(D), after(D)). Figure 9 shows the graphical representation of the two risk graph before(D) (top) and after(D) (bottom) where D is the risk graph shown in Fig. 8.

248

M.S. Lund, B. Solhaug, and K. Stølen

The Semantics of Risk Graphs with Change. Given the syntax of risk graphs with change as deﬁned above, we can deﬁne the semantics as a straightforward generalisation of the semantics of regular risk graphs. The semantics [[(Db , Da )]] of a risk graph with change is deﬁned as def

[[(Db , Da )]] = [[Db ]] ∧ [[Da ]] For a combined representation D of a risk graph with change, the semantics is deﬁned as def [[D]] = [[before(D), after(D)]] 4.3

Reasoning about Likelihoods in Risk Graphs

In this section we introduce rules for calculating probabilities of vertices in risk graphs, and we provide guidelines for consistency checking probabilities that are assigned to risk graphs. The ﬁrst rule is referred to as the relation rule, and captures the conditional likelihood semantics of a risk graph relation. For a vertex v that leads to v , the vertex v v denotes the occurrences of v that happen after an occurrence of v. Rule 1 (Relation). If there is a direct relation from v to v , we have: P

v(P ) v −→ v (v v )(P · P ) The second rule is referred to as the mutual exclusive vertices rule, and yields the probability of either v or v occurring when the two vertices are mutually exclusive: Rule 2 (Mutually exclusive vertices). If the vertices v and v are mutually exclusive, we have: v(P ) v (P ) (v v )(P + P ) The third rule is referred to as the statistically independent vertices rule, and yields the probability of either v or v occurring when the two vertices are statistically independent. Rule 3 (Statistically independent vertices). If vertices v and v are statistically independent, we have: v(P ) v (P ) (v v )(P + P − P · P ) As a small example of probability calculation consider the risk graph in Fig. 6 and assume we want to calculate the probability of v3 from v1 and v2 . By Rule 1

Risk Analysis of Changing and Evolving Systems Using CORAS

249

Table 3. Guidelines for consistency checking likelihoods How to check consistency of likelihoods in risk graphs Exact values in complete diagrams Assigned value: v(p) Calculated value: v(p ) Consistency check: p = p Exact values in incomplete diagrams Assigned value: v(p) Calculated value: v(p ) Consistency check: p ≥ p Intervals in complete diagrams Assigned interval: v([pi , pj ]) Calculated interval: v([pi , pj ]) Consistency check: [pi , pj ] ⊆ [pi , pj ] or, equivalently, pi ≤ pi and pj ≥ pj Intervals in incomplete diagrams Assigned interval: v([pi , pj ]) Calculated interval: v([pi , pj ]) Consistency check: pj ≥ pj

we calculate (v1 v3 )(P1 · Pa ) and (v2 v3 )(P2 · Pb ). Assuming that v1 and v2 , as well as v1 v3 and v2 v3 , are statistically independent, we use Rule 3 to calculate ((v1 v3 ) (v2 v3 ))(P1 · Pa + P2 · Pb − P1 · Pa · P2 · Pb ). Assuming that the likelihood estimates in Fig. 6 are correct, there is still one issue to consider before we can conclude about the likelihood of the vertex v3 . The issue is whether or not the risk graph is complete. If the risk graph is complete, the graph shows all the possible ways in which v3 may occur. In that case we have that v3 = (v1 v3 )(v2 v3 ) and that P3 = P1 ·Pa +P2 ·Pb −P1 ·Pa ·P2 ·Pb is the correct likelihood of this vertex. If the risk graph is incomplete, there may be further scenarios that can lead to v3 . In that case we only know that P3 is the lower bound of the probability of v3 . Consistency checking of risk models is important, as it is a useful means for detecting errors or misunderstandings of the risk estimates that are documented during a risk analysis. The basis for the consistency checking is the likelihood values that are already assigned to the vertices and relations of a risk graph. The guidelines for consistency checking depend on whether the risk graph in question is complete, and whether the likelihoods are given as exact probabilities or as probability intervals. The guidelines are given in Table 3. As an example of consistency checking, consider the risk graph in Fig. 6, assuming ﬁrst that the graph is complete. By the above example, we know that the probability of the vertex v3 is P3 = P1 · Pa + P2 · Pb − P1 · Pa · P2 · Pb given the vertices and relations that lead to this vertex. The assigned probability P3 must therefore equal the calculated probability P3 in order to be consistent with the preceding probability estimates if we are working with exact values. If the P ’s are intervals, we must have that P3 ⊆ P3 for the risk graph to be consistent.

250

M.S. Lund, B. Solhaug, and K. Stølen

Discarding the assumption of the completeness of the graph gives the consistency requirement that the assigned probability P3 must be greater than or equal to the calculated probability P3 , i.e. that P3 ≥ P3 , if we have exact values. On the other hand, if P3 and P3 are intervals P3 = [pi , pj ] and P3 = [pi , pj ], the requirement is that pj ≥ pj . 4.4

Relating Risk Model to Target Description

Risk analysis of changing systems requires means for identifying the parts of a risk picture that are aﬀected by changes to a speciﬁc part of the target (and therefore need to be reassessed), as well as identifying the parts of the risk picture that are not aﬀected (and therefore valid also after the changes). Thus we need techniques for identifying and documenting the relation between the target description and the risk models in a way that gives us traceability between target elements (elements of the target description) and risk model elements. Two key artifacts in a risk analysis are the target model and the risk model. The target model is the core part of the overall target description and documents the events, scenarios and actors that are the subject for the risk analysis. Given these two artifacts we introduce a third artifact in order to establish the relation between the former two: a trace model. The trace model is of a table format that allows the tracing from target model elements to risk model elements, and vice versa. Initially we can think of the trace model as a set of pairs (uid , vid ) of target model identiﬁers uid and risk model identiﬁers vid representing the rows of the table. This of course require that each of the elements have a unique identiﬁer. In the following we assume that we already have a target model and a risk model of elements with unique identiﬁers, since obtaining such models by indexing the elements is a trivial task. From a pragmatic point of view, there are two obvious shortcomings of the table format given above. To make eﬃcient use of the trace model it should convey information about the relations in an intuitive way; the use of possibly tool generated indexes for the model elements is not intuitively informative. Furthermore, in many cases several target model elements are logically understood as a whole. Without some means of grouping several rows of the table into one compound relation, such structures of the target model will be obscured. To mitigate this we introduce a third column in the table for tagging the target model element/risk model element pairs. The grouping of pairs is then conducted by inserting the same tag on several rows. The name of the tag should be chosen by the end-user, and should be a unique name that conveys intuitive information about the grouping. More formally, the trace model is now a set of tuples (uid , vid , t) of a target model identiﬁer, a risk model identiﬁer and a tag. We extend the risk graph notation with a language construct for explicitly specifying the relation to the target model. The construct is used for annotating risk graphs with the tags of the trace model. We understand this construct as a mere visualisation of the trace model in the risk graphs, and not as part of the semantics of risk graphs. An example of the visualisation of a trace model in a risk graph with change is shown in Fig. 10. As with the other elements of

Risk Analysis of Changing and Evolving Systems Using CORAS

251

t1 t2

v1 [P1]

Pa

v2 [P2]/[P2’]

Pb/Pb’

v3 [P3]/[P3’]

Pc/Pc’

v4 [P4]/[P4’]

Pd/Pd’

Pf Pe

v5 [P5]

v7 [P7]/[P7’]

v6 [P6]

t3

Fig. 10. Relating target model and risk model

risk models with change, the target model relations can be speciﬁed as existing before the change only, (for example t1 ), after the change only (for example t3 ), or both before and after the change (for example t2 ).

5

Instantiation of CORAS

In a CORAS risk analysis, threat diagrams are used intensively to facilitate risk identiﬁcation and risk estimation. The diagrams are furthermore used as a part of the documentation and reporting of the analysis results. Figure 11 depicts an example of a threat diagram. In fact, this threat diagram shows the scenarios that are modelled by means of the risk graph in Fig. 10. Threat diagrams describe how threats may exploit vulnerabilities to initiate threat scenarios, how threat scenarios may lead to unwanted incidents or other threat scenarios, and the assets harmed by the unwanted incidents. The language constructs are threats (deliberate, accidental and non-human), vulnerabilities, threat scenarios, unwanted incidents and assets. Only threat scenarios and unwanted incidents may be assigned likelihoods. There are furthermore three kinds of relations in threat diagrams, namely initiates relations, leads-to relations and impacts relations. An initiates relation has a threat as source and a threat scenario or unwanted incidents as target. It can be annotated with a likelihood that describes the likelihood for the threat to initiate the related scenario or incident. A leads-to relation has a threat scenario or unwanted incident as both source and target. It can be annotated with a conditional likelihood. An impacts relation has an unwanted incident as source and an asset as target, and can be annotated with a consequence value that describes the harm of the incident on the asset when the incident occurs.

r3

r2

r1

v5 [P5]

v2 [P2]/[P2’]

v1 [P1]

t3

Pa

Pb/Pb’

t1

Pe

v3 [P3]/[P3’]

t2

Pc/Pc’

v6 [P6]

v4 [P4]/[P4’]

Pf

Pd/Pd’ v7 [P7]/[P7’]

c/c’ a

252 M.S. Lund, B. Solhaug, and K. Stølen

Fig. 11. Instantiation of risk graphs in CORAS

Risk Analysis of Changing and Evolving Systems Using CORAS

253

While all scenarios and relations in Fig. 10 are present in Fig. 11, there are some signiﬁcant diﬀerences between the two diagrams. The threat diagram explicitly shows the initiating threats, distinguishes v7 from the other scenarios as an unwanted incident, and explicitly shows the asset that is harmed. The diﬀerences between threat diagrams and risk graphs are summarised as follows: – Initiate relations and leads-to relations in threat diagrams can be annotated with vulnerabilities, while the relations in risk graphs cannot. – Threat diagrams distinguish between four kinds of vertices, namely threats, threat scenarios, unwanted incidents and assets, while risk graphs only have scenarios. – Threat diagrams distinguish between three kinds of relations, namely initiates relations, leads-to relations and impacts relations, while risk graphs only have leads-to relations. Given the diﬀerences between threat diagrams and risk graphs, the techniques for reasoning about likelihoods nevertheless carry over to the CORAS instantiation. The vulnerabilities are mere annotations on relations, and can be ignored in the formal representation of the diagrams. Moreover, the various vertices and relations of threat diagrams can be interpreted as special instances of the risk graph vertex and relation: – An unwanted incident of a threat diagram is interpreted as a scenario of a risk graph. – A set of threats r1 , . . . , rn with initiates relations to the same threat scenario s is interpreted as follows: The threat scenario s is decomposed into n parts, where each resulting sub-scenario sj , j ∈ {1, . . . , n}, corresponds to the part of s that is initiated by threat rj . The threat rj with initiates relation of likelihood lj to sub-scenario sj is then combined into the risk graph scenario Threat rj initiates sj and the scenario is assigned likelihood lj . – An impacts relation from unwanted incident u to asset a with consequence c in a threat diagram is interpreted as follows: The impacts relation is interpreted as a risk graph relation with likelihood 1; the asset a is interpreted as the risk graph scenario Incident u harms asset a with consequence c.

With this interpretation, we refer to Sect. 4.3 for the techniques for reasoning about likelihoods in CORAS threat diagrams. However, notice that Rule 1 (Relation) applies to the CORAS leads-to relations only and that Rule 2 (Mutually exclusive vertices) and Rule 3 (Independent vertices) apply to the CORAS threat scenarios and unwanted incidents. In order to allow all likelihood reasoning to be conducted directly in CORAS diagrams, we introduce a separate rule for the initiates relation. We let r denote a threat, v denote a vertex (threat scenario or unwanted incident), r − → v denote the initiates relation from threat r to vertex v, and r v denote the occurrences of vertex v that are initiated by the threat r.

254

M.S. Lund, B. Solhaug, and K. Stølen Table 4. List of acronyms Acronym

Meaning

ACC ADS-B AMAN AOIS ATCO ATM ATS CWP FDPS OPS Room SUP

Area Control Center Automatic Dependent Surveillance-Broadcast Arrival Manager Aeronautical Operational Information System Air Traﬃc Controller Air Traﬃc Management Air Traﬃc System Controller Working Position Flight Data Processing System Operation room Supervisor

Rule 4 (Initiates). If there is an initiates relation from threat r to vertex v, we have: P r− →v (r v)(P )

6

Practical Example

In this section we present a practical example of a risk analysis of a changing system under the before-after perspective. As the running example we use an Air Traﬃc Management (ATM) risk analysis case study conducted within the SecureChange project.1 European Air Traﬃc Management is currently undergoing huge changes with introduction of new information systems and decision support systems, as well as the reorganisation of ATM services, as part of the ATM 2000+ strategic agenda [6] and the Single European Sky ATM Research (SESAR) [26]. The case study presented in the following focuses in particular on the introduction of two new systems: Arrival Manager (AMAN) and Automatic Dependent SurveillanceBroadcast (ADS-B). The results of the case study are fully documented in a SecureChange deliverable [17]; in this paper only a selection of the target and the analysis is presented. A list of acronyms from the ATM domain used in the example is given in Table 4. 6.1

Context Establishment

The context establishment includes articulating the overall goals and objectives of the risk analysis, and deciding its focus and scope. This includes making a description of the target of analysis, identifying the assets and deciding the risk evaluation criteria. 1

The case study was conducted in close interaction with ATM personnel with expert knowledge about the target of analysis and the planned changes. For more information on the SecureChange project, see http://www.securechange.eu/

Risk Analysis of Changing and Evolving Systems Using CORAS

255

Goals and Objectives. An important part of Air Traﬃc Management is the services provided by ground-based Air Traﬃc Controllers (ATCOs) located at Area Control Centers (ACCs). One of the main responsibilities of Air Traﬃc Controllers is to maintain horizontal and vertical separation among aircrafts and between aircrafts and possible obstacles. They must ensure an orderly and expeditious air traﬃc ﬂow by issuing instructions and information to aircrafts, and by providing ﬂight context information to pilots, such as routes to waypoints and weather conditions. An important characteristic of the ATM domain of today is that there are limited interactions with the external world, and therefore also limited security problems in relation to information ﬂow to and from the environment. A further characteristic is that humans are at the center of the decision and work processes and the role of automated decision support systems and tools is limited. The planned and ongoing changes raise new security issues and security concerns with immediate impact on safety issues. The overall objective of this risk analysis is to understand, document and assess security risks of ATM with particular focus on the arrival management process with the involved activities, tasks, roles, components and interactions. The party of the analysis is the ATM service provider. Target Description. The target of the analysis is a speciﬁc Area Control Center and the activities of the Air Traﬃc Controllers in the arrival management process. The Area Control Center is a ground-based center with the responsibility of managing the traﬃc of a given airspace. The actual traﬃc management is conducted from the Operation room (OPS Room), which is the operational environment of the Air Traﬃc Controllers. The Air Traﬃc Controllers have different roles, some of which have their own Controller Working Position (CWP). The Controller Working Positions makes a range of tools for surveillance, communication and planning available to the Air Traﬃc Controllers. In the following we ﬁrst document the target of analysis before the introduction of AMAN and ADS-B. Thereafter we describe the planned changes, before we document the target description where the introduction of AMAN and ADSB is reﬂected. The selected part of the target is documented by use of UML structured classiﬁers and activity diagrams. Before Changes. The structured classiﬁer of Fig. 12 shows the structure and communication links of the ATM components, while the structured classiﬁer of Fig. 13 shows the Operation room as consisting of a number of ACC islands that are connected to the ACC network. Each ACC island consists of a number of Controller Working Positions, each of which are operated by exactly one Air Traﬃc Controller, and is divided into sector teams with the responsibility of assigned sectors of the airspace. The Operation room furthermore has a number of Supervisors (SUPs) that communicate with the ACC islands, and that also are connected to the ACC Network via the Controller Working Position CWP SUP. The UML activity diagram of Fig. 14 gives a high-level overview of the various tasks of the arrival management process.

256

M.S. Lund, B. Solhaug, and K. Stølen

class ATM

: Meteostations

: FDPS

: Radar

: AOIS

: Surveillance

: ACC network

: Technical room[1..*]

: OPS room[1..*]

: Adjacent ATS unit[*]

: Aircraft[*]

Fig. 12. ATM before changes

class OPS Room : ACC network

: CWP_SUP

: SUP[1..*]

: ACC island[1..*]

Fig. 13. Operation room before changes

Planned Changes. The changes we are addressing are, in particular, the introduction of the Arrival Manager (AMAN) in the managing of air traﬃc, as well as the introduction of the Automatic Dependent Surveillance-Broadcast (ADS-B). The AMAN is a decision support tool for the automation of Air Traﬃc Controllers’ tasks in the arrival management, such as the computation of arrival sequences for aircrafts approaching an airport. The introduction of the AMAN aﬀects the Controller Working Positions, as well as the Area Control Center as a whole. The main foreseen changes from an operational and organisational point of view are the automation of tasks (i.e. the usage of the AMAN for the computation of the arrival sequence) that currently are carried out by Air Traﬃc Controllers.

Risk Analysis of Changing and Evolving Systems Using CORAS

257

Arrival management tasks

Aircraft data analysis for starting the sequence creation

Controlling the aircraft in the sector

Sequence finalisation

Clearances to the aircraft for building the planned sequence Progressive transfer of the whole sequence to the adjacent sector

Fig. 14. Arrival management tasks before changes

The introduction of the ADS-B is actually independent of the AMAN, but is introduced during the same time frame. ADS-B is a cooperative GPS-based surveillance technique for air traﬃc control where the aircrafts constantly broadcasts their position to the ground and to other aircrafts. After Changes. In order to highlight the changes in the diagrams, we use grey shading to indicate elements that are introduced. The UML structured classiﬁer of Fig. 15 shows the structure and communication links of the ATM components after the changes. At this level we only see the introduction of the ADS-B. The diagram of Fig. 16 shows the internal structure of the Operation room after the introduction of the AMAN. The AMAN is connected to the ACC network, and thereby also to the ACC islands and the Controller Working Positions. The UML activity diagram of Fig. 17 gives a high-level overview of the arrival management tasks after the changes. Assets. The purpose of asset identiﬁcation is to identify the parts, aspects or properties of the target with respect to which the risk analysis will be conducted.

258

M.S. Lund, B. Solhaug, and K. Stølen class ATM

: Meteostations

: FDPS

: ADS-B

: Radar

: AOIS

: Surveillance

: ACC network

: Technical room[1..*]

: OPS room[1..*]

: Adjacent ATS unit[*]

: Aircraft[*]

Fig. 15. ATM after changes class OPS Room : AMAN : ACC network

: CWP_SUP

: SUP[1..*]

: ACC island[1..*]

Fig. 16. Operation room after changes

An asset is something to which a party assigns value, and hence for which the party requires protection. In this analysis, the party is the ATM service provider who owns the Area Control Center in question. The risk analysis addresses security issues. Before the changes, the focus is on the security property Information Provision: The provisioning of information regarding queue management sensitive data by speciﬁc actors (or systems) must be guaranteed 24 hours a day, 7 days a week, taking into account the kind of data shared, their conﬁdentiality level and the diﬀerent actors involved. As explained above the ATM of today has little interaction with the external world, but the planned changes will raise new security issues. Therefore, in the analysis of the situation after the changes, the focus is extended to also include the security property Information protection: Unauthorised actors (or systems) are not allowed to access conﬁdential queue management information.

Risk Analysis of Changing and Evolving Systems Using CORAS

Arrival management tasks

Acquisition of the AMAN provided sequence

Controlling the aircraft in the sector

AMAN sequence monitoring and verification Clearances to the aircraft for building the planned sequence Progressive transfer of the whole sequence to the adjacent sector

Fig. 17. Arrival management tasks after changes

ATM service provider

Compliance

Availability of arrival sequences

Availability of aircraft position data

Airlines’ trust

Confidentiality of ATM information

Fig. 18. Assets before and after the changes

259

260

M.S. Lund, B. Solhaug, and K. Stølen

In the asset identiﬁcation these security properties are covered by the two corresponding concepts of conﬁdentiality and availability. More precisely we deﬁne the assets of the analysis to be Availability of arrival sequences and Availability of aircraft position data before the changes, with the addition of Confidentiality of ATM information, after the changes. In addition we deﬁne the indirect assets Compliance and Airlines’ trust. Indirect assets are assets that, with respect to the target and scope at hand, is harmed only via harm to other assets. Assets that are not indirect we often refer to as direct assets. We use CORAS asset diagrams to document the assets and the relations between them. The asset diagram showing assets both before and after is given in Fig. 18. Availability of arrival sequences and Availability of aircraft position data, as well as the indirect asset Compliance, shown with white shadows, are relevant both before and after, while Confidentiality of ATM information and the indirect asset Airlines’ trust, shown with black shadow, are relevant only after. The arrows of the diagram specify harm relations. One asset is related to another if harm to the former may lead to harm to the latter. In this case, harm to all direct assets may lead to harm to the indirect asset Compliance, but only harm no Confidentiality of ATM information may lead to harm to Airlines’ trust. Risk Evaluation Criteria. The risk evaluation criteria deﬁne the level of risk that the party, i.e. the ATM service provider, is willing to accept for the given target of analysis. Basically, the criteria are a mapping from risk levels to the decision of either accepting the risk or evaluating the risk further for possible treatment. In order to speak of risk levels, we need ﬁrst to deﬁne consequence and likelihood scales and a risk function. The consequence and likelihood scales are partly based on requirements and advisory material provided by EUROCONTROL [5,7], and are given in Tables 5 through 7. The risk function is a mapping from pairs of consequence and likelihood to risk levels and is documented by the risk matrix shown in Fig. 19. We use three risk levels, namely low (light grey), medium and high (dark grey). With these deﬁnitions we deﬁne the risk evaluation criteria as follows: – High risk : Unacceptable and must be treated. – Medium risk : Must be evaluated for possible treatment. – Low risk : Must be monitored. In this analysis, the scales, risk function and risk evaluation criteria apply both before and after the changes, while in the general this is not always case as both scales, risk functions and risk evaluation criteria may change in the transition from before to after. 6.2

Risk Identification

Risk identiﬁcation is conducted as a structured brainstorming involving personnel with ﬁrst hand knowledge about the target of analysis. By conducting a

Risk Analysis of Changing and Evolving Systems Using CORAS

261

Table 5. Consequence scale for availability Consequence Description Catastrophic Major Moderate Minor Insigniﬁcant

Catastrophic accident Abrupt manoeuvre required Recovery from large reduction in separation Increasing workload of ATCOs or pilots No hazardous eﬀect on operations

Table 6. Consequence scale for conﬁdentiality Consequence Description Catastrophic Major Moderate Minor Insigniﬁcant

Loss of data that can be utilised in terror Data loss of legal implications Distortion of air company competition Loss of aircraft information data (apart from aircraft position data) Loss of publicly available data

Table 7. Likelihood scale Likelihood Description Certain Likely Possible Unlikely Rare

A very high number of similar occurrences already on record; has occurred a very high number of times at the same location/time A signiﬁcant number of similar occurrences already on record; has occurred a signiﬁcant number of times at the same location Several similar occurrences on record; has occurred more than once at the same location Only very few similar incidents on record when considering a large traﬃc volume or no records on a small traﬃc volume Has never occurred yet throughout the total lifetime of the system

Likelihood

Insignificant

Minor

Consequence Moderate

Rare Unlikely Possible Likely Certain

Fig. 19. Risk function

Major

Catastrophic

Availability of aircraft position data

M.S. Lund, B. Solhaug, and K. Stølen

Software error

CWP

Duplication of labels

Creation of false alarms The consolidation of data from several radar sources fails

Technical room

ATCO

Lack of awareness

ATCO fails to comply with arrival management procedures

Delays in sequence provisioning

Degradation of aircraft position data

Availability of arrival sequences

262

Fig. 20. Threat diagram before changes

walkthrough of the target description, the risk are identiﬁed by systematically identifying unwanted incidents, threats, threat scenarios and vulnerabilities. The results are documented on-the-ﬂy by means of CORAS threat diagrams. While the various parts of the threat diagrams are modelled and documented, the relations to the target of analysis are identiﬁed and documented at the same time. The approach to risk identiﬁcation is to ﬁrst identify and document risks for the target of analysis before the changes. Once this is completed, we

Fig. 21. Threat diagram before and after changes

Attacker

ADS-B transponder

Surveillance

Software error

Dependence on broadcasting

ADS-B

CWP

The consolidation of data from several radar sources fails

Technical room

Lack of awareness

Spoofing of ADS-B data

Degradation of aircraft position data

Delays in sequence provisioning

ATCO

Critical aircraft position data leakes to unauthorised third parties

ATCO fails to comply with AMAN sequence

Eavesdropping ADS-B communication

ADS-B transponders not transmitting correct information

Duplication of labels

Creation of false alarms

ATCO fails to comply with arrival management procedures

Confidentiality of ATM information

Availability of aircraft position data

Availability of arrival sequences

Risk Analysis of Changing and Evolving Systems Using CORAS 263

264

M.S. Lund, B. Solhaug, and K. Stølen

proceed by identifying and documenting the risks after the changes. Based on the documented relations to the target description, i.e. the trace model, we identify the parts of the threat scenarios that are not aﬀected by the changes and therefore do not have to be addressed again from scratch. Before Change. The threat diagram of Fig. 20 documents unwanted incidents that may arise due to duplication of labels on the Controller Working Position interface. A label depicts an aircraft with its position data, and is derived from radar data. When several radar sources are used, the label is generated by automatically consolidating (merging) the data from the various sources. In some cases, software errors may yield duplicated labels that may lead Air Traﬃc Controllers to believe there are two aircrafts. The duplication may also lead to false near miss alarms. This may lead to two unwanted incidents Delays in sequence provisioning and Degradation of aircraft position data. After Change. Figure 21 is the result of the risk identiﬁcation where the changes have been taken into account. The unwanted incidents documented in Fig. 20 persist under the changes and we therefore ﬁnd them as “two-layered” before-after elements in Fig. 21. However, the incidents Delays in sequence provisioning may be caused by the threat scenario ATCO fails to comply with arrival management procedures only before the changes and by the threat scenario ATCO fails to comply with AMAN sequence only after the changes. Due to the introduction of ADS-B as a means for surveillance, there are also further threats and threat scenarios that are relevant for the unwanted incident Degradation of aircraft position data after the changes. This is documented by the threats ADS-B transponder and Attacker, and the threat scenarios ADS-B transponders not transmitting correct information and Spoofing of ADS-B data. The threat Attacker also initiates a new threat scenario Eavesdropping ADS-B communication which leads to a new unwanted incident Critical aircraft position data leaks to unauthorised third parties that may cause harm to the new asset Confidentiality of ATM information. 6.3

Risk Estimation

The risk estimation basically amounts to estimating likelihoods and consequences for the unwanted incidents. Usually, we also estimate likelihoods for threat scenarios in order to get a better basis for estimating the likelihood of unwanted incidents and to understand the most important sources of risks. To the extent that threat scenarios and unwanted incidents before changes are completely unaﬀected by the changes, the risk estimates need not be conducted twice. However, when scenarios and incidents are aﬀected by the changes, the value of these must be reassessed. Likewise, we must estimate likelihoods and consequences for scenarios and incidents that only are relevant after the changes. Before Change. The estimation of likelihoods and consequences before the changes is documented in Fig. 22.

minor

Availability of aircraft position data

265

Software error

CWP

Duplication of labels [possible]

Degradation of aircraft position data [possible]

Availability of arrival sequences Creation of false alarms [possible]

The consolidation of data from several radar sources fails [possible]

Technical room

ATCO

Lack of awareness

ATCO fails to comply with arrival management procedures [rare]

Delays in sequence provisioning [possible]

minor

Risk Analysis of Changing and Evolving Systems Using CORAS

Fig. 22. Risk estimation before changes

After Change. The threat diagram of Fig. 23 documents the estimation of likelihoods and consequences when the changes are taken into account. The issues in relation to ADS-B are relevant only after the changes and are therefore only assigned after values. The likelihood of the threat scenario Creation of false alarms and the unwanted incident Delays in sequence provisioning furthermore changes under the changes to the target of analysis. The threat scenario ATCO fails to comply with arrival management procedures is not relevant after the changes and are thus only assigned a before value.

Fig. 23. Risk estimation before and after changes

Attacker

ADS-B transponder

Surveillance

Software error

Dependence on broadcasting

ADS-B

CWP

The consolidation of data from several radar sources fails [possible]/[possible]

Technical room

Lack of awareness

Spoofing of ADS-B data [rare]

ATCO fails to comply with AMAN sequence [rare]

Eavesdropping ADS-B communication [certain]

ADS-B transponders not transmitting correct information [likely]

Duplication of labels [possbile]/[possible]

Creation of false alarms [possbile]/[unlikely]

Degradation of aircraft position data [possible]/[possible]

Confidentiality of ATM information

Availability of aircraft position data

Availability of arrival sequences

minor/ minor

major

minor/ minor

Delays in sequence provisioning [possible]/[unlikely]

ATCO

Critical aircraft position data leakes to unauthorised third parties [rare]

ATCO fails to comply with arrival management procedures [rare]

266 M.S. Lund, B. Solhaug, and K. Stølen

Risk Analysis of Changing and Evolving Systems Using CORAS

6.4

267

Risk Evaluation

The purpose of risk evaluation is to decide which of the identiﬁed risks are acceptable and which must be evaluated for treatments. But before we evaluate the risks, we also need to estimate risks to the indirect assets, which until this point have not been dealt with. Indirect assets are assets that are harmed only through harm to other assets. This activity therefore consists of following the harm relations from direct to indirect assets documented in the asset diagram (recall Fig. 18) and estimating how the harm caused by unwanted incidents propagate. The result of this is documented in Fig. 24. A risk is the likelihood of an unwanted incident and its consequence for a speciﬁc asset. From the risk estimation documented in Figs. 23 and 24, we get seven risks when coupling unwanted incidents and assets. – R1:Delays in sequence provisioning (risk before-after toward Availability of arrival sequences) – R2:Degradation of aircraft position data (risk before-after toward Availability of aircraft position data) – R3:Critical aircraft position data leaks to unauthorised third part (risk after toward Confidentiality of ATM information) – R4:Delays in sequence provisioning (risk before-after toward Compliance) – R5:Degradation of aircraft position data (risk before-after toward Compliance) – R6:Critical aircraft position data leaks to unauthorised third part (risk after toward Compliance) – R7:Critical aircraft position data leaks to unauthorised third part (risk after toward Airlines’ trust)

Delays in sequence provisioning [possible]/[unlikely]

Degradation of aircraft position data [possible]/[possible]

Critical aircraft position data leakes to unauthorised third parties [rare]

minor/ minor Availability of arrival sequences minor/ minor

insignificant/ insignificant

minor/ minor

Availability of aircraft position data major

minor

Compliance

moderate Confidentiality of ATM information

Fig. 24. Risk estimation for indirect assets

Airlines’ trust

268

M.S. Lund, B. Solhaug, and K. Stølen

Likelihood

Insignificant Rare Unlikely Possible Likely Certain

R4 R4

Minor R6 R1 R1, R2, R5

Consequence Moderate R7

Major R3

Catastrophic

Fig. 25. Risk evaluation

ATCO

R1: Delays in sequence provisioning [low]/[low] Software error

Availability of arrival sequences

Technical room Surveillance

ADS-B transponder

R2: Degradation of aircraft position data [low]/[low]

ADS-B

Availability of aircraft position data

CWP

Attacker

R3: Critical aircraft position data leakes to unauthorised third parties [medium]

Confidentiality of ATM information

Fig. 26. Risk diagram before and after changes – direct assets

Risk Analysis of Changing and Evolving Systems Using CORAS

Software error

ADS-B transponder

Attacker

269

R4: Delays in sequence provisioning [low]/[low]

R5: Degradation of aircraft position data [low]/[low]

Compliance

R6: Critical aircraft position data leakes to unauthorised third parties [low]

R7: Critical aircraft position data leakes to unauthorised third parties [moderate]

Airlines’ trust

Fig. 27. Risk diagram before and after changes – indirect assets

The ﬁrst step of the risk evaluation is to calculate the risk levels. These are obtained by plotting the risks in the risk function deﬁned during the context establishment (recall Fig. 19) using the estimated likelihoods and consequences. The risk evaluation is conducted separately for the risks before the changes and the risk after the changes. Figure 25 shows the risks plotted in the risk function; we use italic to indicate risks before and bold to indicate risks after. We use CORAS risk diagrams to document the results of calculating the risk levels. These diagrams show the risks together with the threats that initiate them and the assets they harm. The three risks toward direct assets (R1–R3) are documented in Fig. 26, while the risks toward indirect assets (R4–R7) are documented in Fig. 27. By the risk evaluation criteria deﬁned in the context establishment, risks R3 and R7 must be evaluated for possible treatments, while the rest must be monitored. 6.5

Risk Treatment

For changes that are planned and/or anticipated, the risk treatment should ensure that the risk level is maintained at an acceptable level through the changes, and that the risk level of the resulting system is acceptable. Whether or not

Attacker

ADS-B transponder

Surveillance

Dependence on broadcasting

ADS-B Spoofing of ADS-B data [rare] Implement encryption of ADS-B signals

Eavesdropping ADS-B communication [certain]

ADS-B transponders not transmitting correct information [likely]

Implement backup or improve maintenance of the transponder

Fig. 28. Treatment diagram after changes R7: Critical aircraft position data leakes to unauthorised third parties [moderate]

R3: Critical aircraft position data leakes to unauthorised third parties [medium]

R2: Degradation of aircraft position data [low]/[low]

Airlines’ trust

Confidentiality of ATM information

Availability of aircraft position data

270 M.S. Lund, B. Solhaug, and K. Stølen

Risk Analysis of Changing and Evolving Systems Using CORAS

271

the current risks should be subject to treatment depends on the time frame of the change, as well as the priorities of the parties and other stakeholder. If the planned or anticipated changes are very immediate, it may not make sense to invest in treatments for risks that disappear after the changes. Identiﬁed treatment options are documented using CORAS treatment diagrams, as shown in Fig. 28. The identiﬁed treatments focus mainly on the risk picture after the changes and, because only risks R3 and R7 must be evaluated for possible treatments, on the introduction of the ADS-B. One of the treatment options, namely ADS-B encryption, can ensure both conﬁdentiality and authentication, while the second turns out to be relevant only for availability and thus not for R3 and R7.

7

Related Work

The extent to which existing risk analysis methods support the analysis and representation of changing risks depends of course on the relevant perspective of change; recall from Sect. 2 the classiﬁcation of changes into the maintenance, before-after and continuous evolution perspectives. Somewhat more thorough treatments than provided in this paper of the maintenance and continuous evolution perspectives in CORAS are given in [17,20], in particular an approach to the continuous evolution perspective inspired by an application of CORAS in risk monitoring [23]. In principle, any risk analysis method can be used to analyse changing risks from the maintenance and the before-after perspective by staring from scratch and doing complete reassessments. For the management of changing risks to be eﬃcient, however, there should be methodical support for dealing with change in a systematic and focused manner. Most of the established risk analysis methods provide little or no support for analysing changing and evolving risks. The ISO 31000 risk management standard [15] prescribes change detection and identiﬁcation for emerging risks, but provides no guidelines for how to do this in practice. A state-of-the-art methodology like OCTAVE [1] recommends reviewing risks and critical assets, but how the risk analysis results should be updated is not speciﬁed. Some approaches have support for associating elements of risk models to parts of the target description, which may facilitate the identiﬁcation and documentation of risk changes due to target changes. UML based approaches such as misuse cases [28] may utilise built-in mechanisms in the UML for relating elements from diﬀerent UML diagrams. ProSecO [12] relates risks to elements of a functional model of the target. With respect to the before-after perspective, ProSecO provides some support for modelling the various phases or states of a change process; when the models change, elements of the model may be transferred to states that indicate the need for additional risk analysis. ProSecO furthermore provides some support for the modelling of the change process by means of state machines. Other academic studies have focused on either maintenance [18,27] or variants of reassessment [9,16].

272

M.S. Lund, B. Solhaug, and K. Stølen

The continuous evolution perspective is the most general of the three perspectives on change as it incorporates time into the picture and thereby refers to the instance of the evolving risk picture at any given point in time. Among the existing approaches the support for this perspective is virtually non-existent. Some risk modelling approaches do provide support for updating the values that are annotated to diagrams in the sense that by changing the input values, the derived output values can be automatically updated. These includes fault trees [13], Markov models [11], and Bayesian networks [2,8]. Still, CORAS is to our knowledge the sole approach to provide means for predicting the evolution of risks related to an evolving target.

8

Conclusion

A traditional risk analysis considers the target of analysis at a particular point in time and in a particular conﬁguration. The result is the documentation of a risk picture valid for that conﬁguration at that point in time. However, systems and their environment tend to change and evolve over time and the risks toward the system may shift. If changes are not captured and reﬂected in the risk documentation the established risk picture may no longer be valid, with the consequence that we no longer have a full and correct overview of the risks of our target. Conducting a new risk analysis from scratch when the target or its environment has changed is not an approach to be preferred as risk analyses are time and resource consuming and parts of the risk documentation is likely to still be valid. In order to appropriately handle change, risk analysis methods should be supported with techniques and guidelines that are specialised toward modelling, analysing and reasoning about changing risks. However, the nature of the changes have implications the methodological needs. In this paper we have categorised changes into three perspectives referred to as maintenance, before-after and continuous evolution and characterised the methodological needs for each of them. The focus in this paper has been on the before-after perspective which comprise planned and anticipated changes of a radical and extensive nature. The challenge in this perspective is to obtain and present a risk picture that describes both current and future risks without doing double work. This requires methods and guidelines for distinguishing the parts of the risk documentation that are aﬀected by changes from the parts that are unaﬀected. In this paper we handle this by providing modelling support for changing risks and means for relating risk models to target models in such a way that changes in the target models can be traced to the risk models. These are given as extensions to risk graphs, which may be seen as a common abstraction of a variety of diﬀerent risk modelling techniques. In this way, the extension may be instantiated into these diﬀerent techniques. We demonstrate this by providing an instantiation into the CORAS risk modelling language. Further, we demonstrate our approach by a practical example from a risk analysis case study from the Air Traﬃc Management (ATM) domain.

Risk Analysis of Changing and Evolving Systems Using CORAS

273

In diﬀerence from other approaches, we provide a systematic approach to change management in the context of risk analysis. This includes methodological guidelines on how to maintain and update risk analysis documentation to reﬂect both past and future changes to the target without starting from scratch and doing a full reassessment, something that to a large extent is missing in other risk analysis methodologies. Acknowledgements. The work on which this paper reports has been funded by the European Commission through the projects SecureChange (Contract no. 231101) and NESSoS (Contract no. 256980). Thanks to Massimo Felici, Frank Innerhofer-Oberperﬂer, Valentino Meduri, Alessandra Tedeschi, and the ATM experts participating in the analysis, for their contributions to the ATM case study.

References 1. Alberts, C.J., Davey, J.: OCTAVE criteria version 2.0. Technical report CMU/SEI2001-TR-016. Carnegie Mellon University (2004) 2. Ben-Gal, I.: Bayesian networks. In: Ruggeri, F., Kenett, R.S., Faltin, F.W. (eds.) Encyclopedia of Statistics in Quality and Reliability. John Wiley & Sons, Chichester (2007) 3. Brændeland, G., Refsdal, A., Stølen, K.: Modular analysis and modelling of risk scenarios with dependencies. J. Syst. Softw. 83(10), 1995–2013 (2010) 4. Dudely, R.M.: Real analysis and probability. Cambridge Studies in Advanced Mathematics, Cambridge (2002) 5. EUROCONTROL: EUROCONTROL safety regulatory requirements (ESARR) 4 – Risk assessment and mitigation (2001) 6. EUROCONTROL: Air Traﬃc Management Strategy for the years 2000+. vol. 1,2 (2003) 7. EUROCONTROL: ESARR advisory material/Guidance document (EAM 2/ GUI 5) – Harmonisation of safety occurrences severity and risk assessment (2005) 8. Fenton, N., Neil, M.: Combining evidence in risk analysis using Bayesian networks. Agena White Paper W0704/01, Agena (2004) 9. Goel, S., Chen, V.: Can business process reengineering lead to security vulnerabilities: Analyzing the reengineered process. Int. J. Prod. Econ. 115(1), 104–112 (2008) 10. Haugen, Ø., Husa, K.E., Runde, R.K., Stølen, K.: STAIRS towards formal design with sequence diagrams. Softw. Syst. Modeling 4(4), 355–367 (2005) 11. Howard, R.A.: Dynamic Probabilistic Systems. Markov Models, vol. I. John Wiley & Sons, Chichester (1971) 12. Innerhofer-Oberperﬂer, F., Breu, R.: Using an enterprise architecture for IT risk management. In: Information Security South Africa Conference, ISSA 2006 (2006) 13. International Electrotechnical Commission: IEC 61025 Fault Tree Analysis (FTA) (1990) 14. International Electrotechnical Commission: IEC 60300-9 Dependability management - Part 3: Application guide - Section 9: Risk analysis of technological systems - Event Tree Analysis (ETA) (1995)

274

M.S. Lund, B. Solhaug, and K. Stølen

15. International Organization for Standardization: ISO 31000 Risk management – Principles and guidelines (2009) 16. Lee, E., Park, Y., Shin, J.G.: Large engineering project risk management using a Bayesian belief network. Expert Syst. Appl. 36(3), 5880–5887 (2009) 17. Lund, M.S., et al.: SecureChange Deliverable D5.3 Assessment method (2011), http://www.securechange.eu/sites/default/files/deliverables/ D5.3.Assessment Methods.pdf 18. Lund, M.S., den Braber, F., Stølen, K.: Maintaining results from security assessments. In: 7th European Conference on Software Maintenance and Reengineering (CSMR 2003), pp. 341–350. IEEE Computer Society, Los Alamitos (2003) 19. Lund, M.S., Solhaug, B., Stølen, K.: Evolution in relation to risk and trust management. Comput. 43(5), 49–55 (2010) 20. Lund, M.S., Solhaug, B., Stølen, K.: Model-Driven Risk Analysis – The CORAS Approach. Springer, Heidelberg (2011) 21. Nielsen, D.S.: The cause/consequence diagram method as basis for quantitative accident analysis. Technical report RISO-M-1374, Danish Atomic Energy Commission (1971) 22. Object Management Group: Uniﬁed Modeling Language: Superstructure, version 2.1.1 (non-change bar). OMG Document: formal/2007-02-05 (2005) 23. Refsdal, A., Stølen, K.: Employing key indicators to provide a dynamic risk picture with a notion of conﬁdence. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (eds.) IFIPTM 2009. IFIP Advances in Information and Communication Technology, vol. 300, pp. 215–233. Springer, Heidelberg (2009) 24. Robinson, R.M., Anderson, K., Browning, B., Francis, G., Kanga, M., Millen, T., Tillman, C.: Risk and Reliability – An Introductory Text. R2A, 5th edn (2001) 25. Schneider, S.: Attack trees: Modeling security threats. Dr. Dobb’s J. 24, 21–29 (1999) 26. SESAR Consortium: The ATM Target Concept. SESAR Deﬁnition Phase - Deliverable 3 (2007) 27. Sherer, S.A.: Using risk analysis to manage software maintenance. J. Softw. Maint.: Res. Pract. 9(6), 345–364 (1997) 28. Sindre, G., Opdahl, A.L.: Eliciting security requirements by misuse cases. In: 37th International Conference on Technology of Object-Oriented Languages and Systems (TOOLS Paciﬁc 2000), pp. 120–131. IEEE Computer Society, Los Alamitos (2000)

Author Index

Alvim, M´ ario S. 211 Andr´es, Miguel E. 211 Asnar, Yudistira 152 Basin, David 101 Bhargavan, Karthikeyan

Livraga, Giovanni 1 Luccio, Flaminia L. 35 Lund, Mass Soldal 231 Massacci, Fabio 152 Micciancio, Daniele 185

66

Chatzikokolakis, Konstantinos Clavel, Manuel 101

211

Dania, Carolina 101 De Capitani di Vimercati, Sabrina Egea, Marina

Ortiz, Gonzalo 1

101

Palamidessi, Catuscia

211

101 Samarati, Pierangela 1 Solhaug, Bjørnar 231 Steel, Graham 35 Stølen, Ketil 231

Focardi, Riccardo 35 Foresti, Sara 1 Fournet, C´edric 66 Garc´ıa de Dios, Miguel A. Gordon, Andrew D. 66 Gorrieri, Roberto 125

101 Valdazo, Javier Vernali, Matteo

101 125

Our partners will collect data and use cookies for ad personalization and measurement. Learn how we and our ad partner Google, collect and use data. Agree & close