Data Refinement: Model-Oriented Proof Methods and their Comparison (Cambridge Tracts in Theoretical Computer Science (No. 47))

Data Refinement: Model-Oriented Proof Methods and their Comparison The goal of this book is to provide a comprehensive and systematic introduction to the important and highly applicable method of data refinement and the simulation methods used for proving its correctness. The authors concentrate in the first part on the general principles needed to prove data refinement correct. They begin with an explanation of the fundamental notions, showing that data refinement proofs reduce to proving simulation. The topics of Hoare Logic and the Refinement Calculus are introduced and a general theory of simulations is developed and related to them. Accessibility and comprehension are emphasised in order to guide newcomers to the area. The book's second part contains a detailed survey of important methods in this field, such as VDM, and the methods due to Abadi & Lamport, Hehner, Lynch and Reynolds, Back's refinement calculus and Z. All these methods are carefully analysed, and shown to be either incomplete, with counterexamples to their application, or to be always applicable whenever data refinement holds. This is shown by proving, for the first time, that all these methods can be described and analysed in terms of two simple notions: forward and backward simulation. The book is self-contained, going from advanced undergraduate level and taking the reader to the state of the art in methods for proving simulation. Willem-Paul de Roever is Professor of Software Technology at the Institute for Computer Science and Applied Mathematics, Christian Albrechts University in Kiel. Kai Engelhardt is Postdoctoral Research Fellow at the School of Computing Science, University of Technology, Sydney.

Cambridge Tracts in Theoretical Computer Science Editorial Board S. Abramsky, Department of Computing Science, University of Edinburgh P. H. Aczel, Department of Computer Science, University of Manchester J. W. de Bakker, Centrum voor Wiskunde en Informatica, Amsterdam Y. Gurevich, Department of Electrical Engineering and Computer Science, University of Michigan J. V. Tucker, Department of Mathematics and Computer Science, University College of Swansea Titles in the series 1. 2. 3. 5. 6. 7. 8. 9. 10. 11. 12. 14. 15. 17. 18. 19. 21. 22. 23. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45.

G. Chaitin Algorithmic Information Theory L. C. Paulson Logic and Computation M. Spivey Understanding Z A. Ramsay Formal Methods in Artificial Intelligence S. Vickers Topology via Logic J.-Y. Girard, Y. Lafont & P. Taylor Proofs and Types J. Clifford Formal Semantics & Pragmatics for Natural Language Processing M. Winslett Updating Logical Databases K. McEvoy & J. V Tucker (eds) Theoretical Foundations of VLSI Design T. H. Tse A Unifying Framework for Structured Analysis and Design Models G. Brewka Nonmonotonic Reasoning S. G. Hoggar Mathematics for Computer Graphics S. Dasgupta Design Theory and Computer Science J. C. M. Baeten (ed) Applications of Process Algebra J. C. M. Baeten & W. P. Weijland Process Algebra M. Manzano Extensions of First Order Logic D. A. Wolfram The Clausal Theory of Types V Stoltenberg-Hansen, I. Lindstrom & E. Griff or Mathematical Theory of Domains E.-R. Olderog Nets, Terms and Formulas P. D. Mosses Action Semantics W. H. Hesselink Programs, Recursion and Unbounded Choice P. Padawitz Deductive and Declarative Programming P. Gardenfors (ed) Belief Revision M. Anthony & N. Biggs Computational Learning Theory T.F. Melham Higher Order Logic and Hardware Verification R. L. Carpenter The Logic of Typed Feature Structures E. G. Manes Predicate Transformer Semantics F. Nielson & H. R. Nielson Two Level Functional Languages L. Feijs & H. Jonkers Formal Specification and Design S. Mauw & G. J. Veltink (eds) Algebraic Specification of Communication Protocols V Stavridou Formal Methods in Circuit Design N. Shankar Metamathematics, Machines and Godel's Proof J. B. Paris The Uncertain Reasoners Companion J. Desel & J. Esparza Free Choice Petri Nets J.-J. Ch. Meyer & W. van der Hoek Epistemic Logic for AI and Computer Science J. R. Hindley Basic Simple Type Theory A. Troelstra & H. Schwichtenberg Basic Proof Theory J. Barwise & J. Seligman Information Flow A. Asperti & S. Guerrini The Optional Implementation of Functional Programming Languages 46. R. M. Amadio & P.-L. Curien Domains and Lambda-Calculi

Data Refinement Model-Oriented Proof Methods and their Comparison

Willem-Paul de Roever Kai Engelhardt

with the assistance of Karl-Heinz Buth Jos Coenen Paul Gardiner Yassine Lakhnech Frank Stomp

CAMBRIDGE UNIVERSITY PRESS

CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo, Delhi Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridge.org/9780521103503 © Cambridge University Press 1998 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 1998 This digitally printed version 2009 A catalogue record for this publication is available from the British Library ISBN 978-0-521-64170-8 hardback ISBN 978-0-521-10350-3 paperback

Contents

Preface

page viii

Part I: Theory

1

1 1.1 1.2 1.3

Introduction to Data Refinement Goal and Motivation Introduction to Data Refinement Historical Background

2 2 4 17

2 2.1 2.2 2.3 2.4 2.5

Simulation as a Proof Method for Data Refinement Introducing Simulation Soundness and (In)completeness of Simulation Data Invariants, Abstraction Relations, and Normal Variables Towards a Syntactic Characterization of Simulation Historical Background

19 19 22 25 34 46

3 3.1 3.2 3.3 3.4 3.5 3.6

Relations and Recursion Partial Orders and Monotonicity Binary Relations Recursion and Termination — the ^-Calculus Relational Semantics of Recursion — the Continuous ^-Calculus Reasoning about Termination — the Monotone ^-Calculus Historical Background

49 49 50 55 56 62 71

4 4.1 4.2 4.3 4.4 4.5 4.6

Properties of Simulation Composing Simulation Diagrams Implications between Simulations Data Invariants and Totality of Abstraction Relations Soundness of Simulation Maximal Data Types Completeness

73 74 77 80 81 82 83

vi

Contents

4.7

Historical Background

5 5.1 5.2 5.3 5.4 5.5 5.6

Notation and Semantics Introduction Predicates Programs Relational Terms Correctness Formulae Historical Background

87 90 91 93 108 112 115 119

6 6.1 6.2 6.3

A Hoare Logic Proof System Soundness and (Relative) Completeness Historical Background

121 122 128 130

7 7.1 7.2 7.3 7.4 7.5

Simulation and Hoare Logic U-simulation in Hoare Logic L-simulation in Hoare Logic L" 1 -simulation in Hoare Logic U"1-simulation in Hoare Logic Historical Background

132 133 133 143 144 144

8 8.1 8.2 8.3

An Extension to Total Correctness Semantic Model and Basic Fixed Point Theory Interpretation Functions for Total Correctness Historical Background

146 148 160 178

9 9.1 9.2 9.3

Simulation and Total Correctness Simulation An L-Simulation Theorem for Total Correctness Historical Background

181 181 189 193

10 Refinement Calculus 10.1 Lattice-theoretical Framework 10.2 Predicate Transformer Semantics 10.3 Predicate Transformers and Data Refinement 10.4 Predicate Transformers and Partial Correctness 10.5 Historical Background

11 11.1 11.2

194 196 206 215 223 234

? Gallery

236

Part II: Applications

255

Reynolds' Method Introduction Running Examnle: Finding Paths in Directed Graohs

257 257 258

Contents

vii

11.3 11.4

Analysis of Data Refinement a la Reynolds Historical Background

272 287

12 12.1 12.2 12.3 12.4

VDM Introduction Example: Dictionary Analysis of Data Refinement in VDM Historical Background

289 289 291 301 315

13 13.1 13.2 13.3

Z, Hehner's Method, and Back's Refinement Calculus Z Hehner's Method for Data Refinement Back's Refinement Calculus

317 318 327 332

14 14.1 14.2 14.3

Refinement Methods due to Abadi and Lamport and to Lynch Auxiliary Variables and Refinement Mappings Possibilities Mappings Historical Background

340 341 358 361

Appendix A

An Introduction to Hoare Logic

363

Appendix B

A Primer on Ordinals and Transfinite Induction

387

Appendix C

Notational Convention

394

Appendix D

Precedences

402

Bibliography

404

Index

419

Preface

The goal of this monograph is the introduction of, and comparison between, various methods for proving implementations of programs correct. Although these methods are illustrated mainly by applying them to correctness proofs of implementations of abstract data structures, the techniques developed apply equally well to proving correctness of implementations in general. For we shall prove that all these methods are only variations on one central theme: that of proof by simulation, of which we analyze at least 13 different formulations. As the central result we prove that these methods either imply or are equivalent to L-simulation (also called forward or downward simulation in the literature) or a combination of L- with L"1-simulation (the latter is also called backward or upward simulation). Since, as shown by Hoare, He, and Sanders, only the combination of these forms of simulation is complete, this immediately establishes when these methods are complete, namely, when they are equivalent to this combination. Our motivation for writing this monograph is that we believe that in this area of computer science (as well as in various other areas) the duty of universities is not to train students in particular methods, but rather to give students insight in both similarities and differences between methods such as VDM, Z, the methods advocated by Reynolds and Hehner, and methods more directly based on Hoare Logic or predicate transformers. The reason for this conviction is that computer science develops far too quickly for us to believe that any of these methods will survive in its present form. Therefore, it makes more sense to emphasize the general principles behind these methods, since these are more likely to maintain their value when the students of today have become the specialists of tomorrow. This monograph consists of two parts. In Part I we develop the general theory required for comparing model-oriented data refinement methods. Part II concerns applications, such as Reynolds' method, VDM, Z, Hehner's conVlll

Preface

IX

dition for data refinement, Back's refinement calculus, Abadi and Lamport's theory of refinement mappings, and Lynch's theory of possibilities mappings. In the first part of this monograph we develop the relational theory of simulation and a general version of Hoare logic, show how data refinement can be expressed in this logic, extend these results to total correctness, and show how all this theory can be uniformly expressed inside the refinement calculus of Ralph Back, Paul Gardiner, Carroll Morgan, and Joakim von Wright. Chapter Dependencies The chapter dependencies are depicted in the figure below. 1

14

13.2

12—M3.1

13.3

In this diagram, X -» Y means that X is a prerequisite for Y. Using this Book as a Classroom Text We use this book as a classroom text for a one-semester course on techniques for proving data refinement, given to advanced undergraduate and beginning graduate students. In general we move the material on applications forward, e.g., by discussing Chapter 11 on Reynolds' method after going through Chapter 7 which constitutes the technical heart of the book, and summarize the results of Section 9.2 briefly in order to discuss Sections 11.3.2 and 11.3.3 (concerning situations where Reynolds' method only works for total correctness). Depending on the teacher's preference, when focusing on applications we have skipped Chapter 10 on the refinement calculus, and when focusing on the refinement calculus skipped Chapters 13 (with the exception of Section 13.3) and 14. In both cases it is necessary that students have a minimal knowledge of

x

Preface

Hoare logic in order to be able to understand the main applications of our theory, namely Chapters 11 and 12 on Reynolds' method and VDM. Such minimal knowledge is provided by, e.g., the first three sections of Appendix A. Appendix B collects an equally minimal body of knowledge about ordinals, which is needed in Section 8.1 on semantic models for total correctness in order to understand the characterization of the least fixed point of a monotone function on a cpo by means of the least upper bound of its, in general transfinite, approximations. Section 3.5 can be skipped upon first reading. Proof Format This monograph contains many proofs. Their degree of formality varies between rather informal, as found in mathematical textbooks, and formal, mostly for demonstration purposes. Most proofs are presented semi-formally, line by line. Usually we provide a short justification for each of these steps, e.g., by giving the name or number of the lemma or theorem applied in this step. Some of the more tedious proofs have been checked using the Prototype Verfication System. (See [ORS92] for an overview or browse SRFs web site for all things PVS at h t t p : //www. c s l . s r i . com/pvs. html.) Feedback We would love to receive feedback from you, our dear readers. Please email your comments to bkmail l@inf ormat i k . u n i - k i e l . de. As soon as it makes sense, we will compile a list of errata and make it available on the World Wide Web at h t t p : //www. inf ormat i k . u n i - k i e l . de/deRoever/bkerr 1. html. Acknowledgments Travel involved in this work has been partially supported by ESPRIT-BRA projects SPEC (no. 3096) and REACT (no. 6021), and the Forderverein Technische Fakultat. This monograph contains photographs of many of the researchers whose work is discussed. We thank them for their permission to publish these. In particular, we thank Mrs Sophie Bekic and her children for their kind permission to publish a dear picture of their late husband and father Hans Bekic, and for providing it. Also we thank Mrs Joanna Park for her kind permission to publish a picture of her late husband David Park, and Mike Patterson for providing that picture. Finally our gratitude goes to Peter van Emde Boas, photographer

Preface

xi

par excellence of computer scientists, for generously putting his collection at our disposal. We would like to thank those people who contributed parts to this monograph, at various stages of its development. These are: • Karl-Heinz Buth, who has written Section 13.1 on Z. • Jos Coenen, who has been involved in its early stages, and who contributed especially to its planning by helping to write first versions of many sections. We were shocked last year when we heard of his untimely death, realizing how unevenly happiness has been bestowed upon people. • Paul Gardiner, who has made essential contributions to Chapter 10 on the refinement calculus, helped us by writing a first version which appeared as [Gar95], and advised us on later versions. • Yassine Lakhnech, who has contributed to the formulation of Section 3.4 on the relational semantics of recursion, to initial versions of Chapter 9 on simulation and total correctness, and to Chapter 14 on refinement methods due to Abadi and Lamport and to Lynch. • Frank Stomp, who wrote Section 14.2 on possibility mappings together with Yassine Lakhnech and contributed at an early stage to Appendix A on an introduction to Hoare logic. David Tranah and the helpful staff at CUP deserve our gratitude for helping us improve and, finally, publish this book. We would also like to thank: Ralph Back, Rudolf Berghammer, Marcello Bonsangue, Pierre Collette, Marja de Vroome, Loe Feijs, Martin Franzle, John Guttag, Barry Jay, Bengt Jonsson, Jens Knappmann, Lars Kuhne, Justus Kurth, Ben Lukoschus, Nancy Lynch, Helge Marquardt, Oliver Matz, Kees Middelburg, Ulf Milanese, Carroll Morgan, Markus Miiller-Olm, Sascha Ott, Amir Pnueli, Bodo Rosenhahn, Markus Schneider, Jan-Hendrik Scholer, Carsten Scholz, Michael Siegel, Karsten Stahl, Martin Steffen, Anne StraBner, Markus Tiedt, Frank Tonn, Ron van der Meyden, Jan Vitt, Jorg Wiechert, and Job Zwiers for their help, without which writing this book would have been impossible. More important is that life would have been impossible without the support of our wives, Corinne and Claudia. We dedicate this book to them. Christian-Albrechts-Universitat zu Kiel University of Technology, Sydney

W.-P. de Roever K. Engelhardt

Parti Theory

1 Introduction to Data Refinement

1.1 Goal and Motivation During the process of stepwise, hierarchical program development, a step represents a transformation of a so-called abstract higher level result into a more concrete lower level one. In general, this development process corresponds to increasing the amount of detail required for the eventual implementation of the original specification on a given machine. In the first part of this book we develop the relational theory of simulation and a general version of Hoare logic, show how data refinement can be expressed within this logic, extend these results to total correctness, and show how all this theory can be uniformly expressed inside the refinement calculus of Ralph Back, Paul Gardiner, Carroll Morgan and Joakim von Wright. We develop this theory as a reference point for comparing various existing data refinement methods in the second part, some of which are syntax-based methods. This is one of the main reasons why we are forced to clearly separate syntax from semantics. The second part of this monograph focuses on the introduction of, and comparison between, various methods for proving correctness of such transformation steps. Although these methods are illustrated mainly by applying them to correctness proofs of implementations of data types, the techniques developed apply equally well to proving correctness of such steps in general, because all these methods are only variations on one central theme: that of proof by simulation, of which we analyze at least 13 different formulations. We study in particular the similarities and the differences between such widely known methods as the methods advocated by John Reynolds [Ger78, Rey81] and Rick Hehner [Heh93], VDM [J90], Z [Spi92b], and methods more directly based on Hoare logic [Hoa72] or predicate transformers [B78, B88b, Mor89a, Mor90, GM93, MV94, Gar95]. Moreover, we consider methods that,

1.1 Goal and Motivation

3

although primarily developed for proving correctness of parallel programs, have as their main ingredient a nucleus which essentially concerns sequential (albeit nondeterministic) program refinement, such as the method of Martin Abadi and Leslie Lamport [AL91], and the possibilities mappings of Nancy Lynch [LT87, Lyn90]. This study makes sense because in the final analysis a surprisingly uniform picture emerges when comparing these proof methods — notwithstanding their wide differences in formulation. Either they are special cases of the method based on L-simulation (also called forward or downward simulation), and therefore incomplete as shown in Section 2.2.2, or they are equivalent to the combination of L- and L"1-simulation (also called backward or upward simulation), and then complete by a theorem of He, Hoare, and Sanders [HHS87]. This strengthens our conviction that it makes more sense to emphasize the general principles behind these methods, since we believe these will preserve their value when the students of today have become the specialists of tomorrow, even if present day methods are superseded by more modern ones. This first chapter presents intuitive explanations and some definitions of key notions in the field of data refinement, such as (abstract/concrete) data type, observability, operation, (data) refinement. The second chapter then introduces the notions of abstraction relation, representation invariant, and simulation, which are used throughout this monograph. In this way the reader is guided to the main questions answered in this part, viz.:

• What is refinement? What is data refinement? What is a correct refinement step, and how can such correctness be proven? • What is simulation? When one faces a correct case of data refinement, can one always prove its correctness given a particular simulation method? That is, which methods for proving data refinement are complete? Are these methods always sound? • What is the weakest (i.e., most general) concrete specification simulating a given abstract specification with respect to a given abstraction relation? • How can one guarantee that termination is preserved during simulation? The first two chapters serve as motivation and set the scene for the remainder of this monograph, starting with our first technical chapter, Chapter 3. We try to lead gently into the topic and therefore sometimes sacrifice rigor and mathematical precision for intuition.

4

1 Introduction to Data Refinement

1.2 Introduction to Data Refinement 1.2.1 Basic Issues Designing a large and complex program usually involves application of some refinement method providing a way to gradually transform an abstract program, possibly a specification, into a concrete implementation. The main principle of such a method is that if the initial abstract program is correct and the transformation steps preserve correctness, then the resulting implementation will be correct by construction. Because an abstract program is, in general, easier to prove correct than a concrete one, this simplifies the structuring of the verification process. This monograph focuses on methods for proving the correctness of such transformation steps. So we ask ourselves what it means to say that the result of applying a transformation step is regarded as a correct implementation of the construct to which that step is applied. In the context of data refinement, this amounts to the question of when an abstract program P(X) using a data type A is implemented correctly by the more concrete program P(C) obtained from P(Sl) by replacing operations Aj from A by corresponding operations Cj which belong to a more concrete data type C. This question becomes more interesting if we abstract from the particular pair of programs P(X) and P(C) to which this data type transformation is applied. This narrows our subject down to that of data refinement, i.e., formulating when the family of operations (Cj)jej belonging to a more concrete data type C correctly implements the family of operations (Aj)jej of the more abstract data type A. The solution of this problem depends on realizing that Aj and Cj in general constitute programs themselves, which are used as modules inside other, for the moment arbitrary, programs P. Now, intuitively, a concrete program module is a correct implementation of an abstract program module, if no program using the concrete module can observe that it is not using the first. That is, "implementation correctness" means that using the concrete program module does not lead to an observation which is not also an observation of the corresponding abstract program module. Note that this does not imply that the concrete and the abstract program display the same observations. There may be observations of the abstract program which are not observations of the corresponding concrete program. Hence, this definition only implies that the observations of the concrete program are contained in those of the corresponding abstract program. This is called refinement. Example 1.1 (Data refinement step) Consider, for instance, the following two sketches of program fragments in a pseudo-Pascal notation, where S\ and S2

1.2 Introduction to Data Refinement

5

are dummies for program fragments not involving program variables £/, /, and x, and () denotes the empty sequence. begin begin var U : set ofN;U := 0; var/: sequence o/N;/ := (); Si;

Si;

U:=Uu{x};

/ :=append(x,l);

y := a member of U end

y:=first(l) end

At this point nothing more than an intuitive understanding of the operational meaning of these program sketches is required. Any observable behavior in terms of values of the common program variables x and y exposed by the RHS1 program should also be a possible behavior of the LHS program. This refinement step comprises of replacing the variable U (ranging over finite subsets of the natural numbers) and operations on it by the sequence-valued variable / and corresponding operations. ^ Which particular set of observations, i.e., semantics, should characterize a program depends on the particular notion of correctness which a transformation step is intended to preserve. For instance, in the context of relational semantics of programs, this meaning is given by pairs of initial and corresponding final states of its computations. In the case of partial correctness, only terminating computations are represented by such pairs, and then refinement is expressed by inclusion between the corresponding relations. In the case of total correctness nonterminating computations are also made observable through pairs. As explained in Chapter 8, there are then various possibilities for characterizing the meaning of a program relationally, and, consequently, different ways to express refinement. The above account, suggestive as it may be, introduces a number of terms, those in italics, which need further explanation. What is an abstract or a concrete program? The notion of abstractness used here is relative. Program refinement distinguishes an upper level, called abstract, from a lower level, called concrete, in order to indicate the direction in which the process of refinement is taking place. Then, what does "gradually transform" mean in this context? Calling the level at which the transformation starts level 0, this level is transformed into level 1, which may be subsequently transformed into level 2, and so on. Thus, 1

We use RHS and LHS as acronyms for Right (respectively, Left) Hand Side.

6

1 Introduction to Data Refinement

a series of successive transformations or transformation steps takes effect, until a sufficiently concrete level of implementation has been reached (by some external criterion). As already indicated, we consider this process of transformation to consist of several individual transformation steps, each of which is considered to transform an abstract level (say, level n) into a more concrete one (level n + 1 ) . In what sense should the terms data type and abstract data type be understood? Abstract data types are usually defined by a set of operators and a set of axioms, typically given in the form of equations; see e.g. [GH78, LG86, Cle86, BHK89, Par90, Wir90]. As an example of this style we present the equations for an abstract data type stack(Z), where Bool refers to the abstract data type of Boolean values (with the usual propositional operators such as -i, =>, V, A) and Z to the abstract data type of the elements to be stacked (regarded as primitive at this level of specification). Example 1.2 (Characterization of stacks through equations) Name: stack(Z)

Operators: emptystack: —> stack(Z) push : Z x stack(Z) —> stack(Z) pop : stack(Z) —> stack(Z) top : stack{Z) —> Z empty? :stack{Z) —> Bool Axioms: 2. 3. 4.

top(push(z,s)) —z empty? (emptystack) = true empty ?(push(z,s)) = false