The Rubato Composer music software: component-based implementation of a functorial concept architecture

Computational Music Science

Series Editors Guerino B. Mazzola Moreno Andreatta

Gérard Milmeister

The Rubato Composer Music Software Component-Based Implementation of a Functorial Concept Architecture

Dr. Gérard Milmeister Langackerstrasse 49 8057 Zürich Switzerland [email protected] Contributors: Prof. Dr. Guerino B. Mazzola 164 Ferguson Hall Minneapolis MN 55455 USA [email protected]

ISBN 978-3-642-00147-5

Florian Thalmann Helmern 77l 6102 Malters Switzerland

e-ISBN 978-3-642-00148-2

DOI 10.1007/978-3-642-00148-2 Library of Congress Control Number: 2009921145 c 2009 Springer-Verlag Berlin Heidelberg  This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Printed on acid-free paper 9 8 7 6 5 4 3 2 1 springer.com

Foreword

Gérard Milmeister’s thesis, which is now published in Springer’s innovative series Computational Music Science, is a key text for accessing the present version of the R UBATO software. It is also the beautiful trace of a conceptual and technological completion of a development that was initiated in 1992, when Oliver Zahorka and I conceived, designed and implemented the R UBATO sofware for musical performance. This first implementation on NeXT computers, written in Objective C, was driven by the idea to implement Theodor W. Adorno’s theory of an analytical performance, i.e., a performance that is defined upon musical analysis of the given score. The original architecture of R UBATO was therefore modular and split into modules for analysis (rhythmical, melody, and harmonic) and modules for performance. These modules, coined rubettes, were only conceived as organic parts of an overall anatomy. However, the successive developments and also research driven investigations, such as Anja Fleischer’s work1 on rhythmical analysis or Chantal Buteau’s work2 on motivic analysis showed that there is also a legitimate interest in rubettes without being necessarily parts of a given fixed anatomy. Successive work by Stefan Göller3 on geometric concept spaces associated with R UBATO data formats, and Stefan Müller4 on performance and gesture rubettes proved that there is a different approach to R UBATO, which by far transcends the original “hardcoded” anatomy. 1 Fleischer, Anja. Die analytische Interpretation: Schritte zur Erschließung eines Forschungsfeldes am Beispiel der Metrik. Dissertation, Berlin 2003 2 Buteau, Chantal. A Topological Model of Motivic Structure and Analysis of Music: Theory and Operationalization. Dissertation, Zürich 2003 3 Göller, Stefan. Object Oriented Rendering of Complex Abstract Data. Dissertation, Zürich 2004 4 Müller, Stefan. Pianist’s Hands—Synthesis of Musical Gestures. Dissertation, Zürich 2004

v

vi

Foreword

The new requirements had to face different conceptual, soft-, and hardware conditions and challenges. To begin with, the NeXT computer had finished to exist, and the platform-dependent strategies, such as the original Objective C implementation had become obsolete by the now standard Java virtual platform environment. The other point of change was that the rubettes had to become a modular construct that would be of any size and would also be of open usage without much predefined larger anatomical preconditions. It turned out that the decisive requirements were defined by component-driven programming. However, this generic setup also entailed a radical redesign of the data format of denotators, which was invented in the early collaboration with Zahorka. The redesign was however not only affected by the component-driven data architecture, but by a meanwhile dramatic urge to step from naive (zero-addressed) denotators to functorial denotators, i.e., to quit the old-fashioned concept of a point and to introduce functorial points, i.e., morphisms defined on variable address modules and with values in not necessarily representable space functors. All these delicate requirements set up an agenda that could not be realized except by a computer scientist with excellent programming and really solid mathematical competence. Gérard Milmeister was the ideal researcher to bring such a difficult enterprise to its completion. His award-winning doctoral dissertation, which is now in your hands, is the written counterpart of his remarkable programming work, available as a GPL software on http://www.rubato.org. The thesis is not only a clear and concise introduction to the conceptual and mathematical architecture of the R UBATO enterprise, but offers also precise and concrete tutorials for the programming of rubettes and their networks. The success of Milmeister’s work is, last, but not least, documented by contributions from Florian Thalmann and myself, which prove that the R UBATO software may now reach out to compositional tasks that were postponed since the early developments of a geometric composition software presto in the late 80s. Thalmann’s BigBang rubette is the long-awaited extension of R UBATO to gestural strategies in composition, and it is the proof that Milmeister’s work is not only the completion of a long march through the hard- and software meanders and conceptual revolutions, but also is setting a pointer to creative future music software perspectives. Minnesota, October 2008

Prof. Dr. Guerino Mazzola

Preface to the Springer Edition

For this edition published by Springer, I am happy to be able to include as chapter 17 and chapter 18 two contributions by Guerino Mazzola and Florian Thalmann. The first is the description of a sophisticated rubette that provides an extensive gestural interface to manipulate musical structures. The second contribution is the first major application of R UBATO C OM POSER in music theory and computational composition. It resulted in a remarkable piece of music starting from the idea of “analyse créatrice” forwarded by Pierre Boulez. The whole process involves many of the features presented in this book, and, thus, is something of a “proof by construction” of the usefulness of these concepts. I therefore thank both for their energy and ingenuity in putting the R UBATO C OMPOSER system to test and exercising its capabilities. Zurich, December 2008

Gérard Milmeister

vii

Preface

Trained as a computer scientist, I have always been interested in music and musicology. Therefore I took the opportunity offered by PD Dr. Guerino Mazzola to work with his MusicMedia group, then a part of the MultiMedia Laboratory at the Department of Informatics of the University of Zurich, directed by Prof. Dr. Peter Stucki. Thus, first and foremost, thanks go to Guerino Mazzola, who introduced me to mathematical musical theory, most of which I had never heard of before. He gave me the conviction of working at the forefront of music research and taught me the use of modern mathematical methods, such as category theory. He supervised my work with all the enthusiasm and competence one could wish for. I would also like to thank the reviewers Prof. Dr. Bernd Enders of the University of Osnabrück and Prof. Dr. Renato Pajarola of the University of Zurich, who suggested improvements to this thesis. The thesis could not have been accomplished without the backing by Prof. Dr. Pfeifer, whom I like to thank for his willingness to support it as the responsible member of the Faculty of Mathematics and Natural Sciences. Finally, I have to thank the staff of the Department of Informatics for helping me with the tedious administrative tasks that a doctoral student and assistant has to manage. Zurich, November 2006

Gérard Milmeister

ix

Introduction

It is significant that the art and theory of music production and performance have always been connected to the newest developments of the natural and engineering sciences of the time. Indeed, a sophisticated theory of sound and music has been an important part of the earliest mathematics in ancient Greece. The theory of musical intervals set forth by Pythagoras is still a matter of discussion among psychologists of music and theorists alike. On the other hand, since the appearance of digital computers, and the development of computer science as a mathematical and engineering discipline in the late 1940s and early 1950s, music has been among the first applications to appear besides numerical computations on the newly invented machines. At lot has happened since, and there have always been researchers in mathematics who have been trying to apply the newest trends in their disciplines to the explanation of the principles of music, with various degrees of success. Whatever the outcome of each of these developments, the outlook of music as a whole has been changed forever to the open-minded observer. Unfortunately, it is still the case that the intersection of the set of mathematical music researchers and the set of musicologists or music theorists is vanishingly small compared to the combined number of people active in the field. To further the penetration of mathematical music theory into the realm of music production, it is vital to offer a computer-based tool to contemporary composers and music theorists that provides the most advanced ideas from mathematics applied to music. Category theory is the field of mathematics that has crept into almost every mathematical domain and reformulated most basic tenets in a modern language. Computer science is another discipline that has benefited enormously from the exposure to category theory, as has mathematical music theory. It is certainly not exaggerated to assert that the colossal volume The Topos of Music by Guerino Mazzola has brought music theory to a new

xi

xii

Introduction

level by infusing it with advanced and recent ideas from category and topos theory, whence the name. The following work takes the fundamental ideas expounded in that book and describes the design and implementation of a software system, called R UBATO C OMPOSER, that provides the tools to work creatively and analytically with those principles. The software system is both an application development framework, targeted at programmers proficient in mathematics or music theory, or both, and a graphical user interface intended for the composer or the theorist who wants to make use of components created by developers to build an application that embodies his or her musical ideas. The first part presents the concepts and theory. There is neither place nor need for a complete exposition of the theory developed in The Topos of Music. Therefore only those parts that have found their way into the implementation are discussed. The second part is a thorough account of the implementation of the R U BATO C OMPOSER software system. Here the high-level organization as well as some details are covered. This chapter is also of importance to the developer who wants to extend and build on the R UBATO framework. The third part is about the practical aspects of using R UBATO C OMPOSER. A tutorial describes a typical use through a step-by-step tour illustrated with screenshots of the running program. Several uses of the framework by external projects are introduced that exemplify the application developer aspect. The fourth and last part acts as an appendix. The greatest part is taken up by the R UBATO C OMPOSER user’s manual. This manual is intended as a stand-alone reference and also features many details that are not essential to understanding and therefore are not included in the treatment in the main text.

Overview

Part I Concepts and Theory 1

Overview of Music Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2

The Representation of Music . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

3

Architecture of Concepts I: Principles . . . . . . . . . . . . . . . . . . . . . . . 19

4

The Category of Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5

Architecture of Concepts II: Forms and Denotators . . . . . . . . . . . 55

6

Software Components for Computational Theories . . . . . . . . . . . 65

7

Historical Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Part II The Implementation 8

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

9

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

10 Modules and Morphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 11 Forms and Denotators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 12 Tools and Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

xiii

xiv

Overview

13 Rubato Composer GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Part III Rubato Composer in Practice 14 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 15 A Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 16 First Applications in Rubette Construction . . . . . . . . . . . . . . . . . . 167 17 The BigBang Rubette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 18 Creative Analysis of Boulez’s Structures . . . . . . . . . . . . . . . . . . . . . 201 19 Conclusion and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Part IV Appendix 20 User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Contents

Part I Concepts and Theory 1

Overview of Music Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

The Representation of Music . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Types of Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Symbolic Representation of Music . . . . . . . . . . . . . . . . . . . . . . 9 2.2.1 Electronic Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2.2 MIDI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2.3 2.2.4

3

Musical Representation Languages . . . . . . . . . . . . . . 14 Language of General Concepts . . . . . . . . . . . . . . . . . . 18

Architecture of Concepts I: Principles . . . . . . . . . . . . . . . . . . . . . . . 19 3.1

Pure Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Conjunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Disjunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2 3.3

Architecture with Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.1 3.3.2 3.3.3

4

3

19 20 21 21

Macro Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Frequency Modulation . . . . . . . . . . . . . . . . . . . . . . . . . 26 Full Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

The Category of Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1 From Monoids to Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1.1

Monoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

xv

xvi

Contents

4.1.2 4.1.3 4.2

5

4.2.1 4.2.2 4.2.3 4.2.4

Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Functors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Natural Transformations . . . . . . . . . . . . . . . . . . . . . . . Yoneda’s Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41 43 45 48

4.2.5 4.2.6

Limits and Colimits . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Topoi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Denotators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computational Category Theory . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Data Types in Programming Languages . . . . . . . . . . . 5.3.2 The Role of Diagrams . . . . . . . . . . . . . . . . . . . . . . . . .

55 57 58 58 61

Software Components for Computational Theories . . . . . . . . . . . 65 6.1 6.2

7

4.1.4 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Architecture of Concepts II: Forms and Denotators . . . . . . . . . . . 55 5.1 5.2 5.3

6

Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Types of User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Rubato Composer: Computational Theories . . . . . . . . . . . . . 69

Historical Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 presto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 “Classic” R UBATO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Experiments in Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 R UBATO C OMPOSER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

71 71 73 75 76

Part II The Implementation 8

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

9

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 9.1

Overall Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

9.2 9.3

The R UBATO C OMPOSER Universe . . . . . . . . . . . . . . . . . . . . . 83 Java Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

10 Modules and Morphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Contents

10.1

10.2

xvii

Modules and their Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 10.1.1 The Module Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 87 10.1.2 The ModuleElement Interface . . . . . . . . . . . . . . . . . . . . 91 Module Morphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 10.2.1 The ModuleMorphism Interface . . . . . . . . . . . . . . . . . . . 95

11 Forms and Denotators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 Form Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.2 SimpleForm Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105 105 106 107 109

11.2.3 LimitForm and ColimitForm Classes . . . . . . . . . . . . . . 109 11.3

11.4

11.2.4 PowerForm and ListForm Classes . . . . . . . . . . . . . . . . . Denotators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 SimpleDenotator Class . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.2 LimitDenotator Class . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.3 ColimitDenotator Class . . . . . . . . . . . . . . . . . . . . . . . .

110 110 113 114 115

11.3.4 PowerDenotator and ListDenotator Classes . . . . . . . 115 Tools and Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 11.4.1 11.4.2 11.4.3 11.4.4 11.4.5

Construction of Forms and Denotators . . . . . . . . . . . Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Module Mapping and Structural Replacement . . . . . Reforming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Address Changing . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

116 118 119 120 123

11.4.6 List and Set Operations . . . . . . . . . . . . . . . . . . . . . . . . 124 12 Tools and Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 12.1

12.2 12.3 12.4 12.5

Low-Level Mathematical Tools . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.1 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.2 Matrixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Repository and Predefined Universe . . . . . . . . . . . . . . . . . . . .

127 127 128 128

MIDI Sequencer and Synthesizer . . . . . . . . . . . . . . . . . . . . . . . 130 Scheme Interpreter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 XML as File Format for R UBATO C OMPOSER . . . . . . . . . . . . 132

13 Rubato Composer GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 13.1

Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

xviii

Contents

13.2 13.3

The Implementation of Networks . . . . . . . . . . . . . . . . . . . . . . 136 Running a Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

13.4 13.5

Macro Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

13.6

The Plug-In System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Part III Rubato Composer in Practice 14 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 15 A Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 16 First Applications in Rubette Construction . . . . . . . . . . . . . . . . . . 16.1 Rubettes for Macro Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.2 The Wallpaper Rubette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3 The Alteration Rubette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.4 16.5

167 167 170 176

Counterpoint Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Music Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

17 The BigBang Rubette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 17.1

Spontaneous Algorithmic Composition . . . . . . . . . . . . . . . . . 183 17.1.1 Facts about Geometric Composition Strategies . . . . 184

17.2

Gestural Interaction Concept . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.1 Gesture Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.2 Application of Gesture Theory . . . . . . . . . . . . . . . . . . Modular Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17.3

185 185 187 188

17.3.1 View Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

17.4

17.3.2 Note representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.3.3 Basic Functionality and Navigation . . . . . . . . . . . . . . 17.3.4 Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implemented Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

189 193 193 194

17.4.1 Geometrical Transformations . . . . . . . . . . . . . . . . . . . 195 17.4.2 Wallpapers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 17.4.3 Alteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 17.5

The BigBang Rubette in Context . . . . . . . . . . . . . . . . . . . . . . . 199

18 Creative Analysis of Boulez’s Structures . . . . . . . . . . . . . . . . . . . . . 201 18.1

Boulez’s Creative Analysis Revisited . . . . . . . . . . . . . . . . . . . . 201

Contents

18.2 18.3

xix

Ligeti’s Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 A First Creative Analysis of Structure Ia . . . . . . . . . . . . . . . . . 203 18.3.1 Address Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 18.3.2 Primary Parameter Address Changes . . . . . . . . . . . . . 205

18.4 18.5

18.3.3 Secondary Parameter Address Changes . . . . . . . . . . . 18.3.4 The First Creative Analysis . . . . . . . . . . . . . . . . . . . . . Implementing Creative Analysis in R UBATO C OMPOSER . . 18.4.1 The System of Boulettes . . . . . . . . . . . . . . . . . . . . . . . A Second More Creative Analysis and Reconstruction . . . . .

206 208 209 211 213

18.5.1 The Conceptual Extensions . . . . . . . . . . . . . . . . . . . . 214 18.5.2 The BigBang Rubette . . . . . . . . . . . . . . . . . . . . . . . . . . 219 18.5.3 A Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 19 Conclusion and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 19.1 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 19.2 19.3

Things To Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Ideas for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Part IV Appendix 20 User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 20.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

20.2

Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2.1 R UBATO C OMPOSER’s World of Objects . . . . . . . . . . 20.2.2 Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2.3 Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

233 233 234 236

20.2.4 Macro Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.2.5 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using R UBATO C OMPOSER . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3.1 Starting up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3.2 General Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3.3 Main Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

237 238 238 238 238 239

20.3

20.3.4 Main Menu and Toolbar . . . . . . . . . . . . . . . . . . . . . . . 240 20.3.5 Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 20.3.6 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 20.3.7 Scheme Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 20.3.8 Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

xx

Contents

20.4

20.3.9 Recurring User Interface Elements . . . . . . . . . . . . . . . 253 Core Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 20.4.1 Rubette Description Schema . . . . . . . . . . . . . . . . . . . . 257 20.4.2 List of Core Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . 258

20.5 20.6

20.7

Built-in Non-Core Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . Writing Rubettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.6.1 Developing with the R UBATO Framework . . . . . . . . 20.6.2 Rubette Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rubette Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

269 271 271 273 279

20.7.1 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 20.7.2 The LatchRubette class . . . . . . . . . . . . . . . . . . . . . . . . 279 20.8 20.9

20.7.3 Packaging a Plug-In . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Types of Module Morphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 The Rubette Java Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

20.10 Example LatchRubette class . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 20.11 Keyboard Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 20.12 Rubato Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

Chapter 1

Overview of Music Theories

The title of this chapter states Music Theories in the plural and not the singular Music Theory or Theory of Music. Probably no single theory will ever cover the enormous richness of music in the world, although there have been promising endeavors to extract certain principles common to most. This is a boon and a bane at the same time: a bane, because all attempts at reducing music to a single set of rules have failed so far to satisfy adherents of universally valid systems; a boon, since it relieves music lovers from the potential danger of the dreaded reduction of music to a mere system of rules, devoid of all mysticism cherished by many. Even putting aside all aspects of the subjective, this state of affairs promises new vistas for future musical activity, since the reservoir of new and modified theories for generating new music seems inexhaustible. The role of theory in the analysis as well as the production of music has however not been the same at all times in the history of music. To better understand how theories and what theories fit into contemporary music, it is necessary to shed some light on the last few hundred years during which music theories have been perceived as such by musicians. It is necessary to restrict the discussion to theories about Western music. Of course theories of music from other cultural environments such as India exist and its literature is very broad indeed, but including it would by far break the tight limits set by this text. Certainly the most famous theory linking music and natural science known from Antiquity is the description of music intervals using of chord ratios by Pythagoras. However from the post-Antiquity treatise by Boethius through the Middle Ages until 19th century, music theories were rather enshrouded in the philosophical (or theological) frameworks of the day. Those parts relevant to the analysis and composition of music did not much more than describe the state of the art as it was a decade before the publication of the treatises.

3

4

1 Overview of Music Theories

Thus the Gradus ad Parnassum by Johann Joseph Fux published in 1725 [25] laid out the rules of counterpoint, even if those rules did not accurately describe the practice of most advanced contemporary composers, such as Johann Sebastian Bach. This work, well used even into the present, also illustrates features common to similar theoretical treatises: they delivered a set of recipes justified by empirical or psychological phenomena, thus providing weak theories, even in comparison to the studies pursued by Greek mathematicians. The schema behind the history of these theories can be summarized in the phrase: Theory follows Composition. There is hardly any theoretical proposal that would provide the composer with new ways and guidelines to drive his work. Therefore, it is mostly the case that “avant-garde” composers, such as Beethoven in his late string quartets, produce the material that musicologists try to put into theory after the event. The beginning of the 20th century saw new developments of music theorists trying to give the various aspects of music, such as harmonic and formal structure, a more exact and scientific character. This is certainly connected to the rise of abstract mathematics in the 19th century, which relies on new rigorous types of notation, as well as the use of diagrammatic methods. Such theories include the Funktionstheorie by Hugo Riemann and the structural theories by Heinrich Schenker. During the second half of the 20th century, finally, modern mathematics found its way into theoretical considerations of music [5]. Combinatorics is seen as playing a major role throughout the history of musical composition. Dodecaphonic composition and its generalization to serial techniques provide the telling example of this trend. Driven by the popularity of structuralist views, generative theories as they were first proposed by Noam Chomsky for the formalization of language have been applied to musical form as well [38]. This has been a breakthrough in that, for the first time, theories have been devised not least with a view to provide new means for composition. Composers start complementing their musical works by theoretical studies explicating the methods that led to their construction, for example Musiques formelles (1962) by the Greek composer Iannis Xenakis [74]. Several other eminent composers, such as Arnold Schoenberg, Ferruccio Busoni or Paul Hindemith, have published theoretical works that give insight into their works. It has eventually become possible to state that: Composition follows Theory. In the last decades, the role of mathematics in music has been on a steady increase. The invention of digital computers in the late 1940s was soon followed by the creation of music with the help of computers (for example the famous Illiac Suite from 1957 by Hiller and Isaacson [31]) and along with it the development of mathematical theories quite specific to musical applications. It was Milton Babbitt, Hillers teacher of composition at

1 Overview of Music Theories

5

Princeton University, who promoted the use of mathematics, its terminology and, thus, scientific precision in studying theories of music. A music theory should be objective and be stated as a body of definitions, axioms, and theorems, just as in the case of other mathematical investigations. An analogy can be made to the relation of physics to engineering: mathematical music theories play the role of physics, and musical composition the role of engineering. It is obvious that without the theoretical work provided by physics, engineering would be impossible, or, even worse, engineering with a disregard of physics would result in bad work. Cautiously applying the analogy, we would say that contemporary composition without regard to mathematics results in bad music. . . . Two types of composition by electronic means have been predominant. One type is exemplified by the computer programs Project 1 (1964) and Project 2 (1966) developed by the composer Gottfried Michael Koenig. They implement composition algorithms and operate on the level of musical structure. On the mathematical side, aleatoric and combinatorial methods play an important role [36]. Another line of development involving the use of computers for musical composition concerns the production of sound by means of digital processing. In this case music is controlled on a very low level, effectively down to the sound wave. Many parameters, which before were constrained by acoustic instruments, become available. These parameters allow qualities of sound never heard before and thus fruitful for new principles of composition. The pioneering work has flown into the MUSIC-N family of synthesis languages, the prototype program MUSIC having been written by Max Mathews in 1957 at Bell Labs [42]. The most recent descendant in the family, widely used today on many computer platforms, is Csound [10]. The import from mathematics to musical theories comes from all domains of abstract algebra, such as group theory. More recently, category theory has provided comprehensive tools for both analysis and composition. Mathematical studies have been conducted for fields such as counterpoint or the theory of the string quartet. A main event in this direction has been the publication of The Topos of Music by Guerino Mazzola [47], which uses extensive knowledge from category theory to provide theoretical treatments of a broad selection of branches in musical theory. The mathematical approach to music analysis and composition also allows new means for joining the musical past to the future. In accordance with the idea of “analyse créatrice (creative analysis)” put forward by Pierre Boulez [49], Guerino Mazzola undertook the ambitious project of a new composition based on the analysis of the first movement of Beethoven’s sonata op. 106 “Hammerklavier” [43]. The process involved the discovery of mathematical principles buried in the piano sonata and the extraction of significant parameters (“analysis”). The result is an analytical model. After changing the analytical coordinates of this model using mathematical

6

1 Overview of Music Theories

transformations, the model was remapped to produce a new musical work that fundamentally based on, but different from, Beethoven’s composition (“creative”). The process itself is not entirely new, as similar principles have been employed in a vague manner throughout the history of music. But the mathematical approach, the exactness of the procedure, and the awareness of its happening are aspects of a modern phenomenon and open new vistas for future experimentation. The result of the project is the piano sonata “L’Essence du Bleu” which has also been recorded [48]. The sonata has been constructed completely by hand, using the traditional tools available to the composer and mathematician, such as pencil and graph paper. To manage this kind of composition, computer software implementations of the methods used would provide an ideal tool. The development of such software is of paramount importance for future experimentation in this direction. The R UBATO C OMPOSER presented in this thesis is the result of this insight. It is the attempt to bring the mathematical and computational tools to a level that a composer is comfortable with. The mathematics it is based on is category theory, specifically the category of modules. Module theory is very important in mathematical music theory, since it allows the representation of the common objects in music, such as notes and chords, and the description of the usual transformations in the theory of counterpoint and composition, such as transposition, inversion and retrograde, among others. The theory of modules contains the complete theory of linear algebra, and thus Euclidean spaces, which ensures the availability of general geometric manipulations at the hands of the composer. Categorical considerations led to the development of a very general data model, called denotators, which is used throughout R UBATO C OM POSER for the transport of concepts and information.

Chapter 2

The Representation of Music

Theories, whether in mathematics or in music, presuppose a world of objects that we can be theorize upon. In mathematics the notion of objects is quite clear, from obvious number domains to high dimensional manifolds. Musical objects, if we may call them thus, are much more elusive. Identifying the musical objects would be nothing less than determining what music ultimately is. So far, no one really knows what happens in the brain when listening to music, what symbolic structures are generated and and how they are processed. The psychology of music does make some inroads, but without effective results useful to the theorizing we have in mind. We are left with the only possibility of grasping musical objects, namely proposing more or less formal schemes for the representation of music, preferably in a mathematical context. Such a representation should allow the manipulation required by the various methods of musical composition as well as analysis. Certainly, no one format will be able to provide all of this, and, in effect, the formats developed during the centuries of written music targeted overlapping, but different, uses.

2.1 Types of Representation To bring a little order into the various types of representation, it is helpful to borrow from the ontology and semiotics of music. The cube shown in figure 2.1 is an abstract model of locating music within our knowledge system. It considers multiple aspects, which may or may not enter in a specific type of representation. The levels of reality discriminate between the layers where music takes place. The physical and mental levels will be of primary concern in this context. The communication coordinate of the cube refines these layers and relates the musical work to its creator and the recipient.

7

8

2 The Representation of Music

This coordinate is usually not made explicit in the representations that we discuss below. The third coordinate semiosis deals with the semiotics of music. It provides the theory of how expressions (significants) are related to the meaning (significate) of a musical work. The act of relating significants to significates is called signification. The process of signification is the attempt of extracting the content from a representation. An in-depth discussion of musical ontology can be found in [47].

Se io

te ca ifi n tio ign  S ca i f   ni  g  nt i  S  ca ifi gn  sis  Si    

m

  Mental                Reality Psychological                 Physical Creator |

Listener Work {z } Communication

Fig. 2.1: Topographic cube of musical ontology, after [47].

The semiotic status of the various types of music representation is very difficult to assess, since we have to deal with two types of information. On the one hand, there is explicit information (or extension). It is the kind of information that can be controlled by rules, many of which have even been cast into formal clothes, for example counterpoint rules according to Fux [25] or harmonic theories such as Schoenberg’s [63]. On the other hand, there is implicit knowledge (or intension). This knowledge cannot be formalized. It depends on environment, personal preferences, historical background, etc.

2.2 Symbolic Representation of Music

9

[2]. The consequence of this is that, whatever representation we use, the act of signification, and therefore the relation of expressions to meanings, varies from individual to individual, thus making impossible the “ultimate”, completely unambiguous representation. It has already been hinted that the levels of physical and mental realities play an important role in the representation of music. A physical representation aims to reproduce music in terms of sound, often as low-level as air pressure changes as in the physical theory of sound. The theories of sound production as in the acoustics of instruments also belongs to this class. Common formats of representation include analog tape recordings or digital audio file formats such as MP3. It is the quality of sound rather than of music that is transported in this kind of representation. It has been the means of old to preserve musical performances and to archive oral, nonwritten music common in ethnomusicologist studies. More recently, audio has been supplemented by video, thus capturing an important part of performance, namely the gestures involved in making music. However important the interest in the other types of representation is, we are going to concentrate on symbolic representations, which try to capture the mental aspects.

2.2 Symbolic Representation of Music The printed score is the most famous of all representations. Developed from a gestural language, the neumatic notation used in manuscripts of early church music (see figure 2.2 for an example) [19], it was extended and refined over ten centuries to the point of integrating a wealth of information for the musical performer. In its original form it reached its peak during the first decade of the 20th century, when composers such as Gustav Mahler determined every aspect of the performance of their works using extremely fine-grained instructions affecting even single notes. Some structure is indicated and beginning in the Romantic era, content is hinted at by often poetic directions (for example, très doux et pur in figure 2.3 showing the beginning of Scriabin’s 10th Sonata). Other printed representations targeted at specific instruments and types of music have been devised, such as the various tablatures for lute and keyboard music, very common from the 15th to the 18th centuries [23]. Along with common notation, many composers from the second half of 20th century devised notations of their own to communicate the intent of their works or even specific to a single work [17]. In many cases these graphic “scores” have been considered as works of art in themselves.

10

2 The Representation of Music

Fig. 2.2: Neumatic signs describing the melodic motion above the lines of text (St. Gall, 10th century).

Moderato

    
 





 







169 

 p
 
 


  9   16 très doux et pur


 



 

 

Fig. 2.3: The first four bars of Alexander Scriabin’s Sonata No. 10, Op. 70, in traditional notation.

2.2.1 Electronic Scores Early in the development of computers, handling of music has already been a hot topic. Thus it comes as no surprise that many efforts have been made to bring musical scores into electronic form. One aim has been, of course, to follow the way of digital book production and make the production of musical scores entirely digital. A whole industry is devoted to the development of so-called music notation software, and with it, the invention of electronic score representations. Two examples of quite different approaches shall illustrate this idea. The popularity of XML has had its effect on music representation too, and we now have a large number of XML based markup languages for music, such as SMDL (Standard Music Description Language) [65], NIFF XML (Notation Interchange File Format), and many more. One particular scheme is MusicXML [59], which has become the lingua franca for the interchange of electronic scores between many notation pro-

2.2 Symbolic Representation of Music

11

grams, e.g., Finale and Sibelius. As is often the case with XML, even a small musical fragment generates a rather large instance of MusicXML. Therefore only part of the music from figure 2.3 as coded in MusicXML is shown in figure 2.4.

<score-partwise> <part-list> <score-part id="right"> <part-name>Right <score-part id="left"> <part-name>Left <part id="right"> <measure number="1"> 960 9 16 <sign>G 2 <note> 480 1 eighth <note> <step>D 5 240 1 16th <note> <step>A 1 4 sharp 1440 1 quarter <dot/>

<part id="left"> <measure number="1"> 960 9 16 <sign>F 4 <note> <step>F 1 4 sharp <notations> 1440 1 quarter <dot/> <note> <step>F 1 4 <notations> 720 1 eighth <dot/>

Fig. 2.4: MusicXML representation of the fragment figure 2.3.

12

2 The Representation of Music

These XML based formats are mainly intended for interchange and are not meant to allow authoring on the source itself. In contrast, there is a school of typography that favors textual representation of typesetting instructions that are compiled into high quality renderings. The main representative is Donald Knuth’s TEX, which allows for score typesetting using the add-on package MusixTEX. Modern descendants are ABC [70] and the powerful score typesetting software Lilypond [28]. The score fragment in figure 2.3 is effectively the result of compiling the code in figure 2.5. Observe how Lilypond tries to find out a good layout without the author providing precise instructions. The implementation of such layout software requires advanced research and techniques in computer science [27] and the construction of a fully functional and complete software application is a major undertaking. \score { \new PianoStaff > } } \new Staff { \clef bass { ges’4.( ges’8.) es’4. ~ es’8. ~ es’4. ~ es’8. ~ es’4. ~ es’8. } } >> } Fig. 2.5: The Lilypond source code used to generate figure 2.3.

2.2 Symbolic Representation of Music

13

All of these types of representation, different as they may be, serve the main purpose of the visual layout of traditional printed scores. Machine-based manipulation and feature extraction for analysis would be hard, if not impossible. This is to be expected, since they adopt all of the idiosyncrasies of human music notation accrued over the centuries. Some attempt to address the strictly musical aspect and allow the export of a MIDI version of their data (Lilypond for example), but for a more deeply musical understanding, other representations are needed.

2.2.2 MIDI The Musical Instrument Digital Interface (MIDI) was first proposed in 1981 as a means for driving electronic instruments, such as synthesizers. The standard [55] comprises three parts: 1. a specification for a serial cable and plug that establishes the physical connection of electronic devices; 2. a protocol that defines the possible events that are sent over the cable in real-time and are to be interpreted by the receiving instrument (where each instrument is assigned a channel number); 3. a file format that integrates such events, provides them with the times of their occurrence, and is enriched with so-called meta events for changing tempi and selecting programs (a means for choosing among several sound types provided by a device). The most important type of event is Note on/off. The Note on event signifies the pressing of a key. This event requires as parameters the channel (which is the address of the receiving device), the pitch (where the number 60 is defined to be c4 ), and the velocity, which essentially corresponds to the loudness. The Note off event analogously signifies the release of a key. Figure 2.6 lists the MIDI events generated from a simple rendering of the fragment in figure 2.3. MIDI does show its age though, in particular in its hardware limitations. Its affinity to keyboard instruments makes it cumbersome to control musical events that are not based on semitone scales, the workaround being to make use of meta events such as pitch bending. Nevertheless, MIDI can be put to good use when it comes to create fragments of musical performance, since the file format is probably the most common of all among music software and therefore is a sure value for interchanging data. We will come back later to this use when discussing the R UBATO C OMPOSER software.

14

2 The Representation of Music Time Event

Channel

Pitch Velocity

0 0 0 240 360 360 1080 1080 1320 1320 1440 1440 2160 2160 2280 2280 2400 2400 2520 2520 2760 2880 2880 2880 2880 3120 3120 3120 4320 4320

Meta Event 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Meta Event MSPQ = 1666666 program = 0 66 100 74 100 74 127 70 100 66 127 63 100 70 127 69 100 69 127 66 100 66 127 70 100 70 127 72 100 72 127 69 100 69 127 68 100 84 100 84 127 72 100 86 100 78 100 72 127 86 127 78 127 68 127 63 127

Tempo Program Note on Note on Note off Note on Note off Note on Note off Note on Note off Note on Note off Note on Note off Note on Note off Note on Note off Note on Note on Note off Note on Note on Note on Note off Note off Note off Note off Note off

Fig. 2.6: A sequence of MIDI-events. In this example the unit of time is defined to be 1/480 of a MIDI quarter note. The Tempo event specifies the duration of a MIDI quarter note as MSPQ, which means milliseconds per quarter note.

2.2.3 Musical Representation Languages Musical notation and instrument control languages are ultimately surface representation and not conducive to analytical research. They require a musical mind to make sense of the analytical signification looming behind the music they describe. On the other end of the spectrum are the languages explicitly designed for the purpose of unfolding analytical structure. These languages are invented in the context of the development of software for musical composition and analysis. In the world of computer-based manipulation of symbolic data, the programming language LISP assumes a prominent place. Predominantly used

2.2 Symbolic Representation of Music

15

in the research of artificial intelligence, LISP and, in particular, CLOS (Common Lisp Object System) is also the basis of some of the best-known computer-based composition systems. Two particular implementations are Common Music and OpenMusic.

Common Music Common Music or CM [67] is an object-oriented LISP environment for music composition. It represents music as high-level objects, and through programs and algorithms designed by the user transforms them into events that rendered through MIDI. CM is very powerful, but it requires a good knowledge of programming, in particular of LISP, to even get started. It belongs therefore to the realm of programmer-musicians. Figure 2.7 shows an example of a seq object, which is a list of subobjects, in this case a time ordered sequence of eighty MIDI notes with pitches randomly chosen from the specified collection. (new seq :name ’pulse :subobjects (loop with d = .1 for i below 80 collect (new midi :time (* i d) :keynum (pickl ’(c4 d4 ef4 f4 g4 af4 bf4 c5)) :duration d :amplitude (interp (mod i 8) 0 .25 7 .75)))) Fig. 2.7: An example for the definition of a seq object in Common Music.

OpenMusic The more recent OpenMusic software has been developed at the IRCAM in Paris by Gérard Assayag and Carlos Agon [3]. It is also built on top of LISP, but in addition features a graphical user interface (figure 2.8) that separates the composer from the low-level entrails to a certain extent. The interface follows the familiar concept of visual programming by connecting components in networks (patches). OpenMusic has already met with considerable success, as testified by the contributors to [4]. OpenMusic exports the concepts of object-orientation to the user oriented interface. Classes can be defined and instances of classes can be created in a way that is inherited from the underlying CLOS. There are classes

16

2 The Representation of Music

Fig. 2.8: A “patch” in OpenMusic.

for notes, chords and sequences, which are combined and manipulated using the methods provided by the system or programmed by the composer. Thus, while offering appealing visual interaction, OpenMusic requires programming skills do unlock its considerable power.

Humdrum David Huron’s Humdrum Toolkit [32] takes a quite different path. Whereas OpenMusic is primarily a visual environment for composing, Humdrum targets musical analysis and takes advantage of all the features that a UNIX environment offers. This means, first of all, that Humdrum is commandline-based. In fact, the toolkit consists of a number of shell scripts that operate on text-based representations of music. Therefore the standard UNIX text utilities, such as grep, sed, awk, or sort, can be put to good use.

2.2 Symbolic Representation of Music

17

Humdrum does not rely on one unique format of representation, but allows the use of several different formats, each tailored to one specific use. One such format is the so-called **kern representation. This correspond roughly to common practice music notation, with pitches encoded as equally-tempered values at concert pitch (figure 2.9). **kern *staff3 *clefF4 *k[] *M9/16 =14.g-\ . . 8.g-\ =2 [4.e-\ . . 8.e-\ =3 4.e-\ . . . . 8.e-\ . =4 4.e-\ 8.e-]\ *-

**kern *staff2 *clefG2 *k[] *M9/16 =18r\ (16dd\ [4.b-\ . =2 8b-]\ 16b--\ 4.g-)/ . =3 16b-\ 16cc\ 16b--\ [4.a-\ . . . =4 4.a-\ 8.a-]\ *-

**kern *staff1 *clefG2 *k[] *M9/16 =1. . . . =2 . . . . =3 8.r . . 8r (16eee/ 8cc 8ff# 8ddd) 16r =4 . . *-

Fig. 2.9: The first four bars of Sonata No. 10 in **kern notation. The score is subdivided into bars, for example =3 starts the third bar. A single note is specified using its duration and its pitch, for example 4.b- denotes a dotted quarter b flat.

Other representations include **pc for pitch classes instead of absolute pitches, or **harm which encodes Western functional harmony. Many commands have the sole purpose of converting between different representations, others extract information such as the most probable key of the musical fragment or possible patterns that reoccur periodically. The command key applied to our example gives the following result: Estimated key: E-flat minor

(r=0.8370)

confidence:

68.3%

All in all, Humdrum provides some unique features, but expects the user to be familiar with the UNIX way of doing things in general and text processing in particular.

18

2 The Representation of Music

2.2.4 Language of General Concepts The foregoing sections presented a variety of representations, from the very specific, such as notation and MIDI format, to the more general, such as OpenMusic’s classes and objects. It is certainly preferable to design a representation format that allows for the most general variations. The idea is to use a language of general concepts, based on hierarchical abstract structures. This is a generic approach, but it is strongly influenced by structuralist thought. In music, Schenkerian analysis is an example of such an approach. More recently, the principles of Chomsky’s notion of human language grammar have been transferred to music under the title of generative theory of tonal music (GTTM) by the composer Fred Lerdahl and the linguist Ray Jackendoff [38]. This theory has found great favor with a lot of music theorists. Unfortunately, this popularity has masked grave limitations of GTTM. Its rather dubious assumptions, its reliance on unequivocal hierarchies, and its normative statements do not adequately reflect the realities of music [46]. It is at least necessary to consider multiple hierarchies in order to respect the inherent ambiguities present in most types of music. These ambiguities must not be resolved, otherwise the essential richness of music responsible for its intellectual and emotional power is irretrievably lost. In addition to strict hierarchies, it is important to allow circular structures. This permits arbitrarily deep hierarchies, high complexity on every level, and even fractal-like self-similarity relations. In the next chapters, an architecture of concepts is presented and embedded in a complete mathematical theory.

Chapter 3

Architecture of Concepts I: Principles

An architecture of concepts deals with question: “How is a concept built?” To answer the question, the architecture provides the means of construction to build a concept from ground up. We are going to present two version of such an architecture, the first one is pure and works without primitive (or undefined) concepts. The second one supplements the pure architecture with atomic concepts. The pure architecture is self-sufficient with minimal means and is of such generality as to provide the foundations for virtually all of mathematics. As a matter of fact, it is required to even understand what the most basic of all mathematics, the set, is. This approach has been chosen in [52] to build a comprehensive environment of mathematics suitable for computer scientists.

3.1 Pure Architecture The pure architecture of concepts builds on the following principles: 1. A concept has a name. This is essentially different from traditional mathematics, where the building blocks are generally nameless. A mathematical space, such as the Euclidean plane R2 is defined by its construction, but there is no way to distinguish two spaces of exactly the same form, but with different intent. This is in fact a manifestation of the dichotomy of conceptual extension and intension. The mathematical space solely considers extension and disregards intension. There is a difference between a Euclidean space describing time versus position and one describing frequency versus amplitude, although the extension in both cases is R2 . In the language of concepts the first space might be given the name “Route” and the second space the name “Spectrum”. There can be no automatic conversion between these spaces. After all, what would be the

19

20

3 Architecture of Concepts I: Principles

interpretation of a point (x, y) of “Spectrum” as a point in “Route”? Any transfer from the one to the other must be explicitly defined. Thus, the name fixes a concept to a particular intent and discriminates it from other concepts, whatever explicit formal equality there may otherwise be. 2. A concept is built from other concepts, called its components. This is the basic principle of construction, without which no structural relationship could exist. 3. There are three methods of how to build concepts from other concepts: • Conceptual Selection: A selection concept has one component. • Conceptual Conjunction: A conjunctive concept has one or more components. • Conceptual Disjunction: A disjunction concept has two or more components. 4. A concept has instances with the following properties: • An instance has a value. • An instance has a name. The role of the name of an instance is analogous to its role for concepts.

3.1.1 Selection Selection provides the method for building collections. As an example consider the concept “SoftIce” which has the single component “Flavor”. The “Flavor” concept has several instances, say “Strawberry”, “Banana”, “Chocolate, “Vanilla”, etc. How exactly “Flavor” is built itself is of no interest here. How is an instance, say, “mySoftIce”, of “SoftIce” construed? Imagine we enter a soft ice cream parlor and order “mySoftIce”. We may point to the boxes containing the various flavors and say “this one” and/or “that one”. This is exactly the way an instance is created: “mySoftIce” is a selection of a number of instances of “Flavor”. An intuitive example from music is the concept of a score. In a naive view, an instance of the concept “Score” is a selection of some instances of the concept “Note”. A special notation for denoting concepts makes it easier to write about them. The notation for a conceptual selection is the following (here we use the musical example): Score:.Power(Note)

3.1 Pure Architecture

21

The term Power for selection introduced here will be explained in chapter 4 which deals with the mathematical theory. The terminology further uses Limit for conjunction and Colimit for disjunction. Parallel to the notation for concepts a notation for instances is introduced. An instance of “Score”, say “SimpleMelody”, made from instances “FirstNote”, “SecondNote”, “LastNote” of the concept “Note” is denoted as follows: SimpleMelody:@Score({FirstNote, SecondNote, LastNote})

or, leaving out the braces: SimpleMelody:@Score(FirstNote, SecondNote, LastNote)

It is important to observe that there is no order implied in the succession of the arguments.

3.1.2 Conjunction A conceptual conjunction determines a concept based on a fixed number of components. Paradigmatic is the concept of Note which is constructed from the concepts of Onset, Pitch, Duration, and Loudness. Here again we assume that the component concepts have already been suitably defined. This example is written as follows: Note:.Limit(Onset, Pitch, Duration, Loudness)

An instance of Note, say myNote, is determined whenever we know its onset and pitch and duration and loudness: myNote:@Note(myOnset, myPitch, myDuration, myLoudness)

3.1.3 Disjunction For a conceptual disjunction a sequence of at least two components needs to be given. An instance of a disjunction is defined whenever exactly one of these components is defined. In music, a note can be generalized to include rests, which are only determined by their onsets and durations: Rest:.Limit(Onset, Duration)

22

3 Architecture of Concepts I: Principles

A general note GeneralNote, then, is the disjunction of Rest and Note, and an instance of GeneralNote, say myGeneralNote, is determined whenever we know either the note or the rest. In this example, myGeneralNote is a rest: myGeneralNote:@GeneralNote(myRest)

These tools are sufficient to build set theory as in [52]. To get things going, the empty set is defined as the instance of the concept Set:.Power(Set) with an empty collection, i.e., emptySet:@Set(). However, the entire construction is long-winded, and it takes a significant effort to even construct the integers. Therefore we assume that we have a collection of primitive concepts available, such as various classes of numbers, strings, etc. This non-pure architecture is the basis of the further development in this work.

3.2 Architecture with Primitives In the previous section some concepts, such as Onset, have been left undefined. Onset is assumed to be the space of continuous time, modeled by the real numbers R. Such a primitive concept is atomic, i.e., it is no longer further analysed in terms of the architecture of concepts. Their properties come from various mathematical theories, such as module or field theory, assumed to be available to us. The notation uses the type Simple to describe such primitive concepts: Onset:.Simple(R)

An instance of Onset is a real number, for example 5.4, where the unit has been arbitrarily set to be the duration of a quarter note: myOnset:@Onset(5.4)

With the addition of simple concepts, we also have the practical tools available to assemble the various representations useful for handling different aspects of music. Beside the textual notation that has been introduced above, a tree-like twodimensional style for visualizing concept hierarchies is helpful for grasping conceptual structures. For each of the four principles, there is a diagrammatic schema:

3.2 Architecture with Primitives

23

Simple A concept of type Simple, such as Name:.Simple(R2 ), is drawn as: Name R2

Limit A concept of type Limit, such as Name:.Limit(Name1 , Name2 , . . . Namen ), is drawn as: Name Q Name1

Name2

···

Namen

The upper case Greek letter π is a reminder for “product” which is the mathematical name for this construction.

Colimit A concept of type Colimit, such as Name:.Colimit(Name1 , Name2 ,. . . Name3 ), is drawn as: Name ` Name1

Name2

···

Namen

The inverted upper case Greek letter π is a reminder for “coproduct” which is the mathematical name for this construction.

24

3 Architecture of Concepts I: Principles

Power A concept of type Power, such as Name:.Power(Name1 ), is drawn as: Name {} Name1 The symbol {} serves as a reminder for notation used for sets in mathematics. With these rules, the simple score concept can be graphically represented as in figure 3.1. Here an alternative, but equivalent, style of drawing the links has been used. Score {} Note Q Onset

Pitch

R

Q

Loudness Duration Z

R

Voice Z

Fig. 3.1: The conceptual representation of a score.

3.3 Examples In the rest of the chapter, some examples are presented to illustrate how the intuitive principles outlined above can be used to organize common concepts from the realm of music.

3.3 Examples

25

3.3.1 Macro Notes The simple score concept shown in figure 3.1 considers a piece of music only as a bag of notes, without any internal structure. Schenkerian analysis, however, imposes hierarchical structures, from the “Urlinie” to phrases and motifs, down to ornaments, such as trills [38]. A simplified realization of this approach considers macro notes, which consist of a traditional note (the anchor note) and an attached bag of satellite macro notes. In this way arbitrarily deep hierarchies can be built. The graphical representation of a macro score consisting of macro notes (called nodes here) is shown in figure 3.2.

MacroScore {} Node Q Note Q Onset

Pitch

R

Q

Loudness Duration Z

R

Voice Z

Fig. 3.2: The conceptual representation of a macro score.

One important feature to notice in this figure is the presence of circularity. This circularity is in fact the guarantee for hierarchies of arbitrary depth. A particular instance of the MacroScore concept is, however, non-circular. The recursion is ended by specifying a node with a note and an empty MacroScore set at the lowest level. A simple example of such an instance is figure 3.3, where one Node has as anchor the first note d5 and and as satellites (MacroScore again) the notes of the trill (which are again nodes, with empty sets of satellite notes). The second Node has as anchor the dotted half e5 and an empty set of satellite notes.

26

3 Architecture of Concepts I: Principles

Fig. 3.3: Example of a macro score.

The subject of macro objects will be taken up again in section 16.1 as a powerful application of R UBATO C OMPOSER.

3.3.2 Frequency Modulation Another example, still from music, but this time related to the generation of sound, is frequency modulation (FM). Developed my John Chowning in 1973 [18], FM is a method for synthesizing sound by chaining sine generators. A single sine generator is of the form: A sin(2π · F · t + ϕ) where A is the amplitude, F the frequency, ϕ the phase shift and t the time parameter. Fourier synthesis consists of taking some such simple generators: X Ai sin(2π · Fi · t + ϕi ). i

The term 2πFt+ϕ is called the support of a particular generator. A first order FM object is created by adding another simple generator to the support: A sin(2π · F · t + ϕ + B sin(2π · G · t)). By iterating this procedure, higher-order FM objects can be built. This way of recursively defining objects suggests a circular concept similar to the one for macro notes from subsection 3.3.1. In fact, it is a further example of a macro object and already shows a common principle. The FMObject concept is illustrated in figure 3.4. Observe the exact correspondence to the MacroScore concept in figure 3.2.

3.3 Examples

27

FMObject {} Node Q Support Q A

F

Ph

R

R

R

Fig. 3.4: The conceptual representation of frequency modulation. Here the concepts A, F and Ph represent the amplitude, frequency and phase, respectively. The concept FMObject is a defined as a set, and represents the Fourier sum of nodes (no order needs to be implied, since addition is commutative and associative.

Figure 3.5 shows an example for an FM object that follows this definition. Expanded into mathematical notation, it represents the function 1 1 4 1 2 sin(2π·440·t+ π+ sin(2π·620·t))+ sin(2π·860·t− π+ sin(2π·120·t)). 3 2 5 2 3 The emergence of similar principles in two different kinds of objects hints at the possibility of factorizing common operations and manipulations. It opens the way for questions such as: What happens, if a certain, meaningful operation for a MacroScore object is applied to an FMObject? New insights into structure may turn up by the simple recognition of analogies between disparate concepts.

3.3.3 Full Score The simple realization of a score concept as shown in figure 3.1 is ideal for discussing the principles exposed in the following chapters. Its simplicity

28

3 Architecture of Concepts I: Principles fm-object1 {•, •} node1 Q

support1 Q A1

node3 Q fm-object2 {•}

F1 Ph1

1 440

support3 Q

node2 Q

1 3π

support2 Q A2 1 2

fm-object3

A3

F3 Ph3

4 5

860 − 21 π

fm-object4 {•} node4 Q support4 Q

{}

F1 Ph1

A4 2 3

620 0

fm-object5 {}

F4 Ph4 120 0

Fig. 3.5: A instance of the concept from figure 3.4. The tree-like drawing of instances is analogous to the drawing of a concept structure. The main difference concerns sets, where the number of elements is indicated by dots, and the elements themselves branch off from the set object.

is also the reason that current applications in R UBATO C OMPOSER use it as a basis. However, it only considers simple notes and does not deal with the many other parts relevant to music from the real world, such as key signatures, tempo changes, bar lines, and other instructions for articulation. A much more fully featured schema for musical scores has been devised based on a design in [56]. It is shown in figure 3.6. Its complexity, however, prohibits its use for the exemplification of fundamental principles. On the other hand, whenever the full power of musical representation is required, the need for such a full featured score schema is obvious. ❦ The next chapter deals with the embedding of the architecture of concepts in a mathematical framework. With the theory in place, the discussion of concepts continues in chapter 5 by introducing forms and denotators, which are the mathematical realizations of the notions presented in this chapter.

Title

{}

TimeSignature

Q

{}

KeySignature

Q

Q

Z-String

SimpleNote

Z-String

R

R

Z-String

Z-String R

Q

R-String

R

Q

Z-String

Z-String

Voice Onset Pitch Loudness Duration Accidental Articulation Fingering

Q

Q

Pedal

RelativeDynamic Q

`

Dynamic

{}

Dynamics

Q

Z-String

AbsoluteDynamic

R

R

Z-String

Q

SimpleRest

Z-String

R

R

Q

Rest

Z-String

Z-String

Z-String

R

R

Z-String

SimpleNote SatelliteSymbol GeneralNote Voice Onset Duration SimpleRest SatelliteSymbol GeneralNotes

Q

Note

Z-String

Voice Onset AbsoluteDynamicSymbol Voice Onset Duration RelativeDynamicSymbol

Z-String R

GeneralNotes Voice Onset Duration PedalSymbol

Q

Slur

`

GroupArticulation

{}

GroupArticulations

Fig. 3.6: The conceptual representation of a fully featured score. In order not to overload the picture, some links are not shown, but are instead indicated by a small arrow. Occurrences of concepts with the same name always denote the same concept.

R

Voice Onset Duration RelativeTempoSymbol

RelativeTempo

`

GeneralNotes

`

Z-String

R

{}

R

R

GeneralNote

Z-String

Z-String

{}

Tempi

R

Q

Q BarLine Duration

Repetition

BarLine

Onset

{}

{}

Voice

Repetitions

BarLines

Tempo

Z-String

Voice Onset ClefSymbol

Q

R

Z2

Q

Clef

{}

Clefs

AbsoluteTempo

Z-String

Z12 3

Voice Onset KeySymbol Voice Onset TimeSymbol

TimeSignatures

Q

Signatures

KeySignatures

Voice Onset AbsoluteTempoSymbol

Z-String

Date Instruments Edition

Q

Q

Z-String Z-String Z-String Z-String Z-String

Composer

Lines

BibInfo

Q

Score

3.3 Examples 29

Chapter 4

The Category of Modules

Mathematics provides the tools for describing the physical world. A travel deep down to the most elementary physics, the domain of subatomic phenomena, leaves us with more or less pure mathematics. The application of mathematics to the representation of thought and modeling of thought processes seems to lend itself to a similar treatment. Many areas of mathematics qualify for this task. However, two specific domains will form the basis of the precise formulation of an architecture, as proposed in the last chapter: The theory of modules and category theory. Whereas the former is contingent and could be exchanged for any other theory of mathematical spaces, the latter is necessary to hold the whole construction together.

4.1 From Monoids to Modules Abstract algebra deals with algebraic structures, i.e., structures consisting of a base set and one or more operations that satisfy a number of properties. The study begins with very simple structures and proceeds by enriching them with more and more properties. One may say that each additional property is intended solve a certain problem, quite similar to the way that the path from integers to complex numbers goes through the requirement of the solution of certain equations. In this section a selection of some important types of algebraic structures is presented. Observe that many more types exists, in-between and beyond, but these are the ones that play a major role in the mathematics used in R UBATO C OMPOSER. The following discussion is not meant as an overview of algebraic theory, but rather as a refresher. Since the algebra provides the fundamental computational structure of R UBATO C OMPOSER, hints at the

31

32

4 The Category of Modules

implementation discussed in part II are interspersed and preceded by the ➟ symbol. A basic knowledge of set theory will be assumed.

4.1.1 Monoids Definition: A monoid is a pair (M, ∗) where M is a set and ∗ : M × M → M is a binary operation with the following properties: 1. Associativity: For k, l, m ∈ M: (k ∗ l) ∗ m = k ∗ (l ∗ m). 2. Neutral element: There is e ∈ M, such that e ∗ m = m ∗ e = m for all m ∈ M. The monoid is commutative if, in addition, m ∗ n = n ∗ m for all m, n ∈ M.  Since the requirements are modest, most of the familiar structures such as the various number domains (using either addition or multiplication as the operation ∗, and 0 or 1 as the neutral element, respectively) are monoids. An important monoid is the word monoid (W, ∗). This monoid is defined as follows: The set W consists of all finite sequences of letters from a given alphabet A. Its elements are called words or strings and include the empty word ǫ. The operation ∗ is the concatenation of strings and can thus be notated by simple juxtaposition. It is clear that concatenation is associative, but generally not commutative. A word monoid over an alphabet A is only commutative if A is a singleton set or the empty set. The neutral element is the empty word ǫ. The second important concept besides algebraic structures themselves is that of a homomorphism. Given two structures M and N, a homomorphism from M to N is a map that preserves structure. What structure preservation exactly means is to be defined as the case arises. For monoids, we have the following definition: Definition: Given two monoids (M, ∗M ) and (N, ∗N ), a monoid homomorphism f : (M, ∗M ) → (N, ∗N ) is a map of sets f : M → N such that 1. f(m ∗M n) = f(m) ∗N f(n) for all m, n ∈ M, 2. f(eM ) = eN for the neutral elements eM ∈ M and eN ∈ N.  The following terminology is generally useful:

4.1 From Monoids to Modules

33

Definition: An isomorphism is a homomorphism that is bijective. An endomorphism is a homomorphism where the domain and codomain coincide. An automorphism is an endomorphism that is an isomorphism. 

4.1.2 Groups The next step enriches the monoid structure with an additional property, that of invertibility. This property follows from the desire to solve the equations x ∗ m = e and m ∗ x = e for x, where m is any element of the structure. Definition: A monoid (G, ∗) is a group if every g ∈ G is invertible, i.e., there is h ∈ G such that g ∗ h = h ∗ g = e.  Obviously (Z, ·) is not a group, but (Z, +) certainly is, as is (R\{0}, ·). The study of group theory is far-reaching and extends into such diverse scientific fields as botany, computer science and, not least, music theory. Permutations, and combinatorics in general, form a significant part of group theory. The definition of monoid homomorphism extends to that of a group homomorphism, i.e., a group homomorphism is a monoid homomorphism between groups. A group is commutative, or abelian, if it is so as a monoid.

4.1.3 Rings So far, only structures featuring a single operation have been presented. The ring structure adds a second operation and those properties that define how both operations interact. Definition: A ring is a triple (R, +, ∗), such that (R, +) is a commutative (additive) group with neutral element 0R and (R, ∗) is a (multiplicative) monoid with neutral element 1R . The operations of these two structures are linked through the distributivity property: for all x, y, z ∈ R, x ∗ (y + z) = x ∗ y + x ∗ z and (x + y) ∗ z = x ∗ z + y ∗ z. A ring (R, +, ∗) is commutative if the monoid (R, ∗) is so.

34

4 The Category of Modules

A commutative ring where every non-zero element is invertible is called a field.  The definition of ring homomorphisms follows from the definitions of homomorphisms for groups and monoids. Definition: A map of sets f : R → S between rings R is S is a ring homomorphism if f is a group homomorphism of the additive groups of R and S and if f is monoid homomorphism of the multiplicative monoids of R and S, i.e., for all a, b ∈ R f(1R ) = 1S f(a + b) = f(a) + f(b) f(a ∗ b) = f(a) ∗ f(b) It follows that f(0R ) = 0S and f(−a) = −f(a), where −a is the additive inverse of a ∈ R.  Rings are fundamental in the design of the foundation of R UBATO C OM POSER. The complete inclusion chain of number rings is provided, i.e., Z, Q, R, and C (➟ page 91). The last three are also fields. Remark: The rings Zn of integers modulo n are usually constructed as quotient rings. However for reasons of simplicity they are treated as independent rings in R UBATO C OMPOSER, since otherwise a more comprehensive algebraic environment including the theory of ideals would have to provided. This would be against the goal of compactness of the implementation. Such functionality is provided by general-purpose computer algebra systems such as M APLE and M ATHEMATICA. 

Polynomials Polynomials with one or more indeterminates can be formed into a ring structure by building on the word monoid introduced above. However, it is also possible to short-cut and simply define the polynomial ring RhXi with coefficients in the ring R and one indeterminate X as follows: An element p in RhXi is of the form: k

p = ck · X + ck−1 · X

k−1

+ ck−2 · X

k−2

2

+ · · · c2 · X + c1 · X + c0 =

k X i=0

ci · X i

4.1 From Monoids to Modules

35

where ci ∈ R and k ≥ 0. The integer k is called the degree of the polynomial p, if k is the greatest number such that ck 6= 0. The ring operations are: n X

ci · X i +

i=0

n Y

i=0

n X

di · X i =

i=0

ci · X i ·

m Y

n X

(ci + di ) · X i

i=0

di · X i =

i=0

m+n X i=0

 

X

k+l=i



ck dl  · X i

For the definition of the sum, it is assumed that both polynomials have the same degree. If this is not the case, the polynomial with the lower degree can be augmented with terms of the form 0 · X i to increase the “degree”. The additive and multiplicative neutral elements are 0R and 1R , respectively, regarded as constant polynomials (i.e., k = 0). R UBATO C OMPOSER currently provides polynomials with one freely specified indeterminate over the number rings (➟ page 91).

Strings A structure such as a polynomial ring with coefficients in the ring R is called an R-algebra. Another important R-algebra is the class of rings of strings with coefficients in a ring R. String rings are a generalization of polynomial rings and build on the word monoid. However, as in the case of polynomials, a direct definition is straightforward. The ring of R-strings RhUNICODEi over the alphabet of UNICODE characters is defined as follows: An element s in RhUNICODEi is of the form: s=

k X

ci · Wi

i=0

where ci ∈ R and the Wi are pairwise different words in the word monoid over the UNICODE alphabet. String rings are implemented in R UBATO C OMPOSER for all the number rings (➟ page 91). From the computer science perspective an element from a string ring may be viewed as a dictionary (implemented as a hash table) mapping UNICODE (native) strings to elements from the ring R, with the assumption that strings not present in the table implicitly map to 0R . The ring operations are:

36

4 The Category of Modules n X

ci i=0 n Y

· Wi +

ci · Wi ·

i=0

n X i=0 m Y

di · Wi = di · Wi =

i=0

n X

(ci + di ) · Wi i=0 n X m X

ci dj · Wi Wj

i=0 j=0

For the addition, both string elements must be “filled up” with word terms having zero coefficient to match them term by term. The multiplication Wi Wj is the concatenation of the words Wi and Wj . If the result of the multiplication contains terms a · Wi,1 Wj,1 and b · Wi,2 Wj,2 such that Wi,1 Wj,1 = Wi,2 Wj,2 = V, then both are replaced by the single new term (a + b) · V. Observe that, in general, the ring is not commutative. The additive neutral element is the string where all coefficients are 0R and the multiplicative unit is the string 1R · ǫ where ǫ is the empty word (i.e., the multiplicative unit of the word monoid). A few examples shall illustrate the particular ring of strings ZhUNICODEi. Since addition is commutative, we may just as well order the terms lexicographically along the words. a = 9 · ABU − 3 · BE + 7 · POL b = 3 · POL + 2 · REGA a + b = 9 · ABU − 3 · BE + 10 · POL + 2 · REGA a · b = 27 · ABUPOL + 18 · ABUREGA − 9 · BEPOL − 6 · BEREGA + 21 · POLPOL + 14 · POLREGA b · a = 27 · POLABU − 9 · POLBE + 21 · POLPOL + 18 · REGAABU − 6 · REGABE + 14 · REGAPOL c = 3 · AA + 5 · A c2 = 9 · AAAA + 15 · AAA + 15 · AAA + 25 · AA = 9 · AAAA + 30 · AAA + 25 · AA d=2·ǫ c · d = 6 · AA + 10 · A =d·c The embedding of strings in the structure of a ring is important, since, this accomplished, most, if not all, of the essential basic data types used in computer programming can now be regarded from the common perspective of ring theory.

4.1 From Monoids to Modules

37

Product Rings Given two rings (R, +R , ∗R ) and (S, +S , ∗S ), the product ring (R×S, +R×S , ∗R×S ) is defined on the Cartesian product of sets R × S, i.e., the set of pairs (x, y) with x ∈ R and y ∈ S. The rings R and S are the factors of the product R × S. The following properties describe the construction of the product ring R × S. For every x, u ∈ R and y, v ∈ S: (x, y) +R×S (u, v) = (x +R u, y +S v) (x, y) ∗R×S (u, v) = (x ∗R u, y ∗S v) 0R×S = (0R , 0S ) 1R×S = (1R , 1S ) The construction of the product of two rings can be iterated to three and more factors: in fact, (R × S) × T and R × (S × T) are isomorphic, thus both may be identified and simply written as R × S × T. A type of ring homomorphism specific to products is the class of projections. Given a product R1 ×· · · Rn with n factors, the projection pi to the i-th factor (i ∈ [1, n]) is defined as pi : R1 × · · · Rn → Ri : (x1 , . . . xn ) 7→ xi . Product rings provide an easy way to “bundle” several ring structures as one. They are, however, not to be confused with the higher-dimensional module structures, discussed in subsection 4.1.4. Product rings with any number of factor rings (but at least two) and their projections are available in R UBATO C OMPOSER (➟ page 91 and page 99).

4.1.4 Modules In computer programming, one way of constructing types larger than the basic types, which we have dealt with using the language of rings, is the introduction of (fixed-length) arrays with elements from a basic type. This is the starting point of the implementation of linear algebra including the best studied and most efficient algorithms in computer science based on matrix computation. The origin in mathematics is found in the study of vector spaces and, more generally, modules. These include linear and affine spaces, with their rich set of geometrical transformations, for example the two- and three-dimensional Euclidean spaces familiar from everyday perception. The application of such transformations is visible throughout science and art, and music in particular. Many manipulations common in musical compo-

38

4 The Category of Modules

sition can be expressed in the language of modules and module morphisms. The availability of an implementation of modules in R UBATO C OMPOSER therefore provides a necessary and powerful foundation for the development of compositional tools. Definition: Given a ring (R, +R , ∗R ), a (left) R-module is a triple (R, M, · : R × M → M), where (M, +) is an abelian group of vectors, and · is the scalar multiplication, with the following properties: 1. For all m ∈ M, 1R · m = m. 2. For all r, s ∈ R and m, n ∈ M, (r +R s) · m = r · m + s · m, r · (m + n) = r · m + r · n, r · (s · m) = (r ∗R s) · m. The subscripts on + and ∗ are usually omitted and ∗ is also written · if any confusion is unlikely. If R is a field, then M is called an R-vector space.  There are several types of homomorphisms that one can define on modules. Let us begin with the most common and restricted one, the class of R-linear module homomorphisms. Definition: Given two R-modules (R, M, ·M ) and (R, N, ·N ), an R-linear homomorphism f : M → N is a group homomorphism such that for all r ∈ R and m ∈ M f(r ·M m) = r ·N f(m). All in all, the characteristic property of an R-linear homomorphism can be expressed as follows, where all possible notational simplifications have been made: For all r, s ∈ R and m, n ∈ M f(rm + sn) = rf(m) + sf(n).  The set of n-dimensional column vectors over a ring R is made into an Rmodule using the element-wise addition of matrixes as the group operation and the external scalar multiplication. Such a module is then denoted by Rn and n is called its dimension. Any R-module isomorphic to Rn is called a free R-module of dimension n. Due to this isomorphism n-dimensional free R-modules can be identified and simply denoted Rn without any distinction. Most familiar are the Euclidean plane R2 and the Euclidean 3D-space R3 . Both are, of course, vector spaces. Each ring R is a one-dimensional free Rmodule itself. On the other hand, any abelian group G can be regarded as a Z-module, where the module addition is clear and the scalar multiplication

4.1 From Monoids to Modules

39

is defined by n · g = (sgn n)(g + g + · · · g) for n ∈ Z and g ∈ G. | {z } |n| times

So we see that modules encompass a large part of the common algebraic structures and thus serve as a useful container to handle them within one consistent framework. The homomorphisms considered so far are only defined between modules over the same ring R. An extension allows the definition of linear homomorphisms between two different rings R and S. Such a dilinear homomorphism consists of two parts: an S-linear homomorphism and a scalar restriction ϕ. Definition: If ϕ : R → S is a ring homomorphism and N is an S-module, then N[ϕ] (or N[R] if ϕ is clear) is the R-module defined by the scalar restriction ϕ via s · m = ϕ(s) · m, for s ∈ R and m in N. A dilinear homomorphism from an R-module M to an S-module N is a pair (ϕ : R → S, f : M → N[φ] ) consisting of a scalar restriction ϕ and an R-linear homomorphism f.  As an example of a dilinear homomorphism, consider a scaling by 12 from Q2 to R2 . The scalar restriction ϕ : Q → R is the embedding of the rationals in the reals, which is a ring homomorphism. We now have to define the Q-linear homomorphism f : Q2 → R2[Q] . All we have to do, is to express the module elements in terms of the basis vectors e0 = (1, 0) and e1 = (0, 1) of R2 . Then for each (x, y) ∈ Q2 : f(x, y) = ϕ( 21 x) · e0 + ϕ( 12 y) · e1 . In practice and in the implementation of R UBATO C OMPOSER, we have a domain of the form Rm , a codomain of the form Sn , an R-linear homomorphism f : Rm → Rn and a ring changing homomorphism ϕ : R → S. The first step applies f to a vector x in Rm to produce a vector y in Rn . Then ϕ is applied component-wise to y and the result is a vector z in Sn . Analogously to rings, the construction of products of modules leads to new modules based on given ones: Definition: For any family (M Qi )i∈I of R-modules where I is a set of indexes, we have the product module i∈I Mi . This is the Cartesian product of the underlying groups where the addition as well as the scalar multiplication Q are defined coordinate-wise. The subset of all vectors m = (mi )i ∈ i∈I Mi where only a finite number of coordinate mi 6= 0 is also a module which is L then called the direct sum module i∈I Mi . In particular, a product where the set of indexes is finite is a always a direct sum.

40

4 The Category of Modules

 The definition of direct sums entails two specific kinds of linear homomorphisms: L the class of projections and the class of injections. Given a direct sum I Mi we have for each index j ∈ I the projection πj :

M

Mi → Mj : (mi )i 7→ mj

I

and the injection ιj : Mj →

M

Mi : mj 7→ (0, . . . 0, mj , 0, . . .).

I

In addition to linear spaces one often considers affine spaces by specifying two domains, the space of points and the spaces of vectors. An object in an affine space is then an anchored vector. This distinction leads to an unnecessarily elaborate theory. It is also possible to locate the points and vectors in the same module space and extend the definition of dilinear homomorphisms to diaffine homomorphisms which are composed of a dilinear part and a translation part. For an R-module M and an element m ∈ M, the translation by m is the map Tm : M → M : x 7→ x + m. The notation Tm is suggestive of the way translations are composed, i.e., Tm Tn = Tm+n . Definition: Given two rings R and S, an R-module M and an S-module N, a diaffine homomorphism f from M to N is a map of the form Tn ◦ f0 , where Tn is the translation on N by n ∈ N and f0 : M → N is a dilinear homomorphism. If R = S and the underlying scalar restriction is the identity, then f is called an R-affine homomorphism.  In R UBATO C OMPOSER, free modules over all the implemented rings are available (➟ page 87). Product modules are also available, which are always direct sums, since the index set is always finite. There is also a large collection of diaffine homomorphisms (➟ page 95) which can be arbitrarily composed and combined in the ways allowed and guaranteed by the theory.

4.2 Categories

41

4.2 Categories Two concepts occur again and again throughout mathematics: the notion of object, which may be numbers, points in space, etc., and the notion of mapping, which is known under various names, such as function, transformation, morphism, etc. If we abstract these two notions, we get a theory with an extremely simple basis, but with enormous consequences, the theory of categories. Virtually every field of mathematics can be recast into category theory, foremost set theory, but also formal logic. Because of the terminology used it is probably not unexpected that this theory plays an important role in computer science (see [61] and [71] for just two such examples, complete sub-fields of computer science, such as functional programming [8] have been built on category theory). It has also been applied to much broader contexts [37].

4.2.1 Definition The following treatment is based on the discussions in [53] and [47]. The two concepts objects and morphisms can in fact be reduced to the sole concept of morphism. Indeed, if we consider any object x, we can always construct an identity morphism Idx : x → x on this object, and use x and Idx interchangeably. Thus we have the following definition: Definition: A category C is a collection of entities f, g, h, . . . which are called morphisms, together with a partial composition, i.e., for some morphisms f and g, a new morphism f ◦ g of C is defined. An identity or object in C is by definition a morphism e such that, whenever defined, we have e ◦ f = f and g ◦ e = g. In addition, the following axioms are given: 1. If f ◦ g and g ◦ h are both defined, (f ◦ g) ◦ h is defined. 2. Whenever one of the two compositions (f ◦ g) ◦ h or f ◦ (g ◦ h) is defined, both are defined and they are equal; we denote the resulting morphism by f ◦ g ◦ h. 3. For every morphism f there are two identities, a “left” identity eL and a “right” identity eR , such that eL ◦ f and f ◦ eR are defined (and necessarily equal to f). The (unique) right identity of a morphism f is called the domain of f, written dom(f), and the (unique) left identity is called the codomain of f, written codom(f). Therefore one also uses the familiar notation f : a → b where a = dom(f) and b = codom(f). The collection of all morphisms f : a → b

42

4 The Category of Modules

is denoted by HomC (a, b) or C(a, b), or, if the category C is clear from the context, Hom(a, b) or a@b.  The category Sets of sets is the most obvious example. Here the objects (or identity morphisms, depending on the choice of language) are the sets, and the morphisms are the set maps, where composition of morphisms is the usual composition of set maps. The terminology domain and codomain coincides with the familiar terminology known from the definition of functions on sets. The codomain of a function must not be confused with the range of a function. The range of a function is a subset of the codomain of the function and is defined as the set of all values produced by that function. The range is also called the image of the function. In many categories, for example the category of groups or the category of vector spaces, the morphisms are mappings, in the first case group homomorphisms and in the second case linear mappings. This need not be the case, however. The main characteristic of morphisms is their composability, which is a natural property of functions, but can also defined for other concepts that are not directly related to functions. As an example, consider the category of matrixes (over the real numbers, for example). Here the morphisms are the (m × n)-matrixes with m rows and n columns (where m and n are non-negative integers). The composition of morphisms is the multiplication of matrixes, i.e., the composition AB of two matrixes A and B is defined whenever A is an (m × n)-matrix and B is an (n × l)-matrix. The result AB is an (m × l)-matrix. The identity morphisms are the identity (n × n)-matrixes, which have 1 on the diagonal, and 0 otherwise. According to the identification introduced above, they also provide the objects of the category. Another example is the path category of a directed graph Γ . The morphisms of the category are the paths starting a vertex v and ending at a vertex w, for all vertexes v, w ∈ Γ . The identity morphisms are the lazy paths, which are the paths from v to v, for all v ∈ Γ , and containing no arrow at all. Of course, the identity path on v identifies with the vertex v. This example makes it obvious why morphisms are called “arrows” by many treatments of category theory. Path categories will play an important role later in this discussion (subsection 4.2.2). In this context we shall concentrate on the category of modules Mod. The objects are modules over arbitrary rings and the morphisms are the diaffine homomorphisms. Composition is obviously the composition of homomorphisms. A first construction of a category based on a given category is the following: Definition: For every category C we have the opposite category Copp . Its morphisms are the same as those of C, but composition is defined as f ◦opp g = g ◦ f. More simply stated, a morphism f : a → b in C thus becomes f : b → a in Copp , i.e., all directions are reversed.

4.2 Categories

43

 The properties commonly used to characterize functions can be carried over in a more general way to the abstract setting of the morphisms of a category. Definition: A morphism f : a → b is a monomorphism, iff for any two morphisms g : c → a and g′ : c → a, f ◦ g = f ◦ g′ implies g = g′ . A morphism f : a → b is an epimorphism, iff for any two morphisms g : b → c and g′ : b → c, g ◦ f = g′ ◦ f implies g = g′ . A morphism f : a → b is a section, iff there exists a left inverse g : b → a, i.e., g ◦ f = Ida . A morphism f : a → b is a retraction, iff there exists a right inverse g : b → a, i.e., f ◦ g = Idb . A morphism f : a → a is an endomorphism. A morphism f : a → b that is a section and a retraction is an isomorphism. An isomorphism that is also an endomorphism is called an automorphism. 

4.2.2 Functors Going one level higher up, morphisms between categories, also known as functors, are introduced: Definition: If C and D are categories, a functor F : C → D is a function which assigns to every morphism c in C a morphism F(c) in D such that 1. F(c) is an identity if c is so, this implies that F maps objects to objects, 2. if c◦c′ is defined in C, then F(c)◦F(c′ ) is defined and F(c◦c′ ) = F(c)◦F(c′ ). The composition of two functors F : C → D and G : D → E is a functor F ◦ G : C → E. In contrast to the usual covariant functors, a contravariant functor is a functor F : Copp → D. When the combined term contravariant functor F is made explicit, then the notation F : C → D is also used instead of F : Copp → D. Two categories C and D are called isomorphic is there exists a functor isomorphism, i.e., there exist two functors F : C → D and F−1 : D → C such that F ◦ F−1 = IdD and F−1 ◦ F = IdC , where IdC and IdD are the identity functors on C and D, respectively.  An important class of categories is based on directed graphs. In the following, the definition of directed graphs from [52] is used. A directed graph Γ

44

4 The Category of Modules

is specified as Γ : A → V 2 , where V is the set of vertexes and A is the set of arrows, and provided with the usual definitions for the head and the tail of an arrow, i.e., head = pr1 ◦ Γ and tail = pr2 ◦ Γ . A path p in Γ is a sequence of arrows (ai )i=1,...n such that for every i < n, head(ai ) = tail(ai+1 ), where n = l(p) is the length of the path p. A path of length 0, which consists only of a vertex v and has no arrows, is called the lazy path at v and is simply denoted by v. Given two paths p and q, such that the vertex at which p ends is equal to the vertex at which q begins, the composition qp is the path resulting from joining p and q at their common vertex. We have l(qp) = l(q) + l(p). It is clear that a lazy path v provides the identity, if composition is defined, i.e., vp = p or pv = p. Now, given a graph Γ : A → V 2 , the path category Path(Γ ) on Γ is defined using the above definitions as follows: Its elements are the paths on Γ and the composition is the composition of paths. The identities are the lazy paths v for each v ∈ V. Therefore the objects of Path(Γ ) can be identified with the set V of vertexes of Γ . The domain of a path is its starting vertex, the codomain is its ending vertex. If we are given a binary relation ∼ among some paths with equal domain and codomain, we can consider the smallest equivalence relation ∼′ which contains ∼ such that f ∼′ g with dom(f) = dom(g) = d and codom(f) = codom(g) = c and for every h and k with dom(h) = c and codom(k) = c, we have c ◦ f ∼′ c ◦ g and f ◦ d ∼′ g ◦ d. Such a relation gives rise to the quotient category Path(Γ )/∼. Its morphisms are the equivalence classes of paths and the composition is the composition of representatives of these classes. The path categories allow us to introduce a graphical notation, called diagram, to represent relationships in categories. Definition: A diagram in a category C is a functor ∆ : Paths(Γ ) → C, where Γ is a digraph (also called diagram scheme in this context). The diagram ∆ is said to commute with respect to a relation ∼ among the paths of Γ , whenever ∆ factorizes through Paths(Γ )/∼. If the relation is maximal (i.e., it identifies all paths having common start and end vertexes), then the diagram is said to be commutative without specification of ∼.  A simple example shall illustrate the above definition of a diagram. Given objects A, B, C, D, and set maps f : A → B, g : A → C, h : B → D, k : C → D, in Sets we have a diagram ∆ : Paths(Γ ) → Sets on the following graph Γ : A → V 2 with V = {A, B, C, D} and A = {f, g, h, k}:

4.2 Categories

45

A

f

h

g ? C

- B

k

? - D

The diagram ∆ maps the vertexes (or lazy paths) A, B, C, and D to the homonymous sets (or identity maps) in Sets. The arrows f, g, h, and k are mapped to the homonymous set maps. In particular, this diagram commutes whenever h ◦ f = k ◦ g. A commutative diagram is effectively a graphical way of representing an equation or a set of equations. Definition: A functor F : C → D is full whenever F(C(x, y)) = D(F(x), F(y)) for all pairs of objects x and y. It is called faithful whenever F : C(x, y) → D(F(x), F(y)) is injective for all pairs of objects x and y. It is fully faithful if F : C(x, y) → D(F(x), F(y)) is a bijection.  Definition: If C is a category, a subcategory is a subcollection C′ of C such that for each morphism f in C′ its domain and codomain are also in C′ , and such that for any two morphisms f and g in C′ , whenever f ◦ g is defined in C, it is also defined in C′ . A subcategory C′ of C is a full subcategory, if for every pair of objects x and y in C′ every morphism x → y in C is also in C′ . The identity on the morphisms in C′ induces an embedding functor C′ → C. 

4.2.3 Natural Transformations If we consider functors as objects, then natural transformations are morphisms between these objects. Definition: If F, G : C → D are two functors, a natural transformation t : F → G is a system of morphisms t(c) : F(c) → G(c) in D, for each object c in C, such that for every morphism f : x → y in C, we have G(f) ◦ t(x) = t(x) ◦ F(f). This property can also be written as the following commutative diagram in D:

46

4 The Category of Modules

F(x)

F(f) ? F(y)

t(x)G(x) G(f) ? - G(y) t(y)

It is easy to see that we have a category Func(C, D) where the objects are the functors F : C → D and the morphisms are all the natural transformations Nat(F, G) between functors F, G : C → D.  If the collections of morphisms x@y = Hom(x, y) in a category C are sets, i.e., objects in the Sets category, then two related functors can be constructed: For every object x in C, we have the functor x@ : C → Sets with y 7→ x@y. This functor sends every object y in C to the set of morphisms from x to y and every morphism f : y → z to the collection of set functions x@f : x@y → x@z with u 7→ f ◦ u, where u : x → y (see figure 4.1 for an illustration of this relationship).

C

Sets x@y

x@

x

u:x→y y x@f

f z

f◦u:x→z x@

x@z

Fig. 4.1: The covariant functor x@ : C → Sets for each object x ∈ C. The picture also illustrates the characteristic property of a covariant functor F = x@, i.e., F(f : y → z) : F(y) → F(z).

Analogously, we have for every object y in C the contravariant functor @y : Copp → Sets with x 7→ x@y. This functor sends every object x in C to the set of morphisms from x to y and every morphism f : x → z to the collection

4.2 Categories

47

of set functions f@y : z@y → x@y with u 7→ u ◦ f, where u : z → y (see figure 4.2 for an illustration of this relationship).

C

Sets

x@y

@y

y

u◦f :x→y x f@y

f z

u:z→y @y

z@y

Fig. 4.2: The contravariant functor @y : C → Sets for each object y ∈ C. The picture also illustrates the characteristic property of a contravariant functor F = @y, i.e., F(f : x → z) : F(z) → F(x).

Of course, the functors just introduced can be embedded in categories. In particular, the category Func(Copp , Sets) of contravariant set-valued functors on C is denoted by C@ . Its objects are called (set-valued) presheaves over C. Thus, amongst others, for each y ∈ C, @y is a presheaf over C. Ap@ plied to the category of modules, we have the category Mod of presheaves over the modules. For instance, @M, where M is a module, is an object of @ Mod . Thus, for a given module M, the presheaf @M over Mod is a function which maps any module N to the set of all diaffine homomorphisms N → M. Definition: Given two categories C and D and an object a of D, we have the constant functor [a] : C → D with [a](x) = a for all objects x ∈ C and [a](f) = Ida for all morphisms f in C. Given a digraph Γ and a fixed object c in a category C, the constant diagram ∆c = [c] associates every vertex of Γ with c and every arrow with Idc . For a diagram ∆ in C, a natural transformation [c] → ∆ is called a cone on ∆. Similarly, a natural transformation ∆ → [c] is called a cocone on ∆.  The preceding definition is a precise statement of the situation exemplified in figure 4.3. There we have a diagram scheme with the vertexes u, v, w, and y, and the arrows auy , auv , avw , and ayw . A particular natural transformation t : [c] → ∆ is the cone represented in the center of the

48

4 The Category of Modules

figure. The stated fact is that the central figure commutes, when regarded as a diagram. An example is the commutative diagram on the right for the triangle {c, ∆(u), ∆(v)}, which in the form of an equation states that ∆(auv ) ◦ t(u) = t(v) ◦ Idc , or, more simply, ∆(auv ) ◦ t(u) = t(v). c u auv

auy

avw

t(u)

∆(u)

∆c (auv ) = Idc

∆(auv )

∆(y)

∆(u)

v

t(w)

t(v)

ayw

Γ

∆c (u) = c

t(y)

t(u)

y

∆(auy )

w ∆(auv )

∆(v)

∆(w)

∆(ayw )

∆c (v) = c

∆(avw )

(b)

(a)

t(v)

∆(v)

(c)

Fig. 4.3: The digraph Γ for the diagrams ∆ and ∆c (a). A cone on ∆, the reason for the name should be obvious (b). The particular commutative diagram for u and v (c).

Figure 4.4 depicts the analogous example for a cocone instead of a cone. ∆(y)

∆(u)

u auv

v

auy

Γ

∆(auv )

y

∆(v) ayw

∆(auy )

∆(ayw )

∆(u)

t(u)

∆(w)

∆c (u) = c

∆(avw )

t(u)

t(w)

t(y)

∆(auv )

∆c (auv ) = Idc

t(v) avw

w

∆(v) c (b)

(a)

t(v)

∆c (v) = c

(c)

Fig. 4.4: The digraph Γ for the diagrams ∆ and ∆c (a). A cocone on ∆ (b). The particular commutative diagram for u and v (c).

4.2.4 Yoneda’s Lemma All the tools are now available to come to a central point in the theory. Definition: Given a category C where the collections of morphisms x@y are sets, the Yoneda embedding Y is the functor Y : C → C@ : x 7→ Y(x) = @x

4.2 Categories

49

where the natural transformations Y(f : x → y) : Y(x) → Y(y) are defined by u 7→ f ◦ u for u : z → x ∈ Y(x)(z) = z@x. If C = Mod, we write Y(M) = @M for a module M. A functor F in C@ is called representable whenever there is an object c ∈ C such that F is isomorphic to Y(c). Obviously, for every module M the functor @M is representable.  Lemma: For every functor F in C@ and every object c ∈ C, we have the collection of natural transformations Nat(Y(c), F) from the functor Y(c) to F, and the map ǫ : Nat(Y(c), F) → F(c) : h 7→ h(c)(Idc ) is a bijection.  The lemma states that the full subcategory of representable functors in C@ is equivalent to C (i.e., there is a fully faithful functor between C@ and C that is essentially surjective on objects), and that such an equivalence is given by the Yoneda embedding. When we put F = Y(d), we get a bijection ∼ ǫ : Nat(Y(c), Y(d)) → Hom(c, d). Again, if C = Mod, we write F(A) = A@F, even in the case where F is not representable, and we have then the bijection ∼ Nat(@A, F) → A@F. In the case where F is the representable functor @M, ∼ we have Nat(@A, @M) → @M(A) = A@M. In this context, we call A the address, since we look at F (or @M) under all morphisms from the viewpoint of A.

4.2.5 Limits and Colimits Constructions through the characterization of limit and colimit provides us with a class of objects which will be of importance later on, when we take up again the discussion of concepts. We need a few preliminary definitions: Definition: An object e in a category C such that there is a unique morphism ! : c → e for every object c in C, is called a final object, it is usually denoted by 1. A final object i in the opposite category Copp is called an initial object in C, it is usually denoted by 0. This means that for every object c in C, there is a unique morphism ! : 0 → c. 

50

4 The Category of Modules

Definition: Given a “basis” object b of a category C, the comma category C/b has all the morphisms f : x → b for x ∈ C as objects, and for two objects f : x → b and g : y → b we have HomC/b (f, g) = {u | g ◦ u = f}, the set of “commutative triangles above b” with the evident composition. The cocomma category C/opp b is the comma category (Copp /b)opp , i.e., for two objects f : b → x and f : b → y we have HomC/opp b (f, g) = {u | u ◦ f = g}. See also figure 4.5.  uxy y

x

uyx fy

fx uxz

b fz z

Fig. 4.5: The objects b, x, y and z are objects of a category C. The morphisms fx , fy and fz of C are objects of the comma category C/b. The morphisms from fx and fy in C/b are all the morphisms uxy ∈ C such that the triangle formed by fx , fy and uxy commutes. The cocomma category is similar, except that the arrows are reversed.

Given a category C and a digraph Γ , we have the category Func(Path(Γ ), C) of diagrams in C. Given such a diagram ∆, we have the comma category Func(Path(Γ ), C)/∆ and, in this category, the full subcategory Cone(∆) of cones [c] → ∆. Similarly, the subcategory Cocone(∆) of cocones ∆ → [c] is defined in the cocomma category Func(Path(Γ ), C)/opp ∆. Definition: For a diagram ∆ : Path(Γ ) → C, a limit of ∆ is a final object in the category of cones Cone(∆). This object is denoted by lim(∆). Thus, the limit is not an object of C, but a cone, including the constant “top object” of the cone. But often, if the rest is clear, one also writes lim(∆) to denote the top object itself.

4.2 Categories

51

Dually, a colimit of ∆ is an initial object in the category of cocones Cocone(∆). This object is denoted by colim(∆). Again, the colimit is not an object of C, but a cocone, including the constant “bottom object” of the cocone. If the rest is clear, one also writes colim(∆) to denote the bottom object itself. A category C for which all limits (of finite diagrams) exist is called (finitely) complete. A category C for which all colimits (of finite diagrams) exist is called (finitely) cocomplete.  The familiar construction of a product, for example the Cartesian product in set theory, is just a special case of a limit, and serves as an example for elucidating the above definition. Figure 4.3 helps to understand the following commutative diagram which characterizes the product as a limit: c
A
A u A

? A pra ◦ u
a × b A prb ◦ u
A
A @
@ A
@ A pra prb @ A
R @ UA 
a b The cone of the product is the natural transformation pr : [a × b] → ∆. The diagram ∆ is discrete (i.e., it there are no arrows) and represented by the vertexes labeled a and b. The natural transformation pr is defined through the projections pra and prb to the factors a and b, respectively, of the product. The top object of the cone is what is called the product, and therefore denoted by a × b. By the common abbreviation introduced in the definition of the limit, we may write a × b = lim(∆). The condition that lim(∆) is a final object translates to the fact that for every object c there is a unique morphism u : c → a × b such that the diagram commutes. The dual of a product is a coproduct. It is a special case of a colimit. The following commutative diagram characterizes the coproduct:

52

4 The Category of Modules

a

b


A@
A @ ina inb
A @
A @R @
A a⊔b
A u ◦ ina A
u ◦ inb
A u
A
A UA ? 
c The cocone of the coproduct is the natural transformation in : ∆ → [a ⊔ b]. The diagram ∆ is discrete (i.e., there are no arrows) and represented by the vertexes labeled a and b. The natural transformation in is defined through the injections ina and inb from the cofactors a and b, respectively, of the coproduct. The bottom object of the cone is what is commonly called the coproduct, and therefore denoted by a ⊔ b. By the common abbreviation introduced in the definition of the colimit, we may write a ⊔ b = colim(∆). The condition that colim(∆) is an initial object translates to the fact that for every object c there is a unique morphism u : a ⊔ b → c such that the diagram commutes. An important result (without proof) is this: the category of sets Sets has arbitrary limits and colimits, thus it is both complete and cocomplete. The category C@ of presheaves over a category C is complete and cocomplete.

4.2.6 Topoi If we are working with a category such as Mod we have to deal with a dilemma. On the one hand we have the category of sets Sets that provides us with a wealth of principles to construct objects, such as limits, colimits, and powersets. But its morphisms are not characterized in any specific way, they are just arbitrary set maps. On the other hand, we have the category of modules, where the morphisms are the highly characterizable diaffine homomorphisms. In this category, we do not have, however, all the possibilities for the construction of useful new objects as in Sets, such as powersets. The goal is to find a way to provide such a category as Mod with the desired properties and this leads to special categories, called topoi that imitate such constructions, but also retains the particular structure of the morphisms. We shall not go into the details of the study of topoi, which is a farreaching theory. However, the most important result is that the category @ of presheaves Mod is a topos. This means that it allows all limits and col-

4.2 Categories

53

imits, as well as subobject classifiers Ω. The last notion can be intuitively understood when related to the category of sets. There Ω = 2 = {0, 1}, @ and Set(S, 2) is the set of subsets of S, i.e., the powerset of S. In Mod , the subobject classifier Ω plays the role of 2, i.e., S@Ω characterizes the set of of subobjects X ⊂ S. @

Thus, by introducing Mod , we have provided the category of modules with the properties (that Mod did not have) that we need to cast our architecture of concepts in a mathematical mold. The importance of topoi for the mathematics of music may be motivated by the following example: We already stated that the theory of modules, more precisely the category of modules, provides us with a powerful tool to pursue mathematical theories of music. Pitch classes modeled using the ring Z12 of integers modulo 12 is a case in point. It is inevitable that the need for arbitrary sets of pitch classes, and sets of sets of pitch classes (sets of chords), arises. These sets cannot, however, be embedded in Mod. We would have to switch to the Sets category and thereby lose module algebra. Both conflicting requirements, that of powersets and that of module algebra, @ are met by working in the category Mod of presheaves over the modules.

Chapter 5

Architecture of Concepts II: Forms and Denotators

Armed with the necessary tools from module and category theory introduced in the previous chapter, we may now embark on the restatement of the principles from section 3.2 in terms of this theory. We shall see that the concepts of Simple, Limit, Colimit and Power reappear in a natural way.

5.1 Forms The theory of functors allows us to speak of generalized points in space, i.e., addressed points instead of traditional points known from standard mathematical approaches. @

opp

In the following we work with the category Mod of functors Fu : Mod → @ Sets. An argument module M to a functor Fu ∈ Mod is called an address of Fu and a module morphism f : M → N is called an address change. A form is a mathematical space and is formally defined as follows based on the presentation in [47]: Definition: A form F is a quadruple F = (NF, TF, CF, IF) where 1. NF is the name N(F) of F. We shall later see what a name is exactly made of. 2. TF is one of the following symbols: Simple, Limit, Colimit, or Power. It is called the type T(F) of F. 3. CT is one of the following base on the type of F: a. Case Simple: CF is a module M. b. Case Power: CF is a form. c. Case Limit or Colimit: CF is a diagram D of forms.

55

56

5 Architecture of Concepts II: Forms and Denotators

It is called the coordinator C(F) of F. The diagram D is a diagram of functors Fun(Fi ) for a family of (Fi )i of forms (see point 4 below). @

4. IF is a monomorphism of functors IF : Fu → X in Mod with: a. Case Simple: X = @M. b. Case Power: X = Ω Fun(CF) . c. Case Limit: X = lim(D). d. Case Colimit: X = colim(D). It is called the identifier I(F) of F. The domain Fu is called the space (functor) of F and denoted by Fun(F). ➟ In the current implementation Fu is always taken to be X, and the monomorphism is the identity on X. This means that Fun(F) = X in that special case.  @

A remark about diagrams: a “diagram of forms” is a diagram in Mod since @ Fun(F) returns a functor which is an object in Mod . For the notation of forms, we resort to the notation introduced in section 3.1, i.e.: Name:.Type(Coordinator). A form of type Simple with module R and name Reals is thus denoted by Reals:.Simple(R). A form of type Limit with the name L and the coordinate forms Li , with i ∈ {1, 2, 3}, is written L:.Limit(L1 , L2 , L3 ). Observe that the diagram, although essential, is not explicitly mentioned. It has to be provided separately if necessary. The graphical notation, also introduced in section 3.1, can easily be transferred to the representation of forms. This representation has the advantage of providing a means for capturing the diagram, too. If the diagram of the above Limit form L consists of the morphisms f1 : L1 → L2 and f2 : L3 → L1 , the graphical representation of L looks as follows: L Q L1

f1

L2 f2

L3

5.2 Denotators

57

Observe also that we have not yet discussed the structure of names, which may therefore taken to be simple strings, for the time being.

5.2 Denotators A denotator is a (generalized) point in a form. Its formal definition is as follows: Definition: Let M be an address. An M-addressed denotator is a triple D = (ND, FD, CD) where 1. ND is the name N(D) of D. 2. FD is the form F(D) of D. 3. CD is an element of M@Fun(F(D)) called the coordinates C(D) of D.  Again, the notation of denotators is based on the notation from section 3.1. However this time, we have an additional address to deal with: Name:M@Form(Coordinates). We can now also say what a name is, both for forms and denotators: it is just another denotator, of whatever form. For the sake simplicity, however, the names will henceforth just be simple strings. How does a Simple denotator D with address Z of form Reals defined above look like? The name is arbitrary, for example N(D) = r. The form is of course F(D) = Reals. Finally C(D) must be an element of M@Fun(F(D)). In the case at hand Fun(Reals) = @R2 and M = Z, thus C(D) ∈ Z@R. In short, it must be a diaffine module homomorphism f : Z → R. Thus: r:Z@Reals(f) This most simple example illustrates the functorial approach of generalized points in mathematical spaces. Here, the value of a point is no longer a real number, but a module homomorphism with domain Z and codomain R. Of course, we can get back to our system of traditional points by limiting ourselves to constant morphisms, in particular to 0-addressed denotators. A denotator of form Reals that just represents the number 2.3, for example, will be defined using a constant morphism f : R0 → R with f(x) = 2.3. In this case, we may also notate more simply r:R0 @Reals(2.3),

58

5 Architecture of Concepts II: Forms and Denotators

or even r:0@Reals(2.3), where the symbol 0 makes clear that the null module corresponding to the codomain module is meant as the address. We may omit that too and are left with r:@Reals(2.3). These comments naturally apply to denotators of types other than Simple.

5.3 Computational Category Theory Forms and denotators provide us with the means for implementing an important part of what we may call computational category theory. This section elucidates this idea.

5.3.1 Data Types in Programming Languages Forms are a generalization of the concept of data types that have been present in programming languages almost since the beginning of modern computer science. It is therefore useful to explain the relationship between the form types and the different types of data structures as they occur in Pascal [34] and Haskell [9], to take two programming languages, one ancient and one recent. Observe that the term “type” is used in two quite different meanings: In denotator theory, the type is one a few fundamental ways that a form (respectively, a denotator) is constructed, while in the theory of programming languages a type corresponds to a form in denotator theory.

Simple Forms of type Simple correspond to the primitive data types in programming languages, such as integer or real in Pascal, or Int or Char in Haskell. Most commonly, the primitive data types betray a particular closeness to the machine representation, which natively provides for integers, characters and floating-point numbers. Only in higher-level languages, such as Haskell or Lisp, do primitive types also include strings or similar types. This is in stark contrast to the Simple forms, which are based on module theory. Modules encompass all the primitive data types usually found in programming languages. Additionally, strings, vectors, and products belong

5.3 Computational Category Theory

59

to this class. In short, rings and free modules over these rings are available. Haskell does provide a product data type with tuples as objects. However module and ring theory allows consistent algebraic operations throughout, which is not possible in programming languages, where operations are provided ad hoc for each data type separately. Thus, tuples in Haskell only allow construction and projection, and not pair-wise multiplication or addition, for example.

Limit The special case of type Limit, product, is provided by most programming languages. As an example, consider the Note form (the other forms are assumed to have already be defined, this applies also to subsequent data types): Note:.Limit(Onset,Pitch,Duration,Loudness,Voice)

In Pascal, the equivalent to this form would be implemented as a record data type: type Note = record o: Onset; p: Pitch; d: Duration; l: Loudness; v: Voice end

Haskell uses the data construction: data Note = Note { o p d l v }

:: :: :: :: ::

Onset, Pitch, Duration, Loudness, Voice

There are several possible ways of defining such a product in Haskell. In this case labeled fields have been used. This is essentially the same as the following: data Note = Note Onset Pitch Duration Loudness Voice

The second Note is in fact the constructor for objects of this type. To create a Note object, it is invoked as follows: Note 0.0 67 1.0 50 0

60

5 Architecture of Concepts II: Forms and Denotators

An interesting effect is that the Note constructor behaves like a curried function, i.e., partial evaluations, such as Note 0.0 are possible.

Colimit Data types corresponding to the special case of Colimit, the coproduct, are implemented in various ways. Object oriented languages, for example, favor subclassing to provide its functionality. In Pascal, the corresponding data type is called variant record. To illustrate the correspondences, consider the form Event:.Colimit(Note,Rest)

by which an event consisting of either a note or a rest is meant. The corresponding variant record in Pascal uses an additional field kind to discriminate between the two alternatives: type EventType = (NOTE,REST); type Event = record case kind: EventType of NOTE: (note: Note); REST: (rest: Rest) end end

In Haskell, this translates again into a data declaration, this time with the version using alternatives: data Event = NoteEvent Note | RestEvent Rest

Here, NoteEvent and RestEvent are the constructors of the two alternatives of the Event data type. These constructors are also used for pattern matching in the definition of functions which have Event as formal argument type.

Power General purpose languages do not provide data type constructors for setvalued data types. A notable exception is SETL [64], which is built around the handling of sets. However, many higher level languages, in particular all functional programming languages, directly support lists. Lists are preferred to sets, since they map more comfortably to machine representations,

5.3 Computational Category Theory

61

which are naturally ordered. Sets are then implemented using lists by eliminating duplicates and enforcing a canonical order. Therefore sets are less efficient than lists during runtime. Pascal has no direct provision for general lists or sets, except for a very special data type that implements bitsets. It allows for sets containing numbers from 0 to 31, and maps directly to a 32-bit machine word. Haskell has natural support for lists. To handle sets, library functions are provided, which regard lists as sets and take care of the correct management. The well-known Score form Score:.Power(Note)

translates to the Haskell list data type data Score = Score [Note]

A main characteristic common to all these types is the absence of addresses in programming language data types. In fact, in every case, an implicit null address can be assumed. Programming languages do not provide the power of the functorial approach, this is unique to forms and denotators. It is to be noticed that the types Limit and Colimit have correspondences only in the special case of discrete diagrams. The next section discusses the meaning of Limit and Colimit forms that have more general diagrams.

5.3.2 The Role of Diagrams Discrete diagrams in forms of type Limit and Colimit do not raise any questions. This is the case of products and coproducts which are handled the same way as in programming languages. However, whenever arrows are present, we must explain what this means for the concrete realization of denotators of such a form.

Limit Consider again the Limit form introduced above and repeated in figure 5.1 (a). The diagram corresponding to the form L is shown in figure 5.1 (b). This diagram is characterized by the two arrows (or morphisms) f1 and f2 . The universal property of limits requires that the diagram commutes, in particular the bottom part involving L1 , L2 , L3 , f1 and f2 . Consider now a denotator d with this form. It is constructed from coordinate denotators d1 with form L1 , d2 with form L2 and d3 with form L3 . The morphisms of the diagram

62

5 Architecture of Concepts II: Forms and Denotators

c L

u

Q L1

f1

L2 f2

L L3

prL1 prL2 L1

prL3 L2

f1

L3

f2 (a)

(b)

Fig. 5.1: A form of type Limit (a). The corresponding diagram (b).

now specify constraints on the coordinates, i.e., we must have f1 (d1 ) = d2 and f2 (d3 ) = d1 . If these conditions are not fulfilled, the denotator d does not exist, i.e., its construction fails. Constraints can be exploited in two ways: Either one provides all coordinates, and the constraints ensure the consistency of the result. Or one provides only some coordinates, in this case, for example, d3 . The other coordinates d1 and d2 are then inferred from the morphisms f1 and f2 . Diagrams, therefore, effectively provide a means for a certain type of constraint programming.

Colimit Although the Colimit type is dual to the Limit type, the concrete meaning of diagrams is completely different. Consider the Colimit form in figure 5.2 (a). The corresponding diagram is shown in figure 5.2 (b). The graphical representation of a form is not to be confused with its diagram defining its universal property. A denotator d with form L is constructed from either a denotator d1 with form L1 or a denotator d2 with form L2 or a denotator d3 with form L3 . What is the diagram of L used for? Take a denotator D1 with coordinate d1 , a denotator D2 with coordinate d2 , and a denotator D3 with coordinate d3 , for example. The diagram now generates a

5.3 Computational Category Theory

63

f2 L ` L1

f1

L2

f1

L1

L2

inL1 inL2

L3

f2

L3

inL3

L u

c

(a)

(b)

Fig. 5.2: A form of type Colimit (a). The corresponding diagram (b).

partition on the collection of denotators with form L where the equivalence relation ∼ is given by the diagram. In our example, this amounts to the equivalences D1 ∼ D2 iff f1 (d1 ) = d2 and D3 ∼ D1 iff f2 (d3 ) = d1 and, finally, D3 ∼ D2 iff f1 (f2 (d3 )) = d2 . To determine whether two denotators with a given Colimit form L are equivalent involves checking a system of equivalences generated from all relevant paths in the diagram of L. Similarly, to check the consistency of a denotator with a given Limit form L means to check a system of equivalences generated from all paths in the diagram of L. If the diagram contains loops or cycles, this may require the computation of fixed points. It is thus clear that diagrams add considerable complexity to the implementation of forms and denotators.

Chapter 6

Software Components for Computational Theories

We have now described the theory with hardly any allusion to issues of implementation. An important aim of this work is to make available the theory in a computational form as far as it is possible at all. A complete discussion of the actual implementation forms the whole of part II. In this chapter, a few issues about the relation between theories, mathematical or other, and machine based interfaces are commented on. In fact bringing theories to computers involves two related but subtly different aspect: On the one hand, the computational realization of the concerned theoretical objects through data structures and algorithms. In our case, this includes mathematical modules as well as forms and denotators and is the subject of chapter 10 and chapter 11. On the other hand, having the necessary data structures and algorithms is fair enough, at least to the competent programmer. It is, however, not sufficient for the user more interested in getting some interesting work done, based on his knowledge of the theory and not on the knowledge of its implementation. Indeed, skills in programming should rarely be required for handling theoretical knowledge with computers. Thus, an adequate interface must be developed that allows the mathematician, or composer, or musicologist, to communicate with the tool that embodies the theories he or she uses. The design of a user interface is of crucial importance. There are many ways to address this problem, from the basic premises such as the choice between a graphical or textual presentation, to the more intricate details within the chosen interface paradigm itself. The fundamental insight is this: the form of the human-computer interface effectively determines how theories are perceived and shapes the options of handling and manipulating theories, whether it restricts or expands them. This does matter even more at the meeting point of art and abstract theory. Here theories themselves and the way of perceiving them are in constant flux. Interfaces must therefore assist in the experimentation within this fluctuating world.

65

66

6 Software Components for Computational Theories

6.1 Types of User Interface There are many types of user interfaces, although a precise taxonomy is not really possible, since most actual implementations employ hybrid approaches. Leaving aside command-line interface such as realized in Humdrum, we consider only graphical user interfaces that are the most common today. One immediate distinction that can be made is between those interfaces that realize some kind of direct manipulation and those that follow the principles of data flow computation.

Direct Manipulation Direct manipulation is the way almost all graphical design software works, such as P HOTO S HOP for bitmap manipulation or B LENDER for 3D animation. The principle basically involves selecting an object, and then performing an operation on it. The ubiquitous cut-and-paste operation scheme also falls into this category. In the field of music software, this principle also dominates. Audio data editors work by allowing the user to select an extract of the sound wave data shown on-screen, which can then be cut, copied and pasted, as well as transformed using typical operations such as amplification or digital filtering (figure 6.1). In the field of symbolic music manipulation, software MIDI sequencers take a middle position in degree of abstractness. A main use of a sequencer is to allow the placement of MIDI events through various types of visualization, for example mimicking common music notation or the simpler form resembling a piano-roll. Apart from cut-and-paste operations, simple musically meaningful manipulations can be effectuated, such as transpositions, inversions or retrograde transformations (figure 6.2). This kind of approach is commonly regarded as the most intuitive, where the meaning of “intuitive” is left rather unclear. It is true, however, that it emulates the pattern of grasping an object, doing something with it, and putting it back where it comes from or to another place. On the other hand, it becomes difficult, although not quite impossible, to keep a history of all operations and to make changes at a particular step in the process. Thus large-scale experimentation is not the domain of direct manipulation interfaces. Another disadvantage is the near impossibility to automate processes. To remedy this, most software employing the direct manipulation strategy offer the possibility to create scripts or plug-ins using a data flow oriented language.

6.1 Types of User Interface

67

Fig. 6.1: The S WEEP audio data editor.

Data Flow The data flow approach is quite different from direct manipulation. In direct manipulation, the user performs the actions required to accomplish the task one after the other, building on previous results, thereby ordering the steps in real time. In contrast, the principle of data flow takes a bird’s eye view. It requires the tasks to be laid out in advance by devising subtasks and specifying how the result of a subtask fits in as input to another subtask, thus producing a network of nodes and links. Each link is a channel wherein data “flows” from a producing subtask (source node) to a consuming subtask (sink node). The user never or seldom intervenes in the actual computation, in contrast to direct manipulation, where he initiates each task not until the previous one has completed its execution (figure 6.3). The advantage of the data flow approach is that the designer of the process has a clear overview at all times. The configuration of different nodes can be changed at will without touching any other node. This is of importance to composers who may want to change parameters very early in the process to see their effect on the resulting composition. This is difficult if not impossible using direct manipulation, since usually the steps are not closely linked and history management does not allow for arbitrary adjustments. One of the best known data flow languages is L AB V IEW, a software system for data acquisition. Real instruments are symbolically represented and their outputs can be connected to various processing nodes for the analysis

68

6 Software Components for Computational Theories

Fig. 6.2: The R OSEGARDEN MIDI sequencer with a common music notation view (bottom) and a piano-roll (called matrix view in R OSEGARDEN) representation (right).

Fig. 6.3: Direct manipulation requires the user to intervene at each stage in a linear time-directed sequence (left). In the data flow approach the user operations and the dependencies between them using a static network. The actual performance happens without intervention by the user (right).

of the acquired data. This is an example of a real-time system, which is however only feasible if the processing algorithm are fast enough to handle the data as it comes in, or if such algorithms are supported by hardware acceleration, e.g., DSP implementation of FFT. In music applications, many software synthesizers use data-flow networks to link unit generators, filters and other processing elements. In this case it is the audio data in the form of samples that flow through the network. We already mentioned O PEN M USIC in subsection 2.2.3 where networks of interconnected components are called patches. Another application that uses data flow is Pure Data (PD) [62]. In figure 6.4 an example of a PD net-

6.2 Rubato Composer: Computational Theories

69

work is presented. This example also shows a particular application of the data flow principle, commonly called Visual Programming. This term is a misnomer, since the idea is simply to place the usual structural programming elements, such as alternatives, loops and expressions onto a canvas and using links instead of (or rather, in addition to) variable assignment assignments. The usefulness of this “programming” style over traditional textual programming code is, however, doubtful. We prefer the nodes to provide high-level functionality instead of sticking together many primitive operations in a complex network to obtain functionality of medium complexity.

Fig. 6.4: A conditional statement in Pure Data. One of the print statements select1, select-2, or select-3 is executed, depending on the input value (currently 0) being 1, 2, or other.

6.2 Rubato Composer: Computational Theories R UBATO C OMPOSER follows essentially the data flow paradigm. However it takes a particular view on what constitutes a node and also integrates direct manipulation to some extent. R UBATO C OMPOSER is about bringing the previously discussed mathematical theories to the composer. Thus the term COMPOSER in the name of the software refers simultaneously to two ideas: (1) it is a tool with the primary goal of musical composition; (2) the approach used to achieve this goal is by composing theories. What does it mean to compose theories? First, a particular type of theory is meant, namely computational theories. This is not to be confused with theories of computation, an entirely different concept. A computational theory is a theory that contains a significant computational part, or, simply stated, a theory that can be implemented. In this view the data flow approach translates in R UBATO C OMPOSER to mathematical objects, represented through denotators, flowing through theories, represented by computational units,

70

6 Software Components for Computational Theories

thereby being constantly transformed. As a very simple example consider the field of number theory. Here the objects are integers. One computational unit would implement the theory of integer factorization. The input of the unit is an integer and the output the set of its factors. The output of two such factorizations may then possibly be fed into a computational unit implementing the greatest common divisor. Naturally, real computational units would generally not act on such a fine-grained level. In R UBATO C OMPOSER such a theory is implemented as a component called a rubette. Instances of rubettes provide the nodes of the networks in the data flow schema. A rubette is generally not a computational primitive such as the simple addition of numbers, but provides high-level processing, the very implementation of the computation theory. A rubette is also different from the usual realization of nodes in data flow networks, in that they may also provide direct manipulation through two interfaces presented to the user as a view, where data can be rendered in an audio-visual manner, and properties that change the behavior of the rubette. By combining the structural possibilities of the data flow approach and the advantages of intuitive direct manipulation, the user can experiment with theories, discover new relationships, and produce interesting results that would remain hidden in traditional approaches. The R UBATO C OMPOSER system is primarily a platform that attracts two types of users: the developer of theories who implements rubettes, and the designer of networks of theories who combines the theories in a meaningful way. Certainly, the two types do not describe disjoint classes. They collaborate to produce as the final result a software application based on the R UBATO C OMPOSER platform.

Chapter 7

Historical Overview

The development of the R UBATO C OMPOSER software is not an isolated event, but the latest entry in a line of musical composition and analysis systems, starting in the early 1980’s. This chapter presents an outline of the evolution of these systems and summarizes the various aims and the results that have been achieved.

7.1 presto At the Salzburger Musikgespräch 1984, Guerino Mazzola presented an early version of a computer system for musical composition called M(2, Z)\Z2 o-scope, based on principles of mathematical transformations. Figure 7.1 shows the demonstration that he gave to Herbert von Karajan. Encouraged by the impression that this system left on the people present at the demonstration, a software system called presto was developed on the Atari ST computer, at the time one of the leading platforms used for computer music composition and research, not least because it supported MIDI natively. The first version appeared in 1989, the second release was available a few years later. During the evolution of this software, Guerino Mazzola published the theoretical underpinnings in the book Geometrie der Töne [44]. The presto application features a direct manipulation style interface, where the user creates and manipulates the score directly in a main view (figure 7.2). Several modules are included that make use of geometrical operations to generate and transform musical motifs. Figure 7.3 shows the so-called OrnaMagic module. It can be used to create an ornament from a given score fragment by applying translations. The exact pattern of how these transformations are applied is indicated through

71

72

7 Historical Overview

Fig. 7.1: Guerino Mazzola demonstrating the M(2, Z)Z2 -o-scope system to Herbert von Karajan in 1984 (left). The Hardware/Software compound making up this “Urpresto” system (right).

Fig. 7.2: The presto software running on Atari ST.

a grid. A generalized version of this module has been implemented in R U BATO C OMPOSER as the Wallpaper rubette, which will be presented in detail in section 16.2.

7.2 “Classic” R UBATO

73

Fig. 7.3: The transformation grid of presto’s OrnaMagic module.

7.2 “Classic” R UBATO From 1992 onwards, Guerino Mazzola and coworkers (Oliver Zahorka, Daniel Muzzulini, and Marcel Waldvogel) developed a new software that, in contrast to presto, focused on the analysis and performance aspect of music. It allows, for example, the extraction of metrical information from a musical work, such as the piano piece “Kuriose Geschichte” from Robert Schumann’s Kinderszenen. This information is then adjusted according to given rules and the original music is rendered according to new data, resulting in a “performance” of the original uninterpreted “dead-pan” version. Joachim Stange-Elbe has applied this principles extensively to Bach’s Kunst der Fuge [66]. This “classic” R UBATO was implemented in Objective-C on the NEXTSTEP platform. The software architecture featured modules called rubettes, that served as subsystems handling each a specific task. The MetroRubette shown in figure 7.4 generates the metrical-rhythmical analysis alluded to above. There is also the HarmoRubette for harmonic analysis and the PerformanceRubette for generating performances. The classic R UBATO internally works with a fixed set of data structures, mainly corresponding to nested lists of notes, which realized a first instance of the (not yet functorial) language of denotators and forms [75]. Each ru-

74

7 Historical Overview

Fig. 7.4: The R UBATO software running on NEXTSTEP currently showing the MetroRubette.

bette can be configured using predicates to work on a certain part of the global pool of data. From 2001 to 2004, the classic R UBATO has been ported to the then new MacOS X platform, capitalizing on the fact that the application development framework on MacOS X is very similar to the NEXTSTEP framework. This work was done by the group led by Thomas Noll with Jörg Garbers as the main developer at the Technical University of Berlin. Apart from the port itself, the software has been extended in various ways. In particular, a project attempted to make R UBATO interact with other commonly used software in computer music research, such as OpenMusic or Humdrum [26]. This proved however difficult since there is no common language that would simplify communication, and therefore more or less ad hoc solutions have been devised. The new R UBATO C OMPOSER system as a successor to classic R UBATO is intended to remedy this by being completely based on the form and denotator language. Moreover, while the port to a modern platform (MacOS X) has certainly infused new life to the classic software, which would otherwise have died with the disappearing NEXTSTEP platform, a similar situation exists today, since MacOS X is the only supported platform and ports to other platforms are not planned. By using the Java Development Kit for

7.3 Experiments in Java

75

R UBATO C OMPOSER, platform independence has been achieved for all practical purposes.

7.3 Experiments in Java First steps in the implementation of a denotator based software have been made Stefan Müller, Stefan Göller and Gérard Milmeister at the Institute of Computer Science of the University of Zurich under the supervision of Guerino Mazzola. The software produced include the EspressoRubette by Stefan Müller [58] and the PrimaVista browser by Stefan Göller [30]. The EspressoRubette computes performance fields for given performances, realizing a topic of inverse performance theory [45]. The PrimaVista browser is a three-dimensional environment for audiovisual rendering and manipulating denotators (figure 7.5). It has been implemented using Java3D and allowed the use of a SpaceMouse for the navigation. A characteristic of this environment is that audio-visual objects are denotators themselves.

Fig. 7.5: PrimaVista browser showing a specific rendering of a set of denotators.

Chantal Buteau used parts of the Java framework to create the MeloTopRUBETTE [15], an improved version of the MeloRUBETTE contained in the

76

7 Historical Overview

classic R UBATO. By following the principles originating in classic R UBATO, the result is a large, monolithic piece of software, which shows an additional limitation of the original approach. The need to move to a component-based implementation led to the next step in and the current state of the development of R UBATO.

7.4 R UBATO C OMPOSER The developments in Java led to the implementation of an important subset of the theory of The Topos of Music, as far as computationally feasible. While the PrimaVista browser showed promise for certain tasks of visualizing denotators, it has been recognized that a data-flow implementation of the graphical frontend presented to the application user would have the largest benefit. The present result is the R UBATO C OMPOSER software system. In fact, its predecessors presto and classic R UBATO did not have an explicit representation of the history of manipulation. Therefore, especially in presto, it turned out to be difficult to retrace one’s steps and apply changes at an earlier stage without redoing the work. The data-flow orientation of R UBATO C OMPOSER allows fiddling with settings at any stage, since the complete diagram of work-flow is always present. The reinterpretation of the rubette principle as components allows the building of complex applications from simpler and more primitive parts, instead of the closed and rather inflexible approach used in the case of the MeloTopRUBETTE. Topos theory has been applied to various aspects of music, for example by Thomas Noll [60], who is also a member of the group maintaining the MacOS X port of classic R UBATO. Nevertheless the original R UBATO implementation does not natively support category-theoretic approaches at all, and thus does not provide an adequate environment for these theories. The fact that R UBATO C OMPOSER is based on categorical denotator theory ensures that modern mathematical principles gain a natural platform.

Chapter 8

Overview

This part describes the implementation of R UBATO C OMPOSER. A few words are in order about some design choices. The most important is obviously the programming environment. The predecessors presto and classic R UBATO have been targeted at very specific platforms. In particular presto was implemented in C using the framework provided by the Atari ST. Classic R UBATO was implemented in Objective C and required the framework of NEXTSTEP and, later, of MacOS X. The consequence of this commitment is that the software did not outlive the platform, i.e., Atari ST in the case of presto. In the case of classic R UBATO, the software was limited to run on the platforms that supported the OpenSTEP framework, and no attempt to port to a different platform has been made. In both cases, this fixation proved to be a hindrance to a more widespread distribution and use of the software. It is to be noted that OpenMusic, which has been implemented in a Lisp system particular to the Macintosh platform, suffers from a similar drawback, although some effort to port OpenMusic to Linux has been made. To avoid this fate, we decided to base the implementation on Java [6] running on the Java Virtual Machine [33]. Now, a lot has been said on the advantages and shortcomings of using Java, and exaggerations on both sides have been the rule rather than the exception. Most of the development of R UBATO C OMPOSER has been done on Linux using the version 1.5 of JDK (Java Development Kit) from Sun and experience has shown that running the complete system on MacOS X succeeded at first go, albeit with a few expected quirks here and there. Portability is thus mostly a non-issue. Performance is not far behind native implementation either. The remaining point is integration with the operating environment. This is probably the weakest aspect, since the Java GUI is intended to be the much the same on every platform, and thus results in some discrepancies in user interface. In the end, the graphical user interface looks a little better on MacOS X than on Linux, without however affecting the experience of the user in any way.

79

80

8 Overview

As has already been hinted at, the user interface of R UBATO C OMPOSER has been created using Java’s own toolkit, called Swing, which is extremely flexible and allows the creation of interface elements of high complexity without overburdening the developer. For the external representation of data, for example files, XML is used throughout. XML is not the panacea that hype would have had it. Ultimately it is just a standardized way of representing data in a hierarchical way, which is, to put it bluntly, rather too unimaginative to be revolutionary. The emphasis is on standardized, which means that, in particular, the JDK provides extensive support for XML, most importantly XML parsing tools. A last important part of the implementation concerns the rendering of musical data. There are many conceptually different ways to handle this. The lowest-level approach, that of generating audio data, has been postponed for the time being, since R UBATO C OMPOSER is mainly oriented to work with more abstract data such as notes and events, at least for the present. The rendering of audio is not excluded, however, and can be realized through denotators as well, as the concept of frequency modulation introduced in subsection 3.3.2 shows. A system for the rendering of audio data would be a complete subsystem in itself and could be added later on. The main concern at this time is a way to simply produce sound from the notes and scores that are generated and it has been deemed good enough to employ the MIDI framework provided by the JDK. Naturally, MIDI has severe shortcomings, such as the difficulty to control arbitrary frequencies and the limited number of channels. The following chapters offer a detailed description of the implementation of R UBATO C OMPOSER, starting with the overall architecture in chapter 9, the implementation of modules in chapter 10, forms and denotators in chapter 11, and some further tools in chapter 12. These explanations are not just meant to give an abstract view of the architecture of the software, but are an important reference, along with the automatically generated Javadoc, for the developer who wants to use R UBATO C OMPOSER as an application framework and intends to extend it. The last chapter 13 in this part deals with the graphical frontend and is treated in a more cursory way. The software is released under the General Public License (GPL). The source and binary code, as well has documentation, can be accessed at http://www.rubato.org.

Chapter 9

Architecture

9.1 Overall Structure The software architecture of R UBATO C OMPOSER ramifies into a vast network of parts and subparts of different size and complexity. In order to facilitate navigation, this chapter describes a rough division into four important parts. The layers I, II, III and IV of this division are shown in figure 9.1. I. The Java 1.5 environment forms the foundation, including subsystems such as XML, Swing and MIDI frameworks. Some extensions such as helpful user interface components, XML Reader and Writer classes and other utilities also belong to this level. The implementation of rational and complex numbers, as well as other more general utilities are also counted to this fundamental part. Some details on the code found therein are discussed in chapter 12. This basis guarantees that the architecture built on top is for the most part platform-independent. II. The second part is the lowest level of implementation that belongs properly to R UBATO C OMPOSER. In short, this concerns the mathematics on which the whole system is built on, i.e., mathematical modules, their elements and morphisms of modules. The symmetric shapes of the subparts suggest that the types (i.e., classes) of modules and of their elements are in a one-to-one correspondence, whereas module morphisms relate to both structures. The most important fact to retain here is that modules, as well as their elements and morphisms, are proper Java objects or instances. A detailed discussion follows in chapter 10. III. The core of R UBATO C OMPOSER resides in this part, namely the implementation of forms and denotators. Similarly to modules and elements, forms and denotators are instances of classes in a parallel hierarchy, consisting of implementations for each of the types Simple, Limit, Colimit,

81

82

9 Architecture

Fig. 9.1: The overall architecture of R UBATO C OMPOSER.

Power and an additional type List. Forms and, more specifically, denotators support a number of generic operations. These are provided either as methods or as separate classes. Chapter 11 covers this part in detail. IV. While the previous layers are mainly intended for programmers familiar with Java, the top layer is the application layer. It provides the graphical user interface that allows the manipulation of the objects in the R UBATO C OMPOSER universe without extensive knowledge of programming. Here the rubettes, components that package functionality from the layers below, are introduced. These components, in their graphical appearance, are placed into networks and linked together via input and output connectors. Some aspects of the implementation are discussed in chapter 13. The use of the interface itself is presented in the manual in chapter 20.

9.2 The R UBATO C OMPOSER Universe

83

9.2 The R UBATO C OMPOSER Universe To help understand what types of objects exist in the universe of R UBATO C OMPOSER, an example state is examined on the basis of figure 9.2. The first fact to remark is the presence of the five most important object types, namely forms (not named in a meaningful way in this context, but numbered from 1 to 5), denotators, modules, module elements, and module morphisms. This picture must not be confused with a class diagram. In fact, all of the little boxes represent objects (i.e., instance of classes), and the arrows denote the relationship of object reference (i.e., a “has-a” relationship in object oriented terminology). This figure requires a few explanations. To begin with, we concentrate on the upper right subdiagram. In general an arrow from a box A to a box B means, that the object A contains a reference to the object B. Endowed with the theoretical knowledge presented in earlier chapters of this text, it follows that Form 1 references Form 2 and Form 3, which in this case might mean that Form 1 is of type Limit, and Form 2 and Form 3 its coordinate forms. Similarly Form 2 and Form 3 contain Module 2 and Module 3, respectively, which indicates that both forms are of type Simple with the corresponding modules as the codomains. The denotator Denotator 1 is obviously of form Form 1, and references two coordinate denotators that have forms Form 2 and Form 3, respectively, as required by Form 1. Both denotators at the leaves contain a module element, each referencing their module. This modules are, of course, enforced by the respective forms. The lower subdiagram shows a similar situation. Here we find a form Form 4 of type Simple with module Module 1. Two denotators Denotator 4 and Denotator 5 have this form, the second one containing an element belonging to the required module. Denotator 4 shows a new feature, namely that of being non-null addressed. The address is Module 4, indicated by the arrow from Denotator 4. The denotator no longer contains just a module element, but a full-fledged morphism with domain Module 4 and codomain Module 1. Both are of course determined by the address on the one hand, and the module in Form 4 on the other hand. In class oriented programming languages, types and classes are defined and fixed at runtime. This is clearly not the case in R UBATO C OMPOSER, since forms (which are akin to classes and types) are objects just as denotators (playing the roles of instances). This means that forms can be dynamically created, and potentially deleted, or modified, respectively duplicated. Also the problem of identity must be taken care of. The dynamical creation of forms is a feature of R UBATO C OMPOSER, however the cases of deletion and modification raise serious questions of consistency. What happens when a form is modified, but is referenced by some denotators? This would lead to a complete mess and is therefore strictly forbidden. Problems of deletions

84

9 Architecture

Fig. 9.2: The universe of R UBATO C OMPOSER.

do not exist, since there is not explicit way of destroying objects. Duplication and identity, however, demands more attention. When are two forms identical? One way of defining equality of forms is structural equality: Two forms are equal if they are of the same type and their components are equal in turn. The first objection to this scheme is the computational cost, the second problem comes from the possibility of recursive forms, which requires additional complexity to handle equality correctly.

9.3 Java Packages

85

To avoid inconsistencies and to keep complexity low, the following scheme has been adopted: Each form must be given a name. This is in contrast to denotators which can be anonymous. No two different forms can have the same name. Two forms are equal if they are the same object (pointer equality in terms of implementation). To help observe these rules a system repository has been introduced. This is a unique, global object which acts as dictionary that maps names to objects, and, as a special case, to forms. Each form must be registered with the repository, no two forms can be registered under the same name, and existing forms are retrieved by looking up in the repository. In figure 9.2 the repository object is shown on the left side. Every form is referenced by the repository. In addition, named denotators may be also registered, as well as modules, elements and morphisms. For these types of objects the restrictions specific to forms do not apply.

9.3 Java Packages The following chapters are going to be concerned with details of the implementation in Java. Java promotes the division of functional parts of a software system into packages, each package containing a set of classes that are intimately related. The package structure of R UBATO C OMPOSER reflects key points of the overall architecture, and it should therefore be easy to relate the packages to the parts discussed above. The most important packages are presented here in advance to give a synopsis of things to come. The first set of packages implements the content of part I: org.rubato.audio.midi Transforming MIDI to and from denotators and playing, MIDI file I/O org.rubato.scheme Embedded Scheme interpreter org.rubato.util Text processing and other utilities org.rubato.xml XML parsing and writing org.rubato.math.arith Complex and rational arithmetic org.rubato.math.matrix Matrixes over common rings

This set of packages deals with part II: org.rubato.math.module Modules and elements thereof org.rubato.math.module.morphism Module morphisms

The third part and some of the fourth part are covered by these packages: org.rubato.math.yoneda Forms and denotators org.rubato.logeo General operations on forms and denotators org.rubato.base Repository class and interface for rubettes

86

9 Architecture

The hierarchies starting at the following packages contain everything relating to part IV: org.rubato.composer The graphical user interface of R UBATO C OMPOSER org.rubato.rubettes A collection of built-in rubettes

Chapter 10

Modules and Morphisms

We now come to the foundations of R UBATO C OMPOSER: Modules are to the architecture of R UBATO, as primitive types are to programming languages.

10.1 Modules and their Elements It has already been said that modules are instances just like their elements. Therefore each module, be it the module of integer numbers, or a free module over the ring of real numbers, is an object in the universe of R UBATO C OMPOSER. In the case of the number rings, there is one unique instance for each ring, except for the modular integers Zn , where there is a unique instance for each n. The types of modules are arranged in a hierarchy. Since each module contains its corresponding elements, there is an exactly parallel hierarchy of module elements. As example, consider the ring of integers: This module is the unique instance of the class ZRing, and the elements are instances of the class ZElement.

10.1.1 The Module Interface Before embarking on the explanation of Module and ModuleElement Java interfaces, it is useful to discuss the question: What makes a module and what makes a module element? The first question gives rise to several aspects: First, what types of modules are there? Second, what can be done with modules?

87

88

10 Modules and Morphisms

The determination of module types directly results in the hierarchy already mentioned: On top, there is the abstract description of a module, then there are the rings and the free modules over rings. These are the main and the best known types, but there are other more specialized ones. The Module hierarchy determines the hierarchy of ModuleElement. The second aspect is about the operations on modules and module elements. These are essentially the properties of mathematical modules, e.g., the determination of the zero element, the construction of the identity morphism, the retrieval of the factor ring, and, of course, the element-of relation between a module and a module element. This entails the interface which is presented in figure 10.1. public interface Module { public ModuleElement getZero(); public boolean hasElement(ModuleElement element); public ModuleElement cast(ModuleElement element); public ModuleElement createElement(List<ModuleElement> els); public ModuleElement parseString(String string); public public public public public public

Ring getRing(); int getDimension(); Module getComponentModule(int i); Module getNullModule(); boolean isNullModule(); boolean isRing();

public ModuleMorphism getIdentityMorphism(); public ModuleMorphism getTranslation(ModuleElement element); public boolean equals(Object object); public int compareTo(Module object); } Fig. 10.1: The Java interface of Module. Only the essential parts are shown.

A few comments about specific methods are in order. Note that the abstract types ModuleElement and ModuleMorphism are used, as well as Ring which is a subtype of Module. The methods getZero, hasElement and getRing behave as expected. getIdentityMorphism, getTranslation: These two methods create module morphisms that are available for every module. The argument of getTranslation must of course be an element of the module in question.

10.1 Modules and their Elements

89

getDimension, getComponent: These methods return the dimension of the module, and the specified component module of the module, respectively. These have different meanings for free modules and for direct sums. The dimension of the direct sum is the number of components. isRing, isNullModule, getNullModule: The methods isRing and isNullModule test whether the module is a ring and whether it is of dimension zero, respectively. The method getNullModule returns a module with the same coefficient ring but with dimension zero. cast: It is often necessary to convert a module element to an element in another module. This method tries to do just that. This is not always possible, however, and in such a case, it simply returns null. createElement: Often an element of a module can be constructed from several elements in a module or a ring, e.g., the coordinates of an element in a free module are ring elements. This method can be used for such constructions. equals, compareTo: The equality of modules is given by mathematical equality, whereas the order implied by compareTo is somewhat arbitrary. The following can be relied on. The number rings are ordered as Z2 < Z3 < · · · Z < Q < R < C. Free modules are first ordered by their rings, and within free modules over the same ring, the order is on the dimension. The FreeModule interface inherits directly from the Module interface and has two extra methods: isVectorspace is true whenever the factor ring is a field and getUnitElement(int i) returns the unit vector with 1 at coordinate i. The abstract class Ring adds three more methods that are specific to rings. It is shown in figure 10.2.

public abstract class Ring implements FreeModule { public abstract RingElement getOne(); public abstract boolean isField(); public abstract FreeModule getFreeModule(int dimension); }

Fig. 10.2: The Java abstract class Ring.

The getOne method returns the unit of the ring. This is specific to a ring, since the concept of a unit element does not exists for modules. The property isField is true whenever every non-zero element is invertible. Each

90

10 Modules and Morphisms

ring gives rise to a free module over this ring and getFreeModule creates such a module with the given dimension. These are the main components in the module hierarchy. The figure 10.3 presents the full hierarchy of all implemented types of modules. Since each ring gives rise to free modules of arbitrary dimensions over this ring, the corresponding free modules are omitted lest the picture is overcrowded. Note, that every ring is a free module of dimension 1. To distinguish between a ring and free modules over the same ring of dimension different from 1, there is an intermediate class ProperFreeModule from which all proper free modules inherits. Thus the free module over R of dimension 1 is the unique instance of RRing, whereas all other free modules over R are instances of RProperFreeModule. In order to avoid the existence of an instance of RProperFreeModule of dimension 1, instances can only be created using a static make method, which, provided with the dimension, returns the right instance. Thus RProperFreeModule.make(1) returns the unique instance of RRing, whereas RProperFreeModule.make(3) creates an instance of RProperFreeModule. The same principle applies to all other subclasses of ProperFreeModule.

Fig. 10.3: The module hierarchy. An arrow from A to B indicates that A is a subclass of (or an implementation of the interface) B. The lighter gray boxes with dashed borders represent interfaces in the Java sense.

10.1 Modules and their Elements

91

Types of Modules The following list summarizes all implemented types of modules. For each ring there are of course the corresponding free modules. ZnRing, ZRing, QRing, RRing, CRing: The rings of modular integers, integers, rational, real and complex numbers. ZnStringRing, ZStringRing, QStringRing, RStringRing, CStringRing: The ring of strings with UNICODE words and coefficient rings Zn , Z, Q, R and C, respectively. ProductRing: Product rings with at least two factor rings. The factors are arbitrary rings, including product rings in turn. PolynomialRing: Polynomials in one indeterminate with coefficients in an arbitrary ring. A polynomial ring is constructed using the static method PolynomialRing make(Ring coefficientRing, String indet). Multivariate polynomial rings can be built by using a polynomial ring as coefficientRing and a different variable as indeterminate. However, no further support for dealing with multivariate polynomials is currently provided. DirectSumModule: This class implements direct sums of modules over the same ring. As an example, Z3 ⊕ Z5 is created with DirectSumModule.make(Z3,Z5) where the variable Z3 contains a module created by ZProperFreeModule.make(3) and the variable Z5 contains a module created by ZProperFreeModule.make(5). RestrictedModule: A module over a ring R with ring change S → R. This class effectively wraps an R-module in an S-module. Thus the ring of reals R can be put into a RestrictedModule using the embedding Z → R and is henceforth regarded as a Z-module.

10.1.2 The ModuleElement Interface The same questions as for modules can be asked of module elements: What types of module elements are there? This is trivially answered since the types of module elements mirror the types of modules, and these we have already treated. The hierarchy rooted at ModuleElement is exactly analogous to the module hierarchy, therefore it is not repeated here.

92

10 Modules and Morphisms

The second aspect is more interesting: What operations do elements of modules admit? Certainly, the operations that belong to the theory of modules are included here: scaling by a ring element and addition. For module elements that are effectively elements of a ring, the specific ring operations are adjoined.

public interface ModuleElement { public boolean isZero(); public public public public public public public public

ModuleElement scaled(RingElement element); void scale(RingElement element); ModuleElement sum(ModuleElement element); void add(ModuleElement element); ModuleElement difference(ModuleElement element); void subtract(ModuleElement element); ModuleElement negated(); void negate();

public public public public

int getLength(); ModuleElement getComponent(int i); Module getModule(); ModuleElement cast(Module module);

public boolean equals(Object object); public int compareTo(ModuleElement object); } Fig. 10.4: The Java interface of ModuleElement. Only the essential parts are shown.

The isZero method is clear. The module operations come in pairs. Scaling by a ring element is performed using scaled and scale. The first returns a new module element scaled by the ring element, the second scales the module element in-place. Therefore the second operation is destructive and potentially dangerous because of aliasing. In general the destructive operation should only be used for performance reasons and if one is certain about the lifetime of the objects. The same comment applies to addition (sum, resp. add), subtraction (difference, resp. subtract) and negation (negated, resp. negate). The ring element argument of the scaling methods must of course belong to the coefficient ring of the module element. Here and in all other similar cases a RubatoException (package org.rubato.base) instance or an instance of a subclass thereof is thrown if the expected arguments and the actual arguments to a method are incompatible.

10.1 Modules and their Elements

93

The getLength method returns the number of components. In the case of a vector, it is simply the length of the vector. The i-th component can be retrieved using getComponent(int i). The result of getModule is the module that this element belongs to. The cast method is analogous to the homonymous method of Module with roles interchanged, i.e., it tries to convert the module element to the module provided as argument. Equality again follows from mathematical equality. The order induced by compareTo is defined as follows: If elements from different modules are compared, their order is the same as the order of the modules they are elements of. Otherwise the order depends on the particular module. In an ordered ring, for example, it is simply the natural order between ring elements. In other cases, for example complex numbers and elements of free modules, a lexicographical ordering is used. Just as in the case of the class Ring, the corresponding element class RingElement provides additional methods, shown in figure 10.5. public abstract class RingElement implements FreeElement { public abstract boolean isOne(); public abstract RingElement product(RingElement element); public abstract void multiply(RingElement element) public abstract boolean isInvertible(); public abstract RingElement inverse(); public abstract void invert(); public RingElement power(int n); }

Fig. 10.5: The abstract class RingElement.

The operations specific to ring elements are multiplication and inverse (whose result is only if an inverse actually exists). Similarly to the module operations, both are available as non-destructive (product and inverse) and destructive (multiply and invert) versions. The power method returns rn for the ring element, where n is a non-negative integer. The property isOne checks whether the ring element is the unit and isInvertible is true iff the element in question is invertible. Since ring elements can be regarded as elements from the one-dimensional free module over the ring, the abstract class RingElement inherits from the interface FreeElement. This interface is also implemented by the various proper free element class such as ZProperFreeElement. It includes methods for accessing the component elements (using getRingElement(int i)

94

10 Modules and Morphisms

or an iterator, so that it can be put in a for-each loop) and methods for the component-wise multiplication (productCW is the non-destructive version and multiplyCW is the destructive version).

Example: Z-Modules To make the relationship between the different classes clearer, consider the example of the ring of integers. Figure 10.6 shows the complete hierarchies containing ZRing and ZElement. The correspondence between the two hierarchies is evident. The (abstract) class NumberRing is just a marker to distinguish the rings Z, Q, R, and C, which are related by embedding homomorphisms, from the other rings.

Fig. 10.6: The complete hierarchy of Module and ModuleElement restricted to the part concerning the ring of integers Z.

One could ask why there is no Field class or interface that marks the property of being a field in the hierarchy, similarly to NumberRing. Remember that the classes in the hierarchy describe types of rings (resp. modules), and are not the rings themselves. As an example, take ZnRing, whose instances are modular integer rings, and only fields when n is a prime. Hence, a Field superclass could not discriminate between field instances and non-field instances of ZnRing. Therefore a property isField has been introduced instead. In the case of fields such as R (the unique instance of RRing) or C (the unique instance of CRing) this method returns the constant true, but in the case of Zn (instances of ZnRing) it checks whether n is prime.

10.2 Module Morphisms

95

10.2 Module Morphisms Now that we have discussed the implementation of the objects of the category of modules, we proceed to the morphisms of the category. The implementation of morphisms determines which particular category it will be. Of important theoretical interest are the diaffine (called simply affine, from now on) morphisms, and these are going to receive the most attention. However, the category of modules may be regarded as a subcategory of the category of sets, in which case the morphisms will be set mappings. These are necessary since many important functions are not affine. In R U BATO C OMPOSER the morphisms are set mappings in the first place, and an attribute indicates whether the morphism is affine or not.

10.2.1 The ModuleMorphism Interface Morphisms are instances of classes that are embedded in a hierarchy. The interface (which is actually an abstract class) of the root of hierarchy, ModuleMorphism, is shown in figure 10.7. The classes in the hierarchy stand for specific types of morphisms, for example, affine morphisms on free R-modules, which are specified by a real matrix and a real translation vector, etc. Instances of these classes can be composed, added, and scaled to create new morphisms, provided their domains and codomains are compatible. Since the component morphisms are well known, it is possible to infer the properties of the resulting morphisms, i.e., whether they are linear, affine, etc. Using the provided types, a large number of mappings can be created, and further types can be added to the hierarchy in later developments of R UBATO C OMPOSER. All module morphism must inherit from ModuleMorphism and suitably override its methods. Some methods have default implementations, such as sum, which creates a generic SumMorphism instance based on the components. ModuleMorphism(Module domain, Module codomain): The constructor must be called in the constructor of every subclass in order to set the domain and codomain of the morphism. This implements the actual formula of the morphism. It does of course work only if its argument is an element from the domain, otherwise an exception is thrown. compose, sum, difference, scaled: These are the types of function composition that are available on all module morphisms. The domains and codomains must be compatimap:

96

10 Modules and Morphisms

public abstract class ModuleMorphism { public ModuleMorphism(Module domain, Module codomain); public ModuleElement map(ModuleElement x); public public public public public

ModuleMorphism ModuleMorphism ModuleMorphism ModuleMorphism ModuleMorphism

compose(ModuleMorphism morphism); sum(ModuleMorphism morphism); difference(ModuleMorphism morphism); scaled(RingElement element); power(int n);

public public public public public public

boolean boolean boolean boolean boolean boolean

public public public public

Module getDomain(); Module getCodomain(); ModuleMorphism getRingMorphism(); boolean inDomain(ModuleElement x);

isModuleHomomorphism(); isRingHomomorphism(); isRingMorphism(); isIdentity(); isConstant(); isLinear();

public int compareTo(Object object); public boolean equals(Object object); public static ModuleMorphism getIdentityMorphism(Module module); public static ModuleMorphism getConstantMorphism(Module module, ModuleElement value); } Fig. 10.7: The abstract class ModuleMorphism. Only the essential parts are shown.

ble, and, in the case of scaled, the argument must be in the ring of the codomain. power: The power morphism only gives a result when invoked on a morphism f where the domain is equal to its codomain. For an integer n ≥ 0 it returns f n . isModuleHomoMorphism: This property is true whenever the morphism is affine. It characterizes the category of modules and affine morphisms.

10.2 Module Morphisms

97

isRingMorphism, isRingHomoMorphism: The property isRingMorphism is true iff the domain and codomain are rings, whereas isRingHomoMorphism additionally checks if the morphism is a ring homomorphism and effectively characterizes the category of rings and ring homomorphisms. isIdentity, isConstant, isLinear: These properties determine whether the morphism is the identity, or a constant, or a linear morphism, respectively. getDomain, getCodomain: These methods return the domain and the codomain, respectively, of the morphism. getRingMorphism: Since a diaffine (or any other) module morphism possibly features a change of ring, for example for a morphism Z3 → R2 the change of ring from Z to R may be the simple embedding of Z in R, this method returns the ring change morphism. Often, it is just an identity morphism. inDomain: Before applying map to a ModuleElement object this method can be used to check if the given element is in the domain of the morphism. getIdentityMorphism, getConstantMorphism: These static methods create identity and constant morphisms with the specified module as domain. The codomain of an identity is of course the same as its domain. In the case of the constant morphism the supplied constant also determines the codomain.

Types of Module Morphisms The hierarchy rooted at ModuleMorphism is much simpler than the hierarchies rooted at Module and ModuleElement. It is essentially flat, with a collection of classes inheriting directly from ModuleMorphism and representing different types of morphisms. A presentation of some of the more interesting types of morphism now follows. IdentityMorphism: The identity exists for every domain and codomain M, such that Id : M → M, and Id(x) = x. Of course, identities can arise from types of morphisms different from IdentityMorphism. For example an affine morphism with unit matrix and zero vector is an identity morphism. To check for an identity, the method isIdentity should be used. ConjugationMorphism: Specific to complex numbers, conjugation maps x + iy to x − iy. This morphism extends conjugation to complex vectors. Conjugation is not

98

10 Modules and Morphisms

a module homomorphism. It is a ring homomorphism if the domain (and codomain) is the ring of complex numbers. ModuloMorphism: The modulo operation is the map Z → Zn : x 7→ x mod n. As implemented here, it extends to Zm → Zm n. EmbeddingMorphism: An embedding map is an important way of moving from one module to another. Embeddings require that no information gets lost during the translation. Therefore we have the standard embeddings of rings Z ⊂ Q ⊂ R ⊂ C, which extend to the string rings R and the polynomial rings R[X]. The embedding Sn into Rm is also possible provided that n < m and S can be embedded in R. When m is strictly larger than n, the last m − n components are filled with zeros from the codomain ring. A product ring R1 ×· · · Rm can be embedded in a product ring S1 ×· · · Sn whenever m = n and Ri can be embedded in Si . Whether the resulting embedding is a homomorphism or not depends on the mapping: For example, we allow the embedding of the rings Zn in the other number rings. However, these embeddings will no longer be a homomorphisms. CanonicalMorphism: A canonical morphism from M to N is the simplest morphism that takes an element from M to N. If M = N, this is of course the identity. If M can be embedded in N, it is an embedding as described above. Other types include ModuloMorphism, and the trivial morphisms R0 → M and M → R0 . In general, canonical morphisms may be devoid of any interesting properties, such as being a ring or module homomorphism. The direction from a “larger” ring such as as R to a “smaller” ring such as Z is also supported. This is akin to casting in programming languages and similarly discards important information. As long as a canonical morphism f from a ring R to a ring S (for example an embedding or even a cast) can be produced, a canonical morphism from Rm to Sn can also be created. If m = n, the components are transferred using f. If m < n the remaining n − m components of the element in the codomain Sn are set to the zero of Sn , and if m > n, the m − n supernumerary components of the element in the domain Rm are discarded. A canonical morphism can be used as a last resort to move from one space to another with a minimum loss of information. GenericBasisMorphism: A morphism f from a free module Rm to a module M can be defined in the following way: We consider the canonical basis vectors of Rm

10.2 Module Morphisms

99

e0 = (0, 0, 0, . . . 0) e1 = (0, 1, 0, . . . 0) e2 = (0, 0, 1, . . . 0) ··· em = (0, 1, 0, . . . 1) where the zero vector e0 is included, because we deal with affine mappings, and therefore the value of f at e0 is generally not 0M . For each ei , the value f(ei ) = fi in M is given. The fi , where 0 ≤ i ≤ m, are the parameters that determine an instance of GenericBasisMorphism. The value f(x) of an arbitrary element x = (x1 , . . . xm ) in Rm is now calculated as: m X f(x) = f0 + (fi − f0 )xi . i=1

PolynomialMorphism: The evaluation of a polynomial at an element from the coefficient ring of the polynomial is generally not a homomorphism, but provides a very important class of functions. The polynomial to evaluate is an instance of PolynomialElement introduced in the previous section. ProjectionMorphism: The projection f from a product ring R1 × · · · Rn to its i-th component Ri is the extraction of the i-th component of an n-tuple: f((x1 , x2 , . . . , xn )) = xi . ReorderMorphism: Given a product ring R1 × · · · Rn and a list of indexes i1 , . . . im , a reordering morphism f takes an element (n-tuple) from the domain R1 × R2 × · · · Rn and reorders its components according to the indexes ij to result in an m-tuple in the codomain Ri1 × Ri2 × · · · Rim : f((x1 , x2 , . . . , xn )) = (xi1 , xi2 , . . . xim ). A special case of an application of this type of morphism is the expansion of a ring R to the product ring Rn . SplitMorphism: A free module Rn can be regarded P as the direct sum of several free modules Ri1 ⊕ Ri2 ⊕ · · · Rim where m j=1 ij = n.

Now, a morphism from Rn → Rn can be defined as a “direct sum” of morphisms fi1 ⊕ fi2 ⊕ · · · fim : Ri1 ⊕ Ri2 ⊕ · · · Rim → Ri1 ⊕ Ri2 ⊕ · · · Rim ,

100

10 Modules and Morphisms

where we provide all fij : Rij → Rij to create an instance of SplitMorphism. ConstantMorphism: A constant morphism f : M → N maps every element from the domain to a fixed value in the codomain. For a given constant c ∈ N, f(x) = c. To check for a constant morphism, the method isConstant should be used. TranslationMorphism: A translation morphism f : M → M adds a constant to an element from the domain. For a given constant c ∈ M, f(x) = x + c. ProductMorphism: Given two morphisms f : M → R and g : M → R where the codomain R is a ring, a product morphism h : M → R can be created such that h(x) = g(x) · h(X).

Deriving Module Morphisms from Existing Ones A new morphism can be derived from one or more existing ones by calling one of the methods sum, difference, scaled, product, and compose. These methods try to make simplifications and reductions. As an example, consider the sum of two affine morphisms f and g of the form R2 → R2 . Since these morphisms are instances of RFreeAffineMorphism and each is characterized by a matrix and a vector, the call f.sum(g) will also return an instance of RFreeAffineMorphism defined by the sum of matrixes and the sum of vectors. Such a simplification is not always possible, however, and in that case an instance of SumMorphism is returned by the sum method. The same principle applies to the other methods for combining morphisms by creating instances of the general types DifferenceMorphism, PowerMorphism, ProductMorphism, ScaledMorphism, and CompositionMorphism.

Example: Rn → Rm morphisms Common types of morphisms are those of the form Rn → Rn , where R is a number ring, especially affine morphisms. In this case, the hierarchy is a little deeper and provides abstract classes of the form RAbstractMorphism and RFreeAbstractMorphism, where R ∈ {Zn,Z,Q,R,C}. These abstract classes already implement everything that is needed, the only abstract method is mapValue, which must provide the code and must be implemented by subclasses. Additionally, properties can be set and other methods can be overridden for reasons of efficiency. As an example we take R to be the real number ring R and look at RFreeAffineMorphism and RAffineMorphism, which extend RFreeAbstract-

10.2 Module Morphisms

101

Morphism and RAbstractMorphism, respectively. The class RAffineMorphism is shown in figure 10.8 and figure 10.9.

public final class RAffineMorphism extends RAbstractMorphism { public RAffineMorphism(double a, double b) { super(); this.a = a; this.b = b; } public double mapValue(double x) { return a*x+b; } public boolean isModuleHomomorphism() { return true; } public boolean isRingHomomorphism() { return (b == 0) && (a == 1 || a == 0); } public boolean isLinear() { return b == 0; } public boolean isIdentity() { return (a == 1) && (b == 0); } public boolean isConstant() { return a == 0; } . . . continued in figure 10.9 . . . } Fig. 10.8: The class RAffineMorphism. Only the more interesting parts are shown. Continued in figure 10.9 on page 102.

An instance of RAffineMorphism is specified by two real numbers a and b, the parameters determining the affine map x 7→ ax + b, which is implemented in mapValue. The properties in figure 10.8 return a result depending on the values of a and b. For example, the morphism is linear whenever b (the translation part) is 0. The compose method in figure 10.9 is specialized in that it first checks if the second component of the composition is also of type RAffineMorphism. If so, then a new instance of this type can be created by combining the param-

102

10 Modules and Morphisms

public final class RAffineMorphism extends RAbstractMorphism { . . . continued from figure 10.8 . . . public ModuleMorphism compose(ModuleMorphism morphism) throws CompositionException { if (morphism instanceof RAffineMorphism) { RAffineMorphism rmorphism = (RAffineMorphism)morphism; return new RAffineMorphism(a*rmorphism.a,a*rmorphism.b+b); } else { return super.compose(morphism); } } public ModuleMorphism sum(ModuleMorphism morphism) throws CompositionException { ··· } public ModuleMorphism difference(ModuleMorphism morphism) throws CompositionException { ··· } public boolean equals(Object object) { if (object instanceof RAffineMorphism) { RAffineMorphism morphism = (RAffineMorphism)object; return a == morphism.a && b == morphism.b; } else { return false; } } public double getA() { return a; } public double getB() { return b; } private double a; private double b; } Fig. 10.9: The class RAffineMorphism. Only the more interesting parts are shown. Continued from figure 10.8.

eters a and b of both affine morphisms. Otherwise, the superclass version is called which takes care of the generic composition. The sum and difference implementations are analogous. Equality testing consists in checking

10.2 Module Morphisms

103

for equality of the parameters. Note that we cannot test for mathematical equality of functions, i.e., two ModuleMorphism instances may compute the same function, but are of different types, hence not equal. The case of RFreeAffineMorphism is analogous. Instead of the real numbers a and b, a real matrix of type RMatrix and a vector of type double[] are required, from which the domain and codomain dimensions are inferred. One subtle point here is the use of a virtual constructor (or factory method): The constructor of RFreeAffineMorphism is not directly invoked, but through a static method public static ModuleMorphism make(RMatrix A, double[] b); In the case where A is a (1×1)-matrix and b is a 1-vector, an instance of RAffineMorphism is created instead of the more general RFreeAffineMorphism. This principle is often used throughout the system. It allows the selection of the most adequate implementation based on the arguments. This flexibility would not possible with ordinary constructors. The foregoing discussion applies with the corresponding changes to the case where R is Zn, Z, Q, or C, instead of R.

Chapter 11

Forms and Denotators

The implementation of forms and denotators constitute the center of R U BATO C OMPOSER. Together they provide the data model of the system. In general, all data that is passed on in R UBATO exist as denotators. Within a subsystem, pieces of data represented by denotators may certainly be transformed to any one of the many forms of representation devised by computer scientists and particularly suited for the task at hand. But whenever transformed or generated data leaves a subsystem, it is as a denotator. In this way, a common basis for accessing and distributing information is guaranteed. This has also been one of the objectives of XML. However, the superior mathematical foundation of forms and denotators as functorial concepts qualifies them much more for the tasks occurring mathematical music science, which deals with high-dimensional spaces. For the following discussion it is useful to recall and quickly summarize the idea behind forms and denotators, which has been extensively discussed in chapter 5: Forms are highly structured mathematical spaces, whereas denotators are objects in forms, i.e., points in such mathematical spaces. An important qualification concerns the notion of point, which is replaced by the notion of general addressed point, a concept which follows from so-called functoriality.

11.1 Requirements A glance at the discussion in chapter 5 suggests a few elements that will be needed for the implementation of forms and denotators. The several types Simple, Limit, Colimit, Power, and an additional type List, call for a five-fold branching of classes from a common superclass. The same applies to denotators. The additional type List is a concession to the convenience of list data types in computer programming. In terms of com105

106

11 Forms and Denotators

puter engineering, lists are more basic than the sets of type Power. Lists do not add anything fundamentally new: one way of expressing a list is as a set of pairs, with first coordinate an order number and second coordinate the list element, with the provision that there are no duplicate order numbers. Nevertheless, it would be an unseemly omission not to treat lists in a concrete as opposed to an abstract environment. Forms of the types Limit and Colimit require diagrams to describe how their factor and cofactor forms, respectively, are connected through morphisms. These diagrams are implemented through the class FormDiagram. However, the handling of the constraints implied by Limit form diagrams and the equivalence relation implied by Colimit form diagrams is not yet implemented. This is the subject of future work and requires extensive research in order to develop the exact semantics for a correct realization. Therefore Limit and Colimit types are currently used in their simpler guise as products and coproducts. Addresses, on the other hand, are fully implemented. Addresses arise from the module morphisms embedded in the denotators of type Simple. The domain of the morphism determines the address of the Simple denotator. The common address of the coordinate denotators determines the address of denotators of type Limit, Colimit, Power, and List. Notwithstanding the circularity involved through the name of a form being a denotator itself, the discussion starts with the implementation of forms. It is therefore necessary to mention a few details that are only fully discussed later on. As we shall see, names are required to have a determined form NameForm of type List, which is handled specially. We have opted for names to be restricted to this form instead of allowing arbitrary forms as intended by the theory. The reason for this choice is to avoid any issue with circularity. In the future, the use of general name forms may be made possible. It is a detail of implementation and should not affect the developer following the API. If not stated to the contrary, all the classes mentioned in this chapter reside in the package org.rubato.math.yoneda. In the following the word factor is often used interchangeably with the word coordinate for historical reasons.

11.2 Forms The current implementation of forms is not yet fully realized with respect to the complete theory as set forth in [47]. However the supporting infrastructure is present and provides the hooks and interfaces necessary to fit in the missing pieces. In order not to confuse the understanding, the discussion will proceed assuming a simpler version of the actual implementation.

11.2 Forms

107

Not all existing methods are mentioned, only a subset large enough to give a self-sufficient overview. As always, the ultimate reference is the Javadoc generated from the sources, available in electronic form.

11.2.1 Form Class The five types of forms are implemented as five subclasses SimpleForm, LimitForm, ColimitForm, ListForm, and PowerForm, of the abstract class Form. Additionally the class NameForm derives from ListForm to provide the special form for name denotators. It has itself the name Name and can be treated as a form of type List. The common interface for the form classes is shown in figure 11.1 and a simplified UML diagram in figure 11.2.

public abstract class Form implements Comparable { public NameDenotator getName(); public String getNameString(); public int getType(); public String getTypeString(); public Denotator createDefaultDenotator(); public Denotator createDefaultDenotator(Module address); public int compareTo(Form other); public boolean equals(Object object); public int getFormCount(); public Form getForm(int i); public List getForms(); } Fig. 11.1: The interface implemented by all form types. Only the most important methods are shown.

Although the meaning of the methods should be obvious, a few details need to be clarified: getName, getNameString: The name of a form is, of course, a denotator. It is an instance of NameDenotator, but since this class derives from ListDenotator, it acts as denotator of type List. It is of practical use to have an ade-

108

11 Forms and Denotators Form name: NameDenotator getName() getType() getFormCount() getForm(int i) getForms() createDefaultDenotator()

SimpleForm

LimitForm

ColimitForm

PowerForm

ListForm

module: Module

forms: Form[]

forms: Form[]

form: Form

form: Form

getModule()

getForm(String label)

getForm(String label)

getForm()

getForm()

NameForm

Fig. 11.2: Simplified UML class diagram of the form class hierarchy.

quate string representation of the name, so the second method takes care of that.1 getType, getTypeString: The type of the form is one of the constants Form.SIMPLE, Form.LIMIT, Form.COLIMIT, Form.LIST, and Form.POWER. Each subclass of Form returns the appropriate constant. A string representation of the type is available through the second method. It is one of “simple”, “limit”, “colimit”, “list”, “power”. Note, that NameForm is nowhere treated as a type of its own. createDefaultDenotator: It is often useful to produce a “canonical” denotator of the given form. The address of the default denotator is determined by the address of its coordinates. A variant of this method takes an address as an argument. It produces a default denotator with exactly that address. equals: Two Form objects are equal, whenever their names are equal. This works since forms must be registered in the repository (see section 9.2) and therefore are uniquely defined by their names. compareTo: Comparison is first on the form type according to the number returned by getType, then on the form name. This works for the reasons set forth above. 1

In the following, NameDenotator will often be abbreviated to ND for reasons of space.

11.2 Forms

109

getFormCount: This returns the number of coordinate forms. The number is 0 for forms of type Simple, 1 for forms of type List and Power, and the number of coordinates for forms of type Limit and Colimit. getForm: Calling this method on a form of type Simple is an error. For types List and Power only the argument 0 is valid. Types Limit and Power require the argument to be between 0 and the number of coordinate forms −1. getForms: The returned list contains all coordinate forms in order. The number of forms in the list is given by getFormCount. The subclasses of Form each feature specific methods which are now presented. Also, some details on the particular implementation of the superclass methods are provided if necessary.

11.2.2 SimpleForm Class A form of type Simple does not have a coordinate form. Instead it specifies the codomain module for Simple denotators. The following additional methods are provided by this class: SimpleForm(ND name, Module m): A form of type Simple with the given name and the given module is constructed. Module getModule(): The module specified by the form is returned. The default denotator returned is a denotator of type Simple that is nulladdressed and contains the constant zero of the module. Null-addressed means that if the module is an R-module, then the address is R0 .

11.2.3 LimitForm and ColimitForm Classes A form of type Limit specifies a list of coordinate forms. They can be retrieved using the methods from the superclass Form. LimitForm(ND name, List forms): A Limit form is constructed with the coordinate forms as specified by the list forms in the order given by the list.

110

11 Forms and Denotators

LimitForm(ND name, List forms, List<String> labels): This alternative constructor allows the specification of a label for each coordinate form. Thus, the list of labels must have the same length as the list of forms. Form getForm(String label): This additional method allows the retrieval of a coordinate form by its label, if one has been assigned. Otherwise forms can only be retrieved by their positional number. Default denotators are created by recursively invoking the defaultDenotator method on the coordinate forms. The ColimitForm class is essentially the same as the LimitForm class. It also allows the specification of labels. The implementations of Limit and Colimit denotators are different, however.

11.2.4 PowerForm and ListForm Classes The implementation of forms of type Power is very simple. PowerForm(ND name, Form form): A form of type Power has only one coordinate form. Form getForm(): This additional method is provided for convenience. Is is equivalent to getForm(0). The default Power denotator is an empty set. Its address is Z0 by default. The second default denotator method can be used for specifying a different address. The ListForm class is essentially the same as the PowerForm class. The implementations of List and Power denotators are different, however. The Name form is the unique instance of the class NameForm derived from ListForm. It therefore needs to reference a coordinate form. This coordinate form is an ordinary Simple form referencing a ZStringRing. This determines the shape that the names of forms and denotators might have. They are sequences of strings, and can therefore be used to construct namespaces and qualified names.

11.3 Denotators The hierarchy of denotator classes is exactly parallel to that of forms. The root class is Denotator and the classes for the different types of denotators

11.3 Denotators

111

inherit from this abstract class as shown in the UML diagram in figure 11.4. The common interface is shown in figure 11.3.

public abstract class Denotator implements Comparable { public NameDenotator getName(); public String getNameString(); public Form getForm(); public int getType(); public Iterator iterator(); public Denotator at(ModuleElement element) public Denotator atNull(); public Module getAddress(); public boolean nullAddressed(); public Denotator changeAddress(Module newAddress); public Denotator changeAddress(ModuleMorphism morphism); public Denotator get(int[] path); public Denotator replace(int[] path, Denotator d); public Denotator map(int[] path, ModuleMorphism morphism); public ModuleElement getElement(int[] path); public ModuleMorphism getModuleMorphism(int[] path); public int compareTo(Denotator object); public boolean equals(Object object); public boolean isConstant(); public boolean hasForm(Form f); public Denotator namedCopy(NameDenotator name); public Denotator copy(); } Fig. 11.3: The abstract class extended by all denotator types. Only the most important methods are shown.

The methods involving a path argument as well as the methods related to addresses such as changeAddress and at will be discussed later in section 11.4. The other methods are briefly presented in the following. In general, all operations on a denotator are non-destructive, i.e., they do not modify the denotator in-place, but create new objects whenever necessary. This ensures referential transparency and avoids problems with aliasing. The term modify henceforth always has this meaning. There are also methods to perform in-place modifications, which can be used for efficiency reasons. However the use of these is dangerous and relies on the knowledge and responsibility of the programmer. They are therefore not described in this context. More information can be found in the Javadoc.

112

11 Forms and Denotators Denotator name: NameDenotator form: Form getName() getType() at(ModuleElement) getAddress() changeAddress(ModuleMorphism m) get(int[] p) replace(int[] p, Denotator d) map(int[] p,ModuleMorphism m) getElement(int[] p) getModuleMorphism(int[] p) copy()

SimpleDenotator

LimitDenotator

morphism: ModuleMorphism

factors: Denotator[]

getModuleMorphism() getElement() sum(SimpleDenotator d) difference(SimpleDenotator d) getInteger() getReal() getRational() getComplex() getString()

getFactor(int i) getFactor(String label) getFactors()

ColimitDenotator factor: Denotator index: int getFactor() getIndex() getLabel()

PowerDenotator

ListDenotator

factors: Denotator[]

factors: Denotator[]

getFactorCount() getFactors() getFactor(int i)

getFactorCount() getFactors() getFactor(int i)

NameDenotator

Fig. 11.4: Simplified UML class diagram of the denotator class hierarchy.

getName, getNameString: These methods correspond to their counterparts in the Form interface. The comments made there apply here too. getForm, getType, hasForm: The method getForm returns the form of the denotator and getType its type. Observe that we always have getForm().getType() == getType() for every Denotator instance. The hasForm method provides a convenient way for checking the form of a denotator. iterator: This method implements the Comparable interface and allows the use of denotator objects in the Java 1.5 for statement to iterate over the coordinates of a denotator. For example, if myDenotator is a denotator of type Power, then for (Denotator d : myDenotator) { ... d ... } assigns each element of the set of coordinates set to d in the body of the loop. getAddress, nullAddressed: The address, an instance of Module, of the denotator is returned by getAddress. For convenience, the predicate nullAddressed is provided which returns true whenever the address of the denotator is of the form R0 , where R is a ring.

11.3 Denotators

113

copy, namedCopy: It is sometimes useful to work directly on a copy of the denotator. A deep copy is created by copy, i.e., the entire denotator tree is duplicated. The name of the copy is the same as the name of the original denotator. A new name can be assigned by using the namedCopy variant. equals: Two denotator are equal, whenever their forms and their names are equal, and, depending on the types, the coordinates are equal. This means that for Limit and Power denotators, the coordinates are equal, in the case of List and Power, the denotators are equal as lists and sets, respectively. Simple denotators are equal whenever the module morphisms are equal. compareTo: Comparison first considers the order of the forms of the denotators to be compared. Then the names are compared, and, if there is still equality, the coordinates are compared. In the case of Limit, List and Power denotators, lexicographical order is used. For Power denotators, this is well defined, since the coordinates are internally ordered. Each type of denotator (as a subclass of Denotator) provides a few additional methods that are specific to the type.

11.3.1 SimpleDenotator Class A denotator of type Simple wraps a module morphism. Its address is the domain of the morphism. The codomain of the morphism must match the module of the Simple form of the denotator. The module morphism is retrieved using the getModuleMorphism method. The getElement evaluates the morphism at the zero of the domain module and returns the result. In the case of constant morphism, this is simply the constant element. Simple denotators can be created using various constructors: SimpleDenotator(ND name, SimpleForm f, ModuleMorphism m): This is the most general constructor. The denotator is completed determined by its name, its form and the morphism it contains. SimpleDenotator(ND name, SimpleForm f, Module a, ModuleElement el): This constructor is useful for building constant Simple denotators. The constant is specified as well as the address a. SimpleDenotator(ND name, SimpleForm f, ModuleElement el): If one only cares about the constant, this constructor may be used. It automatically sets the address to R0 , whenever the module of the form is an R-module, in short, it makes the denotator null-addressed.

114

11 Forms and Denotators

For convenience, a number of access methods are supplied, for example, getInteger which returns an int. They allow quick handling of numbers and string. These access methods, however, only succeed if the method getElement returns a module element of the correct type, otherwise an exception is thrown. In particular, getInteger succeeds only if getElement returns an instance of ZElement or ZnElement. It is the responsibility of the programmer to ensure the validity of the access. Module operations such as sum and difference can be applied directly to Simple denotators, provided that the addresses and codomains of both operands are equal.

11.3.2 LimitDenotator Class The coordinates of a Limit denotator can be retrieved using the following methods: getFactor(int i): The i-th coordinate is returned. getFactor(String label): The coordinate with the given label is retrieved. The label must be among those specified by the Limit form of the denotator. getFactors(): A list of the coordinates is returned. Of course, this list is also available through the iterator as discussed above. There are two constructors: LimitDenotator(ND n, Module addr, LimitForm f, List c): A Limit denotator with the given name and form is constructed. The coordinates c are modified to have the specified address. If this is not possible, an exception is thrown. The number of coordinates may be different from the number required by the form. If there are more than required, the superfluous coordinates are ignored. If there are less than required, the remaining are filled up with default denotators of the respective forms. LimitDenotator(ND n, LimitForm f, List c): This constructor is similar to the first one, except that from the provided coordinates c a common address is computed, which is then the address of the resulting denotator.

11.3 Denotators

115

11.3.3 ColimitDenotator Class A Colimit denotator contains a single coordinate denotator, retrievable using getFactor. This coordinate belongs to an index corresponding to one of the positions in the denotator’s form. This index is available through getIndex. If a label has been assigned to the index, it can be retrieved using the method getLabel. Two constructors are available, analogous to those of LimitDenotator: ColimitDenotator(ND n, Module a, ColimitForm f, int i, Denotator d): A denotator with the specified name and form is constructed. The single coordinate is assigned to the given index. The coordinate is modified to have the given address a. ColimitDenotator(ND n, ColimitForm f, int i, Denotator d): This is similar to the first constructor. However, the address will be simply the address of the coordinate.

11.3.4 PowerDenotator and ListDenotator Classes Denotators of type Power and of type List are almost identical. However, in contrast to Limit denotators, the coordinates of Power are always sorted and the duplicates removed. This ensures that Power denotators are in a canonical form. The retrieval of the coordinates is done through the following access methods: getFactorCount(): The number of coordinates is returned. getFactor(int i): The coordinate at position i is returned. This has a different meaning for List denotators and Power denotators. The order of the coordinates in a List denotator is determined by construction of the list, while the order of the coordinates of a Power denotator is the canonical one. getFactors(): A list of all the coordinates is returned. Of course, the iterator introduced in the superclass can also be used. The constructors for both Power and List denotators are similar to the constructors of Limit denotators except for the arbitrary number of coordinates. We therefore omit their presentation. Names are handled through the NameDenotator subclass of ListDenotator. A name denotator should be regarded as a Limit denotator (and thereby as

116

11 Forms and Denotators

an instance of ListDenotator) for all practical purposes. The coordinates of a name denotator are regular Simple denotators having the regular String form. This concludes the discussion of the implementation of forms and denotators. Remember that, while some details have been provided, the full extent is still hidden and is far more complicated, because of the provisions made for future extensions adopted from the full theory.

11.4 Tools and Operations The above discussion dealt with the primitive access and construction methods of the classes for forms and denotators. Some utilities are provided by helper classes for constructing forms and denotators in a more comfortable way. Other operations are also implemented by classes external to the form and denotator hierarchy in order not to overload the form and denotator interfaces. Finally a few operations provided by the classes themselves relating to paths and addresses have been left out of the discussion above. They are, in fact, among the most important and are presented in detail in this section.

11.4.1 Construction of Forms and Denotators For the construction of forms and denotators, the correct class constructor must be called and the construction itself is often more involved than necessary. To make this easier, two utility classes are provided in the org.rubato.logeo package: FormFactory for building forms and DenoFactory for creating denotators. The FormFactory provides a number of static methods for constructing forms of the various types. Since they are all located at a single place, there is no need to access the various subclasses of Form directly for the simple creation of forms. The methods describe themselves through their names, which are of the shape makeTypeForm(arguments), where Type is Limit, Colimit, Power, List, or one of several names of modules for the creation of Simple forms. The argument list includes the name (as a string) and, for non-Simple types, one or more coordinate forms. The following lists the available methods: SimpleForm makeZModuleForm(String name) SimpleForm makeZnModuleForm(String name, int p) SimpleForm makeRModuleForm(String name)

11.4 Tools and Operations

117

SimpleForm makeQModuleForm(String name) SimpleForm makeCModuleForm(String name) SimpleForm makeZStringModuleForm(String name) SimpleForm makeModuleForm(String name, Module module) LimitForm makeLimitForm(String name, List forms) LimitForm makeLimitForm(String name, Form ... forms) ColimitForm makeColimitForm(String name, List forms) ColimitForm makeColimitForm(String name, Form ... forms) PowerForm makePowerForm(String name, Form form) ListForm makeListForm(String name, Form form)

The class DenoFactory has a similar purpose, this time for the construction of denotators. The methods are all called makeDenotator and each type is provided in three versions for different ways of specifying the name of denotator: either as a name denotator, or a string, or none, in which case the denotator is anonymous. Denotators of the types Limit, Power and List are constructed using the methods that allow a list of coordinates denotator in the arguments. Colimit denotators are constructed using a set of special methods, since an index has to be provided. For the rest, the usage should be clear from the respective arguments. SimpleDenotator makeDenotator(NameDenotator name, Form form, double d) SimpleDenotator makeDenotator(String name, Form form, double d) SimpleDenotator makeDenotator(Form form, double d) SimpleDenotator makeDenotator(NameDenotator name, Form form, int i) SimpleDenotator makeDenotator(String name, Form form, int i) SimpleDenotator makeDenotator(Form form, int i) SimpleDenotator makeDenotator(NameDenotator name, Form form, int i, int p) SimpleDenotator makeDenotator(String name, Form form, int i, int p) SimpleDenotator makeDenotator(Form form, int i, int p) SimpleDenotator makeDenotator(NameDenotator name, Form form, Rational r) SimpleDenotator makeDenotator(String name, Form form, Rational r) SimpleDenotator makeDenotator(Form form, Rational r) SimpleDenotator makeDenotator(String name, Form form, Complex c) SimpleDenotator makeDenotator(NameDenotator name, Form form, String s) SimpleDenotator makeDenotator(String name, Form form, String s) SimpleDenotator makeDenotator(Form form, String s) SimpleDenotator makeDenotator(NameDenotator name, Form form, ModuleElement el) SimpleDenotator makeDenotator(String name, Form form, ModuleElement el) SimpleDenotator makeDenotator(Form form, ModuleElement el) SimpleDenotator makeDenotator(NameDenotator name, Form form, ModuleMorphism m) SimpleDenotator makeDenotator(String name, Form form, ModuleMorphism m) SimpleDenotator makeDenotator(Form form, ModuleMorphism m) Denotator makeDenotator(NameDenotator name, Form form, List ds) Denotator makeDenotator(String name, Form form, List ds) Denotator makeDenotator(Form form, List ds) Denotator makeDenotator(NameDenotator name, Form form, Denotator ... ds) Denotator makeDenotator(String name, Form form, Denotator ... ds) Denotator makeDenotator(Form form, Denotator ... ds)

118

11 Forms and Denotators

Denotator makeDenotator(NameDenotator name, Form form, int index, Denotator d) Denotator makeDenotator(String name, Form form, int index, Denotator d) Denotator makeDenotator(Form form, int index, Denotator d)

11.4.2 Paths Denotators are trees of a special type. In fact, Simple denotators are the leaves, while Limit denotators are ordinary nodes with n branches or subtrees, where n is the number of coordinates of the Limit denotator. Colimit denotators are nodes with the peculiarity that of the n branches, only one is actually present (i.e., the one given by the index). List and Power denotators are nodes with an arbitrary number of branches. For Power denotators we assume the canonical order, so that the branches of all nodes have a definite order. When a fixed denotator is provided, the path from the root to a particular leaf is clear and can be written down by numbering the branches of each node from 0 to n − 1, where n is the number of coordinates of the denotator at the node. This is all well, however, what if we are given a form (which can be regarded as the pattern for constructing a certain type of trees, the denotators), and based on that form and a node in that form, we want to access the node in a denotator of that form, corresponding to the form node. Whenever the nodes in the form tree are of the type Limit or Simple, there is no problem, since they correspond exactly to the nodes of this type in the denotator tree. However, for the other types, there is a problem: A form of type Colimit has n coordinate forms, a denotator of that form however has only one. A form of type Power or List has one coordinate, the corresponding denotators however arbitrarily many. This problem is solved in that if a path contains the number m for the coordinate at a Colimit node, then this number m must match the index in the node denotator, otherwise the path is invalid for the denotator at hand. At a Power or List node, the number m in the path must always be 0. We have then not a single path, but a set of paths for each element in the list or set. This is illustrated in figure 11.5. Whenever an operation should be performed along a path, it is performed along all the paths in the resulting set. A path is represented as an array [i0 , i1 , . . . im ] where each ij is the number of the branch to follow and, in particular, i0 is the number of the branch to follow at the root. Some operations that make use of paths are presented in subsection 11.4.3.

11.4 Tools and Operations

119

A

Path

a

Q

2

Q

B

C

D

Z

Z

{}

0

b

c

d

−3

42

{•, •, •}

E Q

0

e0

e1

e2

Q

Q

Q

F

G

f0

g0

f1

g1

f2

g2

Z

Z

7

23

−5

0

−1

−2

Fig. 11.5: In the form A on the left side, the path [2, 0, 0] is highlighted. Observe that, at the Power node, the path index must be 0. The right hand side shows the union of the corresponding paths [2, 0, 0] in the denotator a. Thus an operation that uses path [2, 0, 0] on the denotator a would effectively apply to the Simple denotator at the leaves f0 , f1 and f2 . If along one path in the union, an invalid path resulting, for example, from a Colimit with wrong index, were encountered, this element in the union would simply be ignored by the operation.

11.4.3 Module Mapping and Structural Replacement Obviously module morphisms can be applied to denotators of type Simple. Consider a Simple denotator a of form A containing a morphism f : M → N, where M is the address of the denotator and N is the module of the Simple form A. Then applying a module morphism g : N → N to a results in a new denotator a′ containing the morphism g ◦ f. Observe, that the map to be applied must always have the function type N → N, where N is the module of the Simple form. The method map(ModuleMorphism m) of SimpleDenotator implements this operation. This is a special case of a morphism of forms, namely a morphism of the type F → F, where F is a form of type Simple. By using a path, such maps can be applied to the leaves of a denotator. In this case the method map(int[] path, ModuleMorphism m) of the root class Denotator is used. The path is an array of integers as defined above. For the denotator a of figure 11.5 using the path [2, 0, 0] and the morphism f : Z → Z : x 7→ 2x + 3, this means that a new denotator a′ is constructed, which is the same as a, except that instead of the integers 7, −5, and −1 in the leaves at the end of the paths, it contains the integers 17, −7, and 1.

120

11 Forms and Denotators

It is clear that the argument path must be compatible, in that it ends at a Simple, and therefore leaf, node. With the knowledge gained from the previous paragraphs, the methods getElement(int[] path) and getModuleMorphism(int[] path) should be clear. The requirements on the path are the same as above. The first method returns the element at the leaf node by calling getElement() on it, the second returns the module morphism by calling getModuleMorphism(). Currently, it is required that the path union must be a singleton (meaning that Power and List denotators may occur along the path, with the restriction that they contain a single element each), so that the returned values are uniquely defined. The method get(int[] path) is a generalized version of these, it returns whatever denotator is at the end node of the path. The replace(int[] path, Denotator d) methods substitutes the denotator at the node uniquely defined by the path with the supplied denotator d. This denotator first undergoes a change of address to fit in at the place, if necessary. This is a further example of a morphism of forms. All these methods throw an exception whenever a path is invalid, for example if a path index is out of range, or the path does not end in a Simple denotator for the map method.

11.4.4 Reforming Often, it happens that one has a denotator of a given form F at hand, but needs a denotator of another form G. If one is lucky, the form tree of F has exactly the same shape as the form tree of G, and the leaf nodes have the same modules, i.e., the trees are exactly the same, if the labels at the nodes denoting the forms names are omitted. In this case the functionality in the package org.rubato.logeo.reform comes handy. The class Reformer provides a method for transforming a denotator of form F to a denotator of form G if possible (thus reforming the denotator). A reformer is an object created using the static factory method make(Form from, From to) which does its job using the reform(Denotator d) with the expected result. The make method returns null if no appropriate reformer can be created for various reasons such as difference of structure, modules, etc. An example of this type of structural equivalence is shown in figure 11.6. The reformer system is however capable of much more. Not only can forms be mapped that are exactly the same as trees except for the names, but also certain transformations are automatically applied to make the forms match. This capability is limited, of course, but for many forms, which are slightly “out of tune”, reforming may prove very useful.

11.4 Tools and Operations

121

A

T

⇔

{}

{}

B

U

Q

Q

C

D

E

V

W

X

R

Q

Q

R

Q

Q

F

G

Y

Z

R

C

R

C

Fig. 11.6: Structural equivalence of the two forms A and T: only the form names are different.

One transformation handles different modules at corresponding leaf nodes. A Simple denotator with codomain module M can be mapped to a Simple denotator with codomain module N using a canonical module morphism M → N if such a morphism exists (see section 10.2). Other straightforward transformations map Power types to List types, and vice versa. Of course, mapping a List denotator to a Power denotator loses the explicit order and duplicates, whereas the direction from Power to List results in a canonically ordered list. Under certain conditions Limit denotator can be mapped to a Simple denotator and vice-versa. A free module in a Simple denotator is split and distributed among the coordinates of the Limit denotator. In the other directions the coordinates of a Limit denotator are gathered in a single free module to produce a Simple denotator. Canonical module morphisms are applied whenever necessary. An example of such a transformation is shown in figure 11.7. To a certain extent entire subtrees consisting of Limit nodes can be reorganized to make trees match. This is done by unlinking and relinking branches within the tree. However, permutations are currently not considered at all, because of the combinatorial complexity involved in the computation, therefore the order of the leaves is left unchanged. This type of transformation is illustrated in the example of figure 11.8.

122

11 Forms and Denotators

A

E

⇔

Q

R6

B

C

D

R

Q3

R2

Fig. 11.7: A denotator of form A can be reformed to a denotator of form E. The reverse direction does also work. The move to the denotator of form C involves a loss of information, however, because of the casting from R3 to Q3 . This also shows that the reforming process is generally not an isomorphism, although this may often be the case.

A

X

⇔

Q

Q

B

C

D

E

Y

Q

Q

aQ

bQ

Q

D

E

F

G

H

Z

H

aQ

bQ

cQ

dQ

eQ

Q

eQ

U

V

cR

dR

Fig. 11.8: The form A is transformed to X by reorganization of subtrees. The forms D, E and H occur in both A and X at the corresponding locations, and need therefore not be reformed. The correspondence between the leaves is indicated using pre-superscripts. See caption of figure 11.7 on the transformation from Q to R.

These are the most important transformations available through the reformer system. There are other transformations between Limit and Power, Limit and Colimit, etc. However, they are generally more limited and require very particular conditions to be successful.

11.4 Tools and Operations

123

Much more complicated transformations with arbitrary omissions and additions are not supported, first, because of computational complexity (the problem is akin to an undecidable word problem), second, because the result is in most cases not unique and therefore difficult to predict. The operation of reforming denotators of form F to denotators of G is a further example of a morphism of forms, in this case the morphism has type F → G.

11.4.5 Address Changing In subsection 11.4.3 a particular mode of transformation has been introduced, where a module morphism is applied to the codomain of a Simple denotator. It is natural to apply this idea to the domain as well. There is, however, no symmetry between the two modes, contrary to appearances. The first, module mapping, requires the selection of a particular Simple node and must be of the function type N → N where N is the codomain module of the selected node. The second applies to the whole denotator and must be of the form M → N where N is the address of the denotator and M is an arbitrary module. Such a transformation is called address change, since the address of the denotator N is changed to the new address M. For denotators of type Simple containing a morphism f : M → N, an address change by g : L → M, where L is the new address, results in a new denotator of type Simple containing the morphism f ◦ g : L → N. For denotators of types other than Simple, the address change is performed recursively on their coordinates, thus resulting in a new denotator of the same type, but having the new address. The Denotator root class features several methods to perform address changes. They implement various levels of generality, and, in fact, can all be implemented in terms of the first one: changeAddress(ModuleMorphism morphism): This is the most general address changing method. It is the straight implementation of the simple algorithm outlined above. It is recursively called for denotators of all types, except Simple, in which case the specified morphism is composed with the morphism contained in the denotator. changeAddress(Module newAddress): This method creates a canonical morphism from the specified new address to the existing address and calls the first method with this morphism as argument.

124

11 Forms and Denotators

at(ModuleElement element): Evaluation of a denotator at a particular element from the address module is equivalent to an address change by a constant morphism with a null module as the new address. As an example, consider a Simple denotator with morphism f : R → R and f(x) = 2x + 3. The application of the method at(3) results in a new denotator with constant morphism g : R0 → R with g(x) = 9. atNull(): This method implements the special case of the at method, where the argument is the zero element of the address module. With the previous definitions, the application of atNull() results in a denotator with constant morphism h : R0 → R and g(x) = 3. The use of constant morphisms to specify constant denotators is important enough to handle in a special way. The implementation of Simple denotators uses module elements to represent constant morphisms to increase the efficiency of this special, but common, case. Of course, to the outside, no difference can be seen between the two internal representations.

11.4.6 List and Set Operations Denotators of the types List and Power admit the usual list and set operations. These operations are provided in the package org.rubato.logeo by the utility classes Lists and Sets. The Lists class contains the following functions: ListDenotator ListDenotator ListDenotator ListDenotator ListDenotator ListDenotator ListDenotator

concat(ListDenotator d1, ListDenotator d2) concat(ListDenotator ... denoList) appendElement(ListDenotator d, Denotator element) prependElement(ListDenotator d, Denotator element) sort(ListDenotator d) sort(ListDenotator d, Comparator c) removeDuplicates(ListDenotator d)

The names of the methods reflect their function, which should be obvious. A few general remarks are in order, however. The concatenation methods only require that the argument List denotators have the same coordinate form. The resulting denotator will have the List form of the first argument. In general, necessary address changes are performed automatically. The removal of duplicates does only make sense if the argument denotator is sorted. Similarly, the Sets class provides Boolean operations, amongst others:

11.4 Tools and Operations

125

PowerDenotator PowerDenotator PowerDenotator PowerDenotator PowerDenotator PowerDenotator PowerDenotator PowerDenotator PowerDenotator

union(PowerDenotator d1, PowerDenotator d2) union(PowerDenotator ... denoList) intersection(PowerDenotator d1, PowerDenotator d2) intersection(PowerDenotator ... denoList) difference(PowerDenotator d1, PowerDenotator d2) difference(PowerDenotator ... denoList) symmetric(PowerDenotator d1, PowerDenotator d2) symmetric(PowerDenotator ... denoList) cartesian(PowerForm resForm, PowerDenotator d1, PowerDenotator d2) PowerDenotator addElement(PowerDenotator d, Denotator element) PowerDenotator addElements(PowerDenotator d, Denotator ... els) PowerDenotator addElements(PowerDenotator d, List els) boolean subset(PowerDenotator d1, PowerDenotator d2) boolean contains(PowerDenotator p, Denotator d) The remarks that have been made about the methods of the Lists class also apply here, with appropriate adjustements.

Chapter 12

Tools and Utilities

To put the mathematical framework discussed above to work in the real world, support from the programming environment is needed. This includes the implementation of low-level mathematics, the handling of input and output and the management of the objects that are created in the universe.

12.1 Low-Level Mathematical Tools Some mathematical foundations for doing algebra are not provided by the standard library of JDK. These are, for example, rational and complex numbers, numerical matrixes, etc. It is useful to summarize some additions provided by R UBATO C OMPOSER, since they are important for the developeruser interested in doing mathematical computations.

12.1.1 Numbers The Java package org.rubato.math.arith contains the missing implementations of rational and complex numbers. The Rational and Complex classes implement the standard Number interface and are thus made to fit in the Java library. Modular arithmetic is not as well supported by the JDK as it should be. In particular the modulo operation must be reimplemented to deliver the expected behavior. This and other number-theoretic functions such as extended gcd are contained in the utility class NumTheory.

127

128

12 Tools and Utilities

The implementation of elements of string rings of the form R is provided for the cases where R is Zn , Z, Q, R, or C.

12.1.2 Matrixes The JDK does not include any matrix computation package. Instead of using a third-party package, it has been decided to provide just the needed functionality for matrixes over the rings Zn , Z, Q, R, or C, in the package org.rubato.math.matrix. The most important methods are the matrix ring operations and matrix inversion for the case of square matrixes over a field. Matrixes are mainly used for the realization of affine morphisms.

12.2 Repository and Predefined Universe In figure 9.2 the need for a system repository that manages objects in the R UBATO universe has been declared. The global or system repository is a unique (or singleton, in the language of software patterns) dictionary-like object that manages all forms created in R UBATO C OMPOSER. This means that, whenever a new form is created, it must be registered in this repository. Once a form is registered under its name, it cannot be changed anymore. This ensures both consistency and efficiency, since it guarantees that no two forms of the same name exist throughout the running time of the system, and no form of a given name can disappear, thus leaving dangling references. The repository also maintains several namespaces that map names to objects. Apart from forms, the supported kinds of objects are modules, elements, morphisms and denotators. Denotators are registered using their intrinsic names (except for anonymous denotators, which cannot be registered), whereas the names of modules, elements and morphisms are arbitrary. The Repository class and its instance systemRepository are defined in the package org.rubato.base. Figure 12.1 shows its interface. The difference between the Builtin and the non-Builtin versions is that an object registered as built-in is to be regarded as predefined and it can be assumed that it will always be available, when the R UBATO C OMPOSER GUI starts up. Also, when the contents of the repository are dumped using toXML (see section 12.5 below), built-in objects are omitted, since they need not be reconstructed from an external description. Among the predefined (built-in) forms and denotators are the following: Simple forms for numbers and strings, as well as a Boolean form with the instances True and False:

12.2 Repository and Predefined Universe

129

public class Repository { public static Repository systemRepository(); public Form register(Form form); public Form registerBuiltin(Form form); public boolean isBuiltin(Form form); public Denotator register(Denotator d); public Denotator registerBuiltin(Denotator denotator); public boolean isBuiltin(Denotator denotator); public Form getForm(NameDenotator name); public Form getForm(String name); public List getForms(); public Denotator getDenotator(NameDenotator name); public Denotator getDenotator(String name); public List getDenotators(); public public public public public

PowerForm autogenPowerForm(Form baseForm); ListForm autogenListForm(Form baseForm); LimitForm autogenLimitForm(ArrayList factors); ColimitForm autogenColimitForm(ArrayList factors); SimpleForm autogenSimpleForm(Module module);

public public public public

void registerBuiltinModule(String name, Module module); void registerModule(String name, Module module); Module getModule(String name); List<String> getModuleNames();

public void registerModuleElement(String name, ModuleElement el); public ModuleElement getModuleElement(String name); public List<String> getModuleElementNames(); public public public public

void registerModuleMorphism(String name, ModuleMorphism m); ModuleMorphism getModuleMorphism(String name); List<String> getModuleMorphismNames(); List<String> getModuleMorphismNames(Module dom, Module cod);

public void toXML(XMLWriter writer); } Fig. 12.1: The interface of the Repository class with a selection of the most important methods.

130

12 Tools and Utilities

Integer :. Simple(Z) Rational :. Simple(Q) Real :. Simple(R) Complex :. Simple(C) String :. Simple(Z-String) Boolean :. Simple(Z2 ) False :@ Boolean(0) True :@ Boolean(1) Real vectors for dimensions up to 12: Vector2D :. Simple(R2 ) Vector3D :. Simple(R3 ) ... The simple Score form: Onset :. Simple(R) Pitch :. Simple(Q) Loudness :. Simple(Z) Duration :. Simple(R) Voice :. Simple(Z) Note :. Limit(Onset, Pitch, Loudness, Duration, Voice) Score :. Power(Note)

A special mention needs to be made of the autogen methods: Often a computation starts with objects of determined forms and creates a result that is of a type that combines the forms involved. When no obvious candidate for such a form is readily available (for example, by explicitly specifying the result form), a unique form needs to be created that satisfies the requirements. For example, if a form of type Power with coordinate form f of name F is wanted, the method autogenPowerForm(f) delivers such a form with a generated name _Power(F) and, at the same time, registers it. The underscore _ is a reminder that this name is internally generated. Subsequent calls on the method with the same arguments will always return the same registered form. This and the other auto-generating methods guarantee that the generated form is unique for the given arguments. For each form type, there is an auto-generating method. It is possible to create and use new repositories beside the system repository. These can be useful for working in a temporary environment. Forms can then be created without affecting the global repository. When finished, the entries from the temporary repository are then simply moved to the system repository.

12.3 MIDI Sequencer and Synthesizer The package org.rubato.audio.midi provides tools that wrap the implementation of a MIDI sequencer and synthesizer as well as MIDI file I/O in

12.4 Scheme Interpreter

131

the JDK. The class MidiReader is used to read a standard MIDI file. Its most important method is getDenotator which returns a denotator of form Score. The conversion from MIDI to Score is currently very simplistic, because the simple definition of Score does not allow for advanced MIDI events, such as pedals or tempo changes. However, it is good enough so that MIDI files can be used to import raw musical fragments that may then be use for composing in R UBATO C OMPOSER. The counterpart of MidiReader to output is MidiPlayer. This class assumes the two tasks of converting a Score denotator to a standard MIDI file and to control the software MIDI sequencer and synthesizer to play a Score denotator. In both cases the voices of the Score denotators are mapped to different MIDI channels. The class allows setting the program (i.e., instrument) for each channel according to General MIDI instrument numbers and the player also allows interactively changing play-back tempo.

12.4 Scheme Interpreter Working with the R UBATO C OMPOSER system means that either the Java framework is used by a developer to create rubettes, plug-ins or otherwise to extend the system. Or, in the case of a non-programmer, a user creates networks of rubettes in the graphical interface in order to perform the intended task. There is a need for intermediate ways of use. This intermediate type of accessing functionality is often present in complex applications in the form of scripting. Scripting usually involves a high-level language capable of handling the application-specific objects in a simple way. In the case of R UBATO C OMPOSER, this high-level language is a version of Scheme, which is itself a dialect of Lisp. The implementation is hand-crafted and is neither optimized for speed, nor does it strictly follow the requirements of the current Scheme standard [35]. The advantage of this approach is that it made it possible to tightly integrate the language with the R UBATO universe and to dispense with many details normally essential to a full-fledged, stand-alone implementation. The Scheme interpreter is contained in the package org.rubato.scheme. Its implementation uses the ideas from [1]. The interpreter itself can be accessed as a listener from the Tools menu or as an instance of the Scheme rubette. The procedures specific to R UBATO C OMPOSER are described in section 20.12 of the manual.

132

12 Tools and Utilities

12.5 XML as File Format for R UBATO C OMPOSER Forms, denotators, and every other object of the R UBATO universe, including networks and rubettes, must somehow be stored to disk for archiving and exchange purposes. For example, the current state of R UBATO C OM POSER can be saved as a R UBATO C OMPOSER project file (where the file name usually has .rbo as extension). A textual representation of denotators and forms had already been devised before the development of R UBATO C OMPOSER in the form of the DenoteX format [47], which is also used on various occasions in this text. Since the types of objects that must be saved are much more extensive, and DenoteX was not designed to cope with this broadness, another way must be found. An obvious choice is XML, first, because the JDK already provides all the tools necessary to deal with parsing XML text, and, second, the objects can be mapped quite naturally to XML structures. All classes, the instances of which can be represented in XML, implement the two methods toXML(XMLWriter writer) and fromXML(XMLReader reader, Element element) from the interface XMLInputOutput. Each such class knows how to write a representation of itself to a writer and how to parse a representation of itself from a reader starting from the supplied XML element. Implementations of these methods generally make use of the utility classes and functions provided in the package org.rubato.xml. Although the XML representation used by R UBATO C OMPOSER follows certain structural principles it is not possible to design a static DTD (document type definition) or XML Schema, since it is truly extensible: every new type of object (module morphisms for example) or rubette (discussed in the next chapter) requires a specific representation via XML tags that cannot be anticipated by a frozen schema. The set of denotators onset :@ Onset(1.0) pitch :@ Pitch(64/1) loudness :@ Loudness(100) duration :@ Duration(5.0) voice :@ Voice(0) note :@ Note(onset, pitch, loudness, duration, voice)

in DenoteX notation is transformed to the following XML text, where the referenced forms are assumed to be predefined and their definition therefore not also included in XML: <MorphismMap type="ConstantModuleMorphismMap"> <Module type="ZFreeModule" dimension="0"/>

12.5 XML as File Format for R UBATO C OMPOSER

133

<ModuleElement type="RElement" value="1.0"/> <MorphismMap type="ConstantModuleMorphismMap"> <Module type="ZFreeModule" dimension="0"/> <ModuleElement type="QElement" value="64/1"/> <MorphismMap type="ConstantModuleMorphismMap"> <Module type="ZFreeModule" dimension="0"/> <ModuleElement type="ZElement" value="100"/> <MorphismMap type="ConstantModuleMorphismMap"> <Module type="ZFreeModule" dimension="0"/> <ModuleElement type="RElement" value="5.0"/> <MorphismMap type="ConstantModuleMorphismMap"> <Module type="ZFreeModule" dimension="0"/> <ModuleElement type="ZElement" value="0"/> As expected by anyone familiar with XML, the result is unwieldy, especially compared to the preceding compact notation. However, it includes every necessary information, respects addresses (in this case Z0 ) and other details needed for more general denotators, and is easily handled by computer implementations.

Chapter 13

Rubato Composer GUI

The R UBATO C OMPOSER GUI presents the mathematics of forms and denotators to the user in a comfortable way, using visual representations wherever possible. Whereas the low-level, programmer-oriented, application programming interface (API) allows the developer to create and manipulate the objects from the R UBATO C OMPOSER universe in whatever way he likes, the R UBATO C OMPOSER GUI offers a more restricted, but visually more appealing method for working with these objects, following the general pattern of data flow systems. Further, the user interface allows the creation and management of the various types of objects, i.e., modules, module elements, morphisms, forms and denotators, through a number of interactive tools. Thus, the system offers direct manipulation in addition to data flow tools, in this sense making it a hybrid implementation. A complete description of the inner workings of the GUI is beyond the scope of this text. Nevertheless, it is important to point at some details useful to the developer who wants to write components or extend the system itself. More information on the use of the R UBATO C OMPOSER GUI is provided in the manual in chapter 20.

13.1 Terminology A rubette is a component type that embodies a computation which consumes and/or produces denotators. The design of the computational core is left to the rubette developer, provided he respects a fixed interface. This interface is implemented as a proper Java interface Rubette. A rubette instance is the component object, an instance of a class implementing this interface. In short, there is an obvious correspondence between the pairs rubette–rubette instance in R UBATO C OMPOSER and class–object in Java. Often, the distinction between rubette and rubette instance is not made ex-

135

136

13 Rubato Composer GUI

plicit. Then the phrase pattern “. . . use a . . . rubette to . . . ” means “. . . use an instance of a . . . rubette to . . . ”. A rubette consists essentially of three parts: 1. A number of inputs, which may also be 0—in this case, the rubette only produces output. 2. A number of outputs, which may also be 0—in this case, the rubette only consumes input. 3. A run method, which is the computational core and is the place where the developer of a rubette implements the functionality of his rubette. In most cases, it retrieves the input denotators, if any, and creates output denotators, if necessary. The rubette developer may also specify two dialogs that allow the rubette user to customize the functionality (properties) and to visualize the data processed by the rubette (view). The schema in figure 13.1 illustrates the relationship between these parts. The exact Rubette interface and a thorough explanation of how to create new rubette types are given in section 20.6. Many predefined (or, equivalently, built-in) rubettes are implemented in the Java package org.rubato. rubette.builtin. These are always available after starting up R UBATO C OMPOSER. A description of all built-in rubettes is provided in section 20.4 and section 20.5. A network implements the data flow model. It contains a collection of instances of the various types of rubettes. It also specifies how the outputs of rubettes are connected to the inputs of other rubettes through objects called links. To perform the computation embodied in a network, it is run or executed. The principles of how this exactly works are explained in section 13.3. The user interface elements are based on Swing which is the platformindependent toolkit used by the Java Development Kit provided by Sun.

13.2 The Implementation of Networks Networks and rubettes generally follow the model-view architecture. A view is an object responsible for the visual representation, but does not contain specific computational functionality. A model is an object containing the computational core, but does not care about how it is visualized. In the case of rubette components, the architecture is threefold: 1. An instance of a class that implements the Rubette interface is the rubette proper. It is the functionality that the rubette developer creates.

13.2 The Implementation of Networks

137

Fig. 13.1: A schematic view of a rubette with the inputs and outputs, and the computational core (here seen as a fragment of actual Java code such as a rubette developer would create). Also shown are the optional view and properties dialog objects that are provided by the developer to allow further interaction with the application user.

2. An instance of the RubetteModel class contains an object of a class derived from Rubette. It is responsible for interfacing the rubette core with the network. It therefore contains also information about the links to and from the rubette. This is the model side of the model-view split. 3. A JRubette instance is a visual component. Such an instance is uniquely tied to an instance of RubetteModel, and vice versa, and is responsible for drawing the component in a network. This is the view side of the model-view split. The architecture for networks is two-fold, since there is no network analogue to the Rubette interface. A NetworkModel object describes the network of interconnected instances of RubetteModel and a JNetwork object is the visual component responsible for drawing the network containing rubette components and links. Observe the use of the J prefix, which follows the convention used for Swing components. The JLink–Link pair works analogously. Networks themselves are contained in a structure called JNetworkContainer which manages multiple instances of JNetwork in a tabbed container style. Thus, several networks may exist at the same time.

138

13 Rubato Composer GUI

The complete functionality visible to the user is managed by a unique instance of JComposer which is the top-level user interface structure. The important parts are visualized in the UML diagram of figure 13.2. A relationship not obvious from the diagram is the following: The JNetwork– JRubette–JLink triple as a whole acts as a view for the NetworkModel– RubetteModel–Link model triple. JComposer rubetteList: JRubetteList networkList: JNetworkList problemList: JProblemList messageLog: JMessageLog startRun() networkContainer

JNetworkContainer networkList: JNetworkList addJNetwork(a: JNetwork, b: String) removeJNetwork(a: JNetwork) newJNetwork() JNetwork: getCurrentJNetwork()

jnetworks 0..*

JNetwork NetworkModel networkModel

name: String computeDependencyTree()

addLink(a: JLink) removeLink(a: JLink) addRubette(a: JRubette) removeRubette(a: JRubette)

links

0..*

JLink

link

rubettes rubettes 0..*

0..*

RubetteModel rubette: Rubette name: String location: Point

JRubette model

srcRubette

Link srcPos: int destPos: int

propertiesDialog: JDialog viewDialog: JDialog destRubette src dest

Fig. 13.2: UML class diagram of the most important parts making up the network and rubette component system of R UBATO C OMPOSER.

Figure 13.3 shows how these components as classes are related to the user interface elements.

13.3 Running a Network As we have seen, a NetworkModel object contains a complete description of how rubette instances are connected. The idea of the data flow model is that these rubette instances take the inputs from the incoming links and produce

13.3 Running a Network

139

Fig. 13.3: The R UBATO C OMPOSER user interface. Some of the visual interface elements are related to the classes that implement them.

the outputs on the outgoing links. This is what we call executing or running a network. The task of running a network is assigned to a Runner object which is placed in a separate thread, so that the user interface remains responsive. A runner executes the rubettes in a certain order by calling the method run on each of them, thereby taking care of distributing outputs and collecting problem reports. A network is essentially a directed graph, and the order of executing its rubettes must be such that, if a rubette A depends on a rubette B, A must be run after B. Thus, this graph is at the same time the dependency graph of rubette execution. Figure 13.4 shows an example network with ten rubettes. All outgoing links start at the bottom of a rubette and go into the tops of other rubettes. Part (b) of the figure shows the abstract dependency graph, where an arrow from A to E means that E depends on A. Using the method of topological sorting [20], a linear ordering ≺ such as shown in part (c) is generated. Here, besides the properties of linear orderings in general, the only rule is that, if we have A → B in the dependency graph, we must have A ≺ B in the linear ordering. In this order, the rubettes are eventually executed by the runner.

140

13 Rubato Composer GUI

A

C

B E

(a)

F

D

G

H

J

K

(b)

A≺B≺C≺E≺F≺D≺G≺H≺J≺K (c) Fig. 13.4: A network with rubettes A to K (a), the abstract dependency graph (b), and a particular ordering of A to K generated by a topological sort (c).

The resulting ordering is generally not unique. In this sense the choice of a concrete ordering is not deterministic and there should therefore not be put any reliance on a particular ordering. The requirement of a linear ordering and the method of producing such an ordering means that, currently, directed loops are not allowed in the dependency graph, i.e., no circular dependencies in the rubette network. This is not a fundamental limitation. However, allowing loops requires first to establish a method for breaking the circularity in order to reach the required linear ordering, along with a semantics to give some sense to the final result. The foregoing discussion treats of network execution as a whole. For purposes of efficiency, a second mode of execution, namely continuous execution, has been devised alongside full execution. In this mode, which can be toggled by the user through a switch in the user interface, the concept of changed rubette is introduced. A rubette is changed, whenever one of the following cases applies: (1) the properties have been altered by the user, (2) an ingoing link has been added to the rubette, or, (3) an outgoing link has been added to the rubette. In continuous execution, whenever a rubette changes, the subgraph of the complete dependency graph with the changed rubette is computed, and only the rubettes in this subgraph are executed. Continuing the previous example, figure 13.5 shows the case where the rubette F changes. The affected subgraph in part (b) of the figure is shown

13.4 Macro Rubettes

141

in black and the resulting ordering of the rubettes to execute is shown in part (c). The advantage of continuous execution is obvious, since only five of the ten rubettes are affected and therefore only half of the full execution is necessary.

A

C

B E

(a)

F

D

G

H

J

K

(b)

F≺G≺H≺J≺K (c) Fig. 13.5: A network with rubettes A to K, where the rubettes affected by a change in F are highlighted (a), the abstract dependency graph (b), and a particular ordering of the affected rubettes generated by a topological sort (c).

13.4 Macro Rubettes In computer science, an often encountered principle is that of black boxes, where, during implementation, part of the code is abstracted and only considered as an opaque box having certain specified inputs and certain specified outputs. The functional relationship of inputs and outputs is public, but the exact method of relating them, the actual inner working, is private and not visible. This principle can be applied to our networks, too. The concept used to implement this type abstraction is that of a macro rubette. A whole network is packaged into a macro rubette. Instead of the usual run method, such a rubette contains an instance of NetworkModel. The runner knows about this and whenever it encounters a macro, it suspends

142

13 Rubato Composer GUI

the current execution and executes the packaged network, before resuming the previous execution. New macro rubettes appear alongside the other rubettes. Whenever a macro rubette is placed into a network, a copy is made of the enclosed network model. In this sense, a macro rubette is similar to textual macro languages such as the familiar C preprocessor or the M4 macro language. Several instances of a particular macro rubette are therefore completely independent. A simple example shall illustrate the procedure of creating a macro rubette. Figure 13.6 shows a very simple network suitable for making into a macro

Fig. 13.6: A simple network with an instance of a MacroInput and MacroOutput rubette each. The dialog shows the configuration of the MacroInput rubette.

rubette. Two special rubettes are placed in the network, namely one instance of a MacroInput rubette and one instance of a MacroOutput rubette. The first specifies the input connectors of the future macro rubette, the second specifies the output connectors. The MacroInput rubette has been configured with two outputs (an input to a network corresponds to an output from a MacroInput rubette) which are named i0 and i1, respectively. The main rubette performs a Boolean operation on the two inputs, provided they are of the correct form. This network is packaged into a macro rubette, by invoking the command Create macro rubette from network in the network popup menu (by right clicking on the network background). The dialog in figure 13.7 allows the specification of the name of the macro rubette as well as other information.

13.4 Macro Rubettes

143

Fig. 13.7: To create a macro rubette at least a name must be specified.

The new macro rubette is added to the list of rubettes under the name of BooleanTest and can henceforth be used just like any other rubette (figure 13.8).

Fig. 13.8: The newly created macro rubette can be placed in networks just like any other rubette.

144

13 Rubato Composer GUI

13.5 Tools Most of this chapter has been about networks and rubettes. These are supplemented with various tools for the creation and manipulation of types of objects in the R UBATO universe. There are tools for the creation of forms and denotators and tools for creating module morphisms. The latter is an important part of the the R UBATO C OMPOSER GUI as a whole, since this is where the user defines the functions he uses. In chapter 10 the wealth of morphism types has been discussed. The challenge now is to provide a means for constructing such morphisms that is both “intuitive” and flexible. This is a difficult task, and very often there exists no unique solution for each problem set. One important type of morphisms, namely affine morphisms of the form R2 → R2 , i.e., affine transformations in the Euclidean plane, are particularly qualified for a graphical method of specification. This comes in handy, since every affine morphism of the functional type Rn → Rn can be expressed as a product of affine morphisms of the type R2 → R2 . Chapter 13 shows how an affine morphism can be specified through its linear part, a real 2 × 2-matrix, and its translation part, a real 2-vector. Optionally, the transformation can be described graphically (by opening an graphical view using the Graphical button). Here the transformation is indicated by showing how a unit square is transformed. The vertexes of the original and their corresponding images are colored, so that reflections do not get lost. For the same type of morphisms, another method is available. Here, instead of Affine, the type is set to Geometry (figure 13.10). A general affine transformation is now specified as a composition of several primitive transformation. The result is also shown graphically. It is also possible to define an affine morphism based on structural criteria instead of geometric properties. Figure 13.11 shows the dialog for specifying a morphism from Q4 to Q5 by indicating how each component from an element in the domain is mapped to a component of image element in the codomain. The resulting morphism is, of course, linear. Observe that a name is required before a morphism can be created. The new morphism is registered under this name in the global repository (see section 12.2) to be referenced later, and is therefore also saved as part of the project.

13.6 The Plug-In System The built-in rubettes live in the rubato package space (and are built and packed into a common JAR file) and are always loaded whenever the R U -

13.6 The Plug-In System

145

Fig. 13.9: An affine morphism in the Euclidean plane is specified by indicating how the vertexes of the triangle (0, 0), (1, 0) and (0, 1) are transformed, highlighted here with different colors. The triangle is extended to a parallelogram in this view. The transformation is defined by dragging the vertexes or the entire parallelogram to the desired positions.

BATO C OMPOSER GUI is started up. It can therefore be assumed that they are always present. Since rubettes are meant to be components that freely extend the R UBATO C OMPOSER system, there must be a way to load rubettes without disturbing the build process and the independence of the main JAR package.

This is accomplished by using a plug-in system that allows the packaging of external rubette code into separate modules, which are placed as JAR archive files in a determined location (on UNIX systems, for example, this is $HOME/.rubato/plugins) and which are automatically loaded on startup and added to the list of rubettes. Section 20.7.3 describes the procedure of creating a rubette plug-in from the developer’s point of view. It is also possible to create a generic plugins, by simply implementing the interface Plugin in package org.rubato. composer.plugin. It consists of the sole method init that will be called on

146

13 Rubato Composer GUI

Fig. 13.10: Another way to characterize an affine morphism is as the succession of various types of primitive transformations. The mappings are applied from top to bottom.

the startup and initialization of the GUI. Such a plug-in can be used for various purposes. For example, it can provide additional tools for the direct manipulation of objects, besides the ones already provided by R UBATO C OMPOSER. However, except for the external rubettes introduced above and the type of plug-ins described presently, the infrastructure is currently not advanced enough for more powerful extensions. The development of such an infrastructure is a topic for future work. Another special type of plug-ins provided alongside rubette plug-ins is devoted to module morphisms. This is especially useful, since it is not possi-

13.6 The Plug-In System

147

Fig. 13.11: A class of simple affine morphisms can be defined structurally, by specifying how the components are mapped. This type of morphism is called Shuffle in the R UBATO C OMPOSER GUI.

ble to foresee every type of morphism that a user may need and bundle it with R UBATO C OMPOSER. To create a module morphism plug-in, the following abstract class must be extended: public abstract class ModuleMorphismPlugin implements Plugin { public final void init(); public abstract ModuleMorphism getPrototype(); public JMorphismType getJMorphismType(JMorphismContainer c); public ModuleMorphism getModuleMorphism(Module dom, Module cod); public abstract boolean checkSignature(Module dom, Module cod); public abstract String getName(); }

148

13 Rubato Composer GUI

To package module morphisms as plug-ins a little more information than the simple init method is is needed. This information is provided by the methods of ModuleMorphismPlugin, to be implemented by the plug-in developer: • The object returned by getPrototype is a single instance of the morphism class provided by the plug-in. This instance is not used as such, but allows the class to be registered in the XML subsystem. This is needed, so that instances of this morphism type can be parsed from XML later on. • The method getJMorphismType is optional and returns a user interface component of type JMorphismType. This component provides the dialog that allows the construction of morphism of the new types from within the module morphism builder. This may be a simple entry form useful for the specification of numerical parameters. It may also be a more complex graphical input method, such as seen in figure 13.11 or in figure 13.9. • The method getModuleMorphism creates a morphism of the new type with the specified domain and codomain, if possible, otherwise it return null. • The method checkSignature returns true whenever a morphism of the new type with the specified domain and codomain can be created. • The string returned by getName is the text that will be displayed in the GUI whenever this type of morphism is referred to, for example in the type selection of the module morphism builder.

Chapter 14

Overview

We now move on from the theoretical aspects and implementation details of R UBATO C OMPOSER to the perspective of the application user. To the user, the R UBATO C OMPOSER GUI looks and feels about the same whatever platform is used. Currently the most extensively tested platforms are Linux running the Java Development Kit version 1.5 from Sun and MacOS X running the JDK version 1.5 provided by Apple. Apart from a few minor variations concerning the look and feel, no significant differences in the behavior of handling and running have been observed. In this final part of the discussion of R UBATO C OMPOSER, an illustrated tutorial tour gives some hints at how a user would proceed and construct a simple network, in this particular case using a functorial approach to work with twelve-tone series. Then several more extensive applications, involving the implementation of plug-in rubettes, are discussed in some detail. These plug-ins are the result of the work by two contributors, Karim Morsy and Florian Thalmann, who delved into the R UBATO framework for several months. During this time, many bugs have been discovered and fixed, but also ideas for improvement have been suggested and implemented in most cases. Finally, two chapters have been contributed by Guerino Mazzola and Florian Thalmann. The first describes the BigBang rubette and second presents a large-scale use of R UBATO C OMPOSER in computational music theory and composition, which demonstrates many of the features which have hitherto been applied to small examples only.

151

Chapter 15

A Tutorial

A step-by-step presentation of a simple procedure is the best way to illustrate the working of R UBATO C OMPOSER and to show how the theoretical concepts fit into the practical framework. In this tutorial, the aim is use the functorial approach to create a very rudimentary musical fragment, starting from a single non-zero addressed note that represents a dodecaphonic series. The rubettes introduced in this tutorial take a “low-level” approach and are intended to be employed by the mathematically skilled user. Currently, the bigger part of available rubettes is of this kind. In the long term, high-level rubettes will be developed that abstract from the mathematics and present an interface and employ terminology that appeal to the classically trained composer. They will work with musical motives, rhythms, or counterpoint, for example. Operations will then be expressed in terms of transposition, inversion, retrograde or augmentation, instead of (or in addition to) the mathematical transformations used in this tutorial. The following is an overview of the steps that make up the tutorial: 1. A single denotator n1 of form Note is created from its coordinate denotators of forms Onset, Pitch, Duration, Loudness and Voice. All these denotators are Z11 -addressed. 2. The denotator n1 is transformed by inversion at a given pitch and an augmentation, followed by a translation in time, yielding in a second Z11 -addressed denotator n2 . 3. The Z11 -addressed denotators n1 and n2 are both evaluated at the 12 canonical affine basis vectors of Z11 , resulting into two Z0 -addressed Score denotators s1 and s2 . 4. The set union of both denotators s1 and s2 is taken to produce a denotator s3 , again of type Score. 5. The denotator s3 is played using the built-in MIDI sequencer.

153

154

15 A Tutorial

All of the required forms are predefined in R UBATO C OMPOSER. They are: Onset:.Simple(R) Pitch:.Simple(Q) Loudness:.Simple(Z) Duration:.Simple(R) Voice:.Simple(Z) Note:.Limit(Onset, Pitch, Loudness, Duration, Voice) Score:.Power(Note) In the following, the tasks to perform often follow similar procedures. Each type of procedure will be presented in detail once. Repeated applications are then alluded to, but not presented again in detail, in order to keep the discussion reasonably short.

Step 1a — Preparing R UBATO C OMPOSER After starting up, R UBATO C OMPOSER presents itself with a main area in the middle, which will contain the networks, a list on the right where rubettes can be dragged to a network, and, at the bottom an information area, where status and error messages (problem descriptions) resulting from the execution of a network will be shown (figure 15.1).

Fig. 15.1: Empty main window on starting up R UBATO C OMPOSER (left). A new network has been created (right).

15 A Tutorial

155

The first action is to create a new network using the button . Additional networks can be created, which are then shown as tabs, but for this tutorial, only one network is needed.

Step 1b — Creating the denotator The twelve-tone series that makes up the denotator to be constructed is presented in figure 15.2. The only changing coordinates are Onset, Pitch

 












Fig. 15.2: The basic dodecaphonic series.

and Duration. The others, Loudness and Voice, are constant and taken to be 60 and 0, respectively. Before constructing denotator n1 , the coordinate denotators o1 , p1 , l1 , d1 , and v1 must be created. The easiest to start with are the simple denotators l1 and v1 , since they consist of constant, although Z11 -addressed, module morphisms. Denotator l1 of form Loudness: Open the Denotator Builder in the Tools menu. Select the Loudness form and set the address type to Non-Null. The required value is a module morphism which is specified by activating the Create button. When the module morphism builder dialog opens, select Z11 as domain. The morphism type is Constant and the constant value is 60 (figure 15.3). After activating the Create button a new denotator is built and registered under the name l1 (actually l1 since subscripts are not allowed in names) in the system repository. Denotator v1 of form Voice: This is very similar to the construction of l1 and proceeds in the same way as previously, with Voice instead of Loudness, and 0 instead of 60 for the constant value. The voice is a non-negative integer that can be mapped to a MIDI program later on when rendering the score. More interesting is the construction of the denotators o1 , p1 and l1 , since these are affine non-constant morphisms. Considering first pitch p1 , the idea is to map each basis vector of Z11 to one of the 12 notes of the series. Remember that in the affine space Z11 , the canonical basis consists of the 11 unit basis vectors of the linear space and the zero vector, making 12 basis elements in total. To represent the series, we ensure that the initial note g4 is mapped to the value 67 (according to the MIDI standard for

156

15 A Tutorial

Fig. 15.3: Creating the Loudness denotator l1 .

pitch numbers). This is the translation part of the affine mapping. The remaining 11 notes are specified as offsets from the initial note (an offset 1 representing a half-tone step, see also figure 15.4): mp : Z11 → Q : x 7→ (4, −1, −3, −2, 5, 7, 2, 3, 1, 6, 8) · x + 67.

Fig. 15.4: The pitches of the series as offsets relative to the first note.

Onset is defined similarly to pitch. The unit is taken to be the duration of a quarter note (the exact tempo is specified later when rendering the music). If the first note didn’t start at time 0, then a translation part with the initial offset would be required. We have mo : Z11 → R : x 7→ (1, 2, 2.5, 3, 4, 5, 6.5, 7, 8, 9, 10) · x. Finally, the definition of duration is exactly analogous to the definition of onset. The translation part gives the duration of the first note. The dura-

15 A Tutorial

157

tions of the remaining notes are relative to the first note: md : Z11 → R : x 7→ (0, −0.5, −0.5, 0, 0, 0.5, −0.5, 0, 0, 0, 1) · x + 1 It is convenient to define and name these morphisms separately. The process is common to all three, so only the definition of mp is presented in detail. The pitch morphism mp is the composition of two morphisms: the affine mapping mp,1 : Q11 → Q and the embedding mp,2 : Z11 → Q11 , thus giving mp = mp,1 ◦ mp,2 . The Module Morphism Builder is invoked from the Tools menu, first to create mp,1 , then mp,2 , and, finally, the composition mp . These three steps are shown in figure 15.5. This procedure is repeated for mo and md . For both morphisms the required embedding is Z11 → R11 . These definitions can be checked by evaluating each of the mappings mo , mp , and md at the basis vectors e0 = (0, 0, . . . , 0), e1 = (1, 0, . . . , 0), . . . e11 = (0, 0, . . . , 1) of Z11 and comparing the results with the notes from the series. The corresponding denotators o1 , p1 , and d1 are now created using the denotator builder in the same way as for l1 and v1 . The final step is the construction of n1 . After selecting the Note form in the denotator builder tool, the five coordinate denotators are each picked from the list to fill in the fields (figure 15.6). A glance at the Object Browser available from the Tools menu confirms the result of the preceding operations (figure 15.7). The browser provides a view to the system repository and all the registered objects. Besides the predefined denotator True and False of form Boolean, all the denotators constructed above are present. By selecting Module morphisms in the Object type panel, the module morphisms created previously will be listed too. All objects are shown in a more or less detailed textual version. The appearance of the Note denotator is quite complicated due to the use of non-constant morphisms.

Step 2a — Defining the transformations The denotator n1 will be transformed in two steps: 1. The pitch coordinate is transformed by the module morphism inverse : Q → Q, where inverse(x) = 134 − x. This corresponds to an inversion at g4 .

158

15 A Tutorial

Fig. 15.5: Creating the affine morphism mp,1 , the embedding mp,2 , and the composition mp = mp,1 ◦ mp,2 .

2. The onset coordinate is transformed by the module morphism augment-shift : R → R, where augment-shift(x) = 2 · x + 2. This corresponds to an augmentation by a factor of 2 followed by a shift in time by 2 quarter notes. Both morphisms, inverse and augment-shift, are constructed by the method already used above. After selecting Q for both domain and codomain of the first mapping, and R for the domain and codomain of the second mapping, the type of morphism to choose is Affine for both. For simplicity’s sake, the

15 A Tutorial

159

Fig. 15.6: Creating the Note denotator n1 by selecting its coordinates. The list shows the available choices for the voice coordinate.

duration is not given a transformation, so the durations in the result will not be stretched in the same way as the onsets. The result after applying first inverse and then augment-shift to the original dodecaphonic series is shown in common notation in figure 15.8.

Step 2b — Adding Source and ModuleMap rubettes to the network First, a Source rubette is placed in the network that has been created in step 1a. This rubette looks up a denotator by its name in the system repository and puts it on the unique output connector (figure 15.9). This rubette is connected serially to two instances of the ModuleMap rubette. The first instance is configured to map the Pitch coordinate of the Note denotator using the inverse morphism, the second instance is configured to map the Onset coordinate using the augment-shift morphism. This is done by selecting the path to the Simple form, where the mapping should be applied. The output of the second ModuleMap instance (labeled “ModuleMap #3”

160

15 A Tutorial

Fig. 15.7: The object browser showing a textual representation of the n1 denotator.

 


  












 








Fig. 15.8: The transformed dodecaphonic series. Observe that the durations are transferred without change from the original series, whence the rests.

in figure 15.10) is the transformed denotator n2 . The order of the mappings does not matter in this case, since they affect independent parts of the denotator.

Step 3 — Evaluating at basis vectors The denotators n1 coming from “Source #1” and n2 coming from “ModuleMap #3” are passed through an instance of an AddressEval rubette each. This rubette implements the address changing operation introduced in subsection 11.4.5. Here the special case of the at operation is used. Since addressed Note denotators are not suitable for playing, because they are made up of morphisms and not traditional points, it is necessary to generate null-addressed denotators that consist of traditional points in order to

15 A Tutorial

161

Fig. 15.9: The Source rubette is configured by selecting a denotator from the list of all available denotators in the system repository. If the Self-refreshable option is selected, the rubette checks on each run, if the denotator of the configured name has changed, and updates its output accordingly.

be able to hear anything at all. Both rubettes are placed and linked as seen in figure 15.13. Both are configured identically: The Z11 -addressed denotators at the input are evaluated at the 12 basis vectors of Z11 . The output form is set to Score, since the result of evaluating a denotator of form Note at a list of elements is a set or a list of denotators of form Note, and such a form is suitably provided by Score. The result is therefore one denotator of form Score each, henceforth called s1 and s2 , respectively. The module of the elements that the denotators are to be evaluated at is, of course, Z11 . Now one may enter each element using the input form presented in the bottom part of the dialog. In the special case treated here, the button Basis vectors generates all basis vectors of the affine space. In the case where the address is a 2-dimensional free module, such as the Euclidean plane R2 , it is also possible to specify graphically the elements used for evaluation. The button Graphical (grayed out in figure 15.11) displays a canvas representing the 2-dimensional space where points can be placed, moved or removed using mouse actions. If the result form is of type Power, the order of the elements in the list does not matter. In the case of

162

15 A Tutorial

Fig. 15.10: The source denotator n1 is passed through a ModuleMap rubette called “ModuleMap #2” which applies the transformation inverse, then through another ModuleMap rubette called “ModuleMap #3” which applies the transformation augment-shift.

List, however, the result of the evaluation is in the order specified by the list of elements, i.e., from top to bottom. Therefore the order of the elements can be adjusted using the Up and Down buttons.

Step 4 — Creating the union of s1 and s2 Both denotators s1 and s2 are conducted to a Set rubette (figure 15.12). This type of rubette performs set operations on its inputs. The number of inputs can be configured, but must be at least 2. The form of the input denotators must have type Power and the coordinate form must be the same for all. In this case s1 and s2 are of form Score and therefore satisfy the requirement. The operation needed here is Union which takes the set union of the coordinates of all input denotators and constructs an output denotator of the same form as the first input denotator form. This is, of course, Score in this case.

15 A Tutorial

163

Fig. 15.11: The configuration of the AddressEval rubette instance “AddressEval #2”. The list of elements has been generated by the Basis vectors command.

Other operations provided by the Set rubette include Intersection, Difference and Symmetric Difference. It is also possible to add an element to a Power denotator or to perform the big union or big intersection, whenever the unique input denotator has type Power of Power, i.e., it is a set of sets. The combined result in common notation looks as follows:

164

15 A Tutorial

Fig. 15.12: The Set rubette is configured with operation Union and two inputs.

 

 


 





 






      4






   Step 5 — Playing the result Finally a ScorePlay rubette is attached to the result from the union. After the network is executed by pushing the button , the final score is shown in the view dialog of the ScorePlay rubette and can be played via the Java software sequencer and synthesizer (figure 15.13). The play-back tempo can be adjusted using the slider at the bottom of the view. The Voices button opens a dialog, shown on the right side of figure 15.13, where the instrument of each voice can be set. The selection is from the collection of General MIDI instruments provided by the Java synthesizer. Play-back can be controlled using the typical tape deck buttons for playing, pausing and stopping, at the top of the view. A vertical line is constantly updated to show the current position of play-back. The synthesizer provided by the JDK is good enough for previewing the result. For high quality results it is recommended to export the generated denotator to a standard MIDI file by feeding it into a MidiFileOut rubette. This MIDI file can then be imported into synthesizer software driven by high-quality samples, or it can be sent to a hardware synthesizer via the

15 A Tutorial

165

Fig. 15.13: The final network.

Fig. 15.14: The view of the ScorePlay rubette showing the resulting score in a piano roll representation.

standard serial MIDI bus. During export, each voice of the score denotator is mapped to a channel in the MIDI file. ❦

166

15 A Tutorial

All the tasks and procedures presented in this tutorial have been of a rather low-level nature. Everything has been defined explicitly, from the basic module morphisms underlying the denotators to the matrixes responsible for the transformations. A composer certainly will prefer to work with higherlevel procedures using graphical input methods provided by feature-loaded rubettes. The following chapter will present some examples of this kind of higherlevel application in the form of a collection of rubettes. In the end, R UBATO C OMPOSER as an application intended to be used by composers, which are not programmers themselves, will grow by the development and availability of such rubettes.

Chapter 16

First Applications in Rubette Construction

The R UBATO C OMPOSER system that has been presented in past chapters is just a framework. To function as a full application, it is in need of rubettes for the various tasks that arise during the work of a composer or music theorist. Several people already have contributed by studying the system and developing components that implement the results from their respective fields of interest. Section 16.1 describes the work of Karim Morsy, a student of computer science at the TU München [57]. It treats of macro objects, which have already been mentioned in subsection 3.3.1. Section 16.2 and section 16.3 present some details of a part of the master’s thesis by Florian Thalmann, a former student of computer science at the University of Berne [68, 54]. This part provides a fresh look at the ornamental transformations implemented in presto.

16.1 Rubettes for Macro Objects The problem that macro objects address will be illustrated by an example. Consider the melodic fragment at left of figure 16.1. It consists of a trill, followed by an unadorned note. The compositional methods we have in mind include geometric transformations, in particular rotation. The result of a rotation by 15 degrees around the third note of the trill is shown in figure 16.1. This picture shows a literal rotation, which is, however, not possible in reality. The meaning of a rotation in spaces such as scores of notes has to be defined suitably, but such a definition is omitted here since it is not the point of this discussion. If we rotate further, say by 90 degrees, the result will become quite clusterlike. This may or may not be what the composer intended. In the end, however, the character of the trill will be completely lost and other effects 167

168

16 First Applications in Rubette Construction

Fig. 16.1: A melodic fragment (left), rotated by 15◦ (right).

will be introduced, which do not belong to the structure and are therefore probably unwanted. Thus, a flat list of notes does not fulfill the requirements of a structured musical score. Theories such as those put forth by Heinrich Schenker emphasize the role of many-leveled hierarchies, where a primitive skeleton called the “Ursatz” is successively fleshed out down to the lowest level of the hierarchy to result in the fully detailed composition. The above example is one of the simplest that can be used to illustrate a hierarchical structuring. The highest level contains two notes, where the first note expands to the trill during the “fleshing out”. The notes of the trills attached to the anchor notes are also called “satellite” notes for obvious reasons. Given such a hierarchy, every operation can now applied at various levels. For the example, this means that performing the rotation on the highest level produces the result in figure 16.2. The anchor note and the second note of the melodic fragment are rotated. The trill attached to the anchor note, however, retains its character.















Fig. 16.2: Rotation applied to the highest level of a two-level version of the example from figure 16.1.

The common Score form has no provisions for satellite notes. Therefore a special recursive form has to be used. This MacroScore form has already been introduced in subsection 3.3.1. For reference, it is repeated in figure 16.3. The rubettes implemented by Karim Morsy provide the means of handling MacroScore denotators. Two rubettes, ScoreToMacro and MacroToScore, deal with the conversion between Score and MacroScore denotators. In this context, a Score denotator is regarded as a macro score with only one level, i.e., all notes from the original score are adorned with empty satellite sets.

16.1 Rubettes for Macro Objects

169

MacroScore {} Node Q Note Q Onset

Pitch

R

Q

Loudness Duration Z

R

Voice Z

Fig. 16.3: The recursive MacroScore form.

Conversely, to transform a MacroScore denotator to a Score denotator, all satellites are discarded. Given a MacroScore denotator, any operation can be applied to the various levels. One particularly important operation, however, is specific to macro objects. It deals with question of how to reduce hierarchical depth. This operation is commonly called flattening and can be illustrated using lists such as they exist in the Lisp programming language: Consider a nested list (ab(cd(ef)g)h). This list can now reduced in depth by one level and results in (abcd(ef)gh). The procedure can be iterated, until the result does not change anymore (or has converged, in the language of mathematics). The final result is (abcdefgh). In the case of macro objects, this type of flattening is not only possible, but necessary. The situation is a little more complex, however. In the Lisp example, the objects of the nested lists have an absolute character in that they do not depend on their parents. On the other hand, satellite objects depend on their anchors in most cases. For example, the pitches and onset times of trill notes are relative to the anchor note. Simply integrating satellite objects with the level one step above does not produce the desired result. Instead, the satellite objects must be modified in some way to fit in the level occupied by the anchors. In the particular case of notes, this means that pitch and onset of the anchor note must be added to the satellite notes. In general, an arbitrary operation combining the anchor and each satellite note might be specified.

170

16 First Applications in Rubette Construction

The flattening operation is defined as follows (using the definition of the MacroScore form from above): [ Flatten(m) = Flatten(node), node∈m

Flatten(node) = (notenode , ∅) ∪ {(notenode ⊕ notek , Mk ) | k ∈ Mnode }.

The first line defines Flatten on a MacroScore denotator m, where the big union runs over its elements node of form Node. The second line defines the operation on a Node object. Here notenode and Mnode are the two coordinates of node. The operation ⊕ specifies how the anchor is combined with a satellite note. The Flatten rubette implements the flattening operation. Currently the operation ⊕ is restricted to the sum of coordinates Pitch, Onset, Loudness and Duration. Further, the number of successive flattening operations can be specified, where unlimited means that the operations is iterated until the result has converged. For non-circular denotators, this always happens after a finite number of steps. The two remaining rubettes, MacroTreeSelect and MacroPredicateSelect help enrich a macro object coming out of the ScoreToMacro rubette. The first allows the specification of the parts to enrich by direct selection, the second by the use of predicates. Both have as first input a macro score, and as second input the set of satellite notes used for the embellishment. The implementation is currently specific to score type objects. There are, however, neither theoretical nor practical reasons, why the principles discussed in this section should not be applied to a generic setting. But this requires a more in-depth analysis of the meaning of the various operations, so as to be able to create a general implementation that makes sense even in unforeseen situations. This topic must be pursued in future projects. Figure 16.4 shows a R UBATO network that uses a MacroPredicateSelect rubette to adorn the notes of a simple score (created randomly using a Melody rubette) with notes generated from another Melody rubette. Both scores are first given a random rhythmic variation, then transformed with ScoreToMacro rubettes. The result is flattened, then transformed back to Score denotators which can be played. Figure 16.5 shows the two input scores and the output scores as displayed by the ScorePlay rubette.

16.2 The Wallpaper Rubette While the first project addresses the problem of the creation and handling of macro objects, the second project is about the transformation of given mu-

16.2 The Wallpaper Rubette

171

Fig. 16.4: This R UBATO network uses macro scores to adorn a musical fragment with satellite notes. The result is seen in the rightmost part of figure 16.5.

Fig. 16.5: The score that is to be embellished (left), the embellishment (center), and the result (right).

sical objects, including the ideas of ornamental creations already contained in presto’s OrnaMagic module, as well as morphing between two musical structures along specified parameters, e.g., rhythmics or harmonics. This section presents the ideas from its first part, the creation of ornamental structures.

172

16 First Applications in Rubette Construction

In presto, it was possible to generate ornaments based on a motif that is translated in the plane of onset versus pitch. This is easy to see in figure 7.3. The ranges indicate where the translations start and how often they are repeated. The picture also shows the translation vectors. This principle can be generalized by interpreting the schema containing the translation vectors as a diagram of morphisms. Then not only translations can be used to generate ornaments, but also general affine homomorphisms. This generalization is implemented in the Wallpaper rubette by Florian Thalmann. Its input is a motif of the form MacroScore or Score, which is then mapped using a sequence of morphisms (see figure 16.6). Observe that, in contrast to simple translations, the composed morphisms do not commute in general, therefore they are applied in a specified order. In the figure, this order is f2 followed by f1 . For each morphism, the range indicates the number of times and the first time it is applied. For f1 the range is [0, 2] and for f2 it is [0, 1].

f1 f1

f2 f1 f1 Fig. 16.6: A simple example of two transformations f1 and f2 , where f2 is a translation and f1 is a rotation by 180◦ followed by a translation. The original motif is at the bottom left. The following combinations have been applied: f10 f20 , f11 f20 , f12 f20 , f10 f21 , f11 f21 , and f12 f21 .

We have not yet explained how these morphisms are defined. It is clear that such a morphism transforms a denotator of form Note into another denotator of form Note. The Wallpaper rubette allows to select any of the coordinates of a Note denotator and combine them. Let us pick one of the transformations f = f2i f1j to illustrate the procedure. This transformation takes place in the Euclidean plane, i.e., f : R2 → R2 .

16.2 The Wallpaper Rubette

173

The coordinates that are to be transformed have the Simple forms Onset and Pitch. Remember that a Simple denotator contains a module morphism. For each Note denotator, the module morphisms of its Onset and Pitch coordinate are extracted, say o : A → R and p : A → Q, respectively, where A is the common address. The following compositions are now necessary: i1 ◦ o : A → R2 , i2 ◦ e2 ◦ p : A → R2 , where i1 and i2 are the injections R → R2 to the first and second coordinates, respectively, and e2 is the embedding Q → R. To combine both o and p in a single instance of R2 , the sum of these morphisms are taken, so that o is found in the first coordinate and p in the second coordinate of R2 . The transformation f is now applied to the result, and finally, to get the coordinates back, the projection on each coordinate is added. The module morphisms on and pn that go into the new coordinate denotators are: on = p1 ◦ f ◦ ((i1 ◦ o) + (i2 ◦ e2 ◦ p)) : A → R, pn = c ◦ p2 ◦ f ◦ ((i1 ◦ o) + (i2 ◦ e2 ◦ p)) : A → Q, where p1 : R2 → R and p2 : R2 → R are the projections to the first and second coordinates, respectively. Note that for the morphism of the Pitch denotator a cast c : R → Q is necessary. This means that pitches that result from such a transformation are quantized. The exact quantization is determined by how the cast is defined. By default, when converting real numbers to rational numbers, only denominators that are divisors of 3·5·128 = 1920 will be generated. The default can be changed in the Preferences dialog of R UBATO C OMPOSER. Further rounding takes place when exporting the result to MIDI representation, since then only integer pitches are allowed. Currently micro-tonal rendering using MIDI, for example through pitch bending, is not supported. The Wallpaper rubette allows for combinations of more than two transformations. Furthermore, the input is not limited to MacroScore or Score denotators. Any form that is of type Power and contains a form with the shape Limit(A,B,C,D,. . . ), where A, B, C, D, . . . are forms of type Simple, can be used as input. The mappings are not limited to A × B → A × B as in the example above, but can be extended to arbitrary combinations, such as A × B → C × A × D, etc. This rubette is a tool for creating very general patterns, which are not only translation symmetries, such as found in everyday wallpapers, but very general transformations that have so far not often been used for generating patterns.

174

16 First Applications in Rubette Construction

We conclude this chapter with an example of an application that uses the Wallpaper rubette. Figure 16.7 shows the network that generates a pattern from a random melodic fragment. In a particular execution of the network this original fragment looks like the left side of figure 16.10.

Fig. 16.7: A R UBATO network that generates a pattern using the Wallpaper rubette.

The melody is first given a rhythmic variation using the Rhythmize rubette, just as in the example above. The result is fed to the Wallpaper rubette, which is configured as shown in figure 16.8.

Fig. 16.8: The configuration of the Wallpaper rubette.

With the symbolism introduced above, the Wallpaper rubette generates a pattern with two transformation f1 and f2 , both of which can be abstractly regarded as being of the type Onset × Pitch → Onset × Pitch. The type is set using the Coordinates button.

16.2 The Wallpaper Rubette

175

Fig. 16.9: The graphical representation of the affine transformation f1 (left). The translation f2 (right).

The first transformation f1 : R2 → R2 is an affine morphism. It has been specified graphically as seen on the left side of figure 16.9. The range is [0, 2], which means that f1 is applied three times, i.e., f1i , i ∈ {0, 1, 2}. The first application f10 is, of course, the identity, and thus generates the original. The second transformation f2 : R2 → R2 is a simple translation in the Onset × Pitch plane. Its range is [0, 1]. The result of the Wallpaper rubette is the score shown on the right side of figure 16.10.

Fig. 16.10: The original melody (left), and the resulting pattern (right). A comparison with figure 16.6 will make the correspondences clear.

176

16 First Applications in Rubette Construction

The Wallpaper rubette, in an extended version, will play an important role in chapter 18.

16.3 The Alteration Rubette The Alteration rubette provides the functionality for altering one composition to another. The alteration concepts from presto’s OrnaMagic module have been generalized for Power denotators. For the definition of alteration, we need: • two compositions d1 : A@P(c11 , . . . c1m ) and d2 : A@P(c21 , . . . c2n ), where cij are of form C, and P is any form of type Power, • d alteration dimensions with: – S. = (S1 , . . . Sd ) altered Simple forms, Si ∈ SP and Si 6= Sj for i 6= j and 0 ≤ i, j ≤ d, – (a1 , b1 ), . . . (ad , bd ), ai , bi ∈ R pairs of marginal alteration degrees and – R. = (R1 , . . . Rd ) Simple forms, according to which the local alteration degree is calculated, where Ri ∈ SP and embedded in R, therefore linearly ordered, • a distance function (a metric) δ(c1 , c2 ) for denotators c1 and c2 of form C and • a nearest neighbor function NN(c, d) = n, for d : A@P(c1 , . . . ck ) and c of form C so that n = ci , if δ(c, ci ) ≤ δ(c, cj ) for j 6= i, 1 ≤ i, j ≤ k. In presto, nearest neighbor search is realized using an algorithm that starts from a point in the Euclidean plane and spirals outwards until the nearest point is found. This is an effective method, because, in presto, points lie on a grid, and therefore only a finite number of discrete values, namely the (two-dimensional) coordinate points, are visited. However, this is not the case in the actual R UBATO C OMPOSER implementation, since Simple denotators accept values from continuous modules. Even though the number of (machine-representable) numbers used as module values is, in the end, finite, it is clearly impractical to implement such a strategy. We therefore have chosen to use a well-known nearest neighbor search method, the kd-tree algorithm [7]. For more details on this approach, see [68]. Let us now describe formally how alteration works. If we alter the composition d1 towards d2 , we obtain a composition alt : A@P(e1 , . . . em ) of form P with |alt| = |d1 |. Its elements e1 , . . . em are transformed versions of c11 , . . . c1m .

16.3 The Alteration Rubette

177

Which Simple denotator values are altered is specified by the d Simple forms Si , each of them standing for an alteration dimension. As with general wallpapers, we need to accept any Power form as input composition, in particular forms, the elements of which may contain several denotators of form Si in their structural tree. To this end, we define a function SA(S, d) = t, which outputs a denotator t of Simple form S ∈ SF for denotator d : A@F. Practically, SA(S, d) searches the form tree of F for the top level form of type S and returns the first denotator found at this position in the tree of d, i.e., if a Power denotator is passed, the path along its first branch is travelled (see [68, s.4.3.2] for details). This ensures that, for every Si and every c ∈ d1 , just one denotator value is altered, namely SA(Si , c). The amount by which a coordinate c is altered is indicated by its local alteration degree. For its calculation, first the position of c within the composition d1 according to Ri is determined. Ri is another Simple form, which is often the same as Si and which defines the scale on which the position of c is measured. The position positioni ∈ [0, 1] of c is calculated using the function SA(Ri , c) − minRi positioni (c) = , maxRi − minRi where maxRi = max({SA(Ri , c) | c ∈ d1 }) and minRi = min({SA(Ri , c) | c ∈ d1 }) are the highest and lowest elements of Ri denotators in d1 . These elements can only be calculated if Ri is embedded in R, which has been assumed above. Then, the local alteration degree is calculated depending on the marginal alteration degrees (ai , bi ). These degrees indicate how much d1 is altered at its lowest and highest values on the Ri scale, whereby ai defines how much maxRi is altered and bi how much minRi . For ai , bi = 0, d1 remains unchanged and for ai , bi = 1, every element of d1 receives the value in d2 with the smallest distance to it in dimension i, as we will see in the later definitions. If, for example, we choose d1 : A@Score, Ri = Onset, ai = 0 and bi = 1, the first Note of d1 in time is not changed at all, whereas the last note receives a new onset, taken from the note in d2 that shows the least distance from it. For each note in between, the local alteration degree is calculated using linear interpolation, according to its position in time. We define the function degreei (SA(Ri , ck )) for the local alteration degree of ck as follows: degreei (m) = ai + (ai − bi ) · positioni (m). The alteration function for dimension i then uses this degree to alterate a morphism m1 towards another morphism m2 : alterationi (m1 , m2 ) = m1 + (m2 − m1 ) · degreei (m1 ). We have now made the necessary preparations and are ready to define the altered composition alt, which we already mentioned above:

178

16 First Applications in Rubette Construction

alt(d1 , d2 ) = {alteration(ck , NN(ck , d2 )) | ck ∈ d1 }. In this definition, we use the morphism alteration : C × C → C, where alteration(c1 , c2 ) returns a copy of c1 , in which the value of the Simple denotator SA(Si , c1 ) is replaced by the altered morphism alterationi (SA(Si , c1 ), SA(Si , c2 )) for every 1 ≤ i ≤ d.

Fig. 16.11: Alteration in the Onset-Pitch plane. Composition 1 (represented by circles) is alterated in the direction of composition 2 (represented by stars), first by 0%, then 50%, and, finally, 100%.

Here are two significant musical examples for alteration, both inspired by presto’s Easy Ornaments module. For a tonal alteration that increases in time, we choose d1 to be a composition and d2 to be a musical scale, the generation of which we describe later, and perform a one-dimensional alteration with: S1 = Pitch, (a1 , b1 ) = (0, 1), R1 = Onset. For a quantization that decreases in time, the composition d2 must be a rhythmical grid of the same duration as d1 . Such a grid can be created using a wallpaper. Furthermore, we define the following one-dimensional alteration: S1 = Onset, (a1 , b1 ) = (1, 0), R1 = Onset. In practice, these two examples are easily realized using the Alteration rubette. Figure 16.12 shows the minimal network settings for a tonal alter-

16.4 Counterpoint Theory

179

Fig. 16.12: The settings for performing a tonal alteration on a given composition.

ation. The Scale rubette is ideal for providing a scale towards which the notes of the composition are altered. The Alteration rubette preference window in the figure shows the necessary specifications for an alteration gradually increasing in time (low degree 0, high degree 1). To quantize a given composition, we need to define a rhythmical grid. To this end, first a Melody rubette is used to create a number of notes, all having the same pitch. Then, the melody is fed into a Rhythmize rubette, which outputs the rhythmic motif for our grid. Here, we set the Rhythmize rubette’s properties so that the length of the motif is exactly one bar (two time units). The grid is then produced by a Wallpaper rubette, which just replicates the motif in time, i.e., it uses a single morphism that repeatedly displaces the onsets of the motif by 2.0. It is important to make sure that the obtained wallpaper exceeds the duration of the composition to be quantized. Finally, the quantization itself is managed by an Alteration rubette, for which we define a single alteration dimension for onset. Part (a) of figure 16.13 shows the network which we used to quantize the starting pattern of Steve Reich’s Piano Phase (a typical wallpaper, by the way) with a gradually increasing degree (from low 0 to high 1). Part (b) shows the resulting score. Although visually the differences are minimal, they are clearly audible. The originally even rhythm is disturbed and virtually absorbed by the rhythm produced by the Rhythmize rubette.

16.4 Counterpoint Theory In his ongoing doctoral thesis work, Julien Junod investigates the different counterpoint systems that arise from the mathematical formalization of Fuxian counterpoint rules [47]. He tries to relate these “worlds” of counterpoint and find homomorphical mappings between them. The resulting

180

16 First Applications in Rubette Construction

(a)

(b) Fig. 16.13: An example for quantization. The network (a) and the resulting score (b).

theories will be used to implement rubettes that provide operations such as the translation of a piece, say by Bach, into another counterpoint world. Other rubettes will help in generating scores within these counterpoint systems.

16.5 Music Composition Chapter 17 presents the results of a research project on gestural music composition at the University of Minnesota in 2007, which was supported by the Hasler Stiftung Bern.

16.5 Music Composition

181

Chapter 18 documents a musical composition project supported by the Cogito Foundation and the Art Mentor Foundation Lucerne, in collaboration with IRCAM in Paris. The ideas from Mazzola’s sonata and the tools known from presto are implemented as a collection of rubettes. These rubettes are used to create a new musical work from the analysis of a given work. In contrast to the sonata, which used Beethoven’s Hammerklavier sonata as a starting point, the new work is based on Structures I for two pianos by Pierre Boulez (1952). This is a work of integral serialism, and it will be interesting to see what the techniques, that have already been applied to a work of early romanticism, will create. The comparison with Structures II, which was composed by Pierre Boulez in 1961, and which is itself a reconstruction of Structures I, will certainly prove fruitful in future research.

Chapter 17

The BigBang Rubette by Florian Thalmann and Guerino Mazzola

This chapter introduces the example of a complex and convenient rubette that has been created using the aforementioned rubette programming interface. The so-called BigBang rubette makes full use of R UBATO C OMPOSER’s mathematical framework. It was implemented during a research visit by Florian Thalmann at the School of Music of the University of Minnesota and presented at the ICMC 2008 in Belfast [69]. Inspired by the gestural aspect and the immediacy of human improvisation, it provides an interactive visual and aural interface for applying R UBATO C OMPOSER’s complex mathematical operations in real time by using simple mouse gestures. The underlying paradigm encourages a more spontaneous and explorative way of composing.

17.1 Spontaneous Algorithmic Composition Musical composition is situated between spontaneous human improvisation and algorithmic, computer-aided composition that is detached from human interventions. The former stresses gestural immediacy and feedback from the sounding realm, whereas the latter delegates the transformation of abstract rules into sounds to a computational dataflow. The experience of the delay between the rules of composition and the resulting product is well known in serialism, for example, in the huge man-made calculations of Pierre Boulez for his famous structures pour deux pianos [11, 12]. Serialists needed a huge amount of time for calculation and hardly knew how their results were going to sound. As already criticized by Iannis Xenakis [73] and György Ligeti [39], the resulting compositions were too complex to be analyzed by human ears during audition and therefore seemed arbitrarily structured.

183

184

17 The BigBang Rubette

It is certainly not an ideal situation for any composer to delegate his or her thoughts to a complex external process, be it purely mathematical or computer-aided. The ideal would be to have at hand all sorts of complex calculation processes, but, at the same time, to be able to assist to each component without delay and without being separated from the ongoing process stages. This is not only a desire of immediacy, but also of transparency. The result of every compositional operation should not only be observable after its application but also foreseeable. More intuitively speaking, one would like to compose gesturally similarly to free jazz, with the extension that these gestures can also be interactions with “visible” algorithmic processes. This chapter presents the result of a research project that combines both algorithmic and spontaneous composition by using cutting-edge gestural paradigms.

17.1.1 Facts about Geometric Composition Strategies Gesture theory in music has mainly been driven by instrumental interactions [72] and less by gestures dealing with abstract compositional strategies. The latter was addressed in a rather abstract, if not metaphorical, spirit by Boulez in [14], we discuss this approach in [49]. Here, we want to make these ideas more concrete in the case of geometric composition strategies. Is it possible to implement such devices in a gestural way as proposed above? To answer this question, let us first recapitulate the general framework of geometric composition techniques. One is given a set C of notes (part of the preliminary composition to transform) in a module M. Such a module is typically the n-dimensional real vector space Rn . A transformation then is an affine invertible map F = Tt ◦ L, where Tt (x) = t + x is the translation by t ∈ Rn , and L ∈ GL(n, R) is a linear map. We want to calculate the F-image F(C) of C. As it is hopeless to handle a general n-dimensional transformation directly by intuitive gestures, we decompose the map F into intuitively more accessible factors. To do so, two theorems come to our aid. The first theorem states that any F may be written as a concatenation F = F1 ◦ F2 ◦ . . . Fk of transformations Fi , which, each, involve only one or two of the n dimensions and leave the others unchanged. The second theorem tells us that we may choose each Fi to be one of the following five musical standard operations: (1) translations Tt (in music: pitch transposition or da capo in time), (2) reflections Refl at a line l (in music: inversion, retrograde or parameter exchange), (3) rotations Rotc,α by an angle α around the center c (in music: retrograde inversion (rotation by 180 degrees), or general rotations as proposed by Maurizio Kagel or Herbert Eimert), (4) dilations Dilc,λ,µ from

17.2 Gestural Interaction Concept

185

center c by horizontal factor λ > 0 and vertical factor µ > 0 (in music: dilations in time for augmentations), and (5) shearings Shl,α along the line l and by angle α (in music: shearing in time of chords resulting in arpeggi). Putting together these theorems, we obtain the concatenation theorem of Mathematical Music Theory [47, p.160]: Each transformation can be written as a product of a priori musically significant factors, each acting in two dimensions and each being of one of the above five standard types. This fact is however only one half of the geometric construction strategies in music. The other half is that affine transformations mostly act on rather small parts C ⊂ K of a composition K. Transformations are local, and therefore require a preliminary selection of those parts C that one wants to transform. We shall deal with the selection process in this chapter, we shall however not discuss the ways and criteria of defining coverings of the global composition K by subcompositions Cj . This topic is crucial and refers to questions of topological nerves of coverings, see [47, ch. 13.2, ch. 48] for the general theory and [49, ch. 10] for an elaborate example.

17.2 Gestural Interaction Concept Our gestural interaction concept is a new way of making mathematical transformations easily accessible and understandable for non-mathematicians. It has been implemented as a prototype for music composition in R UBATO C OMPOSER. In this section, after giving an overview of mathematical gesture theory, we will discuss how this theory can be applied and what is required to implement it in an optimal way.

17.2.1 Gesture Theory In [50], a mathematical theory of gestures has been presented, which does not rely on the hitherto intuitive concept of a gesture, but sets forth a strictly mathematical concept. Our approach is based upon an ontological critique of the transformational theory of music (and even more of the pre-transformational theory, which focuses on the facts without being involved in processes that lead to such facts). It is argued that the arrows f : X → Y (morphism symbols) used in category theory, which is the formal framework for transformational theory, are misleading graphemes in

186

17 The BigBang Rubette

that nothing is really present within the arrows’ shaft, as was also observed by the French philosopher Gilles Châtelet [16]. The Fregean function concept backing mathematical arrows only knows about the argument x and its image f(x). The process of transporting the argument to its image is compressed, if not annihilated, in an abstract formula. The mathematical theory of gestures closes this gap in defining a gesture as a morphism γ : Γ → I@X of directed graphs, where Γ is called the skeleton of the gesture, and I@X is the digraph of continuous curves c : I → X (the arrows) on the unit interval I = [0, 1] with values in a topological space X. The head of an arrow c is the point h(c) = c(1), while the tail is t(c) = c(0). Intuitively speaking, a gesture is a graphical configuration of continuous curves in a topological space X. This means that we may then ask for values of such curve arrows at any intermediate parameter values 0 < λ < 1. In this text, gestures are used for the simple case of linear skeletons Γ = · → · → · . . . →. The idea is to use the above factorization theorem and to represent each factor Fi of F = F1 ◦ F2 ◦ . . . Fk by one of the arrows of the skeleton. Thus, the essence is to turn the factors into the elementary gestures of continuous curves. We have to investigate the typical factors provided by the five standard transformation types. Let us get off ground with translations. A two-dimensional translation Tt maps a vector x ∈ R2 to the function value Tt (x) = t + x. This action is enriched by letting the resulting value Tt (x) be the endpoint of an entire movement that starts with x, in fact, our intuition suggests that we take the curve c(Tt )(x) : I → R2 : λ 7→ x + λt. Rotations are very nicely turned into gestures, and in fact this example illustrates the crude abstraction encapsulated within the Fregean formulas of linear algebra. A two-dimensional rotation Rotc,α is essentially represented by a 2 × 2 matrix (the center being thought as the coordinate origin), and nothing in this matrix reveals the motion understood by the word “rotation”. This is now added by the parametrized curve c(Rotc,α )(x) : I → R2 : λ 7→ Rotc,λ·α (x). We have a similar and evident curve construction for shearing and for dilation. Only reflection is somewhat trickier: reflecting a point means passing through the reflecting line l, which is a singular moment in that movement. There is a canonical way to circumvent this singularity: we could add one dimension to R2 and then interpret reflection as the projection onto this R2 from a rotation around the reflection line l in R3 . We have not implemented this work-around, but left the singular moment as a critical spot in the gestural interpretation of reflection. We have therefore not attempted to use continuous curves to represent reflections, but implemented reflections as (old-fashioned) Fregean functions of mirror character. We conclude with an important remark about the generic relation between gestures and the factorization method in mathematics: Similarly to the translation gesture, the rotation gesture is a continuous curve for each argument x that is transformed. But more than that emerges from this

17.2 Gestural Interaction Concept

187

parametrization: the above factorization theorem—which is used to understand factorization as a means of defining composed gestures on linear directed graphs—suggests that gestures deal with factorization of elementary curve gestures. But, conversely, such a curve, as in the case of the rotational gesture c(Rotc,α ), appears to define a factorization of the rotation for each parameter value λ, in fact c(Rotc,α ) = c(Rotc,(1−λ)·α )◦c(Rotc,λ·α ). This may be seen as an infinite factorization: one factor for each parameter value λ! This philosophy reveals the mathematical turning point of gesture theory: the continuous curves that trace the path from the argument x to its function value f(x) are just a way of factorizing the function into an infinity of intermediate stages. It is a representation of the entire function as a successive and continuous movement through intermediate factor values. This is the point of view we take in the following implementation: Gestures relate to infinite and continuous factorizations of functional assignments.

17.2.2 Application of Gesture Theory These theoretical reasonings led to a new user interface designed as a R U BATO C OMPOSER rubette, where transformations and structures can be realized using gestural mouse commands. The idea is not only to visualize the generation and transformation of structures as a gestural process but also to simplify and optimize the interface, so that quick combinations of gestures become possible and allow the composer to act in a fluid and spontaneous way. For the realization of this interface, the following points had to be considered.

17.2.2.1 Minimal Number of Gestures For each transformation or operation, the number of successive atomic gestures has been minimized, for the less gestures are needed to perform an operation, the faster it can be applied and the more spontaneous a composer’s interactions can be. With simple and quick gestures, a composer can reach the state of mind called flow [21].

17.2.2.2 Live Transformation During gestural interaction, the result of every operation is shown while it is performed. For example, during score rotation, when the composer drags the mouse across the screen to determine the appropriate angle, at

188

17 The BigBang Rubette

any time, the resulting score is shown as a preview. The transformation is applied to the score definitively only when the mouse button is released. Hereby, every transformation (except for reflection, for the reason given in subsection 17.2.1) gains an intuitive gestural aspect and becomes understandable as a step by step process. Optionally, in every step of such a transformation, the corresponding result is played back immediately. This feature lets composers focus on audible results, not on operations.

17.2.2.3 Undo/Redo When transformations are applied in a fast, easy and understandable way, composition becomes a very spontaneous and exploratory process. Therefore, it is very important to include an undo/redo feature that allows immediate backtracking.

17.3 Modular Views Another new concept in music composition software introduced with the BigBang rubette is the concept of completely modular and customizable score views. This section will present the view concept as well as the BigBang rubette’s non-mathematical basic functionality. The mathematical functionality will be presented in detail in the next section.

17.3.1 View Concept To ensure an intuitive and complete comprehension of the geometric and structural material and operations that the BigBang rubette is dealing with, it is important to provide a simple and understandable yet versatile view of it. At the same time, this view must serve as a user interface, where actions can be defined. This leads to various requirements, especially due to the geometrical aspects of the represented material. Since transformations may take place on every side of the high-dimensional cube (see subsection 17.1.1), the first requirement for a view is to be able to represent any of the 20 sides of the cube, i.e., the planes m1 × m2 , m1 6= m2 in M = {Onset, Pitch, Loudness, Duration, Voice}.

17.3 Modular Views

189

A second requirement is to provide different visual representations of the composition’s notes. This can be very useful for the visualization of specific characteristics of the actual composition. Furthermore, such a variable representation can inspire and facilitate the analysis of given compositions. Even when dealing with two-dimensional transformations, as with the BigBang rubette, it may not be possible to foresee all their consequences for objects in a higher-dimensional space. It can therefore be beneficial to observe a certain transformation simultaneously from different perspectives. Multiple simultaneous views are thus an additional requirement for an optimally advantageous view concept. To fulfill the first two requirements, we developed a generalized design, which allows for arbitrary pairing of screen parameters and musical parameters. This means that we define a number of screen parameters, each of which able to represent any of the given note parameters m ∈ M. These screen parameters will be discussed in detail below. The BigBang rubette defines its own set of note parameters M′ , which is a superset of the set M of all Note form coordinates and enables simple visualization of additional note information. At the present time, we define M′ = M ∪ {SatelliteLevel}, where SatelliteLevel indicates the satellite hierarchy level of a note, which is a positive integer number, i.e., 0 for the top anchor notes, 1 for their satellites, etc. . . The third requirement is realized by allowing an arbitrary number of simultaneous views which can all be customized independently.

17.3.2 Note representation On the views provided by the BigBang rubette, notes are represented as rectangles of different sizes (see figure 17.1), similarly to a traditional piano roll representation. Such a rectangle is defined by six parameters: x-position and y-position on the plane, width, height, color and opacity. These parameters are the screen parameters discussed in the last section. We further set N = {x-position, y-position, width, height, opacity, color}. So each of these can represent any of the note parameters m ∈ M′ . We define a view configuration as a functional graph V ⊂ N × M′ , which means that each screen parameter n ∈ N is associated with at most one musical parameter V(n) that defines its value. For screen parameters not covered by V, we define a default value. For example, to obtain the piano roll representation shown in figure 17.1 we would choose the pairs

190

17 The BigBang Rubette

Fig. 17.1: A parameter pairing corresponding to the traditional piano roll notation

(x-position, Onset) (y-position, Pitch) (opacity, Loudness) (width, Duration) (color, Voice) A completely different example for the same musical material is shown in figure 17.2. The corresponding pairings are (x-position, Onset) (y-position, Loudness) (width, Pitch) (color, Onset) (height, Loudness) In addition to this, every screen parameter can be customized independently at runtime. Depending on the represented note parameter, it can be useful to ensure that a screen parameter’s value does not exceed a specific value range. For example it is neater to limit the rectangle’s height in a way that their areas do not intersect. For each n ∈ N, we optionally define minn and maxn , the minimal and maximal screen values, respectively. We then have

17.3 Modular Views

191

Fig. 17.2: An experimental parameter pairing

two options to define the way note parameters are mapped to the screen parameters. If we choose the conversion to be relative, the minimal and maximal values of the given note parameter minm , maxm , m ∈ M′ are determined for the actual score, and then mapped proportionally so that the note with minm is represented by minn and the note with maxm by maxn . For this, we use the formula vm − minm vn = (maxn − minn ) + minn , maxm − minm where vn is the screen value for the note value vm . On the other hand, absolute mapping means that every value with vm < minn or vm > maxn is mapped to a new value, while all other values stay the same, i.e., vn = vm . For absolute mapping, we have two choices. In limited mapping, the values that exceed the limits are given the minn and maxn values, respectively. The following formula is used:   minn if vm < minn vn = maxn if vm > maxn   vm otherwise. For cyclic mapping, we use the formula

192

vn =

17 The BigBang Rubette

(

(vm mod (maxn − minn )) + minn vm

if vm < minn or vm > maxn otherwise.

This mapping type can be useful for the color parameter for example, where it is reasonable to cycle through the color circle repeatedly to visualize a specific note parameter interval (see figure 17.2, where color visualizes a time interval of length 24, i.e., six 4/4 bars). For absolute mapping in general, it is possible not to define either or both of the limits. Accordingly, we assume minn = −∞ or maxn = ∞. Of course, if none of the limits are defined, the visible screen parameters correspond exactly with the original note parameters. At runtime, a view window’s current pairings are selected using a matrix of checkboxes with a column for each screen parameter and a row for each note parameter (see figure 17.2). In addition to these customizable rectangular note objects, satellite relations may optionally be displayed as lines between the centers of two note objects, i.e., as shown in figure 17.3, every note has lines leading to each of its direct satellites.

Fig. 17.3: A satellite hierarchy displayed by lines between notes and a different note color for each level.

17.3 Modular Views

193

17.3.3 Basic Functionality and Navigation For browsing through one of the BigBang rubette’s view windows and for simple score editing, a number of basic actions are available to the user. To structure these and the numerous geometrical actions, a set of independent application modes has been defined. Each of these modes defines a number of mouse actions that become available as soon as a corresponding toolbar button is pressed. By grabbing the, in principle infinite, score plane and moving the mouse pointer, the user can move it around. The mouse wheel is used o zoom in and out. The current score can be edited in common intuitive ways. Using a note selection tool, an arbitrary part of the score can be selected (see figure 17.4). This part can then be copied, pasted or deleted from the score. All these operations can either be chosen from a popup menu or directly applied using shortcuts. The second simple way to edit a score is to draw new notes. This can be done in way similar to common drawing programs. Whenever and as long as the mouse button is pressed, notes are created at the pointer location. Figure 17.3 shows an example of notes created that way. Furthermore, the operations described in section 16.1 have also been implemented as basic functions that can be selected from a popup menu. Flattening can be applied to any selected set of notes whereby each of these note’s satellite level is reduced by one, i.e., the flattening operation defined above is applied once to every note. If an involved note already is in the top level of the hierarchy, it is ignored. The opposite operation is to build satellites, whereby all selected notes can be assigned to a new anchor note anywhere in the hierarchy.

17.3.4 Layers Compositions commonly contain large independent structures, e.g., different parts or voices. In this case, score modifications should often affect only one or a few of these structures. Inspired by the track concept of sequencers such as L OGIC [41] or L IVE [40] and by the layer concept of graphical programs such as P HOTOSHOP, we incorporated the possibility of defining composition layers for this purpose. These layers are superimposed and transparent and they can have three states: active, visible and invisible. All score operations are restricted to the currently active layers. When using the note selection tool described in subsection 17.3.3, for example, only notes in active layers can be selected.

194

17 The BigBang Rubette

Fig. 17.4: Some notes of a layered composition are selected.

The notes of all visible layers are represented as grey boxes in contrast to the active layers’ notes, which are colored, as described in subsection 17.3.2. To hide a layer, it can be set to invisible. Figure 17.4 shows a composition where the voices are split into different layers of all three states. Above the score area, there is a button for each layer, showing its current state. These buttons can be used to change the states of the corresponding layers. At any time, selected score parts can be copied or moved to new or existing layers.

17.4 Implemented Gestures In this section, we illustrate how the concept of gestural interaction presented in section 17.2 is implemented for geometrical and structural operations and how a musical score can be manipulated in this way. We refer to the movie http://www.encyclospace.org/special/bigbang.m4v for an animated demonstration of the gestural interface.

17.4 Implemented Gestures

195

17.4.1 Geometrical Transformations For each of the five basic geometrical transformations described in subsection 17.1.1, a simple screen tool has been designed, which uses a minimal number of mouse gestures and button clicks. The main part of each transformation gesture consists of a click-and-drag movement. As long as the user keeps the mouse button pressed, a preview score is shown, where the shape and color of each translated note change adequately, depending on the view configuration. To illustrate the current action, a scalable schematic representation of the operation appears at the location of the click. To perform a translation, one of the selected notes can simply be clicked and dragged to its final destination. All other selected notes are translated in the same manner. Rotation is realized with two gestures: one click for the definition of the rotation center followed by a click-and-drag movement, which defines the angle. During this procedure, a schematic rotation tool is drawn, as shown in figure 17.5 (a).

(a)

(b)

Fig. 17.5: The rotation tool with a circular arc showing the rotation angle (a) and the reflection tool with a line showing the reflection axis (b).

All other transformations are controlled with a single click-and-drag gesture. For shearing and dilation, the location of the first click defines the center of the given transformation, while the mouse position during dragging and the release point define slant or size, respectively. Finally, for a

196

17 The BigBang Rubette

reflection the mirror axis is defined by clicking two points. See figure 17.5 (b) and figure 17.6 for the corresponding schematic tools.

(a)

(b)

Fig. 17.6: The shearing tool (a) and the dilation tool (b).

Any of these transformations can be performed as a copy and transform operation by pressing and holding the A L T key. Figure 17.7 shows a composition generated in this way from a single motif using different transformations.

17.4.2 Wallpapers Another way of generating regular structures from a single motif has already been introduced in section 16.2. A so-called wallpaper is defined by a number of morphisms, each defining one of its dimensions. It is generated by first repeatedly applying the first morphism to the original motif and by uniting all intermediate results. Each successive morphism is then applied to the resulting union in the same way. In so doing, we finally obtain a grid-like structure such as the one shown in figure 17.8. Wallpapers can be created directly within the BigBang rubette and since the resulting structures are very complex, we can profit even more from the advantages of immediate visual and aural feedback. To create a wallpaper, we select a motif and press the wallpaper start button. Thereafter, we may perform as many gestural geometrical transformations (described in subsection 17.4.1) as we like. The concatenation of all these transforma-

17.4 Implemented Gestures

197

Fig. 17.7: A composition realized from a single motif using copy and transform operations.

tions then forms the morphism of the first wallpaper dimension. During and after every of these partial transformations, the resulting wallpaper is visible and audible. To add a new dimension to the wallpaper, the wallpaper start button can be pressed again and every following transformation then applies to this new dimension. During these operations, the perspective can be freely changed. The wallpaper in figure 17.8, for example, has two dimensions, the first consisting of a translation and a rotation, both on the Onset × Pitch plane, and the second consisting of a translation and dilation on the same plane, followed by a shearing on the Pitch × Loudness plane. The dimension’s ranges (number of consecutive applications for each morphism) can be modified using spinners any time during wallpaper creation. In our example, the second dimension has the range (0, 6). This means that the morphism is applied from one to six times to obtain six copies. The first dimension even has a negative range (−1, 6), so that additionally the inversion of the morphism is applied once. Negative wallpaper ranges are a new feature that could only be realized with the BigBang rubette, where all transformations are geometrical and can therefore be inverted (with a few exceptions that, however, are not very useful for wallpaper generation anyway).

198

17 The BigBang Rubette

Fig. 17.8: An example of a two-dimensional wallpaper with the original motif drawn with darker colors.

17.4.3 Alteration In section 16.3, we introduced another useful musical transformation, named alteration, which is a generalization of the common musical operations tonal alteration and quantization. One composition acts as a force field for another composition, i.e., each note of a composition C1 is attracted by the nearest note of a second composition C2 . Thereby, an arbitrary number of dimensions is affected, e.g., Pitch and Duration such as in the example in figure 17.9. The BigBang rubette is ideal for visualizing the effect of alteration in real time. In alteration mode, two arbitrary excerpts can be selected from the score (C1 and C2 ). Then, using two sliders, the alteration degrees at the beginning and at the end of C1 can be adjusted. During this adjustment, the notes can be watched as they move, so the gestural concept is virtually projected on these two sliders. The checkboxes to the left of these sliders can be used to select the involved note space dimensions. There are five checkboxes, one for every dimension. In figure 17.9 (b), the second and the fourth are selected (for Pitch and Duration) and the alteration degrees are 0%% at the beginning and 100%

17.5 The BigBang Rubette in Context

199

at the end. Therefore, the later in time the note, the closer in Pitch and Duration it is pulled to its nearest note in C2 . This is even better visible here since Duration is additionally represented by the color screen parameter (see subsection 17.3.2).

17.5 The BigBang Rubette in Context From the above descriptions one might conclude that the BigBang rubette is a standalone music composition software. However, it is just an element that can be even more powerful when used in combination with other rubettes. As with every other rubette, multiple instances of the BigBang rubette can be placed and connected in a R UBATO C OMPOSER network. Whenever the parent network is executed, a BigBang rubette sends its current composition to its output connector and optionally receives a new composition through its input connector (a GUI checkbox defines whether or not this is the case). If a composition is received the previous composition and all previously performed operations are overwritten, since it cannot be ensured that the two compositions are related and the operations could be applied to the new one. Many useful scenarios are possible using these connectors. For example, an exisiting MIDI file could be read in by a MidiFileIn rubette, processed in a BigBang rubette and finally written to a Csound file using the Csound rubette. Or the output denotators of multiple BigBang rubettes could be processed using the set operations of a Set rubette.

200

17 The BigBang Rubette

(a)

(b) Fig. 17.9: A composition (a) is gradually altered (b) in Pitch and Duration towards a second composition (darker colored in (a)).

Chapter 18

Creative Analysis of Boulez’s Structures by Guerino Mazzola and Florian Thalmann

This chapter summarizes the first large-scale application of R UBATO C OM POSER. It is a complete musical composition and is mainly the work of Guerino Mazzola and Florian Thalmann. Since it is a more or less independent insert into the rest of this book, some concepts detailed here are a repetition of material that can be found elsewhere in this text. However, suitable references are included at the pertinent locations.

18.1 Boulez’s Creative Analysis Revisited In [14], Pierre Boulez describes a compositional strategy, which is called “analyse créatrice”, creative analysis, and which is opposed to what he calls “sterile academic” analysis in that the analytical results are used as germs to create new compositions. Before discussing Boulez’s ideas in detail, we should stress that his procedure transcends the purely analytical or compositional activities: he proposes a coherent double activity, which includes both, analysis and composition. This entails that our own discourse in this chapter will deal with both, analysis and composition, the latter more specifically realized by use of the music composition software R UBATO C OMPOSER.

18.2 Ligeti’s Analysis Ligeti’s analysis [39] of Structures Ia exhibits the totality of rows appearing in this section of the composition. It starts from the given serial rows SP for pitch classes and SD for durations (the primary parameters), as well as SL for loudness and SA for attack (the secondary parameters). It then presents and 201

202

18 Creative Analysis of Boulez’s Structures

investigates that central 12 × 12-matrix1 Q = (Qi,j ) which gives rise to all row permutations for all four parameters. Whereas the construction of Q is relatively natural, the subsequent permutations thereof for the primary parameters, and even more radically those for the secondary parameters seem to be completely combinatorial. Ligeti attributes to these constructions the qualification of combinatorial fetishism.2 This is even aggravated when it comes to the secondary parameters, where Boulez applies what Ligeti calls chess board knight paths, a procedure which in Ligeti’s understanding qualifies as a purely numerical game without any musical significance. This (dis)qualification remains valid in Ligeti’s final remarks on the new ways of hearing, which are enforced by this new compositional technique. He compares the result to the flashing neon lights of a big city, which, although being driven by precise machines, generate an overall effect of statistical sound swarms. He concludes that with this radical elimination of expressivity, an expressivity still present in Webern’s compositions, the composition finds its beauty in the opening of pure structures. And Boulez—we follow Ligeti’s wording—in such a “nearly obsessive-compulsive neurosis, strains himself at the leash and will only be freed by his colored sensual feline world of ‘Marteau’.” Ligeti’s main objection to Boulez’s approach is that he makes abstraction from the parameters and plays an empty game of numbers instead. We now want to contradict this verdict and on the contrary show that in the language of modern mathematics, topos theory, to be precise, Boulez’s strategy is perfectly natural, and in fact, the only reasonable one when dealing with such diverse parameters as pitch classes, durations, loudnesses, and attacks. When we say “natural”, we mean mathematically natural, but the fact that a musical construction is only understood by advanced mathematical conceptualization, and not by naive combinatorial music theory, proves that mathematical naturality effectively hits the musical point. A fact that later in this chapter is confirmed by the possibility to implement our findings in the music software R UBATO C OMPOSER in order to comply with the creative part of Boulez’s principle. Music theorists have to learn that, from time to time, conceptual innovations may even enlighten their ossified domains. It is not the music’s fault if they are “dark to themselves.”3 1 Ligeti names it R, but we change the symbol since R is reserved for retrograde in our notation. 2 “. . . schließlich die Tabellen fetischartig als Maß für Dauernqualitäten angewandt. . . ” 3 Title of a Cecil Taylor LP.

18.3 A First Creative Analysis of Structure Ia

203

18.3 A First Creative Analysis of Structure Ia from Ligeti’s Perspective We observe right from the beginning that there is no intrinsic reason to transfer the twelve pitch class framework to the other parameters. If the number 12 is natural in pitch classes, its transfer to other parameters is a tricky business. How can this be performed without artificial constructs? To understand Boulez’s procedure, let us first analyse the matrix Q construction. It yields one pitch class row for every row. The ideas run as follows. We get off ground by a modern interpretation of what is a dodecaphonic pitch class series SP . Naively speaking, SP is a sequence of 12 pitch classes: SP = (SP,1 , SP,2 , . . . SP,12 ). More mathematically speaking, it is an affine morphism4 SP : Z11 → Z12 , whose values are determined by the 12 values on e1 = 0 and the eleven basis vectors ei = (0, . . . 1, . . . 0), i = 2, 3, . . . 12, where the single 1 stands on position i − 1 of that sequence. This reinterpretation yields SP,i = SP (ei ). The condition that a series hits all 12 pitch classes means that the images SP (ei ) and SP (ej ) are different for i 6= j. This construction has already been the basis of the simple example in chapter 15. This reinterpretation of a dodecaphonic series means that it is viewed as a Z11 -addressed point of the pitch class space Z12 in the language of topos theory of music [47]. This language views the series as a point in the space Z12 , but just from the perspective of a particular domain, or address, namely Z11 . In topos theory of music, a space Z12 is replaced by its functor @Z12 , which at any given address B, i.e., module over a specific ring, evaluates to the set B@Z12 of affine module morphisms f : B → Z12 . This means that the address B is a variable and that our dodecaphonic series is just a point at a specific address among all possible addresses. In other words, the change of address is completely natural in this context. What does this mean? Suppose that we have a module morphism g : C → B between address modules. Then we obtain a natural map B@Z12 → C@Z12 that maps f : B → Z12 to the composed arrow f · g : C → Z12 . For example, if we take B = C = Z11 , and if g(ei ) = e12−i+1 , then the new series SP · g is the retrograde R(SP ) of the original series. Our claim is that all of Boulez’s constructions are simply such address change maps, and as such follow a very systematic construction. So the combinatoriality is viewed as a particular technique from topos theory. Of course Boulez did not know this, since topos theory was not even invented at that time, and Yoneda’s lemma, which is the key to all these mathematical constructions, was only published in 1954, one year after the publi4

An affine morphism f : M → N between modules M and N over a commutative ring R is by definition the composition f = Tt · g of an R-linear homomorphism g : M → N and a translation Tt : N → N : n 7→ t + n. See also subsection 4.1.4.

204

18 Creative Analysis of Boulez’s Structures

cation of Structure I. But this makes his approach even more remarkable; one could even state that in view of this temporal coincidence, Boulez’s structures are the Yoneda lemma in music.

18.3.1 Address Change instead of Parameter Transformations The trick that enables Boulez to get rid of the unnatural association of different parameters with the serial setup stemming from pitch classes is this. One observes that for any (invertible) transformation T : Z12 → Z12 , we have a new pitch class series, namely the composition T · SP . For a transposition T = Tn , we get the n-fold transposed series. For an inversion U(x) = u − x, we get the inverted series, etc. Now, it is evident that one may also obtain this effect by an address change, more precisely, if T : Z12 → Z12 is any affine transformation, then there is precisely one address change C(T) : Z11 → Z11 by a base vector permutation such that the diagram

Z11

C(T)- 11 Z

SP ? Z12

SP

T

? - Z12

commutes. Instead of performing a parameter transformation on the codomain of the pitch class row, we may perform an address change on the domain Z11 . Note, however, that the address change C(T) is also a function of the underlying series SP . What is the advantage of such a restatement of transformations? We now have simulated the parameter-specific transformation on the level of the universal domain Z11 , which is common to all parameter-specific series. This enables a transfer of the transformation actions on one parameter space (the pitch classes in the above case) to all other parameter spaces, just by prepending for any series the corresponding address change. So we take the transformation T on Z12 , replace it by the address change C(T) on Z11 and then apply this one to all other series, i.e., building SD · C(T), SL · C(T) and SA · C(T). This means that we now have a completely natural understanding of the derivation of parameter series from address changes, which act as mediators between pitch class transformations and transfor-

18.3 A First Creative Analysis of Structure Ia

205

mations on other parameter spaces. This is the only natural way of carrying over these operations between intrinsically incompatible parameter spaces. We replace the spaces by their functors and act on the common addresses. This is quite the opposite of purely combinatorial gaming. It is functoriality at its best. For a deeper understanding of Boulez’s selection of his duration series, referring to Webern’s compositions, see [24].

18.3.2 The System of Address Changes for the Primary Parameters Now, nearly everything in Boulez’s construction of part Ia is canonical. The most important address change is the matrix Q. It is constructed as follows. Its i-th row Q(i, −) is the base change C(TSP (i)−SP (1) ) associated with the transposition by the difference of the pitch class series at position i and 1. The natural5 number Q(i, j) in the matrix is therefore Q(i, j) = SP (i) + SP (j) − SP (1), a symmetrical expression in i and j. Moreover, we now see immediately from the definition of the operator C in the above commutative diagram that the composition of two permutations (rows) of the matrix is again such a permutation row, in fact, the transpositions they represent are the group of all transpositions. We may now view Q as an address change Q : Z11 ⊠ Z11 → Z11 on the affine tensor product Z11 ⊠ Z11 , see [47, E.3.3], defined on the affine basis (ei ⊠ ej ) by Q(ei ⊠ ej ) = eQ(i,j) . For any such address change X : Z11 ⊠ Z11 → Z11 , and any parameter Z series SZ → ParamSpace with values in parameter space ParamSpace, we obtain twelve series in that space by address change SZ ·X of the series, and then restricted to the i-th rows of X, or equivalently, prepending the address change rowi : Z11 → Z11 ⊠ Z11 defined by rowi (ej ) = ei ⊠ ej . Given any such address change matrix X : Z11 ⊠ Z11 → Z11 , we therefore get twelve series in every given parameter space. So we are now dealing with the construction of specific matrix address changes and the entire procedure is settled. The general idea is this: one specifies two address changes g, h : Z11 → Z11 with g(ei ) = eg(i) and h(ei ) = eh(i) and then deduces a canonical address change g ⊠ h : Z11 ⊠ Z11 → Z11 ⊠ Z11 by the formula g ⊠ h(ei ⊠ ej ) = eg(i) ⊠ eh(j) . So, when X is given, we obtain a new address change of the same type by building the composed address change X · (g ⊠ h). For example, the retrograde matrix in Ligeti’s terminology is just the matrix Q · (Id ⊠ R) deduced from Q by the address change Id ⊠ R. And Ligeti’s U-matrix is deduced from Q by U ⊠ U, where U is the address change associated with the inversion at e♭ , i.e., it is the composite Q · (U ⊠ U). 5

We represent elements x ∈ Z12 by natural numbers 0 ≤ x ≤ 11.

206

18 Creative Analysis of Boulez’s Structures

Now everything is easy: for the first piano, for the primary parameters pitch class P and duration D, and for parts A and B Boulez creates one matrix Q1P,A , Q1D,A , Q1P,B , Q1D,B address change each, all deduced from Q by the 1 1 above composition with product address changes TP,A = U ⊠ Id, TD,A = 1 1 U · R ⊠ U · R, TP,B = U · R ⊠ U · R and TD,B = R ⊠ U via 1 Q1P,A = Q · TP,A 1 Q1D,A = Q · TD,A 1 Q1P,B = Q · TP,B

(18.1)

1 Q1D,B = Q · TD,B

This is quite systematic, but the second piano is now completely straightforward, in fact the product address changes of this instrument differ just by one single product address change, namely U ⊠ U: 2 1 TP,A = (U ⊠ U) · TP,A 2 1 TD,A = (U ⊠ U) · TD,A 2 1 TP,B = (U ⊠ U) · TP,B

(18.2)

2 1 TD,B = (U ⊠ U) · TD,B

18.3.3 The System of Address Changes for the Secondary Parameters For the secondary parameters, loudness and attack, Boulez takes one such value per series—deduced from the given series SL and SA —that was derived for the primary parameters. Intuitively, for each row in one of the above matrixes, we want to get one loudness and one attack value. For loudness, we start with the Q matrix address change for piano 1 and with the U matrix for piano 2. We then take an address change a : Z11 → Z11 ⊠ Z11 for part A, and another c : Z11 → Z11 ⊠ Z11 for part B. These address changes are very natural paths in the given matrix. Path a is just the codiagonal of the matrix, i.e., a(ei ) = e12−i ⊠ ei , while path c is the path shown in figure 18.1. Contradicting Ligeti’s verdict, these paths are by no means arbitrary. They are both closed paths if one identifies the boundaries of the matrix. Path a is a closed path on the torus deduced from Q by identifying the horizontal and vertical boundary lines, respectively. And path c is closed on the sphere obtained by identifying the adjacent left and upper, and right and

18.3 A First Creative Analysis of Structure Ia

207

Fig. 18.1: The two paths a and c for loudness in part A and part B, resp., in Ligeti’s Q matrix for piano 1. The paths are the same in the U matrix for piano 2.

lower boundary lines, respectively. The torus structure is completely natural, if one recalls that pitch classes are identified exactly like the horizontal torus construction, while the vertical one is a periodicity in time, also a canonical identification. The sphere construction is obtained by the parameter exchange (diagonal reflection) and the identification of boundary lines induced by this exchange. For the attack paths, one has a similar construction, only that the paths a and c are rotated by 90 degrees clockwise and yield paths α and γ. Again, piano 1 takes its values on Q, while piano 2 takes its values on the U matrix. So apart from that rotation everything is the same as for loudness. Summarizing, we need just one product address change given by the U transformation for the primary parameters in order to go from piano 1 to piano 2, while one rotation by 90 degrees suffices to switch between the secondary parameter paths. Observe that this rotation is just the address change on the matrix space Z11 ⊠Z11 induced by a retrograde on each factor! It could not be simpler, and barely more beautiful.

208

18 Creative Analysis of Boulez’s Structures

18.3.4 The First Creative Analysis A first transduction is now immediate. Of course, there are many ways to shift from the given analytical data to neighboring data in the space of analytical data. A first way is evident, and it is also the one which we urgently need to remedy the evident imperfection of the given construction, namely the number of instruments. Why only two instruments? In order to obtain a more intrinsically serialist construction, one should not work with two, but with twelve instruments. This is achieved in the most evident way: we had seen that the second piano is derived from the first by taking the U matrix instead of the Q matrix. This suggests that we may now take a number of twelve address changes Ui : Z11 → Z11 , starting with the identity Id, and generate one instrumental variant for each such address change, starting with the structure for the first piano, and then adding variants for each successive instrument. This yields a total of twelve instruments and for each a succession of twelve series for part A and twelve series for part B, according to the twelve rows of the matrix address changes as discussed above. For the i-th series this gives us twelve instruments playing their row simultaneously. Boulez has of course not realized such a military arrangement of series. We hence propose a completion of the serial idea in the selection of the numbers of simultaneously playing series. Observe that the series SP of pitch classes has a unique inner symmetry which exchanges the first and second hexachord, namely the inversion I = T7 · −1 between e and e♭ , i.e., the series defines the strong dichotomy No. 71 in the sense of mathematical counterpoint theory [47, chapter 30]. In part A, we now select the instrument SP (i) from below and then take I(SP (i)) successive instruments in ascending order (and using the circle identification for excessive instrument numbers). For part B we take the I-transformed sequence of initial instrumental numbers and attach the original serial numbers as successively ascending occupancies of instruments. Figure 18.2 shows the result. The next step will be to transform this scheme into a computer program in order to realize such compositions and to test their quality. It is now evident that such a calculation cannot be executed by a human without excessive efforts and a high risk of making errors. Moreover, it is also not clear whether such creative reconstructions will yield interesting results, or perhaps only for special transformational sequences U1 , U2 , . . . U12 .

18.4 Implementing Creative Analysis in R UBATO C OMPOSER

209

Fig. 18.2: The instrumental occupancies in parts A and B, following the autocomplementarity symmetry I = T7 · −1 of the original pitch class series. The lowest instruments are taken according to the series while the occupancies are chosen according to the I-transformed values. For example, for the first column, we have the serial value 3, and its I-transform is 4, so we add 4 increasingly positioned instruments.

18.4 Implementing Creative Analysis in R UBATO C OMPOSER As already mentioned in section 18.1, the concrete realization of a variety of creative analyses in terms of notes is beyond human calculation power, or at least beyond the patience of the artistic creator. Therefore, we have implemented the above mathematical procedure in R UBATO C OMPOSER. This comprises seven new rubettes, specifically developed for our procedure (see also figure 18.3): BoulezInput, BoulezMatrix, Transformation, BaseChange, Chess, SerialSystem, and Boulez2Macro. We call them boulettes in order to distinguish them from general purpose rubettes. The outputs A and B of boulette Boulez2Macro create one zero-addressed denotator for each part A and B of Boulez’s disposition. These two denotators, MA and MB , are not just sets of notes, but more refined in that they include hierarchies of notes. This is the form in which MA and MB live: it is a circular form, a slightly different version of which we already encountered in section 16.1, namely

210

18 Creative Analysis of Boulez’s Structures

MacroScore:.Power(Node) Node:.Limit(Note, MacroScore) Note:.Limit(Onset, Pitch, Loudness, Duration, Voice) Onset:.Simple(R) Pitch:.Simple(Z) Loudness:.Simple(Q) Duration:.Simple(R) Voice:.Simple(Z) Thus the formal notation of these denotators is MA :0@MacroScore(MA,0 , MA,1 , . . . MA,m ) MB :0@MacroScore(MB,0 , MB,1 , . . . MB,n ) with the nodes MA,i and MB,j , respectively. Each node has a note, its anchor note, and satellites, its MacroScore set denotator. Observe that the concept of an anchor with satellites is, grano cum salis, also the approach taken by Boulez in his multiplication of chords, where the anchor is the distinguished note, and where the satellites are represented by the intervals of the other notes with respect to the anchor. The outputs A and B are then united in the Set rubette and its output C is sent to the AllFlatten rubette, which recursively “opens” all the nodes’ satellite MacroScore. How is this performed? Given a node with empty satellite set, one just cuts off the set. Otherwise, one supposes that its satellite MacroScore has already recursively performed the flattening process, resulting in a set of notes. Then one adds these notes (coordinate-wise) to the node’s anchor note. This means that the satellites are given the relative position with respect to their anchor note. A trill is a typical example of such a structure: the trill’s main note is the anchor, while the trill notes are the satellites, denoted by their relative position with respect to the anchor note (see again section 16.1). The output from the Boulez2Macro boulette is given as a MacroScore denotator for strong reasons: we want to work on the output and to take it as a primary material for further creative processing in the spirit of Boulez, a processing, which, as we shall see, requires a hierarchical representation. Moreover, the multiplication of chords used in Structure II implicitly uses a hierarchical construction of the above type: the base point x in the pointed chord ax is precisely the instance of an anchor note, as we shall make precise in the following. Therefore, the chosen MacroScore form acts as a unifier of conceptual architectures in parts I and II of this composition.

18.4 Implementing Creative Analysis in R UBATO C OMPOSER

211

Fig. 18.3: The R UBATO network generating MIDI files (played by the ScorePlay rubette) with arbitrary input from the creative analysis that is encoded in the BoulezInput rubette.

18.4.1 The System of Boulettes But let us see first how the Boulez composition is calculated. We are given the following input data (referring to figure 18.3): Output 1 from the BoulezInput boulette contains the series for all parameters as a denotator Series:Z11 @BoulezSeries(SP , SD , SL , SA ) of the form BoulezSeries:.Limit(P, L, D, A) with the factor forms P:.Simple(Z12 ), D:.Simple(R), L:.Simple(Z) and A:.Simple(R3 ). The attack form A has values in the real 3-space, where the first coordinate measures the fraction of increase of nominal loudness, the second the articulatory fraction of increase in nominal duration, and the third the fraction of shift in onset defined by the attack type. For example, a sforzato attack (sfz) would increase nominal loudness by factor 1.3, shorten duration to a staccato by 0.6, and the third would add to the nominal onset a delay of −0.2 × nominal duration. As discussed in section 18.2, the address Z11 yields the parametrization by the 12 indices required for a serial sequence of parameters. For example, the pitch class series is the factor denotator SP :Z11 @P(3, 2, 9, 8, 7, 6, 4, 1, 0, 10, 5, 11). Output 2 contains the two address changes for retrograde R and inversion U. They are encoded as denotators R:Z11 @Index(R1 , R2 , . . . R12 ) and U:Z11 @Index(U1 , U2 , . . . U12 ) in the simple form Index:.Simple(Z), which

212

18 Creative Analysis of Boulez’s Structures

indicates the indices Ri of the affine basis vectors, which are the images of the basis vectors ei of C(R) and C(U). Output 3 encodes the above sequence U. = (Ui ) of address changes for the instrumental sequence, i.e., a denotator U.:Z11 @Sequ(U1 , U2 , . . . U12 ) with Ui :Z11 @Index(Ui,1 , Ui,2 , . . . Ui,12 ), where we use the list type form Sequ:.List(Index). (In fact this also works for any number of instruments, but we restrict our example to twelve instruments as chosen above.) Output 4 encodes the registers, which must be defined in order to transform the pitch classes into real pitches. We give this information in the same form as the address change sequence, where the coordinates for the i-th sequence are the numbers of the octave, in which the pitches of the respective pitch class series in the corresponding instrument are positioned. Octaves are numbered starting from octave 0 at pitch 60 in MIDI format. This information is also used to position the pitches according to an instrumental range. The Split rubette takes the input series Series and sends its pitch class factor SP to output 5 . This denotator is taken as input of the BoulezMatrix boulette and yields the famous matrix Q at output 6 , which we interpret as a denotator Q:Z11 ⊠ Z11 @Index(Qi,j , i, j = 1, 2, . . . 12). The BaseChange boulette is devoted to the calculation of the address changes on output 8 for the primary parameters described in the four formula groups equation 18.1 and equation 18.2 for the primary parameters of the 12 instruments and take as input the matrix Q from output 6 and the sequence U. of instrumental address changes. The Chess boulette is devoted to the calculation of the corresponding address changes on output 9 for the secondary parameters loudness and attack, as described by the chess board paths. Given the address change systems on input 8 and 9 of the SerialSystem boulette, and taking as a third input the total series from output 1 , the total system of series is calculated according to our formulas described in subsection 18.3.2 and subsection 18.3.3. This yields output 10 , which is finally added as input, together with the input 4 of octaves to calculate the effective parameters. The pitches, nominal durations, and loudnesses are now given, the nominal onsets are calculated to produce the rectangular scheme shown in figure 18.2. The attack data is used to transform the nominal values into the attack-specific deformations, and we obtain the outputs A and B as required. This output is a denotator of form MacroScore. Its nodes in part A and B are 144 series each. The anchor note of each serial node is taken to be the first note in the series. The satellites of this node are the remaining 11 notes with their relative position with respect to the anchor note. Moreover, the output denotators at A and B have one instrumental voice number for each instrument. Taking the union of these parts in output C , we obtain

18.5 A Second More Creative Analysis and Reconstruction

213

a large MacroScore denotator MC = MA ∪ MB . Selecting from this system the series as shown in figure 18.2 yields the final “raw” material, which will now be used to generate more involved creative constructions in section section 18.5. The system as calculated by R UBATO is shown in figure 18.4. The graphical representation is realized using the BigBang rubette for geometric composition. The input to this rubette is the denotator MC , while the selection of the instruments according to figure 18.2 is made by direct graphical interactive manipulation. The functionality of the BigBang rubette is discussed in subsection 18.5.2.

Fig. 18.4: The final “raw” material for 12 instruments. Instruments are distinguished by colors. Satellites pertaining to a given anchor note are connected by rays to that anchor note.

18.5 A Second More Creative Analysis and Reconstruction One of the most creative extensions of techniques in musical composition is the opening of the transformational concept. This was already a crucial argument in Boulez’s own construction of derived Q matrixes, where he invented that ingenious tool of address change in order to extend pitch class transformations to parameters, where such operations would not apply in a natural way. Our extension of Boulez’s approach was presented above and implemented in R UBATO’s boulettes, yielding the denotator MC :0@MacroScore(). In this section, we shall add other extensions of the given transformations and apply them to the construction of huge extensions starting from the present “raw material” MC . There are two threads of extensions, which we

214

18 Creative Analysis of Boulez’s Structures

shall expose: the first is the conceptual extension, i.e., conceiving new types of transformations, while the second deals with the associated concrete manipulation of compositions on the level of graphical interactive gestures. The background of this double strategy is the following general idea: the formulaic rendition of compositional tools, when implemented in software, pertains to what is somewhat vaguely called algorithmic composition. This is what happens in R UBATO’s boulettes. The drawback of such an implementation is that the result is “precooked” in the cuisine of the code and cannot be inspected but as a res facta. A composer would prefer to be able to influence his or her processes in the making, not only when it is (too) late. This is why we have realized a different strategy: the transformations, which are made possible by the BigBang rubette, are immediately visible at the time of definition and can be heard without delay. The general idea backing this approach is that conversely, any algorithm should be transmuted into a graphical, interactive and gestural interface, where its processes would be managed on the fly, gesturally, and while they happen. Why should I wait until rotation of musical parameters is calculated? I want to generate it and while I actually rotate the system by increasing angles, I would like to see the resulting rotated set of note events, hear how it sounds, and then decide on the success or failure of that rotation.

18.5.1 The Conceptual Extensions The conceptual extension of transformations has two components: the extension of the transformations as such and the application of such transformations as a function of the hierarchical structure of the MacroScore form. The serial transformations on the note parameters of a composition usually comprise the affine transformations generated by inversion (pitch reflection), retrograde (onset reflection), transposition (pitch translation), and time shift (onset translation). But it also includes the construction of assemblies of iterated transformations, not just one transformed note set, but the union of successively applied transformations. The latter is typically realized by regular patterns in time, where rhythmical structures are constructed. So we have these two constructions: given a set of notes M and a transformation f, one either considers one transformed set f(M) or else the S union i=0,1,...k f i (M). The latter is well known as a rhythmical frieze construction if f is a translation in time. If we generalize frieze constructions to two dimensions, using two translations f and g in the plane, we obtain a S wallpaper i=0,1,...k,j=0,1,...l f i gj (M).

18.5 A Second More Creative Analysis and Reconstruction

215

18.5.1.1 Extensions of Single Transformations The natural generalization of such transformational constructions is to include not only those very special transformations, but any n-dimensional −→ non-singular affine transformation f in the group GLn (R), whose elements are all functions of the form f = Tt · h, where h is an element of the group GLn (R) and where Tt (x) = t+x is the translation by t ∈ Rn . It is well known from mathematical music theory that any such transformation can be decomposed as a concatenation of musical standard transformations, which, each, involve only one or two of the n dimensions. In view of this result, we have chosen the generalization of the above transformations to these special cases in 2-space: (1) translations Tt , (2) reflections Ref L at a line L, (3) rotations Rotα by angle α, (4) dilation DilL,λ vertical to the line L by factor λ > 0, (5) shearing ShL,α along the line L and by angle α. These are operations on real vector spaces, while we have mixed coefficients in the MacroScore form. The present (and quite brute) solution of this problem consists in first embedding all coefficients in the real numbers, to perform the transformations and then to recast the results to the subdomains, respectively. −→ Given the group GLn (R) of transformations (generated by the above twodimensional prototypes), we now have to deal with the hierarchical structure of denotators in the MacroScore form. How can transformations be applied to such objects? To this end, recall that a MacroScore denotator is6 a set M of nodes N = (AN , SN ), which have two components: an anchor note AN from the (essentially) five-dimensional form Note and a MacroScore formed satellite set SN . Common notes are represented by nodes having −→ empty satellite sets. Given a transformation f ∈ GLn (R) and a MacroScore denotator M, a first operation of f upon M is defined by anchor note action: f · M = {(f · AN , SN ) | N ∈ M}

(18.3)

This type of action is very useful if we want to transform just the anchors and leave the relative positions of the satellite notes invariant. For example, if the satellites encode an embellishment, such as a trill, then this is the right operation in order to transform a trill into another trill. This operation is easily generalized to any set S of nodes in the tree of MacroScore denotator M, such that no two of them are hierarchically related (one being in the satellite tree of the other). The above situation of formula 18.3 referred to the top level anchors. Suppose that S consists of nodes N. For non-satellite nodes, we have the above function. Suppose now that such a node N is a satellite pertaining to a well-defined anchor note A(N). Thinking of that anchor note as a local coordinate origin, we may now 6

All denotators in this discussion will be zero-addressed.

216

18 Creative Analysis of Boulez’s Structures

−→ apply a transformation f ∈ GLn (R) to all selected satellite nodes of A(N) by the above formula 18.3, yielding a transformed set of satellites of the same anchor note. We may apply this operation to each set of satellites of given anchors occurring in S. Since there are no hierarchical dependencies, no contradiction or ambiguity appears, i.e., no note will be transformed together with one of its direct or iterated satellite notes. This means that we are simultaneously applyingFf to all satellite sets of S. In other words, we take the disjoint union S = Sk of satellite sets Sk pertaining to specific anchor notes Ak and then apply a simultaneous transformation f to each of these Sk . We denote this operation by f ⊙ S. There is another operation, which we may apply to a set S with the above properties. It does not take the relative positions of S-elements, but their flattened position and then applies the transformation f to these flattened notes. It is the operation one would apply in a hierarchical context, such as a Schenker-type grouping, but without further signification of the hierarchy for the transformational actions. After the transformation, each of these transformed flattened notes is taken back to its original anchor note. For example, if s = 1, and if N = (AN , SN ) is a satellite of level zero anchor note A(N), then we first flatten the note (once), which means that we take N ′ = (A(N)+AN , SN ), we then apply f to its new anchor A(N)+AN , yielding N ′′ = (f(A(N) + AN ), SN ), and we finally subtract the original anchor, yielding the new satellite N ′′′ = (f(A(N) + AN ) − A(N), SN ) of A(N). This operation is again denoted like the above operation, i.e., by f · S.

18.5.1.2 Extensions of Wallpapers Let us now review the construction of wallpapers in view of a possible creative extension. Mathematically speaking, a wallpaper is a structure that is produced by repeated application of a sequence of translations T· = (Tt1 , Tt2 , . . . , Ttr ) acting on a given motif M of notes. Each Tti of these translations is repeatedly applied in the interval numbers of the sequence I. = (Ii = [ai , bi ]), ai ≤ bi , of integers, which means that the total wallpaper is defined by W(T· , I.)(M) =

[

Tλ1 t1 Tλ2 t2 . . . Tλr tr (M)

(18.4)

ai ≤λi ≤bi

This formula has nothing particular regarding the special nature of the different powers of translations. This means that the formula could be generalized without restrictions to describe grids of any sequence of transfor−→ mations f. = (f1 , f2 , . . . , fr ) for fi ∈ GLn (R), thus yielding the generalized wallpaper formula

18.5 A Second More Creative Analysis and Reconstruction

W(f., I.)(M) =

[

λ

λ

f1 1 ◦ f2 2 ◦ . . . frλr (M)

217

(18.5)

ai ≤λi ≤bi

which also works for negative powers of the transformations, since these are all invertible. In our context, the motif M will no longer be a set of common notes, but a denotator of MacroScore form. Therefore, we may replace the naive application of transformations to a set of notes by the action of transformations on such denotators as discussed above. This means that— mutatis mutandis—we have a two transformation wallpaper for a set S of nodes of a MacroScore denotator with the above hierarchical independence property, the relative one W(f., I.) ⊙ S =

[

f1 1 ◦ f2 2 ◦ . . . frλr ⊙ S

[

f1 1 ◦ f2 2 ◦ . . . frλr · S

λ

λ

(18.6)

λ

λ

(18.7)

ai ≤λi ≤bi

or the absolute one: W(f., I.) · S =

ai ≤λi ≤bi

This generalizes the transformations and the motives in question. A last generalization is evident, when looking at the range of powers of the intervening transformations. To the date, these powers are taken within the Q hypercube D = i Ii of sequences of exponents. However, we may admit more generally any finite “domain” set D ⊂ Zr and make the union according to the sequences of exponents appearing in D: W(f., D) ⊙ S =

[

f1 1 ◦ f2 2 ◦ . . . frλr ⊙ S

[

f1 1 ◦ f2 2 ◦ . . . frλr · S

λ

λ

(18.8)

λ

λ

(18.9)

(λ1 ,λ2 ,...λr )∈D

or else the absolute one: W(f., D) · S =

(λ1 ,λ2 ,...λr )∈D

These are the generalizations we need to describe the transformations in Structures in a uniform way. The situation in Structures I has been described above. Let us make a remark on the nature of Boulez’s so-called multiplication which he uses for the composition of Structures II. We restate Boulez’s general formula for multiplication [13, part I] in the due (and natural) mathematical generality: we are given two pointed sets ax and by of note events, notes being elements in a module M. A pointed set ax is a set a together with a distinguished element x ∈ a, which we call base point hereafter. Then the product ax by is defined by the formula

218

18 Creative Analysis of Boulez’s Structures

ax by =

[

Tz−x a

(18.10)

z∈b

with the right base point y as the new base point. If the module M is totally ordered, e.g., for the integers M = Z that may represent pitch, the distinguished element x is chosen to be the minimal element, i.e., the lowest note, of a. This multiplication is associative, but not commutative. However, we have Ty (by ax ) = Tx (ax by ), (18.11) the transposition being taken on the set elements and on the base points. The product commutes if the base points of its factors coincide. One recognizes that it has the shape of a generalized wallpaper. In fact, we have to represent the pointed set ax by a node (x, ∆x (a)), where the anchor is x, and the satellite set is ∆x (a) = T−x a, the interval set in the classical case of pitch space. We then have the translations Tβ−x , β ∈ b from the right factor, and the domain is just the set of card(b) units {(1, 0, . . . 0), (0, 1, . . . 0), . . . (0, 0, . . . 1)}. Alternatively, this multiplication can also be described in terms of the two nodes (x, ∆x (a)) and (y, ∆y (b)), which rightly formalize the pointed sets and the associated interval satellites: we set (x, ∆x (a)) · (y, ∆y (b)) = (y, ∆x (a) + ∆y (b)). Here the sum of satellite sets must be defined recursively as follows: the sum U + V of two MacroScore denotators U and V is defined by U + V = {(Au + Av , Su + Sv ) | u ∈ U, v ∈ V}. In particular ∅ + V = ∅. Since our denotators all end up with the empty satellite sets down their satellite trees, this definition is complete. This sum is commutative and associative. The above constructions were not specified with regard to the addresses involved in these denotators. Now, the sum of MacroScore denotators is in fact well defined for any address, essentially, we only need the sum of notes, and this works without any restriction of addresses. The Boulez algebra construction is in fact functorial, and we may therefore reconsider multiplication for general addresses, in particular: for the serial address Z11 and its associated affine tensor product Z11 ⊠ Z11 . Musically, this amounts to consider serial compositions and multiplication of chords of series instead of usual chords, and also more general objects in MacroScore form. In the following, we have not yet implemented this functorial point of view in the composition tools of the BigBang rubette, but this is by no means problematic since functorial wallpaper construction has already been implemented (see section 16.2 and [68]).

18.5 A Second More Creative Analysis and Reconstruction

219

18.5.2 A Closer Look at the BigBang Rubette In chapter 17, we introduced most of the functionality of the BigBang rubette. However, when it comes to processing hierarchical structures of the form MacroScore, there is an additional way of performing transformations. Since we use this method later in this chapter, we want to discuss it in this context. Everyone of the five basic transformations (translation, rotation, reflection, dilation and shearing) can be applied as a relative transformation. To illustrate what this means, we first have a closer look at how a simple rotation is performed. After having selected the notes to be rotated (by drawing rectangles with the mouse around the critical note groups), the user defines a rotation center by clicking anywhere on the visible plane. Then, when he clicks and holds the mouse button anywhere away from the selected center and moves the mouse, a rotation tool appears around the center, showing the current angle in gray. As long as the mouse button is pressed, the current result of the rotation is visualized immediately whenever the mouse is moved. The rotated set of notes is also immediately played when the user stops moving the mouse. As soon as the mouse button is released, the rotation is performed definitively. Figure 18.5 shows a rotation applied to the first bars of Beethoven’s op. 106, Allegro.

Fig. 18.5: Rotation of the first bars of Beethoven’s op. 106, Allegro (left). The rotation circle shows the mouse movement on its periphery, the original is also shown.

220

18 Creative Analysis of Boulez’s Structures

After a transformation, the transformed notes remain selected, so that the user may add another transformation to the same group of notes, etc. . . This enables a spontaneous and delayless transformational gesture in musical composition. A geometrical transformation such as rotation can also be applied to any set of anchor and satellite notes in MacroScore structures. Nevertheless, when we decide to transform satellites of different anchor notes, we might require that every selected satellite is transformed relatively to the position of its anchor note. For example, we might want to rotate each satellite around its own anchor note. To do this, the user chooses a arbitrary set of satellites throughout the given composition. Then he chooses one anchor note and defines the center of rotation relative to that anchor. When he starts the rotation, the rotation tool is shown for that center, while each selected satellite note is rotated relatively to its anchor note. Figure 18.6 shows an example, where we have created two satellite groups derived from the composition in figure 18.5. After selecting all satellites and the anchor note of the group to the right, we have defined the rotation center near the right anchor note. In this figure, the original notes are also visible, which leads to an overlapping structure to the right.

Fig. 18.6: Here, a relative rotation is performed on the two satellite sets, with their two anchor notes at the rays’ centers. The original notes are also shown.

18.5 A Second More Creative Analysis and Reconstruction

221

Relative transformations and transformations of hierarchical structures may also be used when creating wallpapers as defined in equation 18.8 and equation 18.9. Figure 18.7 shows an example of a two-dimensional wallpaper whose motif is darkened.

Fig. 18.7: A wallpaper is built from a motif (darkened). Two transformations are used, both are translations followed by a rotation and a shrinking dilation.

18.5.3 A Composition using the BigBang Rubette and the Boulettes Guerino Mazzola and Schuyler Tsuda created a musical composition, appropriately named Restructures, using the above techniques, starting from the raw material MC shown in figure 18.4. Additionally the alteration techniques implemented in the BigBang rubette have also been applied, but these will not be discussed further here. The composition can be downloaded from http://www.encyclospace.org/special/restructures.mp3. This composition has four movements. Each movement is transformed according to a specific geometric BigBang rubette technique, which will be described in the following paragraphs. After executing these operations, the twelve voices of each movement, which are available as twelve separate MIDI files, were elaborated by adequate orchestrations. This was realized by Schuyler Tsuda, who is an expert in sound design. He orchestrated and assigned the MIDI files to specific sounds in order to transform the abstract events into an expressive body of sound.

222

18 Creative Analysis of Boulez’s Structures

The first movement (Expansion/Compression) takes a copy of MC , then “pinches” the satellites (but not the anchors) of part A in the sense that the first (by onset) satellites are altered 100% in pitch direction only to a defined pitch, whereas the last satellites are left as they were (0% alteration). The satellites in-between are pinched by linear interpolation. The same procedure is applied to part B, however this time the pinching is 100% at the end and 0% at the start. This is shown in figure 18.9. The instrumentation is shown in figure 18.8. Voice Instrument 1 2 3 4 5 6 7 8 9 10 11 12

Grand piano Scraped, bowed, rolled and struck suspended cymbals Electronic mallets Solo cello Pizzicato strings Electronic space strings Plucked E-bass Grand piano Electronic percussion Timpani Electronic toms Electronic bells Fig. 18.8: Instrumentation of the first movement.

Fig. 18.9: First movement: variable pinching the satellite onsets. 100% pinching at start and end onsets, no pinching in the meeting of the end of part A and the start of part B.

For the second movement (Space-Time) we take another copy of MC and expand the onsets and the durations of the anchors of the second appear-

18.5 A Second More Creative Analysis and Reconstruction

223

ance to the double, which yields the situation shown in figure 18.11. The instrumentation for the second movement is shown in figure 18.10. Voice Instrument 1 2 3 4 5 6 7 8 9 10 11 12

Strings Flute and horn Grand piano Sine waves Electronic voice Grand piano Trombone and tuba Electronic strings Triangle and finger cymbals Bowed piano Clarinets Electronic bells

Fig. 18.10: Instrumentation of the second movement.

Fig. 18.11: Second movement: expanding onsets and durations.

For the third movement (Rotations), taking again a copy of MC , and focusing first on part A, we apply a retrograde inversion to the anchors, and then in a second operation also to all satellites relative to their anchors. We then take part B and apply a rotation of all satellites, relative to their anchors, by 45 degrees in counterclockwise direction. The result is shown in figure 18.13. The third movement is instrumented as shown in figure 18.12. Finally, for the fourth movement (Coherence/Opposition), taking again a copy of MC , we take part A, and pinch to low pitches the anchors, and dilate their durations, whereas the satellites are pinched to high pitches. In part B, we also operation such separation of pitch of satellites from anchors, but we also execute a progressive pinching of the pitches towards a fixed pitch towards the end of the composition. The result is shown in figure 18.15. The fourth movement is instrumented as shown in figure 18.14.

224

18 Creative Analysis of Boulez’s Structures Voice 1 2 3 4 5 6 7 8 9 10 11 12

Instrument Sine waves Oboe and bassoon Pizzicato strings Marimba Horns Electronic mallets Temple blocks and tam-tam Grand piano Electronic percussion Sine waves Trombones Electronic bells

Fig. 18.12: Instrumentation of the third movement.

Fig. 18.13: Third movement: retrograde inversion of anchors and satellites in part A, rotation of satellites in part B.

Voice 1 2 3 4 5 6 7 8 9 10 11 12

Instrument Glockenspiel and electronic noise Glockenspiel and electronic noise Grand piano and electronic noise Harp, electronic noise and pizzicato strings Sine waves Finger cymbals and timpani rolls Electronic bells Grand piano Chinese opera gong and low and high gongs Bowed cymbals Triangle and bass drum rolls Triangle and bass drum rolls

Fig. 18.14: Instrumentation of the fourth movement.

18.5 A Second More Creative Analysis and Reconstruction

225

Fig. 18.15: Fourth movement: Sucking down the anchors, expanding their durations, lifting the satellites in part A, then progressive pinching notes in part B.

Chapter 19

Conclusion and Outlook

No ambitious project will ever become completed. In this regard, the implementation of R UBATO C OMPOSER makes no exception. This is to be seen as fortunate. How boring a software primarily intended for musical composition would be if ever completed. Would this not mean, that no further interesting discoveries could be made? Looking at it in a different way, the R UBATO concept is based on openness. The use of the language of forms and denotators, as well as the extensibility by rubettes, guarantees that R UBATO applications will not be limited in the future, whatever the current focus may be. The completeness therefore lies in the conceptualization and not in the contents at any one time. The goal of the R UBATO project is to build a community that is interested in furthering mathematical music theory and composition using the R UBATO C OMPOSER software. Therefore, researchers and composers who wish to contribute are invited to visit http://www.rubato.org, where further information can be found. The present chapter presents the conclusion of this thesis. It provides an assessment of the various aspects of the work done. Then those parts of the software that exist, but are not in a functional state, are discussed. Finally, ideas for future research are proposed. These are projects that require a more extensive setup than the one in which the current software was set.

19.1 Lessons Learned The work on R UBATO C OMPOSER results in several important insights: First, it is possible to make accessible an advanced mathematical theory within the limits of a software implementation intended for nonmathematicians. Of course, there are many software packages for the math-

227

228

19 Conclusion and Outlook

ematician, physician, computer scientist, or engineer, which fulfill their purpose in an admirable way, for example general computer algebra systems, such as M ATHEMATICA or M APLE. However, if one offers such tools to a composer with only slight knowledge of mathematics, then he or she will not be able to make use of their power. Anyway, the tools for working with category theory, in particular forms and denotators, would still have to be implemented and made available on these systems. R UBATO C OM POSER tries to provide those mathematical tools in terms of the tasks that a composer expects. The rubettes are designed to deal with such common problems, as the generation of patterns. Certainly, not every aspect is completely successful yet in this respect. Therefore there is still work to be done, as outlined in section 19.2. The second important point is the realization that there is yet another discipline, where the import of category theory can no longer be ignored. After engineering disciplines, such as computer science, have been infiltrated by reformulations through category theory, it is now the arts, specifically music, which is subject to such considerations. The book The Topos of Music, which has been the primary source of this thesis, is proof of the beginning of a new era of mathematical and computational music theory and composition.

19.2 Things To Do This section can be regarded as a to-do list. As noted in previous chapters, some important aspects of the full theory had to be omitted. The infrastructure is, however, available. Therefore the completion of at least some of these parts is immediately possible, given enough time and resources for development. This applies also to various details of implementation that are not in the desired state of generality. Among the things to be done is a fairly complete implementation of diagrams and an enrichment of the list of modules, such as Fourier series. Another important class of projects deals with the relation to presto and classic R UBATO. R UBATO C OMPOSER intends, amongst others, to make available most of tools provided by presto, or better yet, an enhanced version of these tools. The functionality of these tools should find their way in a set of rubettes. The Wallpaper rubette is a case in point of such a reimplementation based on new theoretical insights. The presto tools will then be available on the basis of the functorial denotator-based concepts and thereby gain a high degree of flexibility. Other tools waiting for a reimplementation using the R UBATO C OMPOSER framework include classic R UBATO’s Metro, Melo, Harmo, and Performance rubettes.

19.3 Ideas for Future Work

229

The author is not an expert in interface design. Although the current design seems to be quite usable, some straightforward realizations of the humancomputer interface are rather primitive, and need to be more intuitive. One example is the many ways morphisms can be specified. Transformations in the real Euclidean plane can be defined graphically. This should be extended to three-dimensional spaces. Higher-dimensional spaces need specific approaches, as do other types of modules, such as modular integers and polynomials. There are opportunities for novel ideas and R UBATO C OMPOSER provides the ideal platform for their implementation. The R UBATO framework and the R UBATO C OMPOSER GUI are separate entities. It is therefore conceivable to design an entirely new graphical frontend based on different design principles. However, this would be a project on a much larger scale.

19.3 Ideas for Future Work The reassessment of presto tools transports old concepts to the present. Certainly, the new framework allows for novel composing strategies. It is hoped that experimentation will lead to the implementation of new “theories” and produces new types of musical composition guided by the principles exposed in this thesis. For the implementation of R UBATO C OMPOSER, several design decisions have been made. At each point along the way, where a decision has been made, several alternatives were available. Generally the alternative has been selected after an evaluation that considered the impact it would have on the implementation on the one hand and the effect on the potential user on the other. Altering such a decision may result in restructuring the software, since every decision influences the decision made thereafter. However, it is important not to fix the choices for eternity. Future research should try to introduce different approaches and possibly try to integrate them with the current ones. One such important decision was the choice of the category underlying the concept framework of forms and denotators. Currently, the implementation is based on the category of modules and diaffine homomorphisms. The actual code tells us, of course, that this not entirely true, since we can switch to the category of sets, whenever we need mappings that are not diaffine homomorphisms. Unfortunately, this is necessary in this real, not ideal, world. Nevertheless the foundations of modules are quite strong. This is justified by the enormous computational support of modules. Think of the

230

19 Conclusion and Outlook

vast knowledge of matrix computation and linear algebra, accumulated by computer science and numerical mathematics. Another possible choice for the category would have been the category of topological spaces and continuous mappings. This is extremely important for the theory of gestures which deals with topologies and continuous curves [49]. Obviously, this theory is essential to mathematical music theory, in that it includes the study of performance in addition to the traditional fields of harmony, counterpoint, etc. In contrast to module theory, which is constructive in many essential ways, topology is much more elusive to computational approaches. In order to incorporate this theory in a meaningful and practical way, much research will be necessary. R UBATO C OMPOSER may be the right tool for experimentation. If ever a computational theory of topology is available to R UBATO C OM POSER, it is desirable to be able to use in a way that is not exclusive. The theory of modules is still very important and provides features not available in the theory of topology, and vice versa. The integration of multiple categories is difficult, but important, and should be a main focus in future developments of R UBATO C OMPOSER. The denotator model allows for very complex structures, made even more abstract by the functorial implementation. Common graphical visualization techniques already get into trouble when Euclidean spaces with dimension four and higher come into play. The technique of “folding” realized by Stefan Göller ([29], see also figure 7.5) is one approach to a solution. Representations need not be exclusively graphical (or textual, for that matter), but may also be audiovisual, or even gestural. In general, dynamic types of representation, instead of static ones, will lead to new ways, which need to be explored and cast into rubettes. Of course, specific representations for well known data models should still be used. The ScorePlay rubette, for example, plays a musical score and displays it using a sequencer-like diagram. Common music notation is very important to composers and music theorists. Therefore, rubettes that generate and display common music notation, as well as allow its manipulation, must be implemented. Already before the current R UBATO C OMPOSER implementation, some steps have been made towards a distributed rubette system [30]. Rubettes could be located on different hosts and connected via Java RMI (remote method invocation). Network-enabled rubettes would allow the distribution of resources, such as libraries or music hardware, and even provide a collaborative tool for music research. Development in this direction has stalled for the duration of the implementation of R UBATO C OMPOSER. It should be taken up again, now that the R UBATO framework has been mostly completed.

Chapter 20

User’s Manual

20.1 Introduction This manual is intended as a standalone reference to the use of R UBATO C OMPOSER, as well as a guide to implementing rubettes. Therefore there is some duplication of information with respect to the main text of this work, in particular the short discussion of the concepts underlying R UBATO C OMPOSER.

20.2 Concepts R UBATO C OMPOSER uses elements from visual programming such as components and data-flow links. The components are called rubettes and are linked together and grouped in networks.

20.2.1 R UBATO C OMPOSER’s World of Objects From an abstract point of view, R UBATO C OMPOSER is an environment for creating and manipulating objects. Therefore it is important to know the world of objects represented within R UBATO C OMPOSER. Each object may be stored under a name in a global system repository. The namespaces for the kinds of objects are disjoint. Modules:

Mathematical modules are the fundamental data types. They include integers, reals, vector spaces, etc.

233

234

20 User’s Manual

Module elements: Elements of modules are integers, reals, vectors, etc. Module morphisms: Functions between modules are called morphisms. They include affine morphisms, embeddings, etc. Forms: Forms constitute the general “types” of denotator theory. A form has an intrinsic non-empty name and must be registered with this name in the system repository. Denotators: Denotators are the most general objects. A denotator may be anonymous or it may have an intrinsic name. A denotator with a non-empty name can be stored under this name in the system repository. Rubettes:

A rubette instance is a computational entity, which works on denotators (see also subsection 20.2.2). Rubette instances exist only within networks.

Networks:

Rubettes are placed and interconnected in networks to create data flow schemes.

A R UBATO C OMPOSER project is an environment consisting of a collection of objects of above kinds. A project can be saved to disk in a file with extension .rbo. Thus a project file contains a list of definitions, where a definition is an assignment of a name to an object, such as a form to its intrinsic name, a network to its name given during its creation or a module morphism that has been assigned a name, etc. Saving a project into a file and reading it back has the effect of restoring the environment. Be aware, however, that only objects that have been bound to a name, and objects that these objects recursively depend on, are saved. Normal usage of R UBATO C OMPOSER does not produce non-transient dangling objects, i.e., objects that should be saved, but are not.

20.2.2 Rubettes A rubette is a computational entity consisting of several inputs (including none), several outputs (including none) and a computational core that computes outputs from inputs. In general a rubette may provide a visual interface, of which there are two types: 1. Properties: This consists usually of a dialog that presents the UI elements for controlling the exact behavior of a rubette. In other words, here the fundamental parameters may be tweaked. 2. View: This window produces an audiovisual representation of the informational content of the rubette. For example, a rubette may show a

20.2 Concepts

235

musical score as a piano roll, similar to MIDI sequencer software. It may also allow playing the score through the local sound system or through connected MIDI hardware. Examples of these will be shown in section 20.4 where all core rubettes are described. There are no constraints on the type of computation that a rubette embodies. It may be a simple mathematical calculation, but can also generate graphics or audio data, for example as a MIDI player. However, it cannot be assumed that a rubette runs in real-time. Inputs Icon

Rubette instance name Information text

Properties button Output Fig. 20.1: Graphical representation of a rubette.

Figure 20.1 shows what a rubette looks like as a manipulable object in R U BATO C OMPOSER. This is a simple rubette that takes several input denotators of type power (i.e., sets) and produces a result that is a set-theoretic function of its inputs. The input connectors (three of them in this case) are always on top, the output connectors (one in this case) at the bottom. Connectors are colored blue when connected and red when not (this is seen in figure 20.2). An optional icon indicates the rubette type. The instance name is a string that can be assigned by the user to this rubette instance. By default a new instance has a name derived from the name of the rubette (SetOp in this case) with a number appended. The SetOp rubette has a properties dialog that can be opened by clicking the Properties button. If a rubette has a view, then a second button for opening the view would appear. Generally a rubette can be customized for the operation it performs. In this case, operations include set union and intersection. A rubette may show some information about its currently configured operation by specifying an information text. For example, if the user changed the operation to intersection in the properties dialog, the information text would display “Intersection” instead of “Union”.

236

20 User’s Manual

20.2.3 Networks R UBATO C OMPOSER manages any number of networks, organized in a tabbed panel. A network is a canvas, whereunto rubette instances are placed and connected. A network is just a data flow diagram, similar to other visual programming tools. Figure 20.2 shows an example of such a network: it contains four rubettes, named Source #1, Display #2, Set #3 and Display #4.

Fig. 20.2: Graphical representation of a network.

Links are dragged from an output connector of a rubette to an input connector of another rubette, or vice versa. Several links can start from one output connector, but at most one link can enter an input connector. It is however possible to connect several inputs to one input connector by routing them first through a multiplexer rubette. It is currently not allowed to create directed cycles when building a network. To perform the computation embodied by an network, it is executed, or run. When execution is requested, the scheduler must determine in what order the rubettes are run, since rubette inputs depend on outputs of other

20.2 Concepts

237

rubettes. To this end a topological sort is done, and rubettes are run according to the resulting linear order. In the example from figure 20.2, the order would be Source #1 ≺ Display #2 ≺ Set #3 ≺ Display #4, where the preference of Display #2 over Set #3 is purely dependent on the implementation and should be regarded as random. This mode of execution is known as complete run, since every rubette is scheduled. Another mode is continuous run. Here the network is executed as soon as a rubette’s configuration has been changed by the user. The topological sort only considers the modified rubette and its dependents, and only these are submitted to the scheduler for a partial run. This mode is suitable for a more interactive use of R UBATO C OMPOSER, since any manipulation of a rubette will immediately result in updated views of depending rubettes. Multiple networks are independent. However, a network may be packaged in a macro rubette and can then be used as a regular rubette for building other networks. Of course, a network may generate a denotator and register it globally under a name. This named denotator can then be accessed from another rubette.

20.2.4 Macro Rubettes A network may itself exhibit the behavior of a rubette. To accomplish this, all elements that define a rubette must be present. First, the computational part is assumed by the network configuration. Then, the inputs and outputs are defined by putting rubettes of types MacroInput and MacroOutput, respectively, into the network. The MacroInput rubette takes the place of the rubette inputs (1 to 8). Its output connectors are the inputs to the macro rubette. The MacroOutput rubette stands for the rubette outputs (1 to 8), its input connectors will show up as outputs of the macro rubette. At most one of each may be present. No MacroInput rubette means no input at all, similarly for MacroOutput. Such a network is then packaged into a MacroRubette using a command available from the popup menu (right mouse button). At this occasion, a name and a description can be specified. From then on, this macro rubette is available for inclusion in other networks. When an instance of a macro rubette is placed in a network, a complete copy of the macro network is included, whence the name macro. Two instances of a macro rubette are completely independent. The network implementing the macro instance can be opened and modified like a normal network. However, changes will be reflected in the instance, and MacroInput and MacroOutput rubettes can neither be modified, removed, or added anymore. Closing a macro network just closes the window, the macro network itself is not deleted.

238

20 User’s Manual

20.2.5 Tools Apart from the central features of networks and rubettes, R UBATO C OM POSER provides extensive tools for creating and manipulating the fundamental types of objects: modules, elements of modules, module morphisms, forms and denotators.

20.3 Using R UBATO C OMPOSER This section describes the usage of the graphical interface.

20.3.1 Starting up R UBATO C OMPOSER and the R UBATO framework are implemented in pure Java. At least version 1.5 is required. The user interface is based on Swing, therefore independent implementations like GCJ do not work yet. Hopefully this will change in a not so far future. There are currently no plans for a reimplementation with an alternative toolkit such as SWT. Currently R UBATO C OMPOSER is started by executing the JAR archive named rubato.jar or rubato-DATE.jar, where DATE is the version of the distribution. On most platforms this is done by executing the command line java -jar rubato.jar. There are currently no platform-specific mechanisms for starting the application.

20.3.2 General Usage As a Swing application, the basic graphical elements of R UBATO C OM POSER, such as file dialogs, follow the guidelines of Swing on the specific platforms. Many operations are available from the toolbar as well as from the menubar. Others can be directly invoked using keyboard shortcuts. These shortcuts are summarized in section 20.11.

20.3 Using R UBATO C OMPOSER

239

Main menu Toolbar

Network list Rubette panel

Network

Message log Problem log Statusbar Fig. 20.3: Main window.

20.3.3 Main Window Main window. The overall layout of the main window is shown in figure 20.3. Apart from the main menu and the toolbar, there are three areas for importance. The largest part is taken up by the tabbed panel of networks, where each tab is occupied by one network. Right panel. The right panel consists of two tabs, the rubette panel and the network list. The rubette panel contains all types of rubettes currently available to the application. Rubettes are grouped under various headings. The heading Core is always present and contains the built-in core rubettes (see section 20.4). By clicking on a heading the rubettes in the group are shown or hidden. A double click on a rubette will put an instance of its type into the currently visible network (if there is no network at all, nothing happens). Alternatively a rubette may be selected and then dragged onto the network. Below the list of rubettes, a text areas display a description of the selected rubettes. The Add button performs the same thing as a doubleclick. The Edit and Remove buttons are active only when a macro rubette is selected. The other tab contains the network panel. It is simply a list of all existing networks. A mouse click on a network name will make this network visible. Bottom panel. The bottom panel is reserved for the display of information. It has two views, the problem log and the message log. The message log

240

20 User’s Manual

(figure 20.4) records general activity, such as adding rubettes, loading and saving, indicating any errors that occurred. Messages are of three types,

Fig. 20.4: Message log tab.

info for informational messages such as indications of performed actions (in black), warnings for undesirable but non-fatal occurrences (in orange) and errors indicating unrecoverable failures such as unsuccessful loading of files (in red). The checkboxes at the right side of the log window determine what types should be displayed. The Clear button will erase the list. All generated messages are also shown at the bottom-most statusbar for a few seconds. The problem log lists those errors that occur during the execution of a network. It is cleared at the beginning of each run. Since errors are always relative to a rubette, it is possible to a highlight the responsible rubette by clicking on an error message line. The highlighted rubette is surrounded by a thin red frame.

20.3.4 Main Menu and Toolbar Create a new project. The environment is reset to the fresh state as at startup, i.e., all objects, except for the built-ins, are deleted. File ➞ Open... Open an existing project. A rubato project file (extension .rbo) is loaded into the environment. All previously created objects, except for the built-ins, are deleted. File ➞ Add Definitions... Add definitions from an existing .rbo file. A rubato project file (extension .rbo) is loaded into the environment. In contrast to File ➞ Open, the environment is not cleared, but the definitions from the file are merged into it. File ➞ New

20.3 Using R UBATO C OMPOSER

241

Fig. 20.5: Problem log tab. The second line indicates that there is no input on the red-framed rubette Display #5.

Revert project to a saved state. If the current project has been saved at least once, reload it from the file with clearing the environment first. File ➞ Save Save project. If the current project has been saved at least once, save it under its given name, otherwise a file name (with .rbo extension) is requested. File ➞ Save As... Save project under a new name. A file name is requested regardless whether the project has already been saved once or not. File ➞ Quit Leave R UBATO C OMPOSER. The application is closed. If there are unsaved changes, the user is asked if he wants to save. Rubette ➞ Add Rubette Create new rubette instance. A new instance of the rubette selected in the rubette list is added to the current network if there is any, otherwise nothing is done. Network ➞ New network Create new network. A new, empty network is created and displayed as a new tab. It receives a default name of the form “Network #n” where n is a serial number. Network ➞ Close network Close visible network. The current, visible network is deleted. If the currently visible network is the representation of a macro rubette, it is simply removed from view. File ➞ Revert

242

20 User’s Manual

Run visible network. The execution of the current, visible network is started. This menu item is disabled if there is already an execution in progress. Network ➞ Stop Stop network execution. The current execution is stopped. This menu item is disabled if there is no execution in progress. Tools ➞ Module Builder Open module builder tool. The module builder is shown. There is always at most one module builder window. Tools ➞ Module Morphism Builder Open module morphism builder tool. The module morphism builder is shown. There is always at most one module morphism builder window. Tools ➞ Denotator Builder Open denotator builder tool. The denotator builder is shown. There is always at most one denotator builder window. Tools ➞ Form Builder Open form builder tool. The form builder is shown. There is always at most one form builder window. Tools ➞ Object Browser Open object browser tool. The object browser is shown. There is always at most one object browser window. Tools ➞ Scheme Interaction Open Scheme interaction tool. The Scheme interaction dialog is shown. There is always at most one Scheme dialog window. Tools ➞ Scheme Editor Open Scheme code editor. The Scheme code editor is shown. There is always at most one Scheme editor window. Tools ➞ Preferences Change settings. The preferences dialog is shown where settings that affect the overall behavior of R UBATO C OM POSER can be changed (see subsection 20.3.8). Network ➞ Run

20.3.5 Network The multiple networks are laid out as several tabs in the middle of the application window. New networks can be created using Network ➞ New Network or the button. Networks can be removed again with Network ➞ Close Network, the button or Network.Popup ➞ Close network. The last refers

20.3 Using R UBATO C OMPOSER

243

to the popup menu that is activated by clicking the right mouse button in a network window. All operations in the popup menu are relative to the currently visible network. The network popup menu is shown in figure 20.6. The first line in the menu shows the name of the network (the same name

Fig. 20.6: Network popup menu.

is shown as the title of the tab). The command Network.Popup ➞ Rename network will open a dialog requesting the user to enter a new name for the network. When a network is created, it is assigned a name of the form Untitled #n, where n is a serial number. An additional feature of networks are notes, which work like sticky notes (figure 20.7). They don’t affect the computation of a network, but can be

Fig. 20.7: Sticky note: the lower right handle can be used to resize the note.

useful for adding information and explanations. A new note is created at the cursor position with Network.Popup ➞ Create note. Text can be entered into a note by clicking on its contents. Notes can be resized, their background and foreground colors can be changed and they can be given a title by activating its popup menu with a right mouse click. Of course, notes can be removed again. The procedure of adding rubettes and linking them together has already been discussed. Rubettes can be dragged around on the network and have their own popup menu, which is available by right-clicking whenever the pointer is over the rubette that is indicated by a black border surround-

244

20 User’s Manual

ing the rubette selected in this way. The first line of Rubette.Popup shows the name of the rubette, the same as is shown on the rubette representation itself. The command Rubette.Popup ➞ Rename requests a new name from the user. A rubette is removed from the network by invoking Rubette.Popup ➞ Remove. The command Rubette.Popup ➞ Duplicate creates a copy of the selected rubette with a newly generated name. The duplicate should have the same settings as the original. However, what this exactly means has to be specified by the developer of the rubette. The Rubette.Popup ➞ Move To Front and Rubette.Popup ➞ Move To Back commands have the same effect as similarly named operations in common graphics software. The special operation Rubette.Popup ➞ Pass Through can be used to temporarily disable the selected rubette. It is available only for rubettes with at least one input and at least one output. When this operation is toggled, the color of the rubette changes to orange and the execution of the rubette bypasses the proper computational code and simply copies the first input to the first output, doing nothing with the other inputs and outputs. Several operations are available from the popup menu that opens when right-clicking on a link. The command Link.Popup ➞ Remove Link does the obvious thing. The three remaining options, Link.Popup ➞ Diagonal, Link.Popup ➞ Zigzag as well as Link.Popup ➞ Curved each select one of the three ways of representing the link. A network is run by invoking Network ➞ Run or the button . A running network can be interrupted with Network ➞ Stop or . The effectivity of this operation depends at least in part on the behavior of the individual rubettes and how they react to the stop event. The button toggles the continuous run mode.

20.3.6 Tools The functionality available from the Tools menu can be regarded as an interface to the system repository. Objects are created and registered under a specified name in the repository. The dialogs corresponding to the various types of objects (except forms) may also appear when such an object needs to be specified, for example in the properties of a rubette. In this case, a name need not be given, since the object in question is used immediately.

20.3 Using R UBATO C OMPOSER

245

Module Builder Similarly to other builders, the interface changes dynamically depending on the selected values. Figure 20.8 shows the creation of the module (Z[X] × R)3 . First the ring of the module is specified, in this case Productring is

Fig. 20.8: The module builder.

chosen, which requires further information. The checkbox below indicates whether it is a ring of polynomials. Another element that is always present is the dimension of the free module, here 3. For a product ring, the number of factors is indicated and the factors of the product have to be given. For each factor a module builder is recursively inserted. The first factor, for example, is a ring of polynomials with indeterminate X and coefficients in Z. Finally a name must entered, here ZXR3, before the module is created and registered with this name.

Module Morphism Builder The interface for creating module morphisms is probably the most extensive throughout R UBATO C OMPOSER. The reason is that morphisms come in many types and there are many ways to specify them, even in the case of a single type.

246

20 User’s Manual

As before, only some general hints are given, and a specific example is presented (figure 20.9), since the interface is subject to extension. Besides the

Fig. 20.9: The module morphism builder.

name entry, the two module entries (see also 20.3.9) for the domain and the codomain of the module morphism to be created are always present. The builder may however be configured programmatically to fix one or both of the domain or codomain, for example if a module morphism is required to have a given domain (i.e., the domain is fixed) but an arbitrary codomain (i.e., the codomain must be specified). Finally, the type of morphism must be selected. The list of available types depends of course on the particular domain and codomain. Section 20.8 contains an overview of the currently available types. In the example of figure 20.9, the morphism with name f is of the type R2 → R2 . This is a transformation in the Euclidean plane. The type is Geometry, which allows the definition as composition of geometric transformations. Here it is a rotation by 30◦ followed by a horizontal shearing by

20.3 Using R UBATO C OMPOSER

247

45◦ . Transformations can be added and removed using the corresponding buttons. After the button Apply has been pressed, the resulting morphism is created and its effect visualized on a unit square (vertexes are colored, otherwise reflections might not be recognized as such).

Denotator Builder Building denotators is less complicated than building morphisms, since there are only a few types to consider: Simple, Limit, Colimit, Power and List. Generally the name and the form has to be specified. However in other contexts the form may be programmatically fixed. Simple: The address type is required. If it is Null the dialog accommodates a module element entry. If it is Non-Null, a module morphism must be selected with codomain fixed to the module of the denotator form. In figure 20.10, the denotator to be created is given the name d and should have form Real, which has module R. The denotator is Z-addressed and its morphism is the embedding of Z into R.

Fig. 20.10: Simple denotator builder.

Limit: For a denotator of type Limit, the coordinates must be specified. In figure 20.11, the four coordinates for the denotator of form Note have the forms Onset, Pitch, Loudness and Duration, respectively. In this case the form also features labels for each of the coordinates indicated in square brackets. The resulting denotator has the name n1 and is built from the coordinates o1, p1, l1 and d1.

248

20 User’s Manual

Fig. 20.11: Limit (left) and Colimit (right) denotator builder.

Colimit: A denotator of type Colimit specifies exactly one coordinate. The index of the cofactor must be given. Power: The elements of the set can be added to (buttons Add and Add New) and removed from (button Remove) the list. The elements are shown in a compact representation, i.e., the name of an non-anonymous denotator, or, otherwise, the address and form. Since the content of a denotator can be very extensive, it is not included in the representation, but may be shown in the lower text field by selecting the element.

20.3 Using R UBATO C OMPOSER

249

Fig. 20.12: Power (left) and List (right) denotator builder.

List: The interface for type List is very similar to the Power interface. Since the order of the elements in a List denotator is critical, two further buttons (Up and Down) allow the (re)ordering of the elements.

250

20 User’s Manual

Form Builder The interface of the form builder always features a selection box for the form type (the fixed list consisting of Simple, Limit, Colimit, Power, and List) and the entry for the name. In contrast to denotators, forms must always be given a name. The name entry box also checks whether the name is unique. Invoking the form builder from the Tools menu is the only way to create new forms in the R UBATO C OMPOSER GUI. The interface for the Simple type requires the entry of a module, whereas the interfaces for types Power and List require the entry of the coordinate form. Figure 20.13 shows the interface for creating Power forms, the interface for List is the same.

Fig. 20.13: Limit form builder.

The types Limit and Colimit have similar interfaces (figure 20.14). The factor (resp. cofactor) forms are added to a list and can be reordered. The button Add allows the selection of a form from a list, the button Add New invokes the form builder recursively. When a label is entered before a new form is added, this label will be attached to the new factor (cofactor). The following describes a feature not yet completely implemented. The list of factor (cofactor) forms is shown twice. When one form on the left and one on the right is selected, a morphism between the two can be created using Set morphism. This is in fact the interface for creating non-trivial diagrams.

Object Browser The system-wide repository is a dictionary that associates names to objects of the types module, element, module morphism, form and denotator. All

20.3 Using R UBATO C OMPOSER

251

Fig. 20.14: Limit form builder.

forms must be registered in this repository, for all other objects this is optional. The object browser (Tools ➞ Object Browser) presents a view of the repository. Whenever a selected object can be edited, the Edit button will open an editor to this object.

20.3.7 Scheme Tools Scheme Interaction The Scheme interaction dialog allows the evaluation of Scheme expressions in the global Scheme environment. The top part of the dialog is the input area, where Scheme expression are entered. The expressions in this area are evaluated by pressing C T R L -E N T E R , and the result of the evaluation is appended to the output area in the middle of the dialog. The keys C T R L -U P and C T R L -D O W N navigate through a history of previous expressions. Errors are shown in the bottom line. The Stop button interrupts a running evaluation, the Clear button blanks the output area and the Init button resets the global Scheme environment to the initial state.

252

20 User’s Manual

Fig. 20.15: The object browser, a view of the repository.

Scheme Editor The Scheme editor allows to write Scheme code that can be evaluated in the global Scheme environment. The code text is saved as part of the project.

20.3.8 Preferences The following settings are currently available: Default link type Set to one of Line, ZigZag or Curve determines what shape the links should be drawn in (default: Curve), see figure 20.16. Save main window position and size If set, leaving the application will save the last position and size of the main window. These are restored when the application is restarted (default: not set). Ask before leaving If set, the user is asked when leaving the application and there are unsaved changes (default: set). Default Quantization During conversion from reals to rationals, the generated denominators are always divisors of this integer.

20.3 Using R UBATO C OMPOSER

253

Fig. 20.16: Link types: Line (left), ZigZag (middle), Curve (right).

20.3.9 Recurring User Interface Elements Besides the usual Swing user interface elements, R UBATO C OMPOSER makes use of some custom elements that can be found throughout the system. These elements may be used by the developer of rubettes, therefore the name of the class is indicated for easier reference.

Module Entry (JModuleEntry) A module entry component (i.e., user interface element or widget) allows the user to specify a module. The left field shows the currently selected module, the button Create builds a new one, the button Select shows a list, from which a predefined module can be selected by name.

Form Entry (JSelectForm) The entry of forms looks similar to the entry of modules, without the option to create a new form. The Create button is, however, available within the context of the form builder tool. The form is selected from the list. The form entry can be configured to only allow specific types of forms. The types are shown at the top of the selection window.

Denotator Entry (JSelectDenotator) The denotator entry component is analogous to the form entry component. It can be configured to only allow denotators with a specific form.

254

20 User’s Manual

Fig. 20.17: Module entry component (top), module list selection (bottom).

Module Element Entry (JModuleElementEntry) This component is shown whenever the user is required to specify a module element, i.e., numbers, vectors, strings, etc. It is always configured to accept elements from a specified module. The use is straightforward, a more complicated case is shown in figure 20.20, where the module in question is (Z[X] × R)3 . The module of dimension 3 is over the product ring Z[X] × R, where Z[X] is the polynomial ring with indeterminate X and coefficients in Z. As can been seen, the first factor of the third component is incorrect, since the polynomial indeterminate is not Y. This indicated by an orange background.

Module Morphism Entry (JMorphismEntry) Module morphisms can be built (button Create), selected from a list of predefined morphisms (button Select), and prepared for editing (button Edit). Note that the dialog for editing may be different from the one used for creating the morphism in the first place. The reason for this is that a morphism of a specific type (for example affine) may be created using a variety of methods (matrix, graphical) that can no longer be inferred for the edit operation (in this case, always the matrix method will be used for editing). The Build and Create actions invoke a module morphism builder dialog.

20.3 Using R UBATO C OMPOSER

255

Fig. 20.18: Form entry component (top), form list selection (bottom).

Fig. 20.19: Denotator entry component (top), denotator list selection (bottom).

Inputs/Outputs Sliders (JConnectorSliders) Rubette properties may change the number of input and output connectors. This component provides a slider that can be configured to the minimum and maximum (at most 8) number of connectors.

256

20 User’s Manual

Fig. 20.20: Module element entry for the module (Z[X] × R)3 .

Fig. 20.21: Module morphism entry component.

Path Selector (JFormTree) A denotator can be regarded as a tree, while its form specifies the structure of the tree. It is sometimes necessary to pick a specific node from the form tree. This results in a path from the root to this node. This path can later be used to perform operations on this node in a denotator. The path selector (or form tree) is the corresponding user interface component.

Fig. 20.22: Slider for number of input connectors.

20.4 Core Rubettes

257

Fig. 20.23: Path selector (or form tree), where the form Score is expanded, and the path from the root Score to the form Pitch is selected.

20.4 Core Rubettes 20.4.1 Rubette Description Schema To describe a rubette a common schema is used. The following layout shows all possible components. Not all items have to be present. For example a rubette without view will not include the View item. Name Group: Summary: Inputs: Outputs:

The group that the rubette belongs to. The rubettes in this section all belong to the group core, so it is not specified. A short overview of the rubette. Enumeration and description of all input connectors. Enumeration and description of all output connectors.

Description: An exhaustive description of the rubette. View: An explanation of the view, if the rubette has one. Properties: An explanation of the properties, if the rubette has any. A rubette is implemented as a Java class, for example ExampleRubette, the name is then usually just “Example”.

258

20 User’s Manual

20.4.2 List of Core Rubettes The list is in alphabetical order. It may be possible that the list is incomplete with respect to the latest version of R UBATO C OMPOSER (available from http://www.rubato.org). AddressEval Summary: Inputs:

Outputs:

Evaluates a denotator at a specified address. #1: The input denotator to be evaluated. #2: When Evaluate at input denotator is selected, this is the point at which is evaluated, this must be a denotator of type Simple with the appropriate module. #1: The evaluated denotator. In the case where several evaluations are requested (Evaluate at list of elements), this is a set of evaluated denotators.

Description: This rubette performs address changes of denotators in general. There are several possibilities: a) Evaluate at null: The address of the input denotator is changed to be the null module over the ring of the current address by evaluating the denotator at the zero element of its address. b) Evaluate at module element: This is a slight generalization of Evaluate at null. In this case the element used for evaluation is not the zero element, but an element specified in the properties.

Properties:

c) Evaluate at list of elements: Similar to Evaluate at module element, but with multiple evaluations, one for each element in a list specified in the properties. The result is no longer a denotator with the same form as the input form, but a form of type List or Power with the input form as coordinate form. d) Evaluate at input denotator: Similar to Evaluate at module element, but the module element is not specified in the properties but comes from input denotator #2. e) Change address: The most general alternative requires the specification of the address changing module morphism. Ad b) The module element must be specified. Ad c) A list of module elements must be specified (see figure 20.24). The list can be changed using the buttons at the bottom. The Basis vectors button generates the list of canonical affine basis vectors of the module. In the case of C or R2 , where R ∈ {Z, Zn , Q, R}, the button Graphical opens

20.4 Core Rubettes

259

Fig. 20.24: The properties of the AddressEval rubette, where Evaluate at list of elements is selected.

a dialog, where the elements can be selected as points in the plane. Additionally, the output form must be selected from the forms of type List or Power with coordinate form equal to the form of the input denotator. Ad d) The result form must be specified. Ad e) The address changing module morphism must be specified. In the case of b) and c), the address module must be specified. Boolean Summary: Inputs:

Evaluates a logical expression with the input denotators as variables. #1–#n: The variables (of form Boolean) for the logical expression (default: n = 2).

260

20 User’s Manual

Outputs: #1: The result (of form Boolean) of the logical expression. Description: A logical expression is evaluated, where the input denotators are variables. The syntax of logical expressions uses the parentheses ( and ), the symbols & for logical conjunction, | for disjunction, ! for negation, the constants T and F, the variables #n for input denotators #(n + 1), etc., and the usual precedence rules. Properties: The number of arguments and the logical expression string must be set. Constructor Summary:

Creates a denotator of the specified (non-Simple) form.

Inputs: #1–#n: The coordinates of the result denotator. Outputs: #1: The result denotator of the specified form. Description: A denotator of the specified form is constructed using the coordinate denotators at the input connectors. The meaning of the input connectors changes with the type of the result denotator: Power, List: The input denotators form the elements of the result denotator. At most 8 elements can be put it. Not all input connectors need to be actually connectors. For type List the order is the left to right order of the connectors. Limit: The input denotators are the coordinates of the Limit denotator. Only the first 8 factors can be specified, the excess coordinates are set to default denotators.

Properties:

Colimit: The input denotator is the cofactor of the Colimit denotator. The first non-null denotator will be the actual coordinate of the result denotator. The form of the result denotator must be specified.

Display Summary:

Shows a textual representation of a denotator.

Inputs: #1: The denotator to be shown. Description: The input denotator is shown in a text window of the view. The representation can be either a simple human readable text or the complete XML text that is used for the file format. The Display rubette is useful for debugging purposes. View: The main part is the text window. At the bottom is a switch that selects between simple and XML text.

20.4 Core Rubettes

261

Latch Summary:

Stores an input denotator and distributes it to one or more outputs.

Inputs: #1: The input denotator. Outputs: #1–#n: The output denotators #i are all the same. Description: The denotator at the unique input connector is stored within the rubette and passed on to all of the output connectors. This may be useful for creating more branches from a single link. Properties: The number of outputs can be set from 1 to 8. List Summary:

Performs a list operation on the input denotators.

Inputs:

(Concatenate) #1–#n: The argument denotators of type List (default n = 2). (Concatenate All) #1: A list of denotators of type List. (Append elements, Prepend elements) #1: A denotator of type List. #2–#n: Denotators with form equal to the coordinate form of #1 (default n = 2).

(Sort) #1: A denotator of type List. #1: The result of the list operation. The result form (of type List) is taken from input denotator #1, or in the case of Concatenate All, the coordinate form of the form of input denotator #1. Description: The input denotators are the arguments to a list operation:

Outputs:

Concatenate: All input denotators are concatenated in the order of the inputs. Concatenate All: The elements of the input denotator of type List are concatenated in the order they occur in the input denotator. Append elements (Prepend elements): The input denotator #2–#n (where n can be configured to be a value from 2 to 8), are appended (prepended) to the input denotator #1. The constraint is that these denotators have as form the coordinate form of the form of denotator #1. Sort: The result is the sorted input denotator #1, according to the canonical order of denotators.

262

Properties:

20 User’s Manual

The list operation can be selected and the number of inputs for the operations Concatenate, Append (Prepend) elements can be set.

MacroInput Summary: Outputs:

Retrieves the inputs in a macro rubette. #1–#n: Each output connector #i will act as the input connector #i in the resulting macro rubette (default: n = 1). Description: This type of rubette only makes sense if it is placed in a network that is intended to be made into a macro rubette. It provides the external inputs to the macro rubette. Properties:

The number of output connectors can be specified, as well as a label for each output connector, which will show up later as a tooltip for the corresponding input connector on the macro rubette.

MacroOutput Summary: Inputs:

Provides the outputs in a macro rubette. #1–#n: Each input connector #i will act as the output connector #i in the resulting macro rubette (default: n = 1). Description: This type of rubette only makes sense if it is placed in a network that is intended to be made into a macro rubette. It provides the external outputs from the macro rubette. Properties: The number of input connectors can be specified, as well as a label for each input connector, which will show up later as a tooltip for the corresponding output connector on the macro rubette. ModuleMap Summary:

Maps a module morphism onto a denotator.

Inputs: #1: The input denotator. Outputs: #2: The mapped output denotator. Description: Given a path from the root of a denotator to a node containing a denotator of type Simple, a denotator is produced with the selected node or nodes replaced by the result of mapping a module morphism. Properties: The input form, a path to a node of type Simple, and a suitable module morphism must be selected.

20.4 Core Rubettes

263

Mux Summary:

Selects an input denotator according to the number given by input denotator #1 (also known as multiplexer).

Inputs:

#1: A denotator of type Simple with module Z containing a number. #2–#n: Input denotators, one of which is selected (default: n = 2). Outputs: #1: The selected denotator. Description: Input denotator #1 must contain an integer i between 2 and n (the configured number of inputs). The output is set to the input denotator #i. Properties:

The number of input denotators can be set from 2 to 8.

RealArith Summary: Inputs: Outputs:

Evaluates an expression of real arithmetic with the input denotators as variables. #1–#n: The variables for the expression (default: n = 1). The input denotators must be of type Simple with module R. #1: The result of the arithmetical expression. In the case of a real result, the form of the result is taken to be the form of input denotator #1. In the case of a Boolean result, the form is Boolean.

Description: An expression of real arithmetic is evaluated, where the input denotators are variables. The syntax of expressions uses the parentheses ( and ), real number constants and the binary operators +, -, *, /, and ˆ. A ternary operator if. . . then. . . else is also provided. The condition in this operator is a Boolean expression, with relation symbols , =, =, !=, and Boolean operators &, |, and !, and constants T and F. Some common functions are also provided, sin, cos, tan, log, exp, abs, sqrt, max and min. The function arguments are placed within parentheses and separated by commas. The variables are #n for input denotators #(n + 1), etc.. The common precedence rules are used throughout. The result may be configured to be a Boolean. In that case the top level expression is a Boolean instead of an arithmetical expression.

264

Properties:

20 User’s Manual

The number of arguments (input connectors) is specified. An option indicates whether the result is real or Boolean.

Reform Summary: Inputs: Outputs:

Casts the input denotator to a new form. #1: The input denotator to cast from. #1: The output denotator with the new form.

Description: The input denotator is cast into the configured form. Properties: The output form must be specified. Register Summary:

Registers a copy of the input denotator.

Inputs: Outputs:

#1: The input denotator to register a copy of. #1: The registered denotator.

Description: A copy of the unique input denotator is registered with a specified name in the system repository. Properties: The name can be specified. Scheme Summary: Inputs:

Executes Scheme code on the input denotators. #1–#n: The input denotators, which can be accessed using the get-input Scheme function. Outputs: #1–#n: The output denotators, which can be written using the set-output Scheme function. Description: The specified Scheme code is executed. For example the expression (set-output 0 (get-input 0)), sets the first output denotator to the first input denotator. The code runs in an environment on top of the global environment containing the standard Scheme bindings. The local environment is empty at the begin of each run. See also section 20.12. Properties: The number of input and output connectors, and the Scheme code to execute are specified. Select2D Summary:

Projects a list or set of denotators to one or more planes, and returns a selection of them specified by polygons in the planes.

Inputs:

#1: Input denotator of type List or Power with specified coordinate form.

20.4 Core Rubettes

Outputs:

265

#1: Output denotator of the same form as input #1, but containing only the selected elements.

Description: The elements of the input denotator are projected to one or more planes. On each projection, one or more polygonal selections can be drawn. The selected elements for each plane are the elements whose projected coordinates lie in one of the polygons. The final selection is the intersection of the selections of all planes. Properties:

The input coordinate form must be specified first. Then a selection tab can be created, and the projected coordinates for the x- and y-axes are specified using path selectors. Selection tabs can be added and removed. Polygonal selections can be created (first toolbar button), selected for editing (second button), vertexes added (third button), removed (fourth button) and moved (fifth button). Other operations are: information about a point and zooming (see figure 20.25).

SelectForm Summary:

Retrieves all denotators with a specified form contained somewhere in the input denotator tree.

Inputs: Outputs:

#1: The input denotator. #1: A denotator of type List or Power containing the collection of denotators retrieved from the input denotator. Description: The input denotator tree is traversed and all denotators of the specified form are collected. The output denotator is a list or set containing the resulting collection as elements. It is, however, not the form itself that is specified, but the output form, which can be of type Power or of type List. The query form is then its coordinate form. Properties: The output form of type List or Power must be specified. SetOp Summary: Inputs:

Performs a set operation on the input denotators. (Union, Intersection, Difference, Symmetric Difference) #1– #n: The argument denotators of type Power (default: n = 2). (Add Element) #1: A denotator of type Power. #2–#n: Denotators with form equal to the coordinate form of #1 (default n = 2)

266

20 User’s Manual

Fig. 20.25: The properties of the Select2D rubette. Here the first of two selection tabs is visible. The coordinate form is Note, the projection on the x-axis is the R ring of its Onset coordinate, the projection on the y-axis is the Q ring of its Pitch coordinate. A single polygonal selection has been drawn. Also visible are the current position of the pointer and the extents of the window.

(Big Union (Intersection)) #1: A set or list of denotators of type Power. Outputs: #1: The result of the set operation. The result form (of type Power) is taken from input denotator #1, or in the case of Big Union (Intersection), the coordinate form of the form input denotator #1. Description: The input denotators are the arguments to a set operation: Union, Intersection, Difference, Symmetric Difference: The specified operation is performed on the input denotators. Add Element: Input denotators #2–#n are added to the elements of input denotator #1.

20.4 Core Rubettes

267

Big Union (Intersection): The result is the union (intersection) of all denotators, elements of the input denotator. Properties:

The set operation is selected and the number of input denotators where appropriate.

Simple Summary:

Produces a denotator of type Simple.

Outputs:

#1: A denotator of type Simple.

Description: A denotator of type Simple is stored and put to the output connector. Properties: A form of type Simple is selected. An option specifies if the denotator should be null-, or non-null-addressed. If nulladdressed, the module element, otherwise a module morphism, must be entered (see figure 20.26).

Fig. 20.26: The properties of the Simple rubette. The module in this case is the Euclidean plane R2 .

Source Summary:

Produces a registered denotator at its output.

Outputs: #1: Specified output denotator. Description: A denotator is specified and delivered to the only output connector. The denotator is selected from all registered denotators. Therefore only non-anonymous denotators can be specified.

268

Properties:

20 User’s Manual

The denotator is specified using a denotator entry component. Additionally it can be indicated whether during execution the named denotator is always retrieved anew from the repository (Self-refreshable is selected). This is useful if a new denotator is registered under that name.

Split Summary:

If the input denotator is of type Limit, it is split into its coordinates. If it is of type Colimit, its cofactor is produced. Inputs: #1: A denotator of type Limit or of type Colimit. Outputs: #1–#n: In the case of a Limit denotator, the coordinate denotators, n being equal to the number of coordinates in the input, but at most 8. In the case of a Colimit denotator n = 1. Description: The input denotator of type Limit is split into its coordinates, which are produced on the output connectors. However, if the input denotator has more than 8 coordinates, only the first 8 are produced. For a Colimit denotator, its unique coordinate is produced. This rubette will probably be improved to be more flexible. Properties:

The form (Limit or Colimit type) to be split is set.

Stat Summary: Inputs: Outputs:

Performs a statistical operation. #1: Input denotator containing the values to process. #1: The result (denotator of type Simple with a specified form) of the statistical operation. Description: The set of all denotators with a specified form (of type Simple) is extracted from the input denotator, and a statistical operation is performed on this set. Available operations are Minimum, Maximum, Mean, Variance, Sum and Product. Not all modules allow every operation. Properties: The form (of type Simple to extract, which is also the form of the output denotator) can be specified as well as the type of operation.

20.5 Built-in Non-Core Rubettes

269

20.5 Built-in Non-Core Rubettes R UBATO C OMPOSER contains several non-core rubettes, mostly related to music. These rubettes do not belong to the Core group since they are more application specific. MidiFileIn Group: Summary: Outputs:

Score Reads a MIDI file and converts it to a denotator of form Score. #1: A denotator of form Score.

Description: The specified MIDI file is read and converted to a denotator of form Score. Only the attributes onset, pitch, duration, and loudness of the MIDI note events are considered. Properties:

The MIDI file to load is selected using the usual open file dialog.

MidiFileOut Group:

Score

Summary:

Converts a denotator of form Score to MIDI and writes it to a file.

Inputs:

#1: A denotator of form Score,

Description: The input denotator of form Score is converted and written to the specified MIDI file. Since the MidiFileIn rubette discards information, a MIDI file read in using MidiFileIn will not be the same when written back using MidiFileOut. Properties: The MIDI file to write to is selected using the usual open file dialog. ScorePlay Group: Summary: Inputs: Description:

View:

Score Displays and plays a denotator of form Score. #1: A denotator of form Score. The input denotator of form Score is displayed in the view window as a piano roll and can be played using common media player controls. The greatest part of the view (figure 20.27) is occupied by the score in piano roll representation, familiar from sequencer software. Onset, duration and pitch are obviously coded as

270

20 User’s Manual

bars, loudness by varying color saturation, and the voice of the note by different color hues (red, blue, etc.).

Fig. 20.27: The view of the ScorePlay rubette.

The top line of buttons trigger the actions play, pause and stop, from left to right. During playback a moving vertical line will show the current position in the piano roll window. The Voices button opens a dialog that allows the assignment of General MIDI instruments to voices (figure 20.28). From the top combo box the voice to change can be selected from all voices actually present in the score. At the bottom a slider allows the adjustment of the playback tempo factor.

20.6 Writing Rubettes

271

Fig. 20.28: Assignment of instruments to voices (ScorePlay rubette).

20.6 Writing Rubettes R UBATO C OMPOSER essentially depends on the availability of useful rubettes. The previous sections described a core set that is always present. For specific applications, third-party rubettes designed as plug-ins are expected to enrich the environment. In this section, a short overview and tutorial will introduce the potential developer to the mechanisms of rubette design. Currently the only supported language is Java. This introduction will therefore reference Java packages, interfaces and classes as they are implemented in the R UBATO framework.

20.6.1 Developing with the R UBATO Framework There is no place in this manual for a complete exposition of the R UBATO class hierarchy. It is enough to give pointers to the important packages. For the details, the reader should peruse the extensive Javadoc generated from sources itself, and therefore always up-to-date.

272

20 User’s Manual

• The core framework dwells under the package org.rubato.math. This contains the complete implementation of the mathematical foundation of modules, forms and denotators. • Modules and elements of modules are contained in the package org. rubato.math.module, morphisms between modules in the package org. rubato.math.module.morphism. They make use of several classes such as for complex numbers, modular arithmetic, string rings and matrixes from the packages org.rubato.math.arith and org.rubato.math.matrix. • The package org.rubato.math.yoneda contains the classes for forms and denotators. • The packages starting from org.rubato.logeo collect general operations and algorithms on denotators, such as set operations for power denotators and list operations for list denotators. There are also tools for the simplified construction of denotators and forms. • The developer of rubettes also needs the package org.rubato.base, which contains the Rubette Java interface, the package org.rubato.xml for reading and writing XML representations, as well as the packages org. rubato.composer.dialog containing dialogs for entering the various types of objects, and org.rubato.composer.components for special widgets useful for the implementation of rubette properties and view dialogs. • General utilities can be found in the package org.rubato.util. One remark that applies to the development in R UBATO in general: Before a newly created form can be used to create a denotator, it must be registered. This ensures that names are bound to forms in a unique way. This constraint does not apply to denotators. Since this rule is so important, here is a code example to show how to do it: import org.rubato.base.Repository; import org.rubato.logeo.FormFactory; ··· Repository rep = Repository.systemRepository(); Form f = FormFactory.makeZModuleForm("Integer"); f = rep.register(f); if (f == null) { ··· } ··· The method register returns null in the case that the registration has failed (which may happen when a form of that name has already been registered, for example), thus the return value should always be checked.

20.6 Writing Rubettes

273

20.6.2 Rubette Interface The Java interface that every rubette must implement is shown in section 20.9 on page 287. Before we embark on the description of the methods in details, it is important to note that not every method is intended to be used by the implementor of rubettes, nor is it even necessary to implement all methods. In fact, in almost all cases, a rubette will inherit from the abstract class AbstractRubette, which provides default implementations for some of the methods. The methods that are implemented in AbstractRubette and are not generally reimplemented in a specific rubette are marked with a bullet (•). Those that are given default implementations, but are usually overridden in the implementing rubette class, are marked with a circle (◦). Finally, those methods, that must be implemented are left unmarked. A note about constructors: A rubette must provide a default constructor (i.e., a constructor with no arguments), since this is called when R UBATO C OMPOSER instantiates the prototype of the rubette. The default constructor can be overridden, for example to set default values for the rubette parameters. This and additional constructors can be used for the implementation of the newInstance method, described below. void init() This method is called whenever the rubette prototype is created. This is the place where global structures and variables can be created and initialized. void run(RunInfo runInfo) This method embodies the computational core of the rubette. Typically it starts by retrieving the inputs using getInput, then it checks the inputs for correct form (inputs may be null), and terminates by setting the results using setOutput. The argument of type RunInfo contains information about the current execution. The rubette should check runInfo.stopped() periodically during long calculations. If this method return true, run should abort gracefully and return. Rubette newInstance() Whenever a new instance of this rubette is created by R UBATO C OM POSER, this method is called on a prototype object. It must return a freshly and fully initialized object of this rubette. It is a kind of virtual constructor and must not depend on global state. The simplest implementation uses the default constructor and looks like this: public Rubette newInstance() { return new ExampleRubette(); }

274

20 User’s Manual

Rubette duplicate() This method is called whenever the duplicate operation is invoked on a rubette in the GUI. It should return a copy of the rubette instance, which inherits as many attributes as possible. The default implementation is equivalent to newInstance. void toXML(XMLWriter writer) Every rubette instance must be saveable to and loadable from a stream. The native format is XML and this method is responsible for writing a representation of the rubette instance. It is called by R UBATO C OM POSER and is surrounded by an XML tag Rubette that already identifies the type of rubette. The body of this method is therefore responsible only for the settings specific to this rubette. It can make use of all the utility methods defined by the XMLWriter class from the package org.rubato.xml. As an example, consider a rubette whose configuration only consists of an integer variable a and a string variable s: public void toXML(XMLWriter writer) { writer.empty("A", "value", a); writer.empty("S", "value", s); } If a = 5 and s = "Text", this results in the XML code: <S value="Text"/> Here the empty method is used. Consult the Javadoc for the variety of different methods provided by XMLWriter. Rubette fromXML(XMLReader reader, Element element) To parse the XML representation of a rubette of this type, this method, which is the inverse of toXML, is called. The argument element contains the list of XML elements written by toXML. Taking up the previous example, the code that retrieves a and s would be: Rubette fromXML(XMLReader reader, Element element) { // get element , head of the list Element child = reader.getChild(element, "A"); if (child == null) { // there is no element return null; } // get value of element , use default 0 int a = reader.getIntAttribute(child, "value", 0); // get element <S>, next in list child = reader.getNextSibling(child, "S");

20.6 Writing Rubettes

275

if (child == null) { // there is no element <S> return null; } // get value of element <S> String s = reader.getStringAttribute(child, "value"); ExampleRubette r = new ExampleRubette(a, s); return r; } The class XMLReader from package org.rubato.xml provides many methods for parsing elements besides the ones used here. This method must return a new instance initialized according to the settings retrieved from XML. If there is an error, the return value must be null. Error messages can be set using XMLReader.setError, which works similarly to the addError method described below. String getGroup() Returns the group that this rubette belongs to. Core rubettes belong to the group “Core”. An independent rubette should not belong to “Core”. The return value can otherwise be any string whatever. The default group inherited from AbstractRubette is “Other”. ImageIcon getIcon() Returns the icon that is displayed in the graphical representation of the rubette (see figure 20.1). This must be a fixed value, and not vary during the rubette’s lifetime. The default return value is null, which is equivalent to a blank icon. boolean hasProperties() Must return true if this rubette has a properties dialog. If true the methods getProperties, applyProperties and revertProperties must also be implemented. The default value is false. JComponent getProperties() If hasProperties returns true, this must return a JComponent, for example a JPanel containing various user interface elements, buttons, sliders, etc. This component is placed by R UBATO C OMPOSER in a window as shown in figure 20.29. Four buttons are added: Hide removes the window from view, Revert calls the rubette’s revertProperties method, Apply calls the rubette’s applyProperties method and Apply & Hide is a combination of both operations. boolean applyProperties() This method is linked to the button Apply in the properties dialog. It should make the changes that the user made in the properties dialog permanent.

276

20 User’s Manual

Fig. 20.29: Properties window container.

void revertProperties() This method is linked to the button Revert in the properties dialog. It should replace the values in the properties dialog by the current settings of the rubette. The last two methods imply that changes in the properties dialog must not take effect immediately, but only after the user presses the Apply button. boolean hasView() Must return true if this rubette has a view. If true, the methods getView and updateView must also be implemented. The default value is false. JComponent getView() If hasView returns true, this must return a JComponent, for example a JPanel. This component may be contain whatever is necessary to perform a multimedia rendering of the rubette’s state. Similarly to the getProperties method, the component is inserted by R UBATO C OM POSER in a window as shown in figure 20.30. The only button, Hide, causes the window to disappear. It can be shown again by clicking View on the rubette. void updateView() During the execution of a network, the state of the rubette may change, thus causing a change in its view. R UBATO C OMPOSER calls updateView to notify the rubette that it should update its view.

20.6 Writing Rubettes

277

Fig. 20.30: View window container.

boolean hasInfo() Must return true if this rubette has an information text. If true, the method getInfo must also be implemented. The value must not vary during the rubette’s lifetime. The default value is false. String getInfo() Returns the string for the information displayed on the graphical representation of the rubette (see figure 20.1). This string must be short and must not contain line breaks. The value can vary during the rubette’s lifetime. The default value is null. String getShortDescription() Returns a short description of the rubette. This string is shown as a tooltip when the mouse pointer hovers over the rubette. The string must be short and must not end with a period for consistency. The default value is null, which means no tooltip at all. String getLongDescription() Returns a long description of the rubette. This string is displayed in the description area of the rubette panel when the rubette is selected. This string can be of arbitrary length and must end with a period for consistency. The default value is null. String getInTip(int i) When the mouse pointer hovers over an input or output connector, a tooltip describing the connector can be displayed. This method should

278

20 User’s Manual

return a string for the tooltip of input connector number i, where 0 ≤ i < getInCount(). String getOutTip(int i) Analogous to getInTip, this should return a string for the tooltip output connector number i, where 0 ≤ i < getOutCount(). void setInCount(int n) The number of input connectors is set using this method in the constructor of the rubette. The number can be changed during the lifetime of the rubette by adding such an option to the properties dialog. However if the number is reduced, those links that have been connected to those connectors that will not exist anymore will be removed. int getInCount() Returns the current number of input connectors. There is not much sense in overriding the default implementation. Denotator getInput(int i) Returns the denotator at input connector number i, where 0 ≤ i < getInCount(). This method is normally called for each of the input connectors at the beginning of the run method. void setOutCount(int n) The number of output connectors is set using this method in the constructor of the rubette. The number can be changed during the lifetime of the rubette by adding such an option to the properties dialog. However if the number is reduced, those links that have been connected to those connectors that will not exist anymore will be removed. int getOutCount() Returns the current number of output connectors. There is not much sense in overriding the default implementation. void setOutput(int i, Denotator d) Sets the result denotator at output connector number i, where 0 ≤ i < getOutCount(). This method is normally called for each of the output connectors at the end of the run method. void addError(String msg, Object ... objects) If an error occurs during the calculation performed by the run method, this method should be used to publish the error. It can, of course, be called multiple times, whenever it makes sense to continue after an error has occurred. The error message is specified using the argument msg. This string can contain template variables of the form %n, where n is a digit from 0 to 9. These variables are replaced by the arguments from objects in this order. For convenience, the variable names %%n may be used, in this case the corresponding argument is surrounded by double quotes (") in the resulting string. Thus addError("Input %0 is %1 in %%2", 2, "null", "ExampleRubette") will generate the string "Input 2 is null in \"ExampleRubette\"".

20.7 Rubette Example

279

Denotator getOutput(int i) List<String> getErrors() void clearErrors() boolean hasErrors() void setModel(RubetteModel model) RubetteModel getModel() These five methods are used internally by R UBATO C OMPOSER and are of no use to the rubette implementor. They are declared final and can therefore not be overridden in a subclass of AbstractRubette.

20.7 Rubette Example The example for writing a rubette presented in this section is based on the Latch rubette. It is simple enough to only use the basic functionality of the R UBATO framework, but incorporates most features of a rubette implementation.

20.7.1 Specification The Latch rubette has one input connector and a configurable number of output connectors (from 1 to 8, the default should be 1). On execution, the rubette will forward the input denotator to each of the output connectors. This specification makes clear that the rubette will feature a properties dialog to set the number of output connectors, and no view.

20.7.2 The LatchRubette class The class LatchRubette resides in an arbitrary package, say rubatoexample: package rubatoexample; // here come the necessary import statements public class LatchRubette extends AbstractRubette { ... } The methods that must be implemented are presented in the following.

280

20 User’s Manual

The default constructor is the right place to set the number of input connectors and the default number of output connectors. The init method is left empty, since the rubette does not use any additional data structures that must be initialized beforehand. The newInstance method is implemented in the way that has already been mentioned above. The only configurable attribute is the number of output connectors and this will be transferred to the copy in the duplicate method. 1 2 3 4

public LatchRubette() { setInCount(1); setOutCount(1); }

5 6 7 8

public Rubette newInstance() { return new LatchRubette(); }

9 10 11 12 13 14

public Rubette duplicate() { LatchRubette rubette = new LatchRubette(); rubette.setOutCount(getInCount()); return rubette; }

15 16

public void init() {}

Now for the informational methods. The rubette should be called “Latch” and belong to the group “Examples”. 17 18 19

public String getName() { return "Latch"; }

20 21 22 23

public String getGroup() { return "Examples"; }

The short description explains the function of the rubette in the most concise way, whereas the long description gives an expanded version thereof. In more complex rubettes, the long description usually gives a few hints for the usage of the rubette. The methods specifying the tooltips of the connectors do not contain any special information in the case of this simple rubette. If the connectors played different roles, these methods would give a hint at their meaning. Here, the output connector tooltip depends on the actual position of the output connector.

20.7 Rubette Example 24 25 26

281

public String getShortDescription() { return "Stores an input denotator"; }

27 28 29 30 31 32

public String getLongDescription() { return "The Latch Rubette stores its input denotator"+ " and provides it on its outputs the number of"+ " which can be configured."; }

33 34 35 36

public String getInTip(int i) { return "Input denotator"; }

37 38 39 40

public String getOutTip(int i) { return "Output denotator #"+i; }

For the icon, it is necessary to load an image. If this image (for example a PNG file such as latchicon.png) is stored in the same directory as the package of the rubette, the method loadIcon from the utility class Icons can be used. It takes as first argument the rubette class, and as second argument the file name. The icon is loaded statically and stored in a class variable. It should be a pixmap of size 16 × 16 pixels. 41 42 43

public ImageIcon getIcon() { return icon; }

44 45

private final static ImageIcon icon;

46 47 48 49

static { icon = Icons.loadIcon(LatchRubette.class, "latchicon.png"); }

The most important method is the core method run. In this case it is very simple: it puts the sole input denotator to each output denotator. For illustration purposes the code is slightly extended to make use of the runInfo parameter. The run method returns if it has been detected that execution has been stopped (line 53). 50 51 52 53 54 55

public void run(RunInfo runInfo) { Denotator d = getInput(0); for (int i = 0; i < getOutCount(); i++) { if (runInfo.stopped()) { return; }

282

20 User’s Manual

setOutput(i, d);

56

}

57 58

}

Since there is no view, the corresponding methods can be left untouched. There is, however, a properties dialog for setting the number of outputs. The method hasProperties must therefore return true and the three other methods related to the properties dialog must be implemented. 59 60 61

public boolean hasProperties() { return true; }

62 63 64 65 66

public JComponent getProperties() { if (properties == null) { properties = new JPanel(); properties.setLayout(new BorderLayout());

67

outSlider = new JConnectorSliders(false, true); outSlider.setOutLimits(1, 8); outSlider.setOutValue(getOutCount()); properties.add(outSlider, BorderLayout.CENTER);

68 69 70 71

} return properties;

72 73 74

}

75 76 77 78 79

public boolean applyProperties() { setOutCount(outSlider.getOutValue()); return true; }

80 81 82 83

public void revertProperties() { outSlider.setInValue(getOutCount()); }

84 85 86

private JPanel properties = null; private JConnectorSliders outSlider = null;

The properties dialog is constructed once for each rubette and saved in a field properties. It features a JConnectorSliders component (line 68) that is configured to provide an output connector slider (the second argument is true) and no input connector slider (the first argument is false), since the number of input connectors is fixed to one. Additionally the limits are set to be from 1 to 8 (line 69) and the current value is set to the current number of output connectors (line 70). The component is then placed in a JPanel (line 65) using a BorderLayout (line 71). The configuration is applied by setting the current number of output connectors from the current value of

20.7 Rubette Example

283

the slider (line 77). To revert, the slider value is set from the current number of output connectors (line 82). The last step is the implementation of the methods for writing to and reading from XML. 87 88

private final static String OUTPUTS = "Outputs"; private final static String NUMBER_ATTR = "number";

89 90 91 92

public void toXML(XMLWriter writer) { writer.empty(OUTPUTS, NUMBER_ATTR, getOutCount()); }

93 94 95 96 97 98 99 100 101 102 103 104 105

public Rubette fromXML(XMLReader reader, Element element) { Element child = reader.getChild(element, OUTPUTS); if (child != null) { int n = reader.getIntAttribute(child, NUMBER_ATTR, 1, 8, 1); LatchRubette rubette = new LatchRubette(); rubette.setOutCount(n); return rubette; } else { return null; } }

The only setting that must be saved is the number of output connectors n. A constant string for the tag is defined, as well as the attribute in this tag for the number n (lines 87–88). To write XML, the method empty from the writer instance is used (line 91). If n = 3 for example, this results in the string:

Reading from XML is a little more complicated, since error checking must be performed. The method starts by retrieving the first child of the Rubette element, which is passed as argument to the fromXML method. This child should contain an element with the Outputs tag. If it is null, parsing has failed and null is returned to mark this event as an error. Otherwise the value of the attribute number in element Outputs is retrieved using the method getIntAttribute from the reader object. The two first argument numbers indicate the expected range of the value and the method will clamp the received value to this range. If parsing fails, the third argument number is returned as the default value. If everything has been parsed correctly, an instance of the rubette is created and the number of output connectors is configured on this instance. The complete, fully functional source code is contained in section 20.10.

284

20 User’s Manual

20.7.3 Packaging a Plug-In One possibility to make the rubette available to R UBATO C OMPOSER consists in compiling it into the distribution. To do this, the Java file org.rubato.composer.BuiltinRubettes has to be adapted by extending the array of strings classes with the fully qualified name of the rubette class. To avoid this, the plug-in mechanism should be used. The rubette, including its supporting classes, but not classes from the standard R UBATO framework, is put into a JAR archive. In order that R UBATO can find the rubette, the manifest of the JAR file needs to specify the name of the rubette class. In the case of the Latch rubette a file named manifest with the following content has to be created: Plugin: rubatoexample.LatchRubette Then a JAR file latch.jar is built using the command jar cvfm latch.jar manifest rubatoexample which is executed in the top-level directory of the package tree. To make the plug-in visible to R UBATO C OMPOSER, the JAR file has to be placed in one of several standard locations. On UNIX platforms these locations are $HOME/.rubato/plugins and $HOME/Rubato/Plugins, where $HOME is the user’s home directory. On start-up, the R UBATO C OMPOSER GUI will scan these standard locations for JAR archives, load the plug-ins, and instantiate the rubettes as indicated by the manifest.

20.8 Types of Module Morphisms

285

20.8 Types of Module Morphisms The letters L, M and N stand for arbitrary modules, the letters P, R and S for arbitrary rings. The letters n and m represent integers. If the same letter is used in one table row, it stands for the same module, ring or number. The resulting morphism is always denoted by the letter f.

Type

Description

Affine

Rm → Rn The resulting affine morphism f is defined by f(x) = A·x+b, where A is a n×m matrix and b an element of Rn . If m = n = 2, the transformation can be defined graphically in the Euclidean plane. This is currently only implemented for modules over the rings Z, Zn , Q, R, and C.

Canonical

M→N

The result will be the simplest morphism from M to N, i.e., the morphism that maps a value from M to N with the least change. For example, if M = N, then f is the identify. If M is Z and N is Q, then f is the embedding of the integers into the rationals.

Conjugation Cn → Cn The morphism f maps a complex vector v to its complex conjugate v (conjugation is componentwise). Constant

M→N

A constant element c from N must be specified, then f(x) = c, for every x ∈ M.

Composition M → N

The morphism f is the composition of two morphism g : M → L and h : L → N, both of which must be specified, i.e., f(x) = h(g(x)) or f = h ◦ g.

Difference

M→N

For two specified morphisms g : M → N and h : M → N, the result is the difference of g and h, i.e., f(x) = g(x) − h(x).

Geometry

R2 → R2 This is an alternative way to define a map of the real plane by a sequence of primitive geometric transformations. The primitives are rotation, reflection, translation, scaling, horizontal and vertical shearing.

Identity

M→M

Identity morphism, i.e., f(x) = x, for every x ∈ M.

Modulo

Zn

Canonical homomorphism from the integers to the integers modulo m, i.e., f(x) = x mod m, also extended to free modules.

→

Znm

286

20 User’s Manual

Type

Description

Polynomial

R→R

A polynomial p with coefficients in the ring R must be specified, then f(x) = p(x).

Power

M→M

A morphism g : M → M to be specified is raised to a given power n, i.e., f = gn , or f(x) = g(g(. . . (x) . . .)), where g is applied n times.

Product

M→R

For two specified morphisms g : M → R and h : M → R, the result is the product of g and h, i.e., f(x) = g(x) · h(x).

Scaled

M→N

For a specified constant s from the coefficient ring R of N and a morphism g : M → N, the result is g scaled by s, i.e., f(x) = s · g(x).

Shuffle

Rm → Rn Each component of an element of the domain module is mapped to a component of the value in the codomain module. This is in fact a linear morphism representing a generalized permutation. However the remapping is specified in a graphical way.

Split

Rm → Rm The domain and codomain modules are regarded as Rm1 ⊕ · · · Rmn . On each Rmi a morphism gi is specified, then f = g1 ⊕ · · · gn .

Sum

M→N

For two specified morphisms g : M → N and h : M → N, the result is the sum of g and h, i.e., f(x) = g(x) + h(x).

Translation

M→N

A morphism g : M → N to be specified is translated by a given element t from N, i.e., f(x) = g(x) + t.

20.9 The Rubette Java Interface

20.9 The Rubette Java Interface public interface Rubette { public void init(); public void run(RunInfo runInfo); public String getName(); public Rubette newInstance(); public Rubette fromXML(XMLReader reader, Element element); public void toXML(XMLWriter writer); public Rubette duplicate(); ◦ public String getGroup(); ◦ public ImageIcon getIcon(); ◦ public boolean hasProperties(); ◦ public JComponent getProperties(); ◦ public boolean applyProperties(); ◦ public void revertProperties(); ◦ public boolean hasView(); ◦ public JComponent getView(); ◦ public void updateView(); ◦ public boolean hasInfo(); ◦ public String getInfo(); ◦ public String getShortDescription(); ◦ public String getLongDescription(); ◦ public String getInTip(int i); ◦ public String getOutTip(int i); ◦ public void setInCount(int n); • public int getInCount(); • public Denotator getInput(int i); • public void setOutCount(int n); • public int getOutCount(); • public void setOutput(int i, Denotator d); • public Denotator getOutput(int i); • public void addError(String msg, Object ... objects); • public List<String> getErrors(); • public void clearErrors(); • public boolean hasErrors(); • public void setModel(RubetteModel model); • public RubetteModel getModel(); • }

287

288

20 User’s Manual

20.10 Example LatchRubette class package rubatoexample.LatchRubette; import java.awt.BorderLayout; import javax.swing.ImageIcon; import javax.swing.JComponent; import javax.swing.JPanel; import import import import import import import import import import

org.rubato.base.AbstractRubette; org.rubato.base.RubatoConstants; org.rubato.base.Rubette; org.rubato.composer.RunInfo; org.rubato.composer.components.JConnectorSliders; org.rubato.composer.icons.Icons; org.rubato.math.yoneda.Denotator; org.rubato.xml.XMLReader; org.rubato.xml.XMLWriter; org.w3c.dom.Element;

public class LatchRubette extends AbstractRubette { public LatchRubette() { setInCount(1); setOutCount(1); } public Rubette newInstance() { return new LatchRubette(); } public Rubette duplicate() { LatchRubette rubette = new LatchRubette(); rubette.setOutCount(getInCount()); return rubette; } public void init() {} public String getName() { return "Latch"; } public String getGroup() { return "Examples"; }

20.10 Example LatchRubette class

289

public String getShortDescription() { return "Stores an input denotator"; } public String getLongDescription() { return "The Latch Rubette stores its input denotator"+ " and provides it on its outputs the number of"+ " which can be configured."; } public String getInTip(int i) { return "Input denotator"; } public String getOutTip(int i) { return "Output denotator #"+i; } public ImageIcon getIcon() { return icon; } public void run(RunInfo runInfo) { Denotator d = getInput(0); for (int i = 0; i < getOutCount(); i++) { if (runInfo.stopped()) { return; } setOutput(i, d); } } public boolean hasProperties() { return true; } public JComponent getProperties() { if (properties == null) { properties = new JPanel(); properties.setLayout(new BorderLayout()); outSlider = new JConnectorSliders(false, true); outSlider.setOutLimits(1, 8); outSlider.setOutValue(getOutCount()); properties.add(outSlider, BorderLayout.CENTER); } return properties; }

290

20 User’s Manual

public boolean applyProperties() { setOutCount(outSlider.getOutValue()); return true; } public void revertProperties() { outSlider.setInValue(getOutCount()); } public void toXML(XMLWriter writer) { writer.empty(OUTPUTS, NUMBER_ATTR, getOutCount()); } public Rubette fromXML(XMLReader reader, Element element) { Element child = reader.getChild(element, OUTPUTS); if (child != null) { int n = reader.getIntAttribute(child, NUMBER_ATTR, 1, 8, 1); LatchRubette rubette = new LatchRubette(); rubette.setOutCount(n); return rubette; } else { return null; } } private JPanel properties = null; private JConnectorSliders outSlider = null; private final static String OUTPUTS = "Outputs"; private final static String NUMBER_ATTR = "number"; private final static ImageIcon icon; static { icon = Icons.loadIcon(LatchRubette.class, "latchicon.png"); } }

20.11 Keyboard Shortcuts

291

20.11 Keyboard Shortcuts The following table summarizes the keyboard shortcuts active in the main application window. Key

Operation

C T R L -O Open project C T R L -S Save project C T R L -S H I F T -S Save project under new name C T R L -Q Quit C T R L -A Add new rubette C T R L -N Create new network C T R L -W Close visible network C T R L -E Toggles pass-through on the selected rubette HO M E Moves selected rubette to front EN D Moves selected rubette to back F9 Start visible network execution F10 Stop network execution A L T -S H I F T -M Open module builder C T R L -S H I F T -M Open module morphism builder C T R L -S H I F T -D Open denotator builder C T R L -S H I F T -F Open form builder C T R L -S H I F T -B Open object browser A L T -S H I F T -S Open Scheme dialog

292

20 User’s Manual

20.12 Rubato Scheme The little Scheme interpreter embedded in R UBATO C OMPOSER is close to R5 RS, but is not a complete implementation of this standard. It supports the whole number tower, but omits such features as macros and continuations. However, most of the standard functions are provided. A set of functions that pertain to R UBATO C OMPOSER have been added, which will be presented below. In the following, obj stands for an arbitrary argument, d for a denotator, f for a form, d-or-f for an argument that may be either form or denotator. (form? obj) (denotator? obj) Rubato Scheme defines additional object types for forms and denotators. These predicates test their argument for these types. (get-form obj) (get-denotator obj) The argument obj here is either a string or a symbol and specifies the name of a form, respectively a denotator, to retrieve from the system repository. (get-all-forms) (get-all-denotators) All forms, respectively all denotators, in the system repository are returned as a list. (type-simple? d-or-f) (type-limit? d-or-f) (type-colimit? d-or-f) (type-power? d-or-f) (type-list? d-or-f) The argument obj in these predicates is either a form or a denotator. The predicates test for type of the form, respectively the denotator. (type-of d-or-f) The argument obj in this function is either a form or a denotator. The function returns the type of the form, respectively the denotator, as a symbol, i.e., one of simple, limit, colimit, power or list. In the case of a wrong argument, it returns the symbol none. (name-of d-or-f) This function returns the name of its argument, which may be either a form or a denotator. The result is a string. If the argument is neither form nor denotator, an error occurs.

20.12 Rubato Scheme

293

(form-of d) This functions returns the form of the argument denotator. (forms-of f) In the case of a simple form, this function returns an empty list, otherwise it returns a list of the coordinate forms of the form. (form-count-of f) The number of coordinate forms of the argument form is returned as an integer. (factors-of d) The coordinate denotators of the argument denotator are returned as a list of denotators. (factor-count-of d) The number of coordinate denotators of the argument denotator is returned as an integer. (index-of d) When d is a denotator of type Colimit, this function returns the index of the denotator, otherwise it returns −1. (element-of d) When d is a denotator of type Simple, this function returns the element contained in the denotator, otherwise it is an error. (make-denotator s f obj) This function creates a denotator with name s and the given form f. If s is neither string nor symbol, the new denotator will be anonymous. The obj argument depends on the type of f. If f is of type Simple, the argument must be a basic object corresponding to the module in f, such as integer or real, or a vector of such basic objects in the case of a free module. Type Limit expects the list of coordinate denotators and type Colimit a list of two elements, where the first is the index and the second the coordinate. Both type Power and type List expect a list of coordinate denotators. (register d) If the denotator d has a non-empty name, it is registered under this name in the system repository.

References

[1] Abelson, Harold; Sussman, Gerald Jay and Sussman, Julie. Structure and Interpretation of Computer Programs. Second Edition. The MIT Press, Cambridge, Massachusetts 1996. [2] Agawu, V. Kofi. Playing with Signs. Princeton University Press, Princeton 1991. [3] Agon, Carlos and Assayag, Gérard. “Object-Oriented Programming in OpenMusic.” In: Guerino Mazzola, The Topos of Music, pp. 967–990. [4] Agon, Carlos; Assayag, Gérard and Bresson, Jean (eds). The OM Composer’s Book Volume 1. Editions Delatour, IRCAM 2006. [5] Andreatta, Moreno. Méthodes algébriques en musique et musicologie du XXe siècle: aspect théoriques, analytiques et compositionnels. PhD thesis, Paris 2003. [6] Arnold, Ken; Gosling, James and Holmes, David. The Java Programming Language. Third Edition. Addison-Wesley, Reading, Massachusetts 2000. [7] Bentley, Jon Louis. “Multidimensional binary search trees used for associative searching.” In: Communications of the ACM 18, 9, pp. 509-517, 1975. [8] Bird, Richard and de Moor, Oege. Algebra of Programming. Prentice Hall, 1996. [9] Bird, Richard. Introduction to Functional Programming using Haskell. Second Edition. Prentice Hall, 1998. [10] Boulanger, Richard (ed). The Csound Book. The MIT Press, Cambridge, Massachusetts 2000. [11] Boulez, Pierre. structures, premier livre. UE, London 1953. [12] Boulez, Pierre. structures, deuxième livre. UE, London 1967. [13] Boulez, Pierre. “Musikdenken heute I,II.” In: Darmstädter Beiträge V, VI. Schott, Mainz 1963, 1985. 295

296

References

[14] Boulez, Pierre. Jalons (dix ans d’enseignement au Collège de France). Bourgeois, Paris 1989. [15] Buteau, Chantal. “Motivic Spaces of Scores through RUBATO’s MeloTopRUBETTE.” In: Guerino Mazzola, Thomas Noll, and Emilio Lluis-Puebla (eds), Perspectives in Mathematical Music Theory, pp. 330–342, epOs Music, Osnabrück 2004. [16] Châtelet, Gilles. Les enjeux du mobile. Seuil, Paris 1993. [17] Chew, Geoffrey. “Notation (III, 4–6).” In: Stanley Sadie (ed), The New Grove Dictionary of Music and Musicians, Macmillan Publishers Ltd, London 1980. [18] Chowning, John. “The Synthesis of Complex Audio Spectra by Means of Frequency Modulation.” In: Journal of the Audio Engineering Society, 21 (7), pp. 526–534, 1973. [19] Corbin, Solange. “Neumatic notations (I-IV).” In: Stanley Sadie (ed), The New Grove Dictionary of Music and Musicians, Macmillan Publishers Ltd, London 1980. [20] Cormen, Thomas H.; Leiserson, Charles E.; Rivest, Ronald L. and Stein, Clifford. Introduction to Algorithms. Second Edition. The MIT Press, Cambridge, Massachusetts 2001. [21] Csíkszentmihályi, Mihály. Beyond Boredom and Anxiety. Jossey-Bass, San Francisco 1975. [22] Dannenberg, Roger. “Music Representation Issues, Techniques, and Systems.” In: Computer Music Journal, 17 (3), pp. 20–30, The MIT Press, Cambridge, Massachusetts 1993. [23] Dart, Thurston and Morehen, John. “Tablature.” In: Stanley Sadie (ed), The New Grove Dictionary of Music and Musicians, Macmillan Publishers Ltd, London 1980. [24] Deyoung, Lynden. “Pitch Order and Duration Order in Boulez’s Structure 1a.” In: PNM, 16/2, pp. 27–34, 1978. [25] Fux, Johann Joseph. The Study of Counterpoint (Gradus ad Parnassum). Tr. Alfred Mann. W. W. Norton & Co., New York 1965. [26] Garbers, Jörg. Integration von Bedien- und Programmiersprachen am Beispiel von OpenMusic, Humdrum und Rubato. dissertation.de, Berlin 2004. [27] Gieseking, Martin. Code-basierte Generierung interaktiver Notengraphik. Zur Entwicklung einer dynamischen Notendarstellung in interaktiven Lernprogrammen und musikspezifischen MultimediaApplikationen. Dissertation, epOs Music, Osnabrück 2001. [28] GNU LilyPond — The music typesetter. , accessed June 2006. [29] Göller, Stefan. Object Oriented Rendering of Complex Abstract Data.

References

297

Dissertation, Zürich 2004. [30] Göller, Stefan and Milmeister, Gérard. “Distributed RUBATO: Foundation and Multimedia Rendering.” In: Proceedings of the ICMC 2004, ICMA, Miami 2004. [31] Hiller, Lejaren A. and Isaacson, Leonard M. Experimental Music: Composition with an Electronic Computer. Greenwood Press, Westport 1959. [32] Huron, David. The Humdrum Toolkit: Reference Manual. Center for Computer Assisted Research in the Humanities, Menlo Park, California 1995. [33] Java Development Kit 5.0 (a.k.a. 1.5.0). , accessed June 2006. [34] Jensen, Kathleen and Wirth, Nikolaus. Pascal User Manual and Report. Fourth Edition. Springer, New York 1991. [35] Kelsey, Richard; Clinger, William and Rees, Jonathan (eds). “Revised5 Report on the Algorithmic Language Scheme.” In: Higher-Order and Symbolic Computation, 11 (3), pp. 7–105, 1998. [36] Koenig, Gottfried Michael. “Project 2: A Programme for Musical Composition.” In: Electronic Music Reports, 3, Institute of Sonology, Utrecht 1970. [37] Lawvere, F. William and Schanuel, Stephen H. Conceptual Mathematics. Cambridge University Press, Cambridge 1997. [38] Lerdahl, Fred and Jackendoff, Ray. A Generative Theory of Tonal Music. The MIT Press, Cambridge, Massachusetts 1983. [39] Ligeti, György. “Pierre Boulez: Entscheidung und Automatik in der Structure Ia.” In: Die Reihe IV, UE, Wien 1958. [40] Ableton Live. , accessed December 2008. [41] Logic Pro. , accessed December 2008. [42] Mathews, Max. The Technology of Computer Music. The MIT Press, Cambridge, Massachusetts 1969. [43] Mazzola, Guerino. Gruppen und Kategorien in der Musik. Heldermann, Berlin 1985. [44] Mazzola, Guerino. Geometrie der Töne. Birkhäuser, Basel 1990. [45] Mazzola, Guerino. “Inverse Performance Theory.” In: Proceedings of the ICMC 1995, pp. 533–540, ICMA, Banff 1995. [46] Mazzola, Guerino. “Degenerative Theory of Tonal Music.” In: Bernd Enders and Joachim Stange-Elbe (eds), Global Village—Global Brain— Global Music, pp. 382–394, epOs Music, Osnabrück 1999.

298

References

[47] Mazzola, Guerino. The Topos of Music. Birkhäuser, Basel 2002. [48] Mazzola, Guerino. L’Essence du Bleu. Acanthus, Rüttenen 2002. [49] Mazzola, Guerino. La vérité du beau dans la musique. Delatour, Paris 2007. [50] Mazzola, Guerino and Andreatta, Moreno. “Formulas, Diagrams, and Gestures in Music.” In: Journal of Mathematics and Music, 1, 1, pp. 21-32, 2007. [51] Mazzola, Guerino and Milmeister, Gérard. “Functors for Music: The Rubato Composer System.” In: Proceedings of the ICMC 2006, ICMA, New Orleans 2006. [52] Mazzola, Guerino; Milmeister, Gérard and Weissmann, Jody. Comprehensive Mathematics for Computer Scientists 1. Springer, Berlin 2004. [53] Mazzola, Guerino; Milmeister, Gérard and Weissmann, Jody. Comprehensive Mathematics for Computer Scientists 2. Springer, Berlin 2005. [54] Mazzola, Guerino and Thalmann, Florian. “Grid Diagrams for Ornaments and Morphing.” In: Proceedings of the First International Conference of the Society for Mathematics and Computation in Music, MCM, Berlin 2007. [55] The Complete MIDI 1.0 Detailed Specification. The MIDI Manufacturers Association, Los Angeles 1996. [56] Montiel Hernandez, Mariana. El Denotador: Su Estructura, construcción y Papel en la Teoría Matemática de la Musica. UNAM, Mexico City 1999. [57] Morsy, Karim. Denotatorbasierte Makroevents und Methoden der musikalischen Komposition. Zürich 2006. [58] Müller, Stefan and Mazzola, Guerino. “The Extraction of Expressive Shaping in Performance.” In: Computer Music Journal, 27 (1), pp. 47– 58, The MIT Press, Cambridge, Massachusetts 2003. [59] MusicXML Definition. , accessed June 2006. [60] Noll, Thomas. “The Topos of Triads.” In: Harald Fripertinger and Ludwig Reich (eds), Colloquium on Mathematical Music Theory, Grazer Mathematische Berichte, Bericht Nr. 347, Graz 2005. [61] Pierce, Benjamin C. Basic Category Theory for Computer Scientists. The MIT Press, Cambridge, Massachusetts 1991. [62] Puckette, Miller S. Pure Data: another integrated computer music environment. Proceedings, Second Intercollege Computer Music Concerts, Tachikawa, pp. 37–41, 1996. [63] Schönberg, Arnold. Harmonielehre. Universal Edition, Wien 1966.

References

299

[64] Schwartz, Jacob T. Set Theory as a Language for Program Specification and Programming. Courant Institute of Mathematical Sciences, New York University 1970. [65] Sloan, Donald. “Aspects of Music Representation in HyTime/SMDL.” In: Computer Music Journal, 17 (4), pp. 51–59, The MIT Press, Cambridge, Massachusetts 1993. [66] Stange-Elbe, Joachim. Analyse- und Interpretationsperspektiven zu J. S. Bachs “Kunst der Fuge” mit Werkzeugen der objektorientierten Informationstechnologie. Habilitationsschrift, Osnabrück 2000. [67] Taube, Heinrich. “Common Music: A Music Composition Language in Common Lisp and CLOS.” In: Computer Music Journal, 15 (2), pp. 21–32, The MIT Press, Cambridge, Massachusetts 1991. [68] Thalmann, Florian. Musical composition with Grid Diagrams of Transformations. Master Thesis, University of Bern, Bern 2007. [69] Thalmann, Florian and Mazzola, Guerino. “The BigBang Rubette: Gestural Music Composition with Rubato Composer.” In: Proceedings of the 2008 International Computer Music Conference, Belfast 2008. [70] Walshaw, Chris. the abc musical notation language. , accessed June 2006. [71] Walters, Robert F. C. Categories and Computer Science. Cambridge University Press, Cambridge 1992. [72] Wanderley, Marcelo and Battier, Marc (eds). Trends in Gestural Control of Music. Ircam - Centre Pompidou, Paris 2000. [73] Xenakis, Iannis. “La Crise de la musique serielle.” Gravesaner Blätter, 1, 1955. [74] Xenakis, Iannis. Musiques formelles. La Revue Musicale 253 and 254, Editions Richard Masse, Paris 1963. [75] Zahorka, Oliver. “PrediBase—Controlling Semantics of Symbolic Structures in Music.” In: Proceedings of the ICMC 1995, pp. 203– 206, ICMA, Banff 1995.

Colophon

This text has been typeset using LATEX on Fedora Linux. The typeface used for the titles and the body text is Bitstream Kuenstler 480. The typeface used for monospaced text and computer code is Bitstream Letter Gothic 12 Pitch. The fragments of musical notation have been created using Lilypond. For the drawings the vector graphics software Inkscape, as well as METAPOST, have been used. The UML diagrams have been created with the MetaUML package for METAPOST. Bitmap images have been manipulated using GIMP. Fedora: http://www.fedoraproject.org Lilypond: http://www.lilypond.org Inkscape: http://www.inkscape.org METAPOST: http://cm.bell-labs.com/who/hobby/MetaPost.html MetaUML: http://metauml.sourceforge.net GIMP: http://www.gimp.org

301