F. Preparata ( E d.)
Theoretical Computer Science Lectures given at a Summer School of the Centro Internazionale Matematico Estivo (C.I.M.E.), held in Bressanone (Bolzano), Italy, June 9-17, 1975
C.I.M.E. Foundation c/o Dipartimento di Matematica “U. Dini” Viale Morgagni n. 67/a 50134 Firenze Italy
[email protected] ISBN 978-3-642-11118-1 e-ISBN: 978-3-642-11120-4 DOI:10.1007/978-3-642-11120-4 Springer Heidelberg Dordrecht London New York
©Springer-Verlag Berlin Heidelberg 2011 Reprint of the 1st ed. C.I.M.E., Ed. Cremonese, Roma 1975 With kind permission of C.I.M.E.
Printed on acid-free paper
Springer.com
CENTRO INTERNAZIONALE MATEMATICO ESTIVO (CLM.E.)
I Cielo - Bressanone dal 9 al 17 giugno 197 5
THEORETICAL COMPUTER SCIENZE Coordinatore: Prof. F. PREPARATA
R. E. MILLER
Parallel program schemata
D. E. MULLER
Theory of automata
»
65
R. M. KARP
Computational complexity of combinatorial and graph-theoretic problems
»
97
pag.
5
CENTRO INTERNAZIONALE MATEMATICO ESTIVO (C.I.M.E.)
PARALLEL PROGRAM SCHEMATA
R.E. MILLER
Corso tenuto a Bressanone dal 9 al 17 giugno 1975
-
6 -R. E. Miller
Configurable Computers and the Data Flow Model Transformation
R. E. Miller
In this lecture we discuss a type of computer organization which is based upon the concept of operation sequencing being controlled by operand availability.
Such sequencing differs radically from current computer
organizations in which the sequencing is determined by an explicit or implicit ordering of the instructions which is controlled through an instruction counter which specifies the next instruction to be performed. We will discuss a particular type of data sequenced computer called "configurable computers" [7].
Closely related types of machines have
subsequently also been proposed and studied by others [3, 11, 10, 9]. One of the major problems with unconventional computer organizations that have been proposed in the past is the great difficulty of programming such machines.
In some cases the proposed structures were so complex that
an i~ordinate amount of work was required to specify what each object in the structure was to do, while in other cases the structures were particuiarly suited to only a narrow class of computations, and were ill suited for general purpose use.
A number of approaches for representing the
computational process in a data sequenced manner have recently been proposed [II, 5, 2, lJ.
We will briefly describe one such approach [8, 6]
which is aimed at automatically transforming normal computer programs into
a data
sequenced form suitable for configurable computers.
Even though a
direct representation of an algorithm into a data sequenced form may be able to better exploit the advantages of the data sequenced form, consider· able advantage is gained in automatic transformation -- closely related to optimizing compiler techniques -- which allows the user to continue to use his well known programming languages directly, rather than learning a new complex language immediately.
-
7 -R. E. Miller
Configurable Computers One of the central concepts of configurable computers is to have a machine that changes its structure into the natural structure of the ~lgorithm
being performed, thereby allowing parallel computations and many
pf the speed advantages of special purpose devices. Two particular approaches for attaining this goal are described.
The
pne approach is called Search Mode and the other is called Interconnection ~ode.
These should be viewed as only two ends of a spectrum of possibilities
in which considerable differences in the possible computer control structures exist. Search Mode Configurables The basic organization of a search mode machine is depicted in Figure 1.
Active Operational Units The operational units are thought of as a set of either general or special purpose units which perform the computational, conditional, and data generation aspects of the computation.
When one of these units has
completed a task it requests the searcher to find another task for it to perform.
The searcher, which is essentially a new kind of control, then
inspects memory, or a suitable portion thereof, for a new task for the operational unit.
This organization can thus be adopted to the idea
that a task becomes capable of being performed when its operands have
-8-
R. E. Miller been computed.
An example of machine instruction format aids in seeing in
more detail how the machine could operate.
We give an example for an
arith~
metic operation but, of course, a complete repetoire of instructions and formats would have to be given for a complete specification.
I
Operation Code
I
Status Bits
I
First
Second
Operand
Operand
"Address" for Result
This instruction format contains first a field of bits to specify the type of operation to be performed (the operation code).
Skipping over the
status bits for a moment, it then contains fields for the operand values to be stored.
Within the status bits one keeps track of whether the
operands currently reside in these locations (i.e., have been computed and stored there) and whether the location for the result is available for storing a result.
Finally, the. "addresS for result" field specifies where
the result is to be stored
(i.e.~
as an operand for some other instruction)
Storing a result in an operand field updates the status bits, as does the action of performing an instruction.
Clearly, this type of format
eliminates the need for normal instruction sequencing.
The sequencing is
essentially "data driven." It is the operation code and status bit fields that the searcher inspects to determine readiness of an instruction.
This could be imagined
to be done by a type of associative search, or by a method of building up stacks or queues of instructions that are ready to be performed.
Before
describing how programs can be transformed into such a data sequenced form, we describe the interconnection mode machine idea.
-9R. E. Miller Interconnection Mode Configurab1es In the search mode concept the sequencing of instructions was done through the action of operation results becoming operands of new instructions, and these transfers of information were done by storing results in ~he
proper places of instructions.
With the advent of economical e1ec-
switches more direct connection of result to operand could be
~ronic
~nvisioned. ~iagram
This is the case for the interconnection mode idea.
A block
is shown below. Memory
Setup
Data Access
Control
Control
n x n
Interconnection Networks
Operational Units
Here the interconnection network is used to directly interconnect $utputs of one operational unit to inputs of another operational unit in accord with the result to operand specifications of the algorithm. ~asic
The
steps for such a machine are: 1.
Decompose the program into appropriate size blocks.
2.
Transform each block into a data sequenced form.
3.
Store the blocks, so transformed, in memory as setup instructions for the interconnection network.
4.
Choose a block to be performed (to start with this is the block with the start of the program-- subsequently this is specified
-
10R. E. Miller
by what block the running block exits to next) and set up the interconnections as specified. 5.
Perform the block execution. instuctions
6.
Note that during this time no
only data -- need be stored or fetched from memory.
Termination of block specifies next block to be performed (return to 4).
At this point it should be clear that both the search mode and
inter~
connection mode machines are sequenced essentially by data paths rather than control paths through the algorithm.
Thus, the natural parallelism
of the algorithm can be used to speed up operation.
Also, the machines
have some of the speed advantages of special purpose devices.
This is
especially true of the interconnection mode machine since units are actually directly interconnected as they would be in a special purpose device The operational control -- distributed throughout the machine by data availability considerations -- is also rather simple, and this could be a distinct advantage over other approaches to high performance where very complex control mechanisms are required.
Finally, as we have indicated,
standard programming languages can be used to express the jobs to be done and transformational techniques to provide suitable machine language instructions for these machines should be readily developed using techniques known for compiler optimization.
Other potential advantages for
these machines are given in [7]. Data Sequenced Program Transformation We now outline how a program can be transformed into a form suitable for our configurable computers.
We call this a data flow transformation.
The basic steps of the transformation are:
-
11 -
R. E. Miller
1.
Partition the program into "basic blocks" and name each block.
A
basic block is a contiguous segment of code which can be entered only through the first instruction of the block, must be executed by executing each successive instruction in order, and can be exited only from the last instruction in the block.
I.e., it is a "straight line"
segment of code. 2.
Determine the immediate predecessors and successors of each basic block.
3.
Generate (in arbitrary order) a "data flow segment" for each block. A data flow segment consists of: (i)
input list -- i.e. variables needed by the block.
(ii)
output list -- i.e. result names at end of block execution.
(iii) interconnected modules -- i.e. the operations and flow of data from result to operand between operations in the block. 4.
Interconnect data flow segments.
This uses the predecessor and
successor information and updates input and output lists for data "passing through" a block. Without going into great detail we illustrate the transformation through an example from [6]. An Example of Data Flow Model Transformation
As an example program we consider the problem of evaluating the function
f(x) = aX + bx + c.
We assume that
stored in the symbolic locations assume that
x
x, a, b, and c
is a positive integer.
evaluation is shown below. be self-explanatory.
x, a, b, and c
are inputs
respectively, and also
A simple program to perform this
The program language used is simple and should
-
12R. E. Miller
Program
Comments
1
CLA x
set accumulator to
2
STO COUNT
put
x
in location COUNT.
3
CLA a
put
a
in accumulator.
4
TRA 6
transfer to statement 6.
5
MPY a
multiply accumulator by
6
Decrement COUNT
decrease COUNT by 1.
7
Branch on COUNT (to 5 on "! 0)
conditional transfer.
8
STO T
store
a
9
CLA x
place
x
in accumulator.
10
MPYb
form
bx
in accumulator.
11
ADDT
form
aX+bx
12
ADDC
form
aX+bx+c
13
STO R
store result in
Statement II
x
in
x.
a.
T.
in accumulator. in accumulator.
R.
Applying the notion of basic block to this program we find that instructions
1, 2, 3, 4
start of the program. 8, 9, 10, 11, 12, 13
form a basic block with instruction 1 being the
Similarly instructions form a basic block and
These blocks are depicted and named diagram.
BBl
6, 7 5
through
form a basic block,
alone forms a basic block. BB4
in the following
-
13R. E. Miller
BBI
BB3
{
1
CLA x
2
STO COUNT
3
CLA a
4
TRA 6
(
5
MPYa
{
6
Decrement COUNT
7
Branch on COUNT (to 5 on #)
8
STO T
9
CLA x
10
MPY b
11
ADD T
12
ADD c
13
STO R
-----------------------------------------------BB2
BB4
Step 2 of the algorithm determines immediate successors and immediate predecessors as shown in the following figure.
Step 3 of the algorithm generates a "data flow segment" for each basic block.
The idea here is to generate a list of items needed as inputs to
the block, the outputs created by the block and the operations used to create these outputs along with the flow of data between the operations within the block.
The items have names associated with them through the
program language definition, these names we call "source names."
During
-
14-
R. E. Miller example, basic block 2, (BB2). Decrement COUNT.
This block starts with instruction 6
By definition this instruction needs an input with sourqe
name "COUNT" and produces a new output also called "COUNT" which has a value less than the original value of COUNT.
Thus COUNT is placed on the
input list, and we associate a local data name with this input value. use the name
BB2,1; i.e., the first local data name in BB2.
In the out-
put list we then have COUNT with a new value and name this new value as a local data name.
The operation performed is a SUBTRACT I
operation gets placed in the module structure with input BB2,2.
The second instruction of
BB2
tion uses the current value of COUNT, namely or
fO.
This is indicated in the output list
to two values
COUNT BB2,2TI
test either being outcome
and
TI
or
BB2. BB2
Input List Output List
T2.
This
and outpqt instruc~
and tests it for
=0
changing item COUNT-BB2,4 for the outcome of the
A test module is added to the
T -- with the two indicated outputs.
Our result is summerized belpw. Data Flow Segment
COUNT - BB2,1 -€eYN~-=-BB~,~--
COUNT - BB2, 2 Tl COUNT - BB2,2T2 Module Structure
~s
COUNT BB2,2T2
module structure -- we call it test This completes Step 3 for
BB2,2
BB2~.2
so this
BB2,1
is Branch on COUNT.
We
-
15 --
R. E. Miller Similar calculations are done for each of the other basic blocks producing the following results. BBI Input List
Data Flow Segment x - BBl,l a - BBl,2
-A66BM = BBi,iCOUNT - BBl,l ACCUM - BBl,2
Output List
no modules. Data Flow Segment
BB3 Input List
ACCUM - BB3,1 a - BB3,2
Output List
ACCUM - BB3,3 BB3'1;;f3,2
Module Structure
~3,3 BB4 Input List
Output List
Data Flow Segment
ACCUM x b c T
-
BB4,1 BB4,2 BB4,3 BB4,6
- BB4,1
-A66BM-=-BB4,~-
-A66BM-=-BB4,4-A66BM-=-BB4,5ACCUM - BB4,7 R - BB4,7 Module
~tructure
BB4,~BB4,3
l!!D B4,4
-
16-
R. E. Miller Step 4 of the transformation interconnects these module structures by using successor and predecessor information and making output to input connections through common source names. and inserting test points
The result of making these interconnections
TI
and
T2
conditionally on the outcome of test
for places where data passes only T
is shown in the next figure.
Note
that even in this simple example some possibilities for parallelism are exhibited.
For example, the two multiplications and the subtract I
operations could all be performed
concurre~tly.
x
a
T
R
It should be clear from the descript!ons given for configurable computers that this diagram provides all the essential information needed to specify the instructions for a search mode machine or the interconnections for an interconnection mode machine.
Clearly, the sequencing so
specified differs considerably from how the original program would have run on a conventional computer.
-
17-
R. E. Miller tmFERBNCES [1] A. Bllhrs, "Operation Patterns," Symposium on Theoretical Programming, Novosibirsk, USSR, August 1972, in Lecture Notes in Computer Science, Vol. 5 International Symposium on Theoretical Programming, SpringerVerlag, New York, 1974, pp. 217-246. [2] J. B. Dennis, J. B. Fosseen, and J. P. Linderman, "Data Flow Schemas, , Symposium on Theoretical Programming, Novosibirsk, USSR, August 1972, in Lecture Notes in Computer Science, Vol. 5 International Symposium on Theoretical Programming, Springer-Verlag, New York, 1974, pp. 187-216. [3] J. B. Dennis and D. P. Misunas, "A Computer Architecture for Highly Parallel Signal Processing," Proceedings ACM Annual Conference, November 1974, pp. 402-409. [4] K. B. Irani and C. R. Sonnenburg, "Exploitation of Implicit Parallelism in Arithmetic Expressions for an Asynchronous Environment," Report [5] P. R. Kosinski, itA Data Flow Programming Language," IBM T. J. Watson Research Center Report RC-4264, Yorktown Heights, N. Y., March 1973. [6] R. E. Miller, "The Data Flow Model Transformation," to appear in the Proceedings of the 1972 GMD Summer Seminar on Systems Organization, Bonn, Germany. [7] R. E. Miller and J. Cocke, "Configurab1e Computers: A New Class of General Purpose Machines," Symposium on Theoretical Programming, Novosibirsk, USSR, August 1972, in Lecture Notes in Computer Science, Vol. 5 International Symposium on Theoretical Programming, SpringerVerlag, New York, 1974, pp. 285-298. [8] R. E. Miller and J. D. Rutledge, "Generating a Data Flow Model of a Program," IBM Technical Disclosure Bulletin, Vol. 8, No. 11, April 1966, pp. 1550-1553.
s.
[9]
S. Reddi and E. A. Feustel, "A Restructurab1e Computer System," Report, Laboratory for Computer Science and Engineering, Rice Univ., Houston, Texas, March 1975.
[10]
c. R. Sonnenburg, "A Configurab1e Parallel Computing System," Ph. D. Dissertation, University of Michigan, Ann Arbor, Octobe~ 1974.
[11] J. C. Syre, "From the Single Assignment Software Concept to a New Class of Multiprocessor Architectures," Report, 1975 Department d'Informatique, C.E.R.T. BP4025, 31055 Toulouse Cedex, France.
-
18R. E. Miller
Computation Graphs and Petri Nets R. E. Miller In this lecture we show how a special type of Petri net, widely studied in the literature, can be represented by computation graphs. We then illustrate how results about computation graphs can be translated into results for such Petri nets.
We first introduce the two models.
pomputation Graphs [3] A computation
graph
G is a finite directed graph consisting of:
(i) nodes n 1 ,n 2 , ••• ,n t ; (ii) edges d 1 ,d 2 , ••• ,d t , where any given edge dp is directed from a specified node n. to a specified node n.; (iii) four J
~
non-negative integers A , U ,Wand T , where T > W , associated with p p p p p- p rach edge d p • Nodes represent operations, edges represent first-in-first-out queues pf data, and for an edge d mean:
p
directed from n. to n. the four parameters J
~
A is the initial number of items in the queue, U is the number p
p
pf items added to the queue each time n i fires, Wp is the number of items removed from the queue each time n. fires, and T J
p
is the number of items
required as operands for n. to fire. J
The idea of computation sequences for a computation graph is that an operation can fire whenever it has a sufficient number of operands on each of its incoming edges. ~f
After firing it places results on each
its outgoing edges, and these results may be used later as operands
for other operations.
Some simple examples of computation graphs are
$hown in the following figures: (2,1,1,2)
~
~-1 + ~-2
-
19R. E. Miller
This one node, one edge example shows how a computation graph can realize the computation of successive Fibonacci numbers when the two initial values are each 1.
Note here that although
~
=
2 here W
1 so
that two operands are needed for the add operation·but only one is ~emoved.
Thus the second operand in one firing becomes the first
pperand in the next firing. (200,0,1,1)
(100,0,1,1)
(0,1,2,2)
(0,1,0,0)
This example illustrates combining two lists of numbers A and B to form a list C according to the equation c i = a 2i- 1 + a 2i + b i
for i = 1,2, .•. ,100.
More complex computation graphs are shown in [3 & 5], where they are studied in more detail. Petri Nets [6] Petri nets have become a very popular means of representing parallelism in systems.
Some examples of the literature are [1, 2, 6, 7].
It is a simple directed graph model in which the notion of computation can be easily explained.
More formally, a Petri
consists of a finite set of nodes
~
P
=
(IT,E,R,MO)
IT of places, a finite set of nodes
E
of transitions, a relation R ~ (ITxE) u (ExIT) which indicates directed edges between nodes, and a mapping MO from
IT
to the set of non-negative
-
20R. E. Miller
integers, called the initial marking which places a number of "tokens" in each place of
P.
(See [5] for a more complete definition.)
The
tokens in places are used to control the firing of transitions of a Petri net, and it is these firings that represent computations.
The set
of places having edges directed into a given transition are called the input places for the transition, and the set of places having edges directed out of the transition are called its output places.
A transi-
tion is active if each of its input places contains one or more tokens. An active transition may fire.
This removes one token from each of its
input places and adds one token to each of its output places. in a given place can be used in only one transition firing.
A token A sequence
of transition firings is called a firing sequence and each firing changes the marking (tokens) in the places. A simple Petri net is shown in the next figure.
Here circles
represent places, dots in the circles - tokens, and bars represent transitions.
As shown tl is active.
When it fires it removes the token from PI and
places tokens in P2 and P3
Now t2 and t3 are active.
fire tokens reside in P4 and P5 only.
After they both
Now both t4 and t5 are active,
but only one can fire since there is only one token each in P4 and P5' This is called a conflict.
In such a case an arbitrary choice is made,
.and the firing. places a token ba.ck in 1>-1>- _a&. was
th~case.
at the start.
-21R. E. Miller If we represent firing sequences by sequences of transition letters, then the reader may check that the following are firing sequences for this example: tlt2 t 3t 4tl tlt3 t 2t4 tlt3 t 2 t S tlt2 t 3t S Many special types of Petri nets have been studied.
One such type
restricts each place to have exactly one input transition and one output transition.
These kind of Petri nets are called marked graphs.
A simple
example is shown below.
One can now imagine removing the places from marked graphs and letting tokens reside on edges.
Then the figure simplifies to:
which is a directed graph with only one type of node. Marked Graphs and Computation Graphs It is easy to see [4] that any marked graph can be represented by a computation graph with
up
w
P
= T
P
1
for each edge, and A determined by the initial marking. p
Our example marked graph reduces to the following computation graph:
-
22R. E. Miller
(1.1.1.1)
(1.1.1.1)
~~(0'1'1'1) ~~
(0.1.1.1)
Through this reduction properties of computation graphs are readily translated to properties of marked graphs. d l ,d 2 ••• d t of a computation graph with WI WI" 'W t _ l q(l,N) + --U q(2,N) + ... + U U 2 2'" t the queue length on edge d i •
For example, for any loop Ul"'Ut Wl"'W t
= 1,
it is known that
q(t,N) is constant, where q(i,N) is
This translates to say that:
The token
count around a loop in a marked graph does not change by transition firing. We thus have seen, through a simple relationship between models, how a result can be translated to the other model.
We are now ready
to consider other models. References [lJ
F. Commoner, A. W. Holt, S. Even, A. Pneuli, "Marked Directed Graphs, Journal of Computer and Systems Sciences, ~, October 1971, pp.5ll-23.
[2 J M. Hack, "Decision Problems for Petri Nets and Vector Addition
System," MIT Project MAC, Computation Structures Group Memo 95, March, 1974.
[3J
R.M. Karp and R.E. Miller, "Properties of a Model for Parallel Computation: Determinacy, Termination, Queueing:' SIAM Journal of Applied Math., 1390-1411, November, 1966.
[4J
R.E. Miller, "A Comparison of Some Theoreti~al Models of Parallel Computation," IEEE Trans. on Computers, C-22, 8, 710-717, Aug. 1973_
[5J
R.E. Miller, "Some Relationships B,.ttween Various Models of Parallelism and Synchronization," IBM Research Report, RC 5074, October 1974.
[6J
C.A. Petri, "Communication with Automata," Supplem5!nt 1 to Technical Report RADC-TR-65-337, 1, Griffiss Air Force Base,~w York, 1966 [Translated from German-"Kommunikation mit Automaten;' Univ_ of Bonn, HbZJ
-
23-
R. E. Miller [7J
M. Yoeli, "Petri Nets and Asyrichronous Control Networks," Applied Analysis and Computer Science Research Report CS-73-07, University of Waterloo, Waterloo, Ontario, Canada, April, 1973.
-
24R. E. Miller
Parallel Program Schemata R. E.Miller
Introduction
The parallel program schema is an abstract formulation of parallel programs.
It is a complex model which incorporates some very general notions
of operation sequencing over a finite set of operations which fetch and store data in a random access memory.
It is an abstraction (thus called a
schema) in the sense that it leaves unspecified certain of the functional aspects of the operations and concentrates on the sequencing aspects of the computations and those properties that hold true no matter how the functional aspects are defined.
In this series of five lectures we intro-
duce the model, illustrate some of the properties that can be studied and prove some results indicating the techniques used to prove decidability and undecidability of the properties.
See [1-5] for more detail.
Definitions and Basic Properties A parallel program schema.J = (M, A,
Definition (i)
M, a set of memory locations
(ii)
A, a finite set of operations where for each
(iii)
aEA
A
'1)
consists of
{a, b, c,··· }
we have:
(a)
a positive integer
(b)
D(a)
~
M
a specified set of domain locations,
(c)
R(a)
~
M
a specified set of range locations.
'j
K(a)
called the number of outcomes,
(Q, qo' L, T) , the can trol, where:
Q is a set of states qoEQ L =
is the initial state L. u L t , the event ,alphabet l.
lI {a} ,
the initiation symbols
with
L.
aE A
and
Lt
a~{al,a2'···'~(a)}'
l.
the termination symbols.
-
25R. E. Miller
is the transition function which is a partial function from
t
Q
x
l:
into
Q and which is total on Q x l:t·
A schema is thought to operate in the following manner.
A computatiqn
is a sequence of events -- i.e., initiations and terminations of operations. Upon initiation of an operation a it reads its operand values from locations D(a).
Sometime after initiating the performance of the operation terminates.
This is indicated by some termination symbol results of operation a are stored into the symbol
aj
occurs.
indicates that outcome
Formally termination
aj
j
a.El: • J
R(a)
t
Upon termination the
locations.
Also the
(or conditional branch outcome
j)
creates a change of state in the control
~ and from this new state other initiations may be defined.
To define
these ideas more completely we give several further definitions. Definition: (i)
An interpretation
a function
~A::.'
of
.J
consists of:
C associating a set
C(i)
with each
iEM,
specifying the set of values allowed, (ii)
initial memory contexts
(iii) For each F
a
cOEXiEMC(i).
aEA, two functions: X C(i) iED(a)
-+
X
CO-)
iER(a)
(This is the function that given values from determine values for
R(a)
D(a)
will
locations.)
G
a
(This function determines the outcome of the operation a performance given values from
D(a).)
To define computations as sequences of event symbols from 'let
ex = (c, q, 11)
process.
Here
c
designate an
d -instantaneous
l:
we
description of the
is the current contents of memory; q
is the current
-
26R. E. Miller
state of the control; and the list
v(a)
formances of tion
V is a list for each
aEA where each item in
consists of a tuple
X C(i) of domain values of periED(a) currently in process. The initial instantaneous descrip-
a
a O = (cO' qo' VO)
initial state
consists of the initial memory contents, the
qo' and the
Vo
lists are empty for each operation.
The
basic sequencing of operations then is defined as transitions from one pi-instantaneous description to another as follows: Definition:
The partial function
description, and (1)
a'a, for
an
~-instantaneous
is defined by:
aEL
(Initiation symbol case) a = a, a = (c, q, V): a'a
is defined iff
T(q, il)
a-a = (c', qt, V')
If so:
c' for
where
c, q' = T(q, a)
bi-a
is defined.
v'(b)=v(b)
and and
flt(a)
is the list
fl(a)
with
(2)
X c (i) added to the end of the list. iED(a) (Termination symbol case) (J = a j , a = (c, q, V): a·a.
J
is defined iff
where
~
fl(a)
is nonempty and
denotes the first element in the
In this case
a-a. = (c', q', fl') J
for
iiR(a)
c'(i) = c(i).
(ii)
for
iER(a)
c'(i)
(iii)
q' = T(q, a. )
(iv)
fl' (b) = fl (b)
is the component of
. for
fl'(a)
b of a
*
a-y
and
fl(a) =
is the remainder of
the first element of YE~
Fa(~)
i.
,J
that is,
For
where:
(i)
• corresponding to
G (~) = a., a J v(a) list_
fl(a)
~fl'(a);
v(a)
is deleted.
is defined in the normal way by letting a-xa
This now lets us define schema -?t.computations.
after
-
27R. E. Miller
Definition:
A finite or infinite word
x
over
l:
is an d'::computation
iff:
(i)
for every prefix
(ii)
if
x
y
of
x
is finite, then for all
(iii) (Finite delay property) If
aE l: , aO·xa
is undefined;
is a prefix of
y
the property that for every aO·yza
is defined;
aO·y
z
such that
is defined, then for some
z'
x
yz
yz'a
and
aEl:
with
is a prefix of is a prefix of
Thus and-computation by (i) must be consistent with the
a·x
·x x.
defini-
tion (follow the transition rules), by (ii) can end only if no other event is possible, and by (iii) says that if after some point an event can "constantly" occur, then it eventually does occur after some finite delay. If
x
is an ~(x) =
d
-computation then
0. 0 '
aO·lx, a O• 2 x,···,a O•k x,·.·
is called the history of (~ere
iX
~quence
x.
is the prefix of length of
~(x)
containing
of
x.)
1.
For
a
=
and call this the contents sequence of cell i successive values that appear in location Definition: are
d
A schema
.J
i
(c, q,).I)
for
~
=
x.
~(x)
we let
Qi(x)
gives the
during computation
is called determinate if whenever
-computations for some interpretation \1iEM[Q.(x)
denotes the sub-
~i(x)
and the successive values of
0. 0
that "store values" in location
i
-el,
x
x. and
y
then:
Q.(y)] ~
This is one of the basic properties of schemata. that indicates a kind of "proper behavior."
It is a property
In references [1,2] this
property and others are studied in detail. For our lectures we mainly on the determinacy property.
concentrat~
-
28-
R. E. Miller Before we proceed to prove some results about determinacy, a simple example schema might be helpful.
This example shows the control aspects of a schema with three operations
{a, b, c}.
All initiation symbol transitions are shown, but only
those termination symbol transitions that can actually occur in computations are shown.
Here operation
a
has two outcomes
the other operations have only one outcome. situation. outcome Outcome
a2
Operation
a
a 2 , and
As long as it has
operations can be performed.
occurs we exit from the "a
loop" and allow
b
and
Essentially we FORK at this point, allowing
to be performed and then on termination of both
we JOIN (in state
and
This depicts the following
starts the computation.
only a sequence of
operate in parallel. c
a
al
qIO)'
b
and
Some possible computations are:
c
When to
c
b
and
performances
-
29R. E. Miller
(1)
a a z a al
(Z)
a a l be c l b l
(3)
a al
b bl c cl
C b cl b l
In computation (1) no parallelism occurs, that is only one operation is in execution at any moment. Dperations
band
c
In computations (2) and (3) however,
are, after the fourth symbol of the sequence,
concurrently being performed.
The particular computations which actually
would occur depend upon the domain and range location specifications for each operation, the
F
and
G specifications in the interpretation as
well as other aspects of the interpretation. Necessary and Sufficient Conditions for Determinacy We have just defined a property of schemata called determinacy. Essentially this means that no matter what the interpretation, any computations for that interpretation will provide the same sequence of values to occur in any memory cell.
This, of course, implies that for
terminatin~
computations the final results are independent of the sequencing, but is even stronger than that, insisting that the sequence of intermediate values is also. Theorem:
~
We now wish to prove the following theorem about determinacy. Let::J be a persistent, commutative, lossless schema.
is determinate if and only if for all interpretations
~
Then
condition
(A) holds: (A)
If
aO·uGTI ao·uGTI
where
and
aO·uTIG
are both defined, then
aO·uTIG,
TI, GEL, uEL *•
What does this theorem say intuitively? It says that for a certain type of schema
conditions we will
define -- if we reach a point at which two events can occur " s imultaneouslyl+
-
30-
R. E. Miller
winner of the race. The special conditions of the hypothesis are the following. Definition:
A schema is persistent if and only if whenever
are distinct elements of defined, then
and
and
T(q,cr)
and
T(q,TI)
and
T(q,TIcr) are also defined.
A schema ~ is commutative if and only if whenever
T(q,TIcr)
Definition:
TI
and both
!1
Definition: and
T(q,crTI)
~
cr
T(q,crTI)
are both defined then they are equal.
A schema
~
is lossless if for all
aEA, R(a) f
0.
The proof of the theorem requires a number of definitions and lemmas. The first step is to show that we only need to consider a subclass of all interpretations, called one-one interpretations, to prove the theorem. Essentially, a one-one interpretation is an interpretation that stores in each cell the complete history of events that effect the cell.
The needeq
lemma is then: Lemma 1:
Condition (A) of the theorem holds for every interpretation if
and only if it holds for everyone-one interpretation. A detailed definition and proof appear in [1]. Next, we have to prove some properties of how the
relation
behaves for termination and initiation symbols. Let
Lemma 2:
J
be a persistent commutative lossless schema,
interpretation, and
a = (c,q,]l)
t1 a
one-one
an .::k-instantaneous description. \
Then for each pair of operations and
a'b a
(a)
If
a'a b
(b)
If
a'a b" and a'b"a
a
and
b:
are defined then are defined then
a'a b a'a b"
only if (i) R(b) n D(a) = 0 or
(ii)
b"
is a repetition; i,e, IIR(b) (a)
a'b a a'b
a-
"
i f and
IIR(b) (a'b,,)'
-31R. E. Miller
(c)
Proof:
and
a.ajbR,
If
are defined then:
a'bR,a j
(i)
for
a
f b
a.ajbR,
(ii)
for
a
= b',
j
=
R,
=
if and only if R(a) n R(b)
a'bR,a j
and
a'aR,aR,
=
a'aR,aR,'
The proof is by cases:
(a)
If
a = b
then the result is obvious.
simply has two
~(a)
like objects added to it in each case. If
a f b
(c,q'~)'~b = (c,T(q,~b), ~')
then
and we wish to show that (c,T(q,b~), ~").
Since
c
is unchanged by initiations there is nothing to prove for
this component.
finally we compare For
d
and
~'
=
~'(d)
f a,b
=
~"(a)
T(q,~b) = T(q,b~)
Next
~".
=
~"(d)
~(a), liD (a) (c)
modify memory.
by commutativity.
so
~(d)
~'(a)
Similarly
~'(a)
=
~"(a)
~'(b) = ~"(b)
=
~(a), I1 D(a) (c)
and
since initiations do not so part (a) of the lemma is
proved.
Here
c' = c"
since the memory contents are not changed by
¢ach case termination contents.
bR,
is the only thing that changes the memory
the
d
f a,b
list of
~
b ~"(b)
=
Now
~"(b).
lists.
~
~'(d) = ~"(d)
lly definition u'(b)
T(q,~ bR,) = T(q,b£~).
Next, by commutativity
Again we must check the For
so
~'(b)
is also ~'(a)
a, so in
=
is ~(b)
By definition
~(d).
~(b)
a
does not change
with the first element deleted.
with the first element deleted.
~(a) I1 D (a)(c)
and
~"(a)
=
Thus
~(a) I1 D(a)(c').
-
32R. E. Miller
rhese are equal if and only i f
ITD(a)(c) = ITD(a) (c').
Since the interpretation is one-one c = c'
on
That is,
D(a).
ITD(a) (c) = ITD(a) (c')
D(a) n R(b) =
Next, we must consider case (b) for
c'
=
since
e"
or
~
a
For this case let
is the only termination and the
change the first element in the
is a repetition.
b,q,
= b.
i f and only i f
a
initiation does not
list.
~(a)
can finally be shown equal by an argument essentially identical to that for the case (c)
(ii)
,q,
=
(i)
and
a
If
b
=
then
a.' a j b R,
and
are both defined only i f
a.'bR,a j
j, since the first outcome is uniquely determined by
we have (c)
+ b.
a
If
and obviously this equals
a..aja j a
+b
.ajbR,
(C',T(q,ajbR,)'~')
(c,q,~)
'bR,aj
(c" ,T(q,bR,aj )
a.·a.a .• J J
,~")
Using the one-one interpretation we see that R(b) n R(a)
~,
~"since
and
~(b)
a
let
(c,q,~)
if
Thus
G •
=~.
T(q,qjbR,)
=
T(q,bR,a j )
c ,,= c"
if and only
by commutativity, and
in each case a single' item is removed only from the
~ (a)
lists.
This completes the proof of the lemma. Lemma 3:
Let
J
be a persistent commutative loss less schema,
interpretation, and VEE * , and (a)
a. O the initial
such that vcr~
instantan~ous
a.O·vcr~ = a.O·v~cr.
description.
Then for any
is an ...i-computation if and only if
~a
Vlrcrw
wEE* is.
one-qne
Let U
EW :
-
33-
R. E. Miller The proof of part (a) follows from the definition of the
relation,
and part (b) follows by checking all the cases from Lemma 2. Let ~ be a persistent schema, ~ an interpretation and
Lemma 4:
initial instantaneous description. (a)
If
w U,VEL *,WEL * uLand
Let
is defined, aiw
ClO·ua
Cl O the
and
ClO·UV
is defined, then
ClO·uva is defined. (b)
is defined and
ClO·ua
If
uw
is an J-computation then
Part (a) follows from persistence, and part (b) from persistence and the finite delay property. We now wish to prove the theorem. to prove that
J
is determinate.
there is an interpretation and for some
iEM Qi(x)
~
~
(1)
z (n)
~
Assume
such that
Qi(y).
there is an.d -computation
Suppose (A) holds, then we wish
x
(2)
n(z(n»
= n(Qi(y»
=
n
n can
a contradiction showing that
z(O)
=y
Inductive Assumption:
~o
= tV~+l
~
t(x)
y,
t(x)
~
Qi(x) Qi(x)
=
we obtain
n(Qi(z(n»)
and
differ.
so
is defined, where
(1) and (2)
t
u, ~+liv.
=
n(Qi(x»
This provides
Qi(y).
(1) and (2)
Assume
Q(y)
Then
z(n)
is done inductively on
hold for
n
hold for
= k(z(k»,
and n
n
=k
and
is defined
ClO·(kx)~+l =
n.
= O.
ClO·(kx)~+l
since by the inductive assumption (2) holds for 'z(k)
n
such that:
(otherwise we are done).
ClO·t ~+1
<J~computations
We shall prove that for any
The proof of these two properties of
k+1
are
nX.
so for no
Assume
y
That is,
Qiz(n)
Since this is true for all
~(x) ~
and
z(n) has the same cell sequences as Le.
Basis:
is not determinate.
k.
That is, ~+l appears somewhere
= ClO·t
By Lemma 4
~+l
-
34R. E. Miller
in the sequence since z(k+l) = z(k) v =
w~,
WEL *
and
(1) and (2) hold. Since
~EL.
aO·t~+l
is defined,
If
If
v
v
is null then let
is not null then
is defined it follows from (a) of are defined.
and
Lemma 4 that aO·tw~
is defined.
aO·t~+l
is defined.
aO·t~+l~
Also, since
But by the assumption that
(A) holds
= aO·tw~+l~
aO·tw~~+l
Thus, by Lemma 3,
t~+lrru
contents sequences as
is
an~-computation,
and it has the same cell
z(k), that is, for all i, Qi(z(k»
Now, by identical reasoning we Can continue to "slide" left until we have z(k)
t~+lvu
(A)
='>
determinacy.
~+l
to the
with the same cell contents sequences as
(and therefore the same as
it satisfies (1) and (2).
= Qi(twxk+l~u).
y).
We call this
z(k+l)
and note that
This accomplishes the goal of proving that
(The "sliding argument" is a technique which is
central to a number of proofs about schema properties.) The remainder of the proof of the theorem is to show that determinacy implies condition (A). aO·u0~ Qi(U0~)
+ aO·u~0. and
Assume determinacy but (A) does not hold, that is,
Then by cases of Lemma 2 one can show a difference in
Qi(U~0)
for some
i, but this contradicts determinacy,
completing the proof of the theorem. This theorem shows how determinacy, which is a property on sequences of values in memory cells, is equivalent to
~
type of "commutativity" of
events in forming instantaneous descriptions.
That is, when a "racing"
of several operation performances does not cause a change in behavior. Decidability of Determinacy We now wish to show that, for a special class of schemata, determinacy is decidable.
To do this, we must first digress to a combinatorial
-
35-
R. E. Miller
structure called vector addition systems. show how a certain property of these systems is decidable. and then show the connection of this to schemata. befinition: An r-dimensional vector addition system is a pair where
d
W=
(d.W)
is an r-dimensional vector of nonnegative integers, and
W is a finite set of r-dimensional integer vectors.
RCZ/)) is the set of points in the first orthant
The reachability ~ that can be reached from
d
by successively adding vectors in
W such
that the path of points so formed always remains in the first orthant. As
a
simple example considers r = ~. d = (1,1) and W = {(-2.l), (0.1). (3,-l)t.
Note. for example that (4,2)ER(
).
(4.2) = (1.1) + (3.-1) + (0.1) + (0,1) and the successive points
(4.0), (4.1). (4.2)
Some elementary terminology is needed. x
~
Y if and only if
xi
vector of all zeros when appropriate. and any integer
n, n + w = w.
For
= 1.2.···.r.
Yi' i
~
are all in the first
w
r-vectors We use
0
x
and
ortha~t.
y
to denote the
is a symbol for which. for
A rooted tree is a tree with some designated
vertex called the root.
All vertices but the root have one edge directed
into the vertex.
and
If
directed path from
~
~
to
is a directed edge from tf
n
n
are vertices of a rooted tree with a
n • then we designate this by ~
to
n
n
where
9,(~)
"'j (U1
(1)
'j(W)
root
If there
n
~.
is called an end.
with labelled vertices
9,(0 for
is an r-dimensional vector label having components from
N u {w}.
Definition: ,
-< n.
is called a successor of
is a node with no edge directed out then ForU) we construct a tree
t,
then
~
0
consists of: with
9,(0) = d.
-
36R. E. Miller
(2)
be a vertex of
n
(a)
if, for some vertex
(b)
otherwise successors of for which
Let
'J (1tr)
Let
nw
~(n)
+ VI
1;-:
denote the successor of
associated with
n
WEW, then
~(nw)
is determined as follows: (i)
if there is a
(ii) if no such
We demonstrate the example "7-t) •
1;
1;
-< nw
exists, then
;j ell..)
J - N
(initial value)
1f:
S->-
(II of P's)
"Ii:
S ->- N
(II of V's)
This definition can be viewed as the following connection between any two processes Pi and Pj' semaphore s process Pi v(s) # of V(s)'s
,..
\.1(s) initial value
process Pj 1f(s) # of P(s)'s
Here process Pi (the producer) is assumed to end with v(s) V(s) operations, which increases the value of
s
by
v(s).
Process Pj
(the consumer) is assumed to start with 1f(s) pes) operations.
Thus a
loose sequencing is maintained between p. and p.; i.e .• Pj cannot start ~
without
s
J
being sufficiently large (1f(s) or more) and this may depend
upon process Pi having completed some previous executions. A more general producer-consumer system can be defined consisting of: (i)
a finite set B
{Pl' •••• P~} of processes.
(ii)
a finite set S
{sl ••.• 'St} of semaphores;
(iii)
Three functions: \.1: S ->- N initial value 1f': SxB ->- N
(# of pes) at process)
v': SxB ->- N
(# of V(s) a~ process)
This type of producer-consumer system does not restrict a semaphore to being associated with only one pair of processes.
-
59R. E. Miller
Computation Graphs and Unshared Producer-Consumer Systems By a straightforward construction we can show: groposition:
Any unshared producer-consumer system can be represented
by a computation graph. The computations performed by the computation graph can be seen to simulate those of the unshared producer-consumer system. important that the producer-consumer system is "unshared."
Here it is That is,
that each semaphore is associated only with a single pair of processes. This can be seen by noting that a starting of an operation in the computation graph removes a certain fixed number of operands, in one chunk, from input queues, and on completion places a number of results ~n
a chunk in output queues.
This differs from the producer-consumer
operation in that p's and V's successively decrease or increase the values only by one at a time.
Thus, iteration by P's or
V's from several
processes on a single semaphore could cause some different or unexpected bperation.
For the unshared situation this cannot occur. Producer-Consumer Systems and Generalized Petri Nets
A generalized Petri net is similar to a Petri net except that firing pf transitions may remove or add more than one token at a time to a place. Formally a generalized Petri net
P = (IT,l:,R,MO,Ll1,Ll O)
where IT is a finite set of places, l: is a finite set of transitions
R
~
(ITxl:) (l:xIT) describing the edges, MO:IT
the two functions LlI:ITxl:~N
and
~
: l: x IT
~
N
~
N is the initial marking and
-
60-
R. E. Miller ",ith value 0 i f IT X E or E x IT is not in
R.
It is the 81 and 8 0 functions that determine for any place and transition pair, how many tokens are removed or added to the place due to the transition firing. Now we can obtain the following relationship: Proposition:
Any producer-consumer system can be represented by a
generalized Petri net, and conversely. The representation associates items as follows: P
S processes B
"'-./
transitions E
semaphores S
.~
places IT
j..I
(Sj)
,/'"'oJ
7[' (si,Pj)
,--./
V'(si,Pj)
,~
MO 0 and 6 0 (cr,rr) > O. That is there are no self-loops. Proposition:
There is an isomorphism between vector replacement systems
and generalized Petri nets without equivalent transitions. ~roposition:
There is an isomorphism between vector addition systems
and irref1exive generalized Petri nets without equivalent transitions. In both of these isomorphisms there is a direct correspondence between the firing sequences in the Petri nets and the reachable paths in the vector systems.
These lead to interesting connections of the
model properties. References [ 1]
J. L. Baer, "A Survey of Some Theoretical Aspects of Multiprocessing," ACM Computing Surveys, ~, 1, 31-80, March, 1973.
[ 2]
F. COIllIIloner, A. W. Holt, S. Even, A. Pneuli, ''Marked Directed Graphs," Journal of Computer and Systems Science, Vol. 5, October 1971, pp. 511-23.
[ 3]
E. W. Dijkstra, "Co-operating Sequential Processes," in PrograIllIIling Lan~ages, F. Genuys, Editor, New York, Academic Press, 1968, pp. 43-112.
[ 4]
M. Hack, "Decision Problems for Petri Nets and Vector Addition Systems," MIT Project MAC, Computation Structures Group Memo 95, March 1974.
-
63R. E. Miller
[ 5]
R. M. Keller, "Vector Replacement Systems: A Formalism for Modeling Asynchronous Systems," Princeton University, E. E. Technical Report No. 117, December 1972.
[ 6]
R. M. Karp and R. E. Miller, "Properties of a Model for Parallel Computation: Determinacy, Termination, Queueing," SIAM Journal of Applied Math., 1390-1411, November 1966.
[ 7]
R. M. Karp. and R. E. Miller, "Parallel Program Schemata," Journal of Computer and System Sciences, 1, 2, 147-195, May 1969.
[ 8]
R. E. Miller, "A Comparison of Some Theoretical Models of Parallel Computation," IEEE Transactions on Computer, C-22, 8, 710-717, August 1973.
[ 9]
R. E. Miller, "Some Relationships Between Various Models of Parallelism and Synchronization." IBM Research Report RC-5074 , October 1974.
[10]
w.
[11]
c. A. Petri, "Communication with Automata," Supplement 1 to Technical Report RADC-TR-65-337, 1, Griffiss Air Force Base, New York. 1966 [ Translated from German "Kommunikation mit Automate~, If University of Bonn. 1962.1
[12]
M. Yoe1i, "Petri Nets and Asynchronous Control Networks," Applied Analysis and Computer Science Research Report CS-73-07, University of Waterloo, Waterloo, Ontario, Canada, April 1973.
L. Miranker, "A Survey of Parallelism in Numerical Analysis ,I' SIAM Review, 13, 524-547, October 1971.
CENTRO INTERNAZIONALE MATEMATICO ESTIVO (C.I.M.E.)
THEORY OF AUTOMATA
D.E. MULLER
Corso tenuto a Bressanone dal 9 al 17 giugno 1975
-
66D. E. Muller
THEORY OF AUTOMATA David E. Mu11er*
1.
The title "theory of automata" has become somewhat passe because
the problems of interest to people working in this area have changed, and therefore the subject has split into a number of different disciplines.
Each of these disciplines is called by its own name, but
many of the results and techniques developed in the theory of automata have turned out to be useful.
Our object is to look at a few of these
results and techniques and see what sorts of approaches have been applied to problems involving automata. Historically, when people first began to design computing machines, it became clear that the interaction of the various units was sufficiently complex so that systematic were necessary.
~thods
fot organizing the design
The different parts of the machine were thought of as
units and ways were worked out for specifying the design of a unit in terms of the organization of the machine as a whole.
Also, the design
of a unit had to be systematized. The methods employed were algebraic, and consequently much of the theory of automata has retained an algebraic flavor.
Thus, we design
circuits using switching algebra, we discuss homomorphisms among machines, and we use semigroups in the study of cascade decomposition.
'~Coordinated Science Laboratory and Department of Mathematics, University of Illinois at Urbana-Champaign. This work was supported in part by the Joint Services Electronics Program (U. S. Army, U. S. Navy, and U. S. Air Force) under Contract DAAB-07-72-C-02S9.
-
67D. E. Muller
Also, in the early days many logicians became interested in problems concerning automata.
A possible reason for this interest was
the success of A. M. Turing in 1936 in applying the notion of a machine to characterize an effective computation and thus to obtain some of the original results concerning unsolvability.
In any case, many logicians
turned their attention to the study of automata, and as a consequence the nature of the subject and types of questions which have been asked and answered were profoundly influenced the bent of these early workers. One person who stimulated many others to study automata was John Von Neumann.
He was interested both in designing computers more
intelligently and in designing more "intelligent" computers.
The reason
for applying the word "intelligent" to computers here is that Von Neumann thought of the computer as a model for the brain.
While the intervening
25 years since Von Neumann's work have not seen great advances in forming a computer model of the action of the brain, nevertheless, this idea spurred much of the original investigation of machine theory and is still being actively pursued today. In the model used by Von Neumann the basic organ was a device with several input lines and one output line. carrying signals which were either 0 or 1.
The lines were capable of The signal carried by the
output line was determined by the combination of signals on the input lines after a fixed time delay T.
It was thought necessary to use
synchronizing signals to ensure proper timing in an actual machine, and in the mathematical model such synchronization was achieved by allowing a fixed and constant time delay between the occurrences of the combination of input signals and the resulting output signal. A network composed of such organs in suitable variety and suitably
-
68-
D. E. Muller connected could do "anything."
That is, if the network was able to
control devices capable of writing on and reading from a potentially infinite roll of paper tape then it could be made to carry out any computation which could be effectively described.
The machine of Turing,
or for that matter any other computing device could be designed using the elements of Von Neumann. The elements in the brain were thought to be (1) slow compared with electronic speeds and (2) unreliable compared to manufactured components, but they were much smaller than the devices available in those early years and their organization was much more complicated.
Regular signals
travelling through the nervous system were thought by Von Neumann and others to be synchronizing signals. reliability.
But, there remained the problem of
Was it possible for even the rather unreliable elements
found in the nervous system to produce reliable results? proved that it was.
Von Neumann
As long as the elements were not totally unreliable,
he showed that the network as a whole could be made to function with arbitrarily good, although not perfect, reliability. To produce a reliable automaton using unreliable elements, Von Neumann used an approach which is extravagant in its use of equipment although versions of it still find favor today.
First, a
design is worked out on the assumption that each organ works perfectly, never failing to produce the correct output signal.
From this original
design one obtains the final one by a substitution process.
Each line
of the original design is replaced by a bundle containing some large number n of actual lines in the final one. mUltiple of 3.
Let us assume that n is a
Each organ of the original design is correspondingly
replaced by n similar organs which are connected in like manner to the
-
08-
D. E. Muller
~
lines of the bundles.
Finally, errors occurring in a few of the n
organs are partially corrected in the bundles by placing within each of them n/3 "voting" organs which, when functioning properly, produce an output which agrees with the majority of the three inputs.
The n/3
outputs of the "voting" organs are each split into three lines and the resulting n lines are randomly scrambled.
Von Neumann proved that by
choosing sufficiently large n, the probability of failure of the entire network could be made arbitrarily small, assuming the component organs were not totally unreliable.
In this case, an organ with a reliability
of 50% must be considered totally unreliable, since this means that the output signal is totally uncorrelated with the input combination; the output is equally likely to be right or wrong. Since in the final design single signals were replaced by sets of n signals it was necessary to reinterpret the action of the final network made reliable by replication.
Problems of its communication
with the outside world remained as sources of potential unreliability which could not be solved by Von Neumann's technique.
2.
There is a close connection between the theory of automata and
certain basic concepts of algebra, and by regarding automata from an algebraic point of view it has been possible to apply the theory to many things.
It has been applied to the study of formal languages of the type
used in computer programming, and to the mathematically exact description of algorithms, processes, and other formal systems.
In particular, it
has been used to obtain decidability results about statements in formal systems.
Another recent example of such an application of the theory of
automata is to the simpler Burnside's problem--"Is there an infinite.
-
70D. E. Muller
finitely generated. periodic group?" A short construction using automata obtains the result which had previously been discovered by other methods.
The answer is that for each prime P. there is an
infinite p-group with two generators. Thus. we see that the subject we are investigating has found widespread application outside the realm of computer design. Let us now get down to specifics •. An automaton is an idealized machine. and therefore it will be thought of as a device which receives signals from the outside world through a channel called its input;
It
carries out a transformation of what it has received and- presents the results back to the world through a channel called its output. Generally, the input and output channels are constructed by using several lines yith binary (or two valued) signals.
For example. if
three binary lines are used in a channel one can obtain eight combinations of signals.
The set of combinations actually used with a channel
is called the alphabet associated with that channel.
In describing an
automaton. let us represent the input and output alphabets by the letters l:i! and
r
respectively.
To complete the description of the automaton. let S represent the set of possible internal conditions
or~.
The state of the
automaton is characterized by the combination of signals appearing on the lines internal to the automaton. an alphabet.
Thus. S may also be regarded as
Now, according to the assumption that signals are
synchronized. we may treat time as a discrete variable and assume that all changes must occur at integral times: of the automaton at time 0 is gi,ven.
0.1.2 ••••• and that the state
Then. the state at each time t + 1
and the output at time t are uniquely determined by the input and state
-71D. E. ~1f1uller
that exist at time t.
The two functions which describe the next state
and output are called the transition and output functions and will be written M and N respectively. To summarize, we see that an automaton can be described by
r,
specifying the quantities A,
S. M, and N, where the first three
represent sets, called alphabets and the last two are functions: M: SXA ..... S, and N: SXA .....
r.
In other words, M maps each pair (s,a),
where sES and aEA, to a "next" state M(s,a), abbreviated sa and also in S.
Similarly, N maps each such pair (s,a) to an output N(s,a) in
r.
If we specify the state s(O) in which the automaton is placed at time 0 and also specify the inputs a(O),a(l) •••. ,a(r) which occur at times O,l, ..• ,r, then we can uniquely determine the states s(1),s(2), •.• , s(r+l) which occur at times 1,2, ••• ,r+1 respectively. taking s(t+l) = M(s(t),a(t»
=
This is done by
s(t)a(t) for t = O,l, .•. ,r, recursively.
Similarly, we can determine the sequence b(O),b(l), .•. ,b(r) of outputs at times O,l, .•• ,r respectively by taking b(t) t
= N(s(t),a(t»
for
= 0,1, ...• r.
In actual automata the alphabets must be finite, and hence the functions M and N can always be described.
If no simpler rule is
available, they may be tabulated and their calculation carried out by table look-up.
3.
r,
While an automaton is defined by specifying the five quantities A, S. M, and N, we may choose to ignore the output function N and its
associated alphabet
r,
concentrating just on the alphabets A and Sand
the transition function M.
Such a triple A, S, M is called a monadic
algebra or partial automaton.
To understand the reason that the triple
-
72D. E. Muller
~,
S, M can be regarded as an algebra, we note that the mapping
M : S x:a: ... S may be restricted by fixing an element a E:a: and writing Ma : S ... S.
For each s E S, we define Ma (s) as M(s.a).
mapping Ma for each a E:a: •
There is thus a
Here. as before, we may abbreviate M (s) as a
sa. and the function Ma may be thought of as operating upon the state s to produce the new state sa.
The elements of:a: designate operations of
the algebra which are all unary in the sense that a single state s is needed to produce the result sa. A monadic algebra can be represented as a directed graph with labels on the edges corresponding to the operation they represent. let:a:
= {O,l},
Figure lao
= {sO,sl's2}
S
For example,
and let M be given by the table of
Then, the directed graph of the monadic algebra is shown in
Figure lb.
1
sl
s2
s2
So
sl
s2 M
a
b
Figure 1
If So is a state in S, then we define the subset M(sO) generated from So as the intersection of all subsets S' of S such that (i)
(ii)
So E S
I ,
whenever s E S I and a E:a:, then sa E S I .
Intuitively M(SO) consists of So and all states "reachable" from
-
73-
D. E. Muller So by the application of a finite sequence of operations o,f the algebra to sO'
In the language of automata, M(SO) is the set consisting of all
states to which the automaton may be brought by some sequence of inputs if it is started in state sO' Exercise 1:
Prove that the formal definition of M(sO) is valid by
showing that there exists at least one set satisfying the conditions required of S'. Exercise 2:
Hint:
Show that S itself is such a set.
Prove that the set M(sO) satisfies both conditions (i)
and (ii) for S'. Exercise 3:
Prove that M(sO) has no proper subset satisfying these
two conditions. Exercise 4:
Let E: '" [O,l}, S '" [sO,sl,s2}' and let M be defined by
the following table.
Show that M(SO) '" [sO,s2}' M(sl) '" [sO,sl,s21, and
M(s2) '" [sO,s2}'
o
1
s2
s2
s2
So
So
s2
A monadic algebra with a distinguished initial state or generator is one in which a certain state So is specially designated and is such that the entire set S is generated from it. such that M(sO) '" S. and sO'
In other words, it must be
We describe such an algebra by listing 3, S, M,
Note that in Figure 1, if So is chosen as the distinguished
initial state then the relation M(sO) '" S is satisfied. Although the notion of a monadic algebra was introduced by first describing an associated automaton and hence it is natural to think of
-
74-
D. E. Muller
the set of states as finite, the finiteness restriction may be relaxed as long as we are merely discussing abstract algebras.
As an example
of an infinite monadic algebra, let 3 consist of a single letter, say 3
= ['},
and let S consist of the natural numbers 0,1,2, ••..
Then
define the transition function by the rule that for each state s E S, let s'
=
s+l.
In this example it is possible to choose only 0 as the
distinguished initial state.
Naturally, since S is infinite it is
fmpossible to give a tabulation of the transition function or a directed graph corresponding to the algebra. The following theorems are very useful if we wish to carry out formal proofs of properties of monadic algebras having distinguished initial states.
One can construct parallel theorems about any algebra
with a distinguished set of generators. Theorem 1:
(Induction)
For any monadic algebra (E,S,M) with a
distinguished initial state sO' let P(s) be a proposition when s E S. Suppose that (i) (ii)
P (sO) holds, and whenever P (s) holds and a EE, P (sa) holds.
Then P(s) must hold for all sES. Proof: P(s) holds.
Let S' be the subset of S consisting of all s such that Then
the hypothesis.
So E S
I,
and i f s E S I then sa E S I by (i) and (ii) of
In exercise 3 it was shown that M(SO)
proper subsets satisfying these two conditions. so S I Theorem 2:
=
= S.
S includes no 0
For any monadic algebra (3 ,S,M) with a distinguished
ini.tial state sO' if s E S but s is not sO' then s can be expressed in the form s I a for some s I E S and a E:s • Proof:
Let P(s) be the proposition:
either s
-
75D. E. Muller
expressed in the form s'a.
Then, referring to theorem 1, we see that
(i) P (sO), and (ii) i f P (s) and a E:e:, then P (sa).
We note that in (ii)
the conclusion P(sa) always holds even if it could happen that P(s) did not.
Naturally, this fact does not invalidate (ii).
for all s E S, by theorem 1.
This proves theorem 2.
Thus P(s) must hold 0
The two theorems just obtained are not usually presented in a formal way when discussing automata.
The reason we have done so is that in
order to see the parallel between conventional automata and various generalizations such as tree automata it is convenient to write everything down so that the differences and similarities between the theories are clear. To illustrate this technique of generalization we shall briefly introduce the theory of tree automata.
In our treatment of this theory,
we shall use the notion of a groupoid to replace that of a monadic algebra while keeping the theory as similar as possible in all other respects. A groupoid is an algebra with a single binary operation.
There are
no specialized properties such as associativity orcommutativity required of all groupoids and there is no alphabet such as S associated with a groupoid.
Briefly therefore, a groupoid consists of a set S and a
mapping M : 8 X 8 ..... 8.
To
emphasi~e
the parallel between the two systems,
we shall call 8 the set of states and M the transition function.
For
convenience and to avoid confusion, we shall abbreviate the binary operation M(s,s') as sys'. As in the case of monadic algebras, define the subset M(sO) of states generated from that
So
as the intersection of all subsets 8' of 8 such
-
76D. E. Muller
(i) (ii)
So E S', and whenever s, s' ES', then sys' ES'.
Results similar to those of exercises 1, 2, and 3 may be proved in exactly the same way. A groupoid (S,M) with a distinguished generator So must satisfy the relation M(SO)
= S,
Exercise 5:
just as in the case of a monadic algebra.
State and prove the theorems analogous to theorems 1
and 2 for the case of a groupoid.
We shall call these theorems l' and
2' •
We now return to our original treatment of monadic algebras and define a free monadic algebra on an alphabet input alphabet
~
~
as a monadic algebra with
and a distinguished initial state So which has the
following two properties. (1)
For all s E S and a E~, sa # sO' and
(2)
for all s, s'ES and a, a'ES, if sa
s'a', then s
s' and
a = a'.
Although it is customary to define free algebras in a different way and involving the notion of a homomorphism, we shall see that this definition turns out to be equivalent to the usual one. Lemma 1: Proof:
There exists a free monadic algebra on any alphabet S. Let us assume that we are given the alphabet S, then we
proceed to construct a monadic algebra we shall call 3' GS) using its alphabet by the following process.
~
as
Let A be a symbol which is not
a member of S and take A as the initial state of 3'(S).
Define the
transition function M by the general rule that if t is any quantity, then for a ES, take M(t,a) to be the ordered pair (t,a).
Finally, let
the state set T of 3'(S) consist of all elements generated by M from A.
-77-
D. E. Muller It is clear from the definition that
J~)
as defined here is a
monadic algebra with the distinguished initial state A.
To see that it
is free in the sense we defined previously, we note first that A is -=merely a symbol and hence cannot be an ordered pair of the form tao Hence (1) is satisfied.
=
(t,a)
Second, let us assume that ta
= t'a'.
Thus,
(t',a'). and since two ordered pairs are equal iff their first
and second members are respectively equal, we have t Hence, (2) is satisfied and
J~)
is free.
= t'
and a
0
Although we have used a particular method for constructing shall now show that isomorphism.
J(~)
= a'.
is the only free monadic algebra on
~
J~)
we
up to
We also will see that the elements of T may be regarded as
finite sequences of letters
This result is important because we
from~.
recall that such finite sequences were applied at the rnput channel of an automaton. We begin by defining a mapping which will be important in our proof.
Let a be a monadic algebra
the free monadic algebra
MU : SX T
~
(ii)
of lemma 1.
and, as before, let
J~)
be
Then, define the mapping
S by the following two rules, where st is written as an
abbreviation for (i)
~,T,M)
(2:,S,~)
~(s,t).
For all sES, sA = s. For all sES, tET,
aE~,
s(ta) = (st)a.
If rule (ii) were written without using our abbreviations it would appear as
"tt (s ,M(t ,a»
=
Ml ~ (s, t) ,a).
It is necessary to prove thatma is well defined by the above definition which is of an inductive nature. t E T there is exactly one product st. imagine that s is fixed.
That is, for each s E Sand
To carry out this proof we
-
78-
D. E. Muller
First, let t be A. the product st.
Then, rule (i) above may be used for defining
By rule (1) in the definition of a free monadic algebra,
we see that t cannot be of the form t'a.
Thus, rule (ii) above cannot
be applied, so the product st is uniquely given by (i). Second, we use induction (theorem 1) as applied to ;1 0;:).
We make
the inductive assumption that for given t E T the product st is unambiguously defined, i.e. is unique. is true when t is A.
We just saw that this assumption
Now consider the product s(ta).
we see that one value for this product is (st)a.
By (ii) above,
Since rule (1) tells
us that ta cannot be A, we see that (i) is not applicable in this· case. Thus, for s(ta) to be more than single valued it would be necessary to apply rule (ii) a second time. that s(ta)
This would require that ta
s(t'a') = (st')a'.
= a',
so (st')a'
so
But rule (2) in the definition of a
free monadic algebra tells us that ta a
= t'a'
= t'a'
implies that t
= t'
and
(st)a and the two values we assumed for s(ta) must
be the same and it is unambiguously defined. inductive step and the function
ma is
This completes the
well defined.
The functionm a may be regarded as an extension of the transition function Ml of the monadic algebra time t, where
a
a.
Let s be the state of
is now regarded as a partial automaton.
a
at some
Then, we have
seen that sa is the state at time t+l if the input a is applied at time t.
Now, we shall see that each element t of T corresponds to a sequence
a l ,a 2 ... a n of letters from 1;;.
If these letters are applied to the input
successively during the present and next n-l time units, then the state at time t+n will be just st.
These facts will be stated more formally
in a later theorem. In the definition ofma the two monadic algebras ;1(1;;) and
a
were
-
79D. E. Muller
both involved.
Taking
a
to also be 3' (:S), we define \
(:S) : TXT - T.
For any two elements t, t' E T, we abbreviate ~(:s)(t,t') as tt'.
Then,
we can prove the following. Theorem 3:
Let
a
be any monadic algebra (E,S,Ml ) and let t, t' be
any two elements of T, the state set of 3'(:S) , then (st)t' = s(tt'). Proof:
It is clearly true i f t' = A by property (i) in the
definitions of ~ and m3'(:S)' Using induction, assume it holds for given t' ET.
Then, for aEE, we have (st)(t'a) = «st)t')a = (s(tt'»a
s«tt')a) = s(t(t'a», using property (ii) in the definitions of ~b and m3'(:S)'
Hence, it holds for all t' E T.
Theorem 4: Proof: m3'(:S) ,
o
In 3' (:S), for all t E T, At = t.
It is true when we take t = A by (i) in the definition of
Assuming inductively that i t holds for given t and taking aE:S,
we have A(ta)
=
(At)a
= tao
0
This last theorem makes it reasonable to introduce of Aa as a.
~he
abbreviation
Although Aa is a member of T and a is in:S, no confusion
will result from this abbreviation because we have adopted the same notation for Ml and for may always be used.
mao
Since s(Aa)
=
(sA)a
= sa,
the new notation
Algebraically we have identified the set E with
the subset {Aal a E:S} of T, and we shall henceforth regard:S as a subset of T. Exercise 6:
Using the monadic algebra of figure 1, construct the
product s (10) for each state s E S. the case (sOO)(lO)
Hence, show t.h'at theorem 3 holds for
= sO(O(lO».
Results analogous to theorems 3 and 4 are not convenient to state in the groupoid case.
It is possible, however, to define a free
groupoid with a distinguished generator
So
as a groupoid having the
-
80D. E. Muller
following two properties. (1)
For all s,s' E 8, sys' ". sO' and
(2) 'for all s,s' ,s",s'" E 8, if sYs'
s ''Vs '", then s
s" and
s' = s"'.
The analog of lemma 1 may then be obtained in which it is shown that a free groupoid J l exists. Exercise 7: groupoids.
8tate and prove lemma 1', the analog of lemma 1 for
Define the transition function of J l by the rule that sYs'
is the ordered pair (s,s').
4.
Returning to the case of monadic algebras, define a homomorphism
from a monadic algebra (E,8,M) to another monadic algebra (E,8',M') on the same input alphabet E as a mapping h : 8 .... 8' s~ch that for all s E 8 and a EE, h(sa) = h(s)a.
In this definition we have used our standard
abbreviation in an unambiguous way. equation h(sa)
= h(s)a
Theorem 5:
Without the abbreviation the
beco,mes h(M(s,a»
= M'(h(s),a).
If h is a homomorphism, then for all s E 8 and t E T,
h(st) = h(s)t. Proof:
The theorem is clearly true when t
= A.
on J(E), assume inductively that it holds for given t. h«st)a)
= h(st)a =
(h(s)t)a
= h(s)(ta).
step and the result is proved. Theorem 6:
Then h(s(ta»
This completes the inductive
0
Let (E,8,M) be a monadic algebra with a distinguished
initial s,tate sO' such that h(A)
Using induction
Then there is a homomorphism h from J(E) to (E,8,M)
= sO'
This homomorphism is unique, onto, and given by
the rule h(t) = sot for all t E T. Proof:
Let us define the mapping h : T .... S by the rule that
-81D. E. Muller h (t)
=
sot. '·To prove that this mapping is a homomorphism we observe
that for any tET and aES, we have h(ta) Clearly, h(A)
(sot)a = h(t)a.
= sO.
To proVe uniqueness, let h' : T that h'(A)
= sO(ta) =
= sO.
~
S be any other homomorphism such
By theorems 4 and 5, h'(t)
= h'(At) = h'(A)t = sot.
Since this holds for arbitrary t, we see that h' = h. To prove that h as given above is onto, we use induction on the algebra (S,S,M). namely A.
Note that So is the image under h of some member of T,
Assume inductively that s E S is the image of t E T.
is the image of ta since h(ta)
= h(t)a = sa.
induction step so therefore each Corollary:
5
Then sa
This completes the
E S must be the image of some t E T.
a
Each state s E S can be expressed in the form sot for
some t E T. Exercise 8:
If (S .M) and (S ',M') are two groupoids, then a
homomorphism from (S.M) to (S',M') is defined as a mapping h : S such that for all
51'5 2
E S, the relation h(sl'Ys2)
= h(sl)'Yh(s2)
~
S'
holds.
Now, let (S,M) be any groupoid with a distinguished generator So and let ¢ be the generator of the free groupoid there is a homomorphism
from~l
Theorem 7:
Prove that
= So
and that
Hint, define h(t) by the rules:
= h(t l )'Yh(t2 ).
Let (S,S,M) be any free monadic algebra with the
distinguished initial state sO. ~(S)
of exercise 7.
to (S.M) such that h(¢)
this homomorphism is unique and onto. (1) h(¢) = sO' and (2) h(t l 'Yt2 )
~l
Then there is an isomorphism from
onto (S,S,M) which maps A to sO' Proof:
By theorem 6, we know there is a homomorphism h from
onto (S,S,M) such that h(A)
= So
~(S)
and that this homomorphism is unique.
It merely remains, therefore, to show that h is one-to-one.
-
82D. E. Muller
We begin by showing that if h(t) contrary.
= tl a
By theorem Z, t
So = h(t) = h(tla) = h(tl)a, monadic algebra.
Thus
So
= sO'
then t
= A.
Assume the
for some tl E T and a E E.
Then,
which contradicts property (1) of a free
has just one pre image under h, namely"A.
Now,
we assume inductively that a given s E S has just one preimage, say t E T. Let t z be a second pre-
For any aEE, clearly ta is a preimage of sa. image of sa. a' EE.
= t' a'
h(ta), and since h(t'a')
Hence, h(t 'a')
= sa,
h(t)a
Since t z #- A, we may write t z
we have h(t')a'
= sa. =t
= h(t')a'
and h(ta)
Since (E,S,M) is a free monadic
algebra, it satisfies property (Z) and h(t') by our inductive hypothesis t'
for some t' E T and
=
s, while a'
and we obtain t'a'
that sa has just one preimage, namely tao
a.
= tao
Thus,
This shows
This completes the inductive
step and by theorem 1, each state in S has just one pre image under h. Hence h is an isomorphism.
0
This result shows that all free monadic algebras on E are isomorphic.
We may therefore speak of the free monadic algebra on E,
and we shall continue to denote it by
~(E).
A similar result may be
proved analogously for groupoids. We have seen that by letting
a
be ~(A) in the definition of ~,
the mapPing~(E) defines a binary operation on T.
By theorem 3, applied
to this case, we see that the binary operation is associative.
Hence,
we say that~(E) defines a semigroup, and we shall denote it by~~(A)). From the definition of ~(A)' part (i), we see t~at in this semigroup tA
=t
for all tET.
identity element in ~~
Also, by theorem 4, we have At ~~
(E») are identical.
(E)).
=
t.
Hence A is an
We notice that the elements of
~
(E) and
The difference lies in the fact that ~ (E) is an
algebra with a number of unary operations while
~~
CE)) has a single
-
83D. E. Muller
binary operation. Although we shall not continue the detailed development, it can be shown by techniques similar to those we have already used that the free monoid on
~
.
~(J(~))
is
This is the same as the free semigroup to which an
identity element A has been added.
As mentioned earlier, the elements
of T may be regarded as the finite sequences or strings of letters from the alphabet
~
with A representing the sequence of length zero.
binary operation of and t '
= ai ••. a~,
operations of
then tt'
~(~)
is thus concatenation, so that i f t
~(J (~))
= al ••• anai •.. a~.
then ta
= al ••• ana.
a=
(~,S,M)
be a monadic algebra, then in
Let
a l •• .a n
may be shown to correspond to the operation of
= al .•. a n ,
5.
=
Furthermore, the unary
attaching a letter to the right hand end 6f a string. t
The
Thus, if
~(~)
each element
t E T yields a mapping 77l t : S .... S by the rule that the image of each s E S under 77lt is st.
The set of all such mappings is closed under functional
composition since for t, t'E T and s E S we have 77l?it ' (s) st't
= 77lt
, t (s).
= 77l t (st ') =
Hence the set of all such mappings forms a semigroup
called the semigroup of a which we shall denote by
~
en).
We note that
this semigroup is a semigroup with an identity, called a monoid, since 77lA is the identity map on S.
We note that if S is finite
then~(a)
be finite also because there are only finitely many mappings from a finite set to itself. Exercise 9: Exercise 10:
If ~~) is finite must S be finite also?
GiVe an example in which
~(alVand ~(aZ)
isomorphic yet a l has two states whilea Z has three states.
are
must
-
84D. E. Muller
Exercise 11:
Explain
whyJ~(~»
with our original definition of
as defined here is consistent
J(J(~».
In the following example, let
~
= [O,l}
and S
defined by the table
o
1
51
52
52
82
51
52
We construct the semigroup of this monadic algebra by calculating each mapping 7Jl t for strings t E T until no new mappings can be constructed. This is done in the following table in which each mapping7Jl t is labeled with its string t. t 5
o
1
00
51
52
52
8
52
s2
51
52
000
01
10
11
1
52
8
1
s2
51
82
52
51
52
52
51
52
51
52
The subscripts giving distinct members of the semigroup are 00, 10}.
It has the following table.
o
1
00
10
A
0
1
00
10
o
0
00
1
0
10
1
1
10
1
1
10
00
00
0
1
00
10
10
10
1
1
10
10
[A, 0, 1,
-
85-
D. E. Muller Theorem 8:
If h is a homomorphism from a monadic algebra
(E,S,M) onto another monadic algebra
a'
a=
= (E,S',M') then there is a
mapping H from,..P(a) onto,..P(a') such that the image of711 t of,..p(a) is the corresponding element 'l71.~ of,..P(a') and which is a semigroup homomorphism. Proof:
If we attempt to define H by the rule that Hon t ) =711~, then
we must show that this definition is unambiguous.
Suppose two distinct
members t, t' of T yield the same element of"'p (a) so that 'l71. t = 711 t , .
Then
we must show that they also yield the same element of"'p (a'), i. e. that
"", - "", lilt - "l t"
Let s' be any state in S'.
state s ES such that h(s) = s'.
Since h is "onto" we can find a
Now, 711~(s')
h(st') = h(s)t' = s't' = chosen arbitrarily, we see that 711~ = 711~" defined by the rule H('\) = 711~.
s't = h(s)t = h(st) 711~,
(s').
Since s' was
so there is a mapping H
This mapping is clearly "onto."
We next must show that it is a semigroup homomorphism. and 711 t (2) be any two members of ,..P(a).
Then He?lt(2) 711 t (1»
= 711~(1)t(2) = 711~(2) 711~(1) = Hon t (2»H('l71. t (1»·
Hon t (1)t(2»
Let'l71.t(l)
Thus H is
a semi group homomorphism from,..P (a) onto,..P (a') and the proof is complete.
0
Exercise 12: h is not.
Give an example such that H is an isomorphism but
If h is an isomorphism must H be one also?
A monadic algebra
a
=
(E,S,M) with a distinguished initial state
So is called a semigroup monadic algebra if whenever sot = sot' for two elements t,t'ET then for all sES, st = st'. Theorem 9:
If
a
is a semigroup monadic algebra there is a one-to-
one mapping f from ,..P(a) onto S such that f(7l1t) = sot. Proof:
Clearly, for each element 'l71. t of ,..P (a) the image
-
86D. E. Muller
f(~)
= sot = mt(so)
for let f O71 t )
=
monadic algebra
is unique.
f
y,
Whenever such a cross-link occurs,
where the vertices are numbered in PUSH order.
The fact
that cross-links always run from a higher numbered vertex to a lower-
-
107 -
R. M. Karp numbered one is a special property of depth-first search.
Below we show
a digraph, and exhibit the result of a depth-first search. 5 ,,"!
4
I
I
\ \
2
3 -, ,
2
\ I
/
"
The edge
(2,4)
is a reverse frond, and
(6,5)
is a cross-link.
We shall use depth-first search as the basis of an algorithm to find the strong components of a digraph. given any two distinct vertices y.
A digraph is strongly connected if,
x and y,
there is a path from
x to
Thus, if we think of the vertices of a digraph as representing the
states of a system, and the edges as representing transitions, then strong connectedness means that, no matter what state the system is in, every state remains reachable.
A strong component of a digraph is a maximal
strongly connected subgraph.
The strong components of the following
digraph are circled. ,/-----.,,, /
/
/
,
I
I~------~~--~~ \
\
......
_-_ .....
Note that every vertex is in exactly one strong component, and that an edge is in a strong component if and only if it is part of a directed cycle. Let G be a digraph.
For an}, vertex.
.x.~
let. R(x). denote the set
-
108R. M. Karp
Of vertices reachable from x. itself, so x
R(x).
E
In particular, the null path takes
We define the equivalence relation ;
by:
x to x; y
if and only if x and yare in the same strong component of G. an easy exercise to show that x
=y
if and only if R(x)
=
It is
R(y).
We now discuss the computation of the strong components of G.
For
convenience assume there is a vertex s from which every vertex is reachable (we leave it as an exercise to extend the discussion to the case where such an
s does not exist).
first search of G. x,
including x
Proof.
(~)
Recall that Anc(x)
conduct a depth-
denotes the set of ancestors of
x itself.
=y
Lemma.
Starting at s,
min c(x,i() flows f a l l s-t cuts X,X But we have previously established the reverse inequality, and thus it will follow that max = min c(X,X) flows f a l l s-t cuts X,X First we show that computing a maximum flow is not quite as simple as one might first assume. units of flow along the edge
In the following network, suppose we send 5 (s,A),
units flow along the path A,B,t, A,D,t.
and then divide it so that four
and one unit flows along the path
Then we get the flow pattern indicated by the circled numbers. 3
The edges
s,A and B,t are saturated, so that there is no way to send
more flow along any directed path from s to t.
And yet this is not a
maximum flow, since the next figure indicates a flow of value 8.
5
®
4
®
-
117 R. M. Karp
This secund flow is maximum, since its value is 8, which is equal to the capacity of the s,t cut (X,X), where
x = {s,A,B}
X
and
{C,D,t}.
Comparing the second flow with the first, we see that it was necessary to decrease (or cancel) three units of flow on the edge get a flow of greater value.
(A,B)
in order to
Thus, any way to compute a maximum flow by
successive flow augmentations must permit the possibility of cancelling some flow. Consider an edge
o< f
Thus
1>
f(X.X) - f(X.X)
c(X.X).
must be a maximum flow.
We are now ready to prove the max-flow min-cut theorem.
Notice that
the process of computing successive augmentations uses only the operations of addition and subtraction.
Thus every number computed is an integer if
the initial capacities are integer and we start with the zero flow. Hence each augmentation increases
1>
by at least one at each iteration.
and the process must terminate with a flow whose value is equal to the
-
119 -
R. M. Karp capacity of some s-t cut. are rational.
A similar argument applies when the capacities
Thus, when the data is rational, the max-flow min-cut
theorem holds.
But max ¢ and min c(X,X) are both continuous functions X,X of the data, and hence equality must hold for real data as well. This completes the proof of the max-flow min-cut theorem. The following example shows that the wrong choice of flow augmenting paths can cause the algorithm to perform badly.
The value of a maximum flow is obviously 2 x 10 6 , but the computation can take 2 x 10 6 iterations if we insist on augmenting alternately along the paths s,A,B,t and s,B,A,t.
The choice of these paths seems unnatural,
of course, since they are longer and permit smaller augmentations than the paths s,A,t and s,B,t.
Next we will investigate suitable rules
for the choice of flow augmenting paths. We consider a method of computing maximum flows in which, at each step, a shortest augmenting path (i.e., one with fewest edges) is chosen. We show that the total running time of a max-flow algorithm based on this choice is 0(\E\.\V\2). Suppose we start with some initial flow fO,
and by repeated augmen-
tations along shortest augmenting paths, determine a sequence of flows Uk},
each of larger value than its predecessor. Let the augmentation network used in passing from fk to fk+l be denoted Gk (V, Ek). and let pk denote the pa th from s to t in Gk along which augmentation
-
120R. M. Karp
takes place.
For any vertex
u,
let l1"(u)
to
u in
Gk,
of the shortest path from
u to
t
shortest path from
Al so, if
Proof. Gk+ 1.
(w,x)
s
denote the length of a k and let a (u) denote the length
in
Gk.
Then
is an edge in
Consider any edge
(x,w)
in a shortest path from
There are two cases: (a) (x ,w) E: Ek , so l1 k(w) .:. 1 + l1 k(x) k (b) (w ,x) E: Pk, so l1 k(x) =l+l1(w),
l1 k(w)
and
s
IMI.
It follows that
G*,
denote the edges
N-M as solid lines
has more solid edges than wiggly edges.
N are matchings, each vertex of G*
one wiggly edge and at most one solid edge. nent of G*
[Here al denotes
is obtained by deleting from M the ele-
Let N be a maximum matching.
INI > IMI.
Mal P
=
Also,
since
is incident with at most It follows that each compo-
is either an isolated vertex, an alternating cycle (i.e.,
one consisting alternately of solid and wiggly edges) or a simple alternating path.
One component must contain more solid edges than wiggly
edges, and it can only be an alternating path beginning and ending with a solid edge. Exercise.
Such a path is an augmenting path relative to
M.
o
Suppose we start with some initial matching and repeatedly
augment along a shortest augmenting obtained be
Pl 'P 2 "" ,P k,... .
path.
Let the successive paths so
Then
(a)
length(P k):5.. length(P k+l ),
k
(b)
no edge occurs in two augmenting paths of the same length.
=
1,2, ... ,
This suggests that we can compute a maximum matching in a series of stages,
in each of which we exhaust paths of a given length.
In the
bipartite case we can give an efficient implementation based on this idea; not surprisingly, the implementation is very similar to the one we used in the computation of maximum flows.
The main difference is that, when
we augment along a path, all the edges of the path are popped from the 'stack.
Because of this feature, the number of PUSH and POP operations
-
126R. M. Karp
during a single stage,
in which we exhaust all the augmenting paths of a
given length, is O(IEI). We can also show that the number of stages will be only about
/2fVT.
We can easily extend the proof of Berge's Theorem to yield the following lemma. Lemma.
If M is a matching which is not maximum, and the cardinality of
a maximum matching is x, M containing at most
then there is an augmenting path relative to
x~I~1
edges from M.
It follows from this Lemma by setting
IX
stages produce a matching of size
stages are needed altogether. required is ~
~
IMI
x - IX that the first
=
x - IX;
Since x ~
l¥l,
hence at most 21>< the number of stages
/2TVT, and the running time is O(IEIITVT). Tarjan has
given examples showing that this upper bound on the growth rate of the running time for this algorithm is actually tight. To complete our discussion of maximum matchings in bipartite graphs (or, equivalently, maximum sets of independent x's in matrices), we show how to construct a set of lines of minimum cardinality which cover all the x's.
Let us call the two types of vertices in a bipartite graph the boys
and the girls; these correspond to the rows and columns in the matrix. Suppose we have constructed a maximum matching M.
Consider the struc-
ture of the unsuccessful search for an augmenting path relative to M.
G be
Let
the set of girls reachable from som~ free boy by an alternating
path, and let B be the set of boys not reachable from any free boy by an alternating path.
Then
G covers
(where B is the set of all boys), so IGI
=
every x covered by a boy in B-B
GU B
IB-BI - (number of free boys), whence
covers all the x's. IGUSI
=
Also,
IBI - (number of
-
127 -
R. M. Karp
free boys)
IMI.
Example.
ABC
0
E
F
x
x @
2
x
x
3
x
@
x
4
x
@
x
@
5
x
6
The free boys are 2 and 6.
G=
The search:
{C,E}
B={l,4}
The set of lines .{1,4,C,E} covers all the x's. Now we apply augmenting path ideas to the solution of a frequently occurring optimization problem called the assignment problem. have n men and n jobs, and have a cost c ij job j.
Suppose we
of assigning man
i
to
We seek a 1-1 assignment of men to jobs which has minimum cost.
In other words, we require that permutation TI of {1,2, ... ,n} which minimizes
ICi,TI(i)'
The first observation on the way to a solution method is that the optimum choice of a permutation is not affected by adding constants to the rows and
6j
ui
to the columns of the cost matrix to produce a new
-
128-
R. M. Karp
cost matrix (c ij ), where cij n
Ic.
i=l
c ij + a i + 8j . Given any permutation
n
n
n
(.)+ Ia.+ I8(.)
1,1f 1
i=ll
i=l
I c.
i=l
1f 1
n
1f
n
(.) + I a. + I 8·
1,1f 1
i=ll
j=l
J
Thus, the transformation adds the same constant to the cost of every permutation. Secondly, suppose we succeed in transforming the cost matrix to a nonnegative matrix with a set of n independent zeros.
Then we can use
those zeros to form a permutation of cost 0, which is clearly optimum. Given any nonnegative cost matrix, suppose that the maximum number of independent zeros is lines.
k < n.
Then we can cover the zeros with k
Clearly, new zeros must be created among the cells that are not
covered by anyone of these
k lines.
If we subtract e from every row
not in the cover, and add e to every column in the cover, then the net effect is to subtract e from
~very
add e to every doubly covered cell.
uncovered cell in the matrix, and to We choose e as the least value
occurring in any uncovered cell. Exercise.
If S is a maximum set of independent zeros, then no element
of S is doubly covered. Example.
Suppose the cost matrix is the one shown in (a) below.
A con-
venient way to start is to subtract the minimum element in each row from all elements of the row, and then do the same to the columns. duces the matrix shown in (b).
This pro-
The rest of the display shows the succes-
sive matrices reached in the computation.
-
129R. M. Karp
ABC 0
5 2 2 1 2
1 6 7 4
3
1 6 5 3
4
1 897
tn
80@1 [ O@l 0 o 1 l@ @2 34
2
22
677 e=3 (b)
(a)
3 4 e =1 (c)
(d)
In this example progress was misleadingly smooth, since the maximum number of independent zeros increases at each iteration. This is not the case in general.
We give two proofs that the algorithm terminates.
The first proof is by noting that every iteration reduces the sum of the entries in the matrix by an integer (assuming the matrix was originally all-integer).
The second proof stems from the fact that the set of
columns reachable from a free row by an alternating path, in which the edges correspond to cells marked zero, strictly grows at each iteration. The rows and columns reachable before the iteration step continue to be reachable, and the new zero is so placed that it makes at least one more column reachable.
We illustrate by exhibiting the reachable rows and
columns in matrix (b), and then in matrix (c) (before the 3-D element is ci rcled).
(b)
(c)
Since we either augment the matching or reach a new column at every iteration, an augmentation must occur every n iterations, and the entire computation must terminate within n2 iterations.
If, for each row, we
-
130R. M. Karp
use a priority queue to represent the elements of that row which are not covered by a column in the minimum cover, then each iteration can be carried out in O(n log n) primitive steps (addition, subtraction, comparison), and hence the assignment problem can be solved in 0(n 3log n)
steps.
While the above algorithm (called the Hungarian method) is a nice application of the Konig-Egervary theorem, it is not the most efficient algorithm known for the solution of the assignment problem.
We next give
another method, based on shortest-path calculations, which solves the problem in 0(n 3 )
steps.
Let G = (V,E)
be a digraph with V = {l ,2, ... ,n},
First we review a few facts about shortest paths.
have weight dij
~
from 1 to
Then, clearly, for each edge
i.
let each edge
( i ,j )
and let II~1 denote the minimum weight of a path
0,
II~ + 1
d.. lJ
II~ > J -
( i ,j ) ,
a,
.and II~ + d.. - II~ = 1
if (i,j)
lies in a
lJ
minimum~weight
J
a
path from
to
j.
Now suppose we are given a nonnegative n x n matrix (c ij ) in which a set of independent zeros is circled.
We construct an associated di-
graph with 2n ·vertices, of which n correspond to the rows and the columns.
If cell
weight c ij
from row
an edge of weight
a
i-j
n,
to
is not circled, then we have an edge of to column j.
from j
to
i.
If i-j
is circled, then we have
In this digraph, we may use the
Dijkstra algorithm to compute shortest paths from the set of free rows to all rows and columns.
This requires a slight modification, since the
original algorithm is given for a single source, and now everyone of the free rows is treated as a source,
Let Yi
denote the weight of a
-
131 -
R. M. Karp
shortest path to row i, and let OJ denote the weight of a shortest path to column j.
Now consider the matrix
(Cij)
= (cij+Yi -OJ). By
the foregoing remarks about shortest-path problems, this will be a nonnegative matrix, and all the cells that lie in shortest paths will become O.
Also, the cells that were circled will remain at O.
entry in a shortest path to a free column becomes 0,
Every
so every such path'
may be used as an augmenting path, and thus the size'of the matching (i.e., the set of circled independent zeros) can be increased by at least 1. Thus, after n iterations, we obtain n independent zeros, and the assignment problem is solved.
Each shortest-path calculation takes
0(n 2) steps, and thus the entire process is completed in 0(n 3) steps. We give a computational example. At each step each row is labelled with Yi
and each column with OJ.
A matching is indicated in each
matrix, and the augmenting path used to get that matching from the previous one is shown.
0 0 0 0
1221
o4
5221 1 674 1 653 189 7
4 4@0 0 0 @4 5 3 0 o 4 ,3 2 0 o6 7 6
32
(b)
(a)
O---@ 1: 0
1: \
@
I
o'---@
I
2
3 (d)
0 8@1 2 0 @O 2 1 o O@O 0 024 2 (c)
9---
