This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
u,for some v E Z},
or, equivalently, by means of the basic open neighbourhoods N;;={VEA*lu 2. Let X = xaiAw C xAw = Y. In case Xc Z C Y, where Z = uAw E P, we must have x
f(x)} is c.e.
f : A * ---+ R+ be semi-computable from above and 9 : A * ---+ R+ be semi-computable from below. Then show that the set {x E A* I g(x) < f(x)} is c.e.
4. Let
5. Show that if A = Nand p is defined by p(n) probabilistic model. 6. Show that if A = Nand p is defined by p(n) a probabilistic model.
=
=
2- n -1, then (A,p) is a
(n+I)I(n+2), then (A,p) is
7. Let P be the uniform probability distribution on A = {O, 1}1O. What is the probability of the events: a) Xl = {x = Xl ... XlO I X2 = 0, X5 = I}, b) Xl = {x = Xl.· .XlO l2:i~1 = 3}? 8. Prove the following properties of probability distributions: a) P(0)
=
0 and P(A)
=
1.
b) If (Xi)i=l, ... ,n are disjoint subsets of A, 2:~=1 P(Xi)' c) If Xc Y, then P(Y \ X)
=
then P(Uf=1 Xi)
P(Y) - P(X).
d) For every X, YeA, P(X U Y) = P(X)
+ P(Y) -
P(X n Y).
9. We toss a fair coin four times. Assume that we know that at least one time we have got 1. What is the probability that we have got 1 in the first toss, i.e. Xl = I? Compare this probability to the probability of the event Xl = 1. 10. Prove that the following pairs of events are independent, (X, Y), (X, Y), (X, Y), provided X and Yare independent. 11. Show that if (X I, Y) and (X 2, Y) are independent and X I, X 2 are disjoint, then (Xl U X 2 , Y) are independent.
20
1. Mathematical Background
12. Show that if Xl, X 2, ... , Xn are independent, then so are the events Yl , Y2,.·., Yn provided Yl E {Xl, Xl}, Y2 E {X2' X 2}, ... , Yn E {Xn,Xn}. 13. Show that if Y n Z = 0 and Y, X 2, . .. , Xn and Z, X2, .. . ,Xn are independent, then so are (Y U Z), X 2 , ••. , X n . 14. Show that if Xl, X 2, ... , Xn are independent, then so are (Xl n X 2 ), •.. ,Xn ·
15. (Bernoulli scheme with finitely many tosses) Consider A = {O,l}n, a = ala2 ... an E A and define
p(a) = p~umber of Os in a p?umber of Is in a, where Po
+ Pl
=
1, PO,Pl :::=: O.
Prove:
a) For every 1::::; il < i2 < ... < ik ::::; nand bl ,b2 , ••. ,bk E {0,1}, we have: P( {a E A I ail = bl , ai2 = b2, ... ,aik = bd) = Pb l Pb2 ... Pb k· b) The events {a E A I,ail = bl }, {a E A l,ai2 = b2}, ... ,{a E A I,aik = bk} are independent. 16. Show that every measure p, is additive (for all pairwise disjoint sets (En)o~n~m, p,(U~=oEn) = 2:~o P,(En)), monotone (if E c F, then p,( E) ::::; p,( F) ), su b-additive (for every sequence of sets (En )n>O,
p,(Un"?oEn) ::::; 2:n"?O P,(En)). 17. A null set is a set of measure zero. Show that a countable union of null sets is a null set. 18. Let X be a set for which i) p,(X) exists and ii) for every c > 0 there exists a set Y such that X c Y and p,(Y) ::::; c. Then, prove that X is a null set. 19. (Bernoulli scheme with infinitely many tosses) Consider A = {O, I}, Po Pl = 1,PO,Pl :::=: O. The measure
+
n
p,(xAW) =
II Pai' i=l
where x = ala2 ... an E An, gives the probability of getting a particular sequence ala2 ... an of Os and Is in the first n tosses in which 0 appears with probability Po and 1 appears with probability Pl. If Po = Pl, then we get the Lebesgue measure. 20. Let y E AW be fixed. Define
if x
0 has a representation in S of log n + 2log log n + 1 bits.
Example 2.5.
Furthermore, by replacing 0 by al and 1 by a2 we can consider that the function bin takes values in {aI, a2}* C A*. The set {d(x) I x E A*} c A* is prefix-free, where d( x)
= bin(lxl)x
is the self-delimiting version of the string x E A *.
2.2
Instantaneous Coding
Consider two alphabets Y = {YI, Y2, ... , YN} and A = {aI, a2, ... , aQ} such that 2 ::; Q < N. If Y is the alphabet of a given initial information source and A is the input alphabet of a communication channel, then in order to transmit the letters (i.e. strings on Y) through the given channel an encoding process has to be developed, even if we assume that there is no noise on the communication channel. Definition 2.6. i} A (finite) code is an injective function i.p : Y ---> A *. The elements of i.p(Y) are called code-strings. ii} An instantaneous code or prefix code is a code i.p such that i.p(Y) is prefix-free.
= {YI, Y2, Y3, Y4} and A following functions defined on Y,'
Example 2.7. Let Y
YI i.pl
i.p2 i.p3 i.p4
Y2
Y3
=
{O, I}. Consider the
Y4
00 01 10 11 10 110 1110 11110 10 10 110 1110 01 011 0111 01111
The codes i.pl, i.p2 are instantaneous while the code i.p4 is not (i.p4 (Y) is not prefix-free); i.p3 is not even a code.
In what follows we will be concerned with instantaneous codes. Their main property is the uniqueness of decodability: a code is uniquely decod able if for each source sequence of finite length (i.e. string), the corresponding sequence of code-strings does not coincide with the sequence of
25
2.2 Instantaneous Coding
code-strings for any other source sequence. In other words, the (unique) extension of r.p to y* is injective. For example, the sequence
0010001101 in code r.p1 can be split as
00,10,00,11,01 and decoded as
Not every uniquely decodable code is instantaneous (e.g. r.p4), but as we shall see later, such a code can always be converted into an instantaneous code. The advantage of the prefix-free condition resides in the possibility to decode without delay, because the end of a code-string can be immediately recognized and subsequent parts of the message do not have to be observed before decoding is started. A simple way of building prefix codes is given by the following theorem.
= 1,2, ... , N, be positive integers. These numbers are the lengths of the code-strings of an instantaneous code r.p: Y ~ A* iff L:~I Q-n i :::; 1.
Theorem 2.8 (Kraft). Let (ni), i
Proof Let r.p : Y ~ A * be an instantaneous code such that !r.p(Yi)! = ni,l :::; i :::; N. Let ri be the number of the code-strings having length i. Clearly, rj = in case j > m = max{ nl, ... , nN}. As the tode is
°
instantaneous, the following relations hold true: rl r2 r3
rm
< Q, < (Q - rl)Q = Q2 - rIQ, < ((Q - rl)Q - r2)Q = Q3
.)
= x}.
Hu(x).
c) The canonical program defined with respect to Chaitin's universal computer U is x*
= min{u
E A* I U(u,>.)
=
x},
where the minimum is taken according to the quasi-lexicographical order on A* induced by al < a2 < ... < aQ.
By definition, x* is the most compact way for U to compute x: the computation U(x*) = x produces x by freeing Ixl - Ix* I bits of memory. What is the least thermodynamic cost of generating a string x from the canonical program x*? Zurek [454] has proven that the computation U(x*, >.) = x can be achieved reversibly, with no cost in terms of entropy increase. Let us note that a reversible computation, i.e. a computation which can be undone, can be performed only by using computer memory to keep track of the exact logical path from input to output (see further Calude and Casti [65]): thermodynamic irreversibility is inevitable only in the presence of logically irreversible operations. Corollary 3.5. For every computer
'
(3.3)
(3.4)
rt dom(U)..).
0
Definition 3.8. a) The Kolmogorov-Chaitin conditional complexity (for short, the conditional complexity) induced by the computer
. In view of the Invariance Theorem one has
H(x,y)
= H«
for some constant c
x,y »::; Hc«
x,y »
+ c = H(y,x) + c, o
> O.
If f : A* ---7 A* is a computable bijection, then H(f(x)) = H(x) + 0(1). Indeed, we can use the Chaitin computer C(u, >.) = f(U(u, >.)). In the proof of Proposition 3.12 we have used the function
Remark.
f(x) = < (xh, (xh > . Lemma 3.13. The following two formulae are true:
H(x/x) = 0(1),
(3.8)
H(string(H(x))/x) = 0(1).
(3.9)
3. Program-size
40
Proof We have only to prove that conditional program-size complexity induced by a universal computer is bounded above. For (3.8) we use Chaitin's computer C(A,U) = U(U,A),U By (3.3), C(A, x*) = x, so Hc(x/x) Invariance Theorem.
E
A*.
= O. Formula (3.8) follows from the
For the second formula we construct Chaitin's computer
D(A, u)
= string(lul),
if U(u, A)
< 00.
Again by (3.3),
D(A,X*) = string(lx*l) = string(H(x)), HD(string(H(x))/x) = 0, and the required formula follows from the Invariance Theorem.
D
Lemma 3.14. There exists a natuml c such that for all x, yEA * one
has H(x) ::; H(x, y)
+ c,
(3.10)
H(x/y) ::; H(x)
+ c,
(3.11)
+ H(y/x) + c, H(x, y) ::; H(x) + H(y) + c.
H(x, y) ::; H(x)
Proof First we use the Chaitin computer C(u, A) H(x)
< Hc(x) + c < min{lull u E A*, (U(u, A)h H(x,y)
= x, (U(u, A)h = y}
proving (3.10). For (3.11) we can use the Chaitin computer D(u, v) = U(u, A) :
= H(x).
(3.13)
= (U(u,A)h:
+ c,
HD(X/Y)
(3.12)
+c
3.2 Computers and Complexities
41
To get (3.12) we construct a Chaitin computer C satisfying the following property: if U( u, x*) = y, then C(x*u, >.) = < x, y > . For the construction we use the c.e. (infinite) set V = dom(U>..). The computation of C on the input (x, >.) proceeds as follows: 1.
Generate all elements of V until we find (if possible) a string v E V with v
.)
=
try to compute U(w,v).
< U(v,>.), U(w, v) >.
Clearly, C is a p.c. function and C(u, v) = 00, for v i=- >.. Assume that x, y E dom( C>..) and x
..), Wx
E
dom(Uux )' Wy
E
dom(Uuy )
such that
x = UxW x , Y = UyWy. Since U x and u y are both prefixes of y and they belong to the prefix-free set dom(U>..) , it follows that Ux = uy. Moreover, {wx, wy} C dom(Uu ) , where u = Ux = uy and uW x , uWy are prefixes of y; we deduce that Wx = w y, i.e. x = y. So C is a Chaitin computer. Next we show that C satisfies the condition cited above. Let v = x*u and assume that U(u, x*) = y. Obviously, x* E V; during the first step of the computation of C( ux*, >.) we get x*; next we compute u and U( u, x*) = y < 00. According to the third step of the computation,
C(x*u, >.) = < U(x*, >'), U(u, x*) > = < x, y > . In the case H(yjx) = lui one has U(u,x*) exists a natural c such that
H(x,y)
= y and consequently there
=
H«x,y»5oHc«x,y»+c < Ix*ul + c= H(x) + H(yjx) + c.
As concerns (3.13),
H(x, y) 50 H(x) by (3.12) and (3.11).
+ H(yjx) + Cl 50 H(x) + H(y) + C2, 0
42
3. Program-size
Proposition 3.15 (Sub-additivity). The following formula is true:
H(xy) S H(x)
+ H(y) + 0(1).
Proof We use Chaitin's computer C(w,>') the relation (3.13).
=
(3.14)
(U(w,>')h(U(w,>')h and 0
Definition 3.16. The mutual algorithmic information of the strings x and y, according to Chaitin's computer C, is
Hc(x : y) = Hc(y) - Hc(y/x). Also, H(x : y)
= Hu(x : y).
Proposition 3.17. There is a constant c> 0 such that
H(x : y) :2 -c, H(x : y) S H(x)
(3.15)
+ H(y) - H(x, y) + c.
(3.16)
Proof The inequality (3.15) follows from (3.11). By (3.12) we get
H(x : y) = H(y) - H(y/x) ::; H(y) + H(x) - H(x, y) + c.
o
Lemma 3.18. The following formulae hold true:
H(x : x) = H(x)
+ 0(1),
(3.17)
H(x : >.) = 0(1),
(3.18)
H(>' : x) = 0(1).
(3.19)
Proof Formula (3.17) comes from (3.8). By (3.15), H(x : >.) :2 -c, for some positive constant c. Furthermore,
H(x: >.) < H(x) + H(>') - H(x, >.) + Cl < H(x) - H(x, >.) + C2
< because H(x, >.)
C3
= Hc(x), where C(u, >.) = (U(u, >')h.
Finally, using (3.15) and the Chaitin computer D(v, >.) can prove (3.19).
= (U(v, >')h
we 0
3.3 Algorithmic Properties of Complexities
3.3
43
Algorithmic Properties of Complexities
We begin this section by considering the set of canonical programs CP={x* IxEA*}
(see Definition 3.4b). We shall prove that CP is an immune set, i.e. CP is infinite and has no infinite c.e. subset. Theorem 3.19. The set of canonical programs is immune.
Proof The set CP is clearly infinite, as the function x - t x* is injective. We now proceed by contradiction, starting with the assumption that there exists an infinite c.e. set SeC P. Let S be enumerated by the injective computable function f : N - t A *. We define the function 9 : N - t A * by g(O) = f(O), g(n + 1) = f(minj[lf(j)1 > n
+ 1]).
It is straightforward to check that 9 is (total) computable, Sf = g(N+) is c.e. infinite, Sf C Sand Ig( i) I > i, for all i > O. Using the prefix-free set in Example 2.5 we can construct a Chaitin computer C such that for every i :2 2, there exists a string u such that C(u, >.) = g(i) and
lui::; log i + 2 log log i ::; 3log i. By the Invariance Theorem we get a constant
H(g(i)) ::; Hc(g(i))
+ Cl
::;
Cl
such that for all i EN,
3logi + Cl.
(3.20)
We continue with a result which is interesting in itself:
Intermediate Step. in CP, one has
There exists a constant
H(x) :2
C2
:2 0 such that for every x
Ixl- C2·
(3.21)
We construct Chaitin's computer D(u,>.)
= U(U(u,>.),>.)
and pick the constant C2 coming from the Invariance Theorem (applied to U and D). Taking x = y*, z = x*, we have D(z, >.)
= U(U(z, >'), >.) = U(U(x*, >'), >.) = U(x, >.) = U(y*, >.) = y,
44
3. Program-size
so
HD(y) ::; H(x),
= IY*I = H(y) ::; HD(Y) + C2 ::; H(x) + C2· 1, if g(i) E CP, then Ig(i)1 > i, so by (3.20) and (3.21) Ixl
For i
~
i -
C2
< Ig(i)l- C2
::;
H(g(i)) ::; 310gi + Cl,
and consequently only a finite number of elements in S' can be in CPo
D
Remark. In view of (3.21), the canonical programs have high complexity. We shall elaborate more on this idea in Chapter 5. Corollary 3.20. The function f : A *
-+
A *, f (x) = x* is not com-
putable. Proof The function f is injective and its range is exactly CPo
D
Theorem 3.21. The program-size complexity H(x) is semi-computable from above, but not computable.
Proof We have to prove that the "approximation from above" of the graph of H(x), i.e. the set {(x,n) I x E A*,n E N,H(x) < n}, is c.e. This is easy since H (x) < n iff there exist yEA * and tEN such that Iyl < nand U(y,.x) = x in at most t steps. For the second part of the theorem we prove a bit more, namely:
Claim. There is no p.c. function
Qn(1- Q1-c /(Q - 1)).
(3.25)
Proof Take E = An in Proposition 3.24.
Proposition 3.26. If F : A * H(x) :::; F(x) + 0(1), then
--t
0
N is an arbitrary function such that
#{x E A* I F(x) < m} < Qm+O(l). Proof Clearly, {x E A* I F(x) < m} C {x E A* some constant c> O. Consequently,
logQ #{x E A* I F(x) < m}
I H(x)
0 such that for all natural m > 0 #{x E A* I F(x) then H(x) :::; F(x)
+ 0(1).
< m} < logm + q,
47
3.5 Halting Probabilities
Proof. Let {(Xl, md, (X2' m2), ... } be an injective computable enumeration of the c.e. set {(x,m) E A* x N I F(x) < m}. We construct Chaitin's computer C by the following algorithm: 1.
All strings yEA * are available.
2.
For i = 1,2, ... generate (Xi, mi), choose the first available Yi E Alogmi+q and put C(d(Yi),.x) = Xi.
3.
The string Yi is no longer available.
Recall that d comes from Example 2.5. In view of the hypothesis, we have "enough" elements to run every step, so in case F(x) < m there exists Y E A1ogm+q with C(d(y),.x) = X, i.e. Hc(x):::; 10gm+2loglogm+0(1). In particular, F(x) < F(x) + 1, so
Hc(x) :::; 10g(F(x) + 1) + 2 log 10g(F(x) + 1) + 0(1) :::; F(x) + 0(1). Finally, we use the Invariance Theorem.
3.5
o
Halting Probabilities
It is well known that the halting problem for an arbitrary (Chaitin) computer is unsolvable (see Section 9.2). Following Chaitin, we switch the point of view from a deterministic one to a probabilistic one. To this end we define - for a given Chaitin computer - the halting probabilities.
Definition 3.28. Given a Chaitin computer C we define the following ((probabilities" : Pc(x) = {UEA*IC(U,A)=X}
Pc(x/y)
= {UEA*Ic(u,y*)=X}
In the case C = U we put, using the common convention, P(x) Pu(x), P(x/y) = Pu(x/y). We say that Pc(x) is the absolute algorithmic probability of Chaitin's computer C with output X (it measures the probability that C produces x); Pc(x/y) is the conditional algorithmic probability.
3. Program-size
48
The above names are not "metaphorical". Indeed, P is just a probability on the space of all sequences with elements in A, i.e. AW, endowed with the uniform distribution. See Section 1.4 for more details and specific notation. As a consequence, for every Chaitin computer C,O :::; Pc(x) :::; 1 and 0 :::; Pc(x/y) :::; 1, for all strings x, y. Actually, we can prove a bit more. Lemma 3.29. For every Chaitin computer C and all strings x and y,
nC = L
(3.26)
Pc(x) :::; 1,
XEA*
L
(3.27)
Pc(x/y) :::; 1.
xEA*
Proof. For (3.26) we can write
nC =
L
Pc(x)
xEA*
=
L
L
Q-[u[
XEA* {uEA*[C(U,A)=X}
=
L
Q-[u[:::;
1,
uEdom(C)..)
the "series" still being a probability. The same argument works for (3.27).
o Remark. The number nc = LXEA* Pc(x) expresses the (absolute) halting probability of Chaitin's computer C. Lemma 3.30. For every Chaitin computer C and all strings x, y, Pc(x) ~ Q-Hc(x) ,
(3.28)
Pc(x/y) ~ Q-Hc(x/y).
(3.29)
Proof. One has· Pc(x) = {UEA*[C(U,A)=X}
and Hc(x)
= lui, C(u,.x) = x.
o
In the case of the universal Chaitin computer, neither the absolute nor the conditional algorithmic probability can be 0 or 1.
49
3.6 Exercises and Problems Scholium 3.31. For all x,y E A*,
0< P(x) < 1,
(3.30)
0< P(xly) < 1.
(3.31)
Proof In view of Lemma 3.30, with C = U, P(x) 2:: Q-H(x) = Q-1x*1 > O. Using (3.26), 2:xEA* P(x) :::; 1 and the fact that each term of the series is non-zero we deduce that P(x) < 1. A similar reasoning works for (3.31).
o Proposition 3.32. For every Chaitin computer C and all naturals n, m 2:: 1, the following four formulae are true:
#{x E A* I Hc(x) < m} < (Qm - 1)/(Q - 1),
(3.32)
#{x E A* I HC(xly) < m} < (Qm - 1)/(Q -1),
(3.33)
#{x E A* I Pc(x) > nlm} < min,
(3.34)
#{x E A* I Pc(xly) > nlm} < min.
(3.35)
Proof For (3.32) we use Lemma 3.23. For (3.34) let 8 = {x E A* Pc(x) > ~} and assume, by absurdity, that #8 2:: ~. Then, by (3.26): 1 2::
L xEA*
Pc(x) 2::
L xES
Pc(x) > !2.#8 2:: 1, m
o
a contradiction.
3.6
I
Exercises and Problems
1. Show that every prefix-free c.e. set of strings is the domain of some Chaitin
computer. 2. Show that there exists a natural c such that for all x E A *, H (x* / x) :::; c, and H(x/x) :::; c. 3. Consider Ackermann-Peter's computable and non-primitive recursive function a : N x N --t N,
a(O,x) = x + 1, a(n+1,x) =a(n,1),
3. Program-size
50 a(n + 1, x + 1) = a(n, a(n + 1, x)).
Show that for every unary primitive recursive function f there exists a natural constant c (depending upon f and a) such that f(x) < a(c, x), for all x ~ c; see Calude [51] for other properties of a. For every natural n define the string s(n) = 1a(n,n). a) For every n EN, K(s(n)) = K(string(n)) + 0(1). b) There is no primitive recursive function f : N
f(K(s(n)))
~
-t
N such that
a(n, n).
4. Fix a letter a E A. Show that there exists a constant c > 0 such that K(an/n) ~ c, for all natural n, but K(a n ) ~ logn-c, for infinitely many
n. 5. Show that there exists a natural c such that for all x E CP, H(x) (Hint: use Chaitin's computer C(u,'x) = u,u E dom(U)..).)
< Ixl +c.
6. (Chaitin) Show that the complexity of a LISP S-expression is bounded from above by its size + 3. 7. Show that the conditional program-size complexity is semi-computable from above but not computable. 8. (Chaitin) Show that H(x) ~ Ixl +loglxl +2logloglxl +c; furthermore, one can indefinitely improve the upper bound (3.22). (Hint: use Chaitin's computer C(bin(lbin(lxl)l)x,'x) = x.) 9. The function H(x/y) is not computable; is it semi-computable from above? 10. If yEA *, mEN and SeA * is a prefix-free set such that EXES Q-Ixi ~ Q-m /(Q _ 1), then there exists an element xES such that H(x/y) ~ Ixl-m. 11. Show that the halting set K = {x E A * I 'Ilx (x) < oo} and the selfdelimiting halting set K S = {x E A* I Cx(x) < oo} ((Cx ) is a c.e. enumeration of all Chaitin computers) are readily computed from one another, i.e. there exists a computable bijection F : A* - t A* such that F(K) = K S . 12. (Levin) Show that the following statements are equivalent: a) The function F : A* - t N is a function semi-computable from above + and K -< F, b) #{x E A* I F(x) < m} < Qm+O(1). 13. (Chaitin) A sequence x E AW is computable iff K(x(n)) ~ K(string(n)). Show that the equivalence is no longer true in case the formula on the right-hand side is valid only for infinitely many n.
+
+
14. Show that K -< H -< K + 2logK.
3.6 Exercises and Problems 15. Show that H ~ K upper bound.
51
+ log K + 2 log log K;
one can indefinitely improve this
16. Let f : N - t A* be a computable function such that If(n)1 = n, for all n ~ O. Then, H(x/ f(lxl)) :::; Ixl + 0(1).
17. Show that K(st'ring(n)) :::; logQ(n)
+ 0(1).
18. Show that there exist infinitely many n such that K (string( n)) 19. Show that if m
~
logQ (n).
< n, then m + K(st'ring(m)) < n + K(string(n)).
20. (Kamae) Prove that for each natural m there is a string x such that for all but finitely many strings y one has K(x) - K(x/y) ~ m. 21. Show that the above statement is false for H(x/y). 22. (Chaitin) An information content measure is a partial function H : N ~ N which is semi-computable from above and Ln>o 2- H (n) :::; 1. In case H(n) = 00, as usual, 2- 00 = 0 and this term contributes zero to the above sum. Prove: a) The Invariance Theorem remains true for the information content measure. b) For all natural n
H(n) H (n) H (n)
< 2logn + c, < log n + 2 log log n + c', < log n + log log n + 2 log log log n + c" ,
c) For infinitely many natural n
H(n) H(n) H(n)
> logn, > logn + log log n, > logn + log log n + log log log n,
23. Reformulate the results in this chapter in terms of information content measure. 24. (Shen) Show that for all strings x, y, z of length less than n
2H(x, y, z) :::; H(x, y)
~
+ H(x, z) + H(y, z) + 0(1).
lone has
52
3.7
3. Program-size
History of Results
The theory of program-size complexity was initiated independently by Solomonoff [373], Kolmogorov [259] and Chaitin [110]. Chaitin refers to the Kolmogorov-Chaitin complexity as blank-endmarker complexity. The importance of the self-delimiting property was discovered, again independently, by Schnorr [361], Levin [278] and Chaitin [114]; however, the theory of self-delimiting complexity was essentially developed by Chaitin (see [122]). Related results may be found in Fine [197], Gacs [199], Katseff and Sipser [249], Meyer [313]. The proof of Theorem 3.19 comes from Gewirtz [208]. The halting probabilities have been introduced and studied by Chaitin [114]; see also Willis [435]. For more historical facts see Chaitin [122, 131, 132, 134], Li and Vitanyi [282]' Uspensky [407]. Overviews on program-size complexity can be found in Zvonkin and Levin [455], Gewirtz [208], Chaitin [118, 121, 122, 125]' Schnorr [361], MartinLof[301], Cover and Thomas [152]' Gacs [203], Kolmogorov and Uspensky [261]' Calude [51], Li and Vitanyi [280,282]' Uspensky [407, 408], Denker, Woyczynski and Y cart [173], Gruska [217], Delahaye [164], Ferbus-Zanda and Grigorieff [195], Sipser [368], Yang and Shen [445, 446].
Chapter 4
Computably Enumerable Instantaneous Codes Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. A ntoine de Saint Exupery
In this chapter - which is basically technical - we present two main tools used to design Chaitin computers and consequently to establish upper bounds: the extension of the Kraft condition (see Theorem 2.8) to arbitrary c.e. sets and relativized computation. New formulae, closely analogous to expressions in classical information theory, are derived.
4.1
The Kraft-Chaitin Theorem
We devote this section to the proof of the following important result. Theorem 4.1. Let
,)=y,C(Z,w)=x}
H(x/y; w) = Hu(x/y; w), P(x/y; w) = Pu(x/y; w).
The following relations are obviously true for all x, y, wE A *: Hc(x/y)
= Hc(x/y; y*), Pc(x/y) = Pc(x/y; y*),
0:::; Pc(x/y; w) :::; 1,
L
Pc(x/y; w) :::; 1,
XEA*
Pc(x/y; w) 2: Q-Hc(x/y;w), 0 < P(x/y; w) < 1.
We refer to Hc(x/y; w) as the (Chaitin) relativized complexity of x, y with respect to wand Chaitin computer C. Similarly, Pc(x/y;w) is the relativized probability. Theorem 4.4. For every Chaitin computer C there exists a constant c > 0 (depending upon U and C) such that for all x, yEA * one has H(x) ::; -logQ Pc(x)
+ c,
H(x/y) ::; -logQ Pc(x/y)
+ c.
(4.5) (4.6)
Proof A simple dovetailing argument shows that the set T = {(x, n) E A*xN I Pc(x) > Q-n} isc.e. Let B = {(x,n+1) E A*xN I (x,n) E T} and put M = Q-(n+l) = Q-l Q-n.
L
L
(x,n+1)EB
(x,n)ET
We shall prove that M ::; 1. To this end we first introduce a piece of notation: for every real a, if Qn < a :::; Qn+l for some integer n, then put n = IgQa (lgQ = flogQ a 1 - 1). The following relations hold true:
62
4. G.E. Instantaneous Codes
< a, 2) if a > 0, then 19Qa < 10gQ a S 19Qa + 1, 1)
if a> 0, then QIgQa
3)
if a is a positive real and m is an integer, then
The first two relations are direct consequences of the definition of 19Q. If a > and m is an integer, then from Qn < a S Qn+l and 19Qa 2:: m we deduce m S 19Qa = n = 10gQ Qn < 10gQ a. Conversely, if 10gQ a > m, Qn < a S Qn+l, then Qn+l 2:: a > Qm, so n+ 1 > m, i.e. n = 19Qa 2:: m (n,m E Z).
°
Next we define the sets
N x = {n E N
I Pc( x) > Q-n},
x E A *.
Since n E N x implies n + 1 E N x it follows that N x is infinite. Moreover,
M=Q-l {nENxlxEA*} and
n E Nx
~
Pc(x) > Q-n
~
10gQ Pc(x)
~
19QPc(x) 2:: -no
> -n
Accordingly,
L
Q-n L n2:- gQPc(x)
Q-n
nENx
l
QIgQPc(X)+l/(Q _ 1)
< Q . Pc(x)/(Q - 1) < Q. Pc(x), and finally
M=Q-l
L L xEA*nENx
Q-n
s L
Pc(x) S 1.
XEA*
Using the Kraft-Chaitin Theorem we construct a Chaitin computer D : A * x {A.} ~ A * satisfying the following property
63
4.2 Relativized Complexities and Probabilities For every (x, n) E T there exists a string v E A * such that D(v, >.) = x and Ivl = n + 1. We prove that D satisfies the relation
Notice that D(v, >.)
=x ~
(x, Ivl) E B ~ Pc(x)
> QI- Iv l
and
HD(X)
= =
min{lvll v E A*,D(v,>.) = x} min{lvll v E A*,Pc(x) > QI- Iv l } min{lvll v E A*, Ivl ~ 1 -lgQPc(x)} 1 -lgQPc(x).
For the conditional case we extend D on a c.e. subset of A * x A +. To this end we let v = U(w,>'), x E A* and define the c.e. sets T:
= {(x,n)
B::/
E A* x N
I Pc(x/v;w) > Q-n},
= {(x, n + 1) E A * x N I (x, n)
E T:}.
It should be noted that in case w = v* E CP (U(v*, >.) = v) one has
T;;* = {(x, n)
E A*
x N I Pc(x/v) > Q- n }.
A similar counting argument shows that M (w, v) :s; 1, where
M(w, v)
= (x,n+1)EBi,"
Indeed, since it follows that
XEA* nENW v,x
64 and
4. G.E. Instantaneous Codes
M(w, v) = ~
Q-
L
~
QIgQPc(x/v;w)
XEA*
L
Pc(x) ~ 1.
XEA*
Using the Kraft-Chaitin Theorem again we extend D on a c.e. subset of A* x A+ such that
HD(X/Y) = 1 -lgQPc(x/y). The computation of D proceeds as follows: if U(w, >..) = v and (x, n+1) E B;;\ then there exists y E A* with D(y, w) = x and Iyl = n + 1. In case U(w, >..)
= v, one has D(y, w)
= x {:} Pc(x/v; w) > QI-Iyl.
Indeed,
D(y,w)
= x {:} (x, Iyl + 1) E B":} {:} Pc(x/v;w) > Q-(lyl-1) = Q1-lyl.
Next let w = v* (U (v* , >..) = v). One can easily check that
HD(x/v)
= x} min{lyll y E A*, Pc(x/v) > Q1- lyl }
min{lyll y E A*,D(y,v*)
min{lyll y E A*, Iyl ~ 1 -lgQPc(x/v)} 1 -lgQPc(x/v). Formulae (4.5) and (4.6) can now be derived from the Invariance Theorem. 0 Remark.
In view of the relations
PD(x)
= Q-1.
L
Q-n,
nENx
PD(X/Y)
= Q-1.
L
Q-n, y*
nENy,x
it follows that PD(X)
< Pc(x) and PD(X/Y) < Pc(x/y).
Corollary 4.5. For every Chaitin computer C there exists a constant c > 0 (depending upon U and C) such that for all x, yEA *
P(x)
~
Q-c Pc(x) ,
(4.7)
P(x/y)
~
Q-c Pc(x/y).
(4.8)
4.2 Relativized Complexities and Probabilities
65
Proof. The constant c comes from Theorem 4.4 (formulae (4.5) and (4.6)). It follows that
Pc(x) :::; Qc-H(x) , Pc(x/y) :::; Qc-H(x/ y ). Using Lemma 3.30 (with C
= U)
we get
Q-cpc(x) :::; Q-H(x) :::; P(x), Q-c Pc(x/y) :::; Q-H(x/ y )
:::;
P(x/y).
o
Theorem 4.6 (Chaitin). The following formulae are true:
H(x)
= -logQ P(x) + 0(1),
H(x/y) = -logQ P(x/y)
(4.9)
+ 0(1).
(4.10)
o
Proof. We use Theorem 4.4 and Lemma 3.30.
Remark. Actually, we have proven a bit more than stated in (4.9) and (4.10): namely, there exists a constant c > 0 such that
0:::; H(x)
+ logQ P(x)
:::; c, 0:::; H(x/y)
+ logQ P(x/y)
:::; c.
As a by-product we are able to show that there are only a few minimal programs.
Corollary 4.7. For every x,v E A*
#{y #{y
E
E
A* I U(y,'\)
= x, Iyl :::; H(x) + n} < Qn+O(l),
A* I U(y,v*) = x,
Iyl :::; H(x/v) + n} < Qn+O(l).
(4.11) (4.12)
Recall that is a computable bijection between A* x A* and A* (with Oi,i = 1,2, as inverses) and P(x,y) = P« X,y ».
Theorem 4.8. One has
P(x):::::
2:= yEA*
P(x, y).
(4.13)
4. G.E. Instantaneous Codes
66
Proof. The Chaitin computer C(x, A) = (U(x, A)h has the following property: if U(y, A) = < u, v>, then C(y, A) = u. We compute PC(x) = {yEA*IC(y,A)=X}
uEA* {yEA* lU(y,A)= <x,u>}
All terms of the series above are positive and for every string yEA * with (U(y,A)h = x there is a unique string u E A* such that U(y,A) = < x, u > (because u = (U(y, A)h and is one-to-one). So,
PC(x)
L
=
P(x, u)
UEA*
and
P(x) ~ Q-cpc(x)
= Q-c ( L
P(x,u)).
uEA*
For the converse relation we define the Chaitin computer
D(z, A)
= < U(z, A), U(z, A) >,
we evaluate the sum of the series
L
PD(x,y)
yEA*
yEA* {zEA*I}
L
Q-Izi
{ZEA* IU(Z,A)=X}
P(x), and we get a constant d > 0 with
P(x, y) ~ Q-d PD(x, y). Finally,
P(x) =
L
PD(x, y) :::; Qd(
yEA*
L
P(x, y)).
o
yEA*
Theorem 4.9. There exist a Chaitin computer C and a constant c
>0
such that for all strings x, y one has Hc(y/x) = H(x, y) - H(x)
+ c.
(4.14)
4.2 Relativized Complexities and Probabilities
67
Proof First we prove the existence of a constant c > 0 (depending upon U) such that QH(x)-c P(x, y)) :::; 1. (4.15)
(L
yEA*
From (4.9), H(x) = -logQ P(x) that for all x E A*
+ 0(1), so we can find
H(x) :::; -logQ P(x)
a natural n such
+ n,
or, equivalently, 1 < __
Q H(x)-n
From (4.13) we can get a real a
1/ P(x)
- P(x)·
> 0 such that
:::; a(
L
P(x, y))-l.
yEA*
Accordingly, QH(x)-n :::;
a(
L
P(x, y))-l
yEA*
and we may take in (4.15) c = n + flogQ a 1+ 1. For every x E dom(U>..) , x = U(u, )..), we generate the c.e. set B~
= {Iv I - lui + c I v E A *, (U (v, )..) h =
x} C Z
(c comes from (4.13)). In case u
= x* (U(x*,)..) = x) we have Bx
= B;* = {lvl-lx*1 + c I v E A*, (U(v, )..)h = x} = {Ivl- H(x) + c I v E A*, (U(v, )..)h = x}.
We then compute the sum of the series:
L
Q-(Ivl-H(x)+c)
{VEA*I(U(v,>")h=x} QH(X)-C {VEA*I(U(v,>")h=x} QH(x)-c(
L vEA*
< 1,
P(x, v))
68
4. G.E. Instantaneous Codes
by (4.15). It is worth noting that in the general relativized case U(u, A) = x we cannot claim the validity of the inequality
L
Q-(Ivl-lul+c) :; 1
{vEA* ,(U(v,.\))t=x}
because Ivl-Iul+c may be negative for some values ofu,v E A*. To avoid this difficulty (which prevents us using the Kraft-Chaitin Theorem) we shall proceed as follows. For every string u E A* with U(u, A) = x t= 00 we generate the elements of the set B'!); = {IVll- lui + c, IV21- lui + c, ... } and we test, at every step t ;:: 1, the condition t
L
Q-(lvil-lul+C) :; 1.
i=l
At the first failure we stop the generation process. Now we are in a position to make use of the Kraft-Chaitin Theorem to get the uth section of a Chait in computer C satisfying the property if U(u, A) = x and (U(y, A)h = x, then C(v, u) = (U(y, A)h, Ivl
= Iyl- lui + c.
It is clear that in the special case u = x*, the Kraft-Chaitin inequality is fulfilled; however, for U( u, A) = x we cannot decide, during the execution of the algorithm, if u = x*, since C P is immune. Next we are going to prove formula (4.14). If Hc(yjx) = lvi, then C(v, x*) = y, i.e. there exists a string w such that (U(w, A)h = x, C(v, x*) = (U(w, A)h = y and Ivl = Iwl-lx*1 +c = Iwl- H(x) +c. So,
x
= (U(w, A)h, y = (U(w, A)h,
U(w, A) = < x, y >, H(x, y) :; Iwl,
= Ivl = Iwl- H(x) + c;:: H(x, y) - H(x) + c. H(x) = Ix*l, H(x, y) = Iwl, U(w, A) = < x, y >.
Hc(yjx)
Conversely, let Clearly, Iwl - H(x) + c E Bx = Bit and the Kraft-Chaitin Theorem applies producing a string v such that Ivl = Iwl - H(x) + c with C(v, x*) = y. Accordingly,
Hc(Y/x) :; Ivl
=
Iwl- H(x) + c = H(x, y) - H(x) + c.
0
69
4.2 Relativized Complexities and Probabilities Theorem 4.10. The following formulae are valid:
H(x, y) H(x : y)
H(x)
=
+ H(yjx) + 0(1),
= H(x) + H(y)
H(x : y)
=
H(y : x)
P(yjx) H(yjx)
- H(x, y)
+ 0(1),
+ 0(1),
P(x)
P(x,y)
(4.19)
+ 0(1),
P(x, y) H(x : y) = logQ P(x)P(y)
(4.17) (4.18)
~ P~~~~) ,
= logQ
(4.16)
+ 0(1).
(4.20)
(4.21)
Proof. For (4.16) we construct a Chaitin computer C and a natural c> 0 such that Hc(yjx) = H(x, y) - H(x) + c.
(See Theorem 4.9.) Accordingly,
+ H(x) -
H(x, y) = Hc(yjx)
c ~ H(yjx)
+ H(x) + 0(1)
(we have applied the Invariance Theorem). To get the converse inequality we rely on Lemma 3.14 (formula (3.12)). From (4.16) we easily derive (4.17)
H(x : y)
= H(y) - H(yjx) = H(y) + h(x) - H(x, y) + 0(1).
The same is true for (4.18):
H(x: y)
= H(x)+H(y)-H(x,y)+0(1) = H(x)+H(y)-H(y,x)+0(1),
by virtue of Proposition 3.12. For (4.19) we note that
H(x, y)
= H(x) + H(yjx) + 0(1),
H(x) = -logQ P(x)
+ 0(1),
H(yjx) = -logQ P(yjx)
+ 0(1);
70
4.
e.E.
Instantaneous Codes
we have used Theorem 4.6. By virtue of the same result we deduce the existence of some constant d > 0 such that -d :::; H(yjx)+logQ P(yjx) :::; d. On the other hand, there exists a natural m such that
P(yjx) :::; mP(x, y)j P(x), P(x, y) :::; mP(yjx)P(x) (see (4.19)). Combining the "left" inequalities we get
-d:::; H(yjx)
+ logQ P(yjx) :::; H(yjx) + logQ P~~~~) ,
P(x) H(yjx) ~ logQ P(x,y)
+ 0(1).
From the "right" inequalities we infer
P(x) H(yjx) :::; logQ P(x, y)
+ 0(1),
thus proving formula (4.20). Finally, (4.21) is a direct consequence of 0 formulae (4.10) and (4.20). Corollary 4.11. One has H(x, string(H(x))) = H(x)
+ 0(1).
Proof We use Lemma 3.13 and Theorem 4.10: H(x, string(H(x)))
4.3
H(x) H(x)
+ H(string(H(x))jx) + 0(1) + 0(1).
o
Speed-up Theorem
We define the halting probability of a Chaitin computer and we prove a result asserting that there is no "optimal" universal Chaitin computer, in the sense of the best halting probability. We fix a universal Chaitin computer U and let U( w,.\) define the halting probability of C on section y to be O(C, y; w)
=
L xEA*
Pc(xjy; w).
= y, y I- .\. We
4.3 Speed-up Theorem In case y
71
= A, the absolute halting probability is O(C) =
L
Pc(x).
xEA*
Finally, if C
= U,
then we put 0
= O(U).
The inequalities will be derived in Corollary 7.3. Theorem 4.12 (Speed-up Theorem). Let U and V be two universal Chaitin computers and assume that U(w, >..) = y. Furthermore, suppose that 1- Ql-k < O(V,y;w) < 1- Q-k,
for some natural k > O. Under these conditions we can effectively construct a universal Chaitin computer W satisfying the following three properties. For all x E A *,
Hw(x/y; w) ::; Hv(x/y; w).
(4.22)
For all but a finite set of strings x E A *,
Hw(x/y; w) < Hv(x/y; w),
(4.23)
O(W,y;w) > O(V;y;w).
(4.24)
Proof We fix y with U(w, >..) B
= {(x,n)
E A* x N
= y and let
I V(z,w) = x, Izl = n,
for some z E A*}.
Since Vw is surjective, it follows that B is c.e. and infinite. We fix a one-to-one computable function f : N + ---t A * x N such that range(J) = B. We denote by Oi (i = 1,2) the projection of A* x N onto the ith coordinate. A simple computation shows the validity of the formula
O(V,y;w) =
L
Q-n.
(x,n)EB
In view of the inequality
O(V; y; w) > 1 _ Ql-k
4. G.E. Instantaneous Codes
72
we can construct enough elements in the sequence (J(i)h, i ally we get an N > 0 such that N
L
Q-(f(i))2
> 1_
~
1; eventu-
Ql-k.
i=l
N ext we claim that
#{i E N Ii> N, (J(i)h :::; k}:::; Q. Indeed, on the contrary, LQ-(f(i)h
D(V, y; w)
i~l
N
>
L
Q-(f(i)h
+ Ql-k
i=l
> 1-
Ql-k
+ Ql-k = 1.
Consequently, there exists a natural M > N (we do not have any indication concerning the effective computability of M) such that for all i ~ M, (J(i)h > k. On this basis we construct the computable function 9 : N+ ---t A* x N by the formula
(') 9
2
=
{!(i), if i :::; N or (i > N, (J(i)h :::; k), ((J(i)h, (J(i)h - 1), otherwise,
and we prove that L
Q-(g(i)h :::;
1.
i~l
First, we consider the number N
S = L
Q-(g(i)h
+
L
Q-(g(i)h,
N+1~i~M,(f(i)h~k
i=l
where M is the above bound. It is seen that N S> L i=l
Q-(g(i))2
=
N L i=l
Q-(f(i)h
>1_
Ql-k.
(4.25)
4.3 Speed-up Theorem
73
Now, a simple computation gives
L Q-(g(i))2
S+Q {i>N,(f(i)h>k}
i21
S + Q. (O(V,y;W) - S) Q. O(V,y;W) + (1- Q)S < Q(l - Q-k) + (1 - Q)(l _ Ql-k) 1 - (Q - 2)Ql-k
< 1. In view of the Kraft-Chaitin Theorem there exists (and we can effectively construct) a Chaitin computer W such that for all i 2:: 1 there is a string Zi E A* of length (g(i)h with W(Zi' w) = (g(i)h = (f(i)h. In the case n = Hv(x/y; w) we deduce that (x, n) E B, i.e. (x, n) = f(i), for some i 2:: 1. In case f(i) = g(i), W(Zi'W) = x, for some Zi E A*, IZil = (g(i)h = n; otherwise (i.e. in case f(i) =1= g(i)) W(Zi' w) = x, for some string Zi E A*, IZil = (g(i)h = n - 1. In both cases Hw(x/y; w) n, which shows that W is a universal Chaitin computer and (4.22) holds. Furthermore, the set {i E N I f(i) = g(i)} is finite, so the inequality H w (x / y; w) < n is valid for almost all strings x.
:s:
Finally,
O(W, y; w)
=
L Q-(g(i))2 i21
QO(V, y; w) > O(V,y;w),
+ (1 -
Q)S
o
proving (4.24). (The number S comes from (4.25).)
Corollary 4.13. Let U be a universal Chaitin computer such that 1 - Ql-k < O(U) < 1 _ Q-k,
for some natural k. Then we can effectively find a universal Chaitin computer W satisfying the following three properties. For all x E A *, Hw(x)
:s: Hu(x).
( 4.26)
For all but a finite set of strings x E A *, Hw(x) < Hu(x),
(4.27)
O(W) > O(U).
(4.28)
74
4. G.E. Instantaneous Codes
Remark. A similar result can be deduced for conditional complexities and probabilities.
4.4
Algorithmic Coding Theorem
In this section we prove the universality of the representation formula (4.9) in Theorem 4.6, i.e. we show that it is valuable not only for the probability P, but also for a class of "semi-measures".
Definition 4.14. a) A semi-measure is a function v satisfying the inequality v(x) ~ 1.
A*
---+
[0,1]
L
XEA*
b) A semi-measure v is enumerable if the graph approximation set of v, {(r, x) E Q x A* 11' < v(x)} is c.e. and computable if the above set is computable.
Example 4.15. The function v : A*
---+
[0,1] defined by
v(x) = 2- lxl - 1 Q-lxl is a computable semi-measure.
Definition 4.16. Let
~
be a class of semi-measures. A semi-measure
Vo E ~ is called universal for ~ if for every semi-measure v E ~, there exists a constant c > (depending upon Vo and v) such that Vo (x) 2:: cv( x),
°
for all strings x E A *.
Theorem 4.17. The class of all enumerable semi-measures contains a universal semi-measure. Proof. Using a standard technique we can prove that the class of enumerable semi-measures is c.e., i.e. there exists a c.e. set TeN x Q x A * such that the sections Ti of T are exactly the graph approximations of the enumerable semi-measures. We denote by Vi the semi-measure whose graph approximation is Ti. Finally we put m(x) =
L n:;::O
Tn-1vn(x).
4.4 Algorithmic Coding Theorem
75
We first show that m is a semi-measure, i.e.
L
m(x) xEA'n~O
XEA*
n~O
r iff L:j=12-nj-lVnj(X» r, for some k 2:: 1,nl, ... ,nk 2:: o. Finally, m is universal since D
In what follows we fix a universal enumerable semi-measure m.
Theorem 4.18 (Algorithmic Coding Theorem). The following formulae are true:
H(x)
= -logQ P(x) + 0(1) = -logQ m(x) + 0(1).
Proof The equality
H(x) = -logQ P(x)
+ 0(1)
is exactly Theorem 4.6. So, we shall prove the formula 10gQ m(x) = 10gQ P(x)
+ 0(1).
Since P = Pu is an enumerable semi-measure and m is universal it follows that m(x) 2:: cP(x), for some positive natural c. To show the converse inequality we make use of the Kraft-Chaitin Theorem and we prove the inequality H(x) :s; -logQ m(x) + 0(1). To this end we consider an injective computable function f : N - t A * x N+ such that feN) = {(x, k) E A* x N+ I Q-k-l < m(x)}. We put f(t) = (:X;t, kt ). It is seen that
4. G.E. Instantaneous Codes
76
L
L
Q-k-l
XEA* Q-k<m(x)
Q-k-l xEA* k>-log Q m(x)
xEA* k~-lgQm(x)
L
QIgQm(x) /(Q
- 1)
xEA*
Q- n }.
L
-n
L
Q-n =
n>l-log Q P(x)
For every x we have Q
n2:1-lgP(x)
=
Q IgP(x) Q 1 -
< P(x),
so condition (i) in Theorem 4.22 is satisfied. Condition (ii) holds for c = 1. Hence by (4.29) we get 0:; LoM,p -7-{p =
L
P(x) . (HM(X)
+ 10gQ P(x))
:; 2.
D
x
Corollary 4.27. Assume that f : A * ---t N is a function such that the set {(x,n) I f(x) < n} is c.e. and Lx2-f(x) :; 1. Let P(x) = Q-f(x). Then P is a semi-measure semi-computable from below, and there exists a Chaitin computer M (depending upon f) such that for all x, HM(X) :S 1 + f(x).
( 4.33)
Minimal programs for M are almost optimal: the code C M satisfies the inequalities 0:; LoM,p - 7-{p :; 1. There exists a universal Chaitin computer U (depending upon f) such that the code Cu satisfies the inequalities
o :; Lou,P -
7-{p :; 2.
4. G.E. Instantaneous Codes
82
Proof We take S = {(x, n) In> f(x)}. Clearly, S = {(x, n) Q-n}. The first condition in Theorem 4.22 is satisfied as
L
I P(x) >
Q-n = P(x)1 :S P(X),
n>f(x)
Q-
for every x, and the second condition is satisfied for
C
= o.
o
Remark. When the semi-measure P is given, an optimal prefix-code can be found for P. However, that code may be far from optimal for a different semi-measure. For example, let A = {O, 1} and C be a prefixcode such that IC(x)1 = 2 Ixl +2 , for all x. Let a > 0 and consider the measure Two radically different situations appear: if a :S 1, then
but if a
> 1, then Le,p"" - 1ip""
Q-n} and then apply Theorem 4.22 to the set S = {(x, -lgP(x)) I x E A*} and constant c=O. 0 Corollary 4.30. Let P be a computable semi-measure. Then, there exists a universal Chaitin computer U such that
Hu(x) :S: 1 -logQ P(x). We are now in the position to characterize all Chaitin computers satisfying the Algorithmic Coding Theorem and to construct a class of (universal) Chait in computers for which the inequality is satisfied with constant c =
O. Proposition 4.31. Let M be a Chaitin computer and c lowing statements are equivalent:
~
O. The fol-
84
4. C.E. Instantaneous Codes ~
+ c) -logQ PM(X).
(a)
For all x, HM(X)
(b)
For all non-negative n, if PM (X) > Q-n, then HM(x)
Proof From HM(X)
~
(1
Q-n
(1
+ c) -logQ PM(x) < PM(X)
~
and PM(x)
~
n + c.
> 2- n we deduce
Q(1+c)-HM(X).
Conversely, we have D
Remark. For any Chaitin computer M satisfying one of the equivalent conditions in Proposition 4.31, the Algorithmic Coding Theorem holds:
(4.34) In fact, a Chaitin computer M satisfies (4.34) iff condition (b) is satisfied. Every universal Chaitin computer U satisfies condition (b), but not all Chaitin computers satisfy this condition. Indeed, to construct such an example, consider the following enumeration: for every string x enumerate Q1x l copies of the pair (x, 31xI + 1). Use the Kraft-Chaitin Theorem to construct a Chait in computer M such that for every string x there exist Q1x l different strings u~, all of length 31xI + 1, such that
M( u i) x -- x, 2. -- 1, 2, ••• , Q1xl • It is seen that PM(x) = Q- 2 Ix l- 1 , so taking nx = 21xI + 2 we get PM(x) > Q-n x , but there is no constant c such that HM(X) ~ nx + c, for all strings x.
Some Chait in computers satisfy condition (b) with c = 0, so their canonical programs are almost optimal. A class of (universal) such computers is provided in the next proposition. Proposition 4.32. Let M be a Chaitin computer such that for all pro-
grams x x,
I- x' with M(x) = M(x') we have Ixl I- Ix'i. Then, for all (4.35)
4.5 Binary
VB
Non-binary Coding (1)
Proof. Consider the set S
85
= {(x, Iyl) I M(y) = x}, PM(X) =
L
and note that
Q-n,
(x,n)ES
as programs producing the same output have different lengths. In view of the hypothesis,
PM(X) > Q-n
~
3(x,k 1 ) E S[(kl < n) V (k 1 1'13 k2(k2
-I kl
1\
=n
(x, k 2) E S))],
hence the second condition in Theorem 4.22 is satisfied with c = O. Using Theorem 4.22 we deduce the existence of a Chaitin computer M' such that HMI(x) :::;; 1 -logQ PMI(X), for all x. Inequality (4.35) follows from HM(x) = min{n I (x,n) E S} = HMI(x). 0
Remark. Not every universal Chaitin computer satisfies the hypothesis of Proposition 4.32. However, if V is a universal Chaitin computer, then one can effectively construct a universal Chaitin computer U such that programs producing the same output via U have different lengths and Hu(x) = Hv(x), for every x; Pu(x) :::;; Pv(x), for all x. Indeed, enumerate the graph of V and as soon as a pair (x, V (x)) appears in the list do not include in the list any pair (x', V(X')) with x -I x' and V(x) = V(x ' ). The set enumerated in this way, which is a subset of the graph of V, is the graph of the universal Chaitin computer U satisfying the required condition.
4.5
Binary vs Non-binary Coding (1)
The time has come to ask the following question: "Why did we choose to present the theory in an apparently more general setting, i.e. with respect to an arbitrary alphabet, not necessarily binary?" It seems that there is a widespread feeling that the binary case encompasses the whole strength and generality of coding phenomena, at least from an algorithmic point of view. For instance, Li and Vitanyi write in their book [282]:
[the} measure treated in the main text is universal in the sense that neither the restriction to binary objects to be described, nor the restriction to binary descriptions (programs) results in any loss of generality.
86
4. C.E. Instantaneous Codes
The problem is the following: does there exist a binary asymptotically optimal coding of all strings over an alphabet with q> 2 elements? Surprisingly, the answer is negative. We let q > p ~ 2 be naturals, and fix two alphabets, A, X, having q and p elements, respectively. The lengths of x E A * and y E X* will be denoted by IxlA and Iylx, respectively. We fix the universal computer 'Ij; : A* x A* ~ A* and the universal Chaitin computer U: A* x A* ~ A*. We denote by K the Kolmogorov-Chaitin complexity induced by 'Ij; and by H the Chaitin complexity associated with U. We shall prove that the following two problems have negative answers: 1. Does there exist a computer T/ : X* x A* ~ A* which is universal for the class of all computers acting on A *, i.e. a computer T/ for which there exists a constant c > 0 such that for every yEA *, if 'Ij;(x,),) = y, then T/(z,),) = y, for some Z E X* with Izlx :::; IxlA +c? 2. Does there exist a Chaitin computer C : X* x A* ~ A* which is universal for the class of all Chaitin computers acting on A *? We begin with a preliminary result. Lemma 4.33. Consider the function f(n)
f :N
---t
N defined by
= l(n + 1) logqpJ + 1.
gq P D ror every natura1 n> ll+10 1-1og q P J + 1 one has
Proof. Clearly, qf(n)
> pn+l. The inequality pn+l ~ pf(n)
is true for all natural n >
l1+
10g q PJ 1-1og qP
+ 1.
+ pn o
The next result says that complexities cannot be optimized better than linearly, i.e. the Invariance Theorem is the best possible result in this direction.
4.5 Binary
VB
Non-binary Coding (1)
87
Lemma 4.34. Fix a real number 0 < a < 1. There is no computer rJ : A* x A* ~ A* and no Chaitin computer C : A* x A* ~ A* such that for all computers
2 and g(n) = llogQ_l nJ, one has ' " Q-g(n)
6
n2:1
< Q -
1 '"
6
n2:1
.
n1ogQ-1Q
< 00 '
so H(string(n)) S llogQ-l nJ
+ 0(1).
Remark. Chaitin's complexity H can be characterized as a minimal function, semi-computable in the limit from above, that lies on the borderline between the convergence and the divergence of the series
L
Q-H(string(n)).
n2:0
We are now able to analyse the maximum Chaitin complexity of strings of a given length.
Theorem 5.4. For every n EN, one has max H(x)
XEAn
= n + H(string(n)) + 0(1).
Proof In view of Theorem 4.10, for every string x of length n,
+ 0(1) S H(string(n)) + H(x/ string(n)) + 0(1).
H(x) S H(string(n), x)
To get the relation max H(x) S n
xEAn
+ H(string(n)) + 0(1)
we shall prove that for every string x of length n,
H(x/ string(n)) S n + 0(1). We fix n 2:: 0 and define the Chaitin computer Cn : An
Cn(x, y) =
X
if U(y, >.) /::
A* ~ A* by
00,
for x E An, y E A*. Accordingly, U((string(n))*,>.)
H(x/string(n))
X
= string(n) and
< Hcn(x/string(n)) + 0(1) min{lzll z E A*, Cn(z, (string(n))*) = x}
+ 0(1) < n + 0(1).
5.2 Cbaitin's DeEnition of Random Strings
105
To prove the converse relation we need the following:
Intermediate Step. For every n 2: 0, #{x E An I H(x) < n + H(string(n)) - t + 0(1)}
m. The next example models the following simple idea (see the second example discussed in Section 5.1): if a binary string x has too many ones (zeros), then it cannot be random.
Example 5.10. The set
V -- {( x,m ) E A* x N+
I II;r Ni (x) -
1 I > Qm JlXT' 1 } Q
where Ni (x) is the number of occurrences of the letter ai in x, is a MartinLaf test. Proof Clearly, V is c.e. and satisfies condition 1). In view of the formula
# {X
E
An
I
I Ixl
Ni(X) _ ~I Q >
}
e;::;
Qn-2(Q -1) ne;2
'
one gets
# {X
E
I Ixl
An I Ni(x) _
~I
Q >
Qm_1 }
JlXT
Qn-2(Q _ 1) Q2m Qn-2-2m(Q _ 1) Qn-m
< Q-1'
D
Example 5.11. Let
0, then mV(cp)(x) = Ixl Kcp(x) - 1 S mV(1f;)(x) + t, i.e. mV(1f;)(x) ~ Ixl - Kcp(x) - (1 + t). If mV(1f;) (x) = 0, then K1f;(x)-d S Ixl S Kcp(x)+I+t. IfmV(1f;)(x) > 0, then mV(1f;)(x) = Ixl-K1f;(x) -1 ~ Ixl- Kcp(x) -t-l, so K1f;(x) S Kip(x) +t. We set c = d + 1 + t; then K1f;(x) S Kcp(x) + c. So, 'ljJ is a universal computer. 0
s
118
5. Random Strings
Theorem 5.26 (Martin-Lof asymptotical formula). Let 't/J be a universal computer and U be a universal Martin-Leif test. Then there exists a constant c (depending upon 't/J, U) such that for all x E A *
Ilxl- K1j;(x) -
mu(x)
I:::; c.
Proof In view of Theorem 5.25 ('t/J is a universal computer and U is a universal Martin-Lof test) we can pick q and t such that for all x E A *
and
mV(1j;)(x) :::; mu(x)
+ t.
We are now using Proposition 5.14. If mV(1j;) (x) = 0, then Ixl- K1j;(x)1 :::; and mu(x) ~ -t ~ -t + Ixl - K1j;(x) - 1. If mV(1j;) (x) i= 0, then mu(x) ~ mV(1j;)(x) - t = Ixl - K1j;(x) - 1 - t. Finally we take
°
c=max(q,1+t).
D
Theorem 5.27. We fix tEN. Almost all strings in RANDf will be declared eventually random by every Martin-Laf test.
Proof If x E RANDf, and 't/J is a universal computer, then K'I/J(x)
>
Ixl - T, for all natural T - O(logQ T) ~ t, by virtue of Corollary 5.8. We fix now a Martin-Lof test V. There exists a q > 1 such that
for all i = 1,2, ... ; so, x €/. VT+q' So, if random.
Ixl
~
T
+ q,
then V declares x D
Corollary 5.28. Every deficiency of randomness function 8 is bounded on every set RAN Df . Comment. Theorem 5.27 says that all Chaitin t-random strings pass all possible effective tests of stochasticity. We have here a first (and strong) argument supporting the adequacy of Chaitin's definition.
5.5 A Computational Analysis
5.5
119
A Computational Analysis
We pursue the analysis of the relevance of Chaitin's definition by confronting it with a natural, computational requirement: there should be no
algorithmic way to recognize which strings are random. First we show that the absolute complexity H is not computable. Theorem 5.29. There is no p.c. function
. (v) then m :::; Ivl + n + c + 1. Indeed, f(a~a2v)
In particular, if
Ivl = H(e),
Ivl + n +
c + 1.
= e and < w, m > EWe,
= 'Pstring(n) (v) = 00.
then m :::; H(e) + d, where d = c+ n + 1.
0
Next we get a stronger version of Theorem 5.31. Corollary 5.34. Ifg: N -+ N is a computable function withg(n) :::; n~t and limn-+oog(n) = 00, then {w E A* I H(w) > g(lwl)} is immune.
Proof Let We
C
{w
E
A* I H(w)
> g(lwl)}. We put
Ve = {< w,g(lwi)
> I w EWe}.
123
5.6 Borel Normality Clearly, Ve is c.e. and Ve A*. So, Ve
= Wf(e)
= Wf(e) , for some computable function f : A*
C
{< w,m >
E
A* x N
~
I H(w) > m}
and in view of Theorem 5.33 from < w, g(lwl) > E Ve = Wf(e) we deduce g(lwl) ~ H(f(e)) + d, i.e. Ve is finite. This shows that We itself is finite. D
Scholium 5.35. If 9 : N ~ N is a computable function which converges computably to infinity, limn->oo g( n) = 00, (i. e. there exists an increasing computable function r : N ~ N, such that if n :::: r(k), then g(n) :::: k) and the set S = {w E A* I H(w) > g(lwl)} is infinite, then S is effectively
zmmune. Proof. In the context of the proof of Corollary 5.34, #We if wE We C {u E A* I H(u) > g(lul)}, then
g(lwl)
< H(w)
= #Wf(e) and
~ H(f(e)) + d ~ If(e)1 + 2 log If(e)1 + d + c + c'.
If Iwl :::: r(lf(e)I+2log If(e)l+d+c+c'), then g(lwl) > If(e)I+2log If(e)l+ d + c + c', so w (j. We. Accordingly, if w E We, then Iwl < r(lf(e)1 + 2 log If(e)1 + d + c + c'), i.e. #We ~ (Qr(lf(e)I+21o glf (e)I+d+c+c') - l)/(Q -1),
and the upper bound is a computable function of e.
D
Corollary 5.36. For all t :::: 0, RANDf is effectively immune. Proof. An infinite subset of an effectively immune set is effectively imD mune.
Corollary 5.31. The set {< w,m
> IH(w)
~
m} is c.e., but not com-
putable.
5.6
Borel Normality
Another important restriction pertaining to a good definition of randomness concerns the frequency of letters and blocks of letters. In a "truly
5. Random Strings
124
random" string each letter has to appear with approximately the same frequency, namely Q-1. Moreover, the same property should extend to "reasonably long" substrings. Recall that Ni (x) is the number of occurrences of the letter ai in the string x; 1 .::; i .::; Q. We now fix an integer m > 1 and consider the alphabet B = Am = {Y1, ... , YQm} (#B = Qm). For every 1 .::; i .::; Qm we denote by N im the integer-valued function defined on B* by Nim(X) = the number of occurrences of Yi in the string x E B*. For example, we take A = {O, I}, m = 2, B = A2 = {OO, 01,10,11} = {Y1, Y2, Y3, Y4}, x = Y1Y3Y3Y4Y3 E B* (x = 0010101110 E A*). It is easy to see that Ixl2 = 5,lxl = 10, N'f(x) = 1, N?(x) = 0, Nl(x) = 3, Nl(x) = 1. Note that the string Y2 = 01 appears three times in x, but not in the right positions. Not every string x E A* belongs to B*. However, there is a possibility "to approximate" such a string by a string in B*. We proceed as follows. For x E A* and 1 .::; i .::; Ixl we denote by [x;i] the prefix of x of length Ixl-rem(lxl, i) (i.e. [x; i] is the longest prefix of x whose length is divisible by i). Clearly, [x; 1] = x and [x;i] E (Aj)*. We are now in a position to extend the functions NF from B* to A*: we put
in case
Ixl
is not divisible by
m.
Similarly,
Ixlm= l[x;m]lm. For x E Aoo and n ~ 1, x(n) = X1X2 ... Xn E A*, so Ni(x(n)) counts the number of occurrences of the letter ai in the prefix of length n of x.
Definition 5.38. A non-empty string x E A * is called c-limiting (c is a fixed positive real) if for all 1 .::; i .::; Q, x satisfies the inequality
Ni(X) - Q- 1 < 1
Ixl
1
- c.
(5.3)
Comments. i) Since 0 .::; Ni(x) .::; lxi, the left-hand side member of (5.3) is always less than (Q -l)/Q. ii) In the binary case Q = 2, a string x is c-limiting iff the inequality (5.3) is satisfied for some i = 1,2. This is because IN1(x)/lxl - 2- 1 1 = IN2(x)/lxl- 2- 11. Definition 5.39. Let c > 0 and m
~
1.
125
5.6 Borel Normality a) We say that a non-empty string x E A* is (c,m)-limiting if
for every 1 ::; i ::; Qm.
b) A non-empty string x E A* is called Borel (c, m)-normal if x is (c,j)-limiting, for every 1 ::; j ::; m.
Definition 5.40. i) A non-empty string x
A* is called m-limiting if x is
E
(V(1ogQ Ixl)/Ixl, m) -limiting, i.e.
for every 1 ::; i ::; Qm. ii) If for every natural m, 1 ::; m ::; logQ logQ say that x is Borel normal.
lxi, x
is m-limiting, then we
We now use a simple combinatorial formula (see Natanson [318]).
Fact 5.41. For all naturals i, m
Lemma 5.42. For every c
Proof In (5.4) put x
k
°
and real x
> 0, 1 ::; m ::; M
> 0,
and 1 ::; i ::; Qm,
= Q-m,i = lM/mJ:
LtJ (lM/mJ) ( k=O
~
k
_ Q-m)2lM/mJ2(Qm _ 1)LM/mJ-k
lM/mJ
= lM/mJQm LM/mJ-2m(Qm - 1).
5. Random Strings
126 Next define the set
On one hand:
#
{x E AM II [!;~j - Q-ml > c} = L #{x E AM I Nim(X) = k} =
kET Qrem(M,m).
L #{x E AM I Ni(x) = k} kET
=
Qrem(M,m).
L (lM{mJ) (Qm _l)LM/mJ-k. kET
On the other hand:
lM/mJQmLM/mJ-2m(Qm >
1)
L c2lM/mJ2 (lM/mJ) (Qm _l)LM/mJ-k kET
k
= c2lM/mJ2Q-rem(M,m)#{XEAM Remark.
I!~~~)
_Q-m\
>e}.
D
For every 1 :S m :S M, 1 :S i :S Qm,
Comment. In case m = 1 and 1 :S i :S Q, Nl(x) = Ni(X), formula (5.5) becomes #{XEAM
I!N~)
_Q-l!
>c}:S
QM-~~_l),
127
5.6 Borel Normality
and the inequality (5.6) reduces to
In view of Definition 5.40, a string x E AM is not Borel normal in case
for some 1 ~ m
~
logQ logQ M, 1 ~ i
~
Qm.
Lemma 5.43. We can effectively compute a natural N such that for all naturals M ~ N,
# {x E AM
I
x is not Borel normal} ~
QM
.
(5.7)
VlogQM Proof We put
s = {m E N I 1 ~ m ~ logQ logQ M}. Using formula (5.6) we perform the following computation:
#{ x
E AM I x is not Borel normal}
for sufficiently large M.
o
128
5. Random Strings
Corollary 5.44. There exists a natural N (which can be effectively computed) such that for all M ~ N one has
#{x E A* I N ~
Ixl
~ M,x is not Borel normal} ~
VQM+3 logQM
(5.8)
Proof By Lemma 5.43 we get a bound N for which the inequality (5.7) is true. Accordingly, using a proof by induction on M we can show the inequalities #{x E A* I N ~
Ixl
~
M,x is not Borel normal}
Qi
M
QM+3
~L ~~ i=N V logQ i VlogQ M
.
D
Theorem 5.45 (Calude). We can effectively find two natural constants c and M such that every x E A* with Ixl ~ M and which is not Borel normal satisfies the inequality
K(x)
~
1
Ixl- "2logQ logQ Ixl + c.
(5.9)
Proof We define the computable function f : N+ ---+ A* by f(t) = the tth string x (according to the quasi-lexicographical order) which is not Borel normal and has length greater than N. (The constant N comes from Corollary 5.44.) In view of (5.8),
t
0 it is enough to show that . vlOgQn Q-2> ___ , n
which is true because
and 1 2'H _Q 2
1
1 2'2+ l 1 > -2 =' 42 > 2i + _. - 2
2
o
Theorem 5.48 answers only one question concerning various (potential) possibilities to extend arbitrary strings to random/non-random strings. To settle all these questions we shall use a topological approach.
5.7 Extensions of Random Strings
5.7
131
Extensions of Random Strings
In this section we deal with the following problem: to what extent is it possible to extend an arbitrary string to a Chaitin random or non-random string? We shall use some topological arguments. Let < be a partial order on A* which is computable, i.e. the predicate "u < v" is computable. We denote by T( A * such that for all natural numbers i,j if s(i)
< x, s(j) < x, for some string x, then i =
j.
Then we can find a rare set which is not computably rare. Illustrate the above situation with examples. Show that the above condition is preserved under computable bijections. 25. Suppose that < is a computable and unbounded partial order on A * and for all strings x, yEA * there exists a string z with x < z, y < z. Then, i) every rare set is computably rare, ii) every non-rare set is dense. Illustrate the above situation with examples. 26. Prove that for every natural t, the set RAN Df is computably rare with respect to the topologies r( <m), r( oo #{x E B Ilxl ::::; n}/n = 0. For example, the set of all binary strings which have twice as many zeros as ones is meagre. Show that if B is computable and meagre, then for each natural t there are only finitely many strings x E B n RAN
Df .
30. Let f : A* x A* ----; [0,00) be a non-negative semi-computable function from below. Show that then
H(y/x) ::::; -logQ f(x, y) + 0(1), for all strings x E A* such that
I:yEA*
f(x, y) ::::; 1.
31. Let A = {O, I} and put for every string x E A* rx
= N 1 (x)/lxl,
1fx
= r~h(x)(l_ rx)No(x),
where Ni(x) is the number of occurrences of i = 0,1 in x. Next define the function 8 : A* ----; [0,00) by 8(x) = log2(1fx ) + Ixl -log2(1 + Ixl). a) Show that 8 is a deficiency of randomness function. b) Express 8 in terms of the entropy function h. (Hint: 8(x) = Ixl(lh(1fx)) -log2(1 + Ixl).) c) Show that there exists a constant c > such that if
°
Irx -
1/21 >
then K(x) ::::; IxI32. (Kramosil) Let (X n k:':l be a sequence of strings such that K(x n ) and IX n I = n. Show that for every natural n ~ 1 . Ni(xn) 11m n->oo
In/mJ --
Q-m
~
Ixnl-t
,
for every 1 ::::; i ::::; Qm. 33. (Chaitin) For every natural t, show that there are infinitely many natural n for which all strings of length n have the property
H(x) < Ixl + llogQ IxlJ - t. 34. Let
x
=
be a random string and random?
X1X2 ... Xn
X1aX2a ... aX n
a
E
A.
Is the string
35. Let x = X1X2 ... Xn be a random string over the binary alphabet {O, I}. Construct the new binary string y = Y1Y2· .. Yn, where Y1 = Xl, Yj = Xj EBXj-1, for j = 2,3, ... ,n and EB is the modulo-2 addition. Is Y random? 36. Show that the maximal number of consecutive ones in an m-random binary string of length n is m + o (10g2 n).
5.10 History of Results
145
37. (Staiger) If C c AQ and x E CW, then there is a constant c > 0 such that for all finite prefixes w
oo
lim #{x k->oo
EA* Ilxl ~ k,x does contain a} = 1 #{x E A* Ilxl ~ k}
which shows that almost all reals, when expressed in any scale Q ~ 2, contain every possible digit a E {O, 1, ... , Q - 1}. The case of strings of digits can be easy settled just by working with a large enough base. For instance, if the string 957 never occurs in the ordinary decimal for some number, then the digit 957 never occurs in base 1000. 0 Theorem 6.1 suggests that for sequences, like strings, randomness refers to typicality; in particular, "almost all sequences" should be "random". To go on we introduce some new notation. The set of all sequences over the alphabet A is denoted by AW, i.e. A W = {x I X=XIX2 ... Xn ... ,Xi E A}.
For every sequence x
=
XIX2 ... Xn ... E AW we put:
a) x(n) = XIX2 .. 'Xn E A*,n > 0, b) xm,n = xmxm+l ... x n , in case n remaining cases.
~
m
> 0, and
xm,n
= A, in the
For every x = Xl ... xm E A* and y = Yl'" Yn'" E AW we denote by xy the concatenation sequence Xl ... XmYl'" Yn' .. ; in particular, >.y = y. For X c A*, we put
6.1 From Random Strings to Random Sequences In case X is a singleton, i.e. X
149
= {x}, we write xAw instead of XAw.
Encouraged by Theorem 6.1 we may define a "random sequence" as a sequence whose prefixes are "c-random". Let us first interpret the above definition in terms of the complexity K: a sequence x E AW is "random" iff there exists a constant c such that for all natural n :2 1, K1fJ(x(n))
>n-
c.
Here 1jJ is a fixed universal computer. To get an image of the nature of the above definition let us consider the binary case (i.e. A = {a, 1}) and denote by Nf)(x) the number of successive zeros ending in position n of the sequence x. A result in classical probability theory says (see Feller [192]' p.210, problem 5) that with probability 1
N,n(x) = 1.
lim sup _0_ _ n-+oo log2 n
This means that for almost all sequences x E AW there exist infinitely many n for which x(n) :=::l x1,n-lognologn, l.e.
The above result suggests that there is no sequence satisfying the above condition of randomness. In fact, we shall prove that the above result is true for all sequences (not only with probability 1)! Hence, the complexity K is not an adequate instrument to define random sequences. We start with a technical result, which is interesting in itself.
Lemma 6.2. Let n(l), n(2), ... , n(k) be natural numbers, k :2 1. The following assertions are equivalent:
i) One has k
L Q-n(i) :2 1.
(6.1)
i=l
ii) One can effectively find k strings s(l), s(2), ... , s(k) in A * with Is(i)1 n(i), for all 1 SiS k and such that
=
k
U s(i)AW = AW. i=l
(6.2)
6. Random Sequences
150
Proof i) =} ii) We may assume that the numbers n(I), n(2), ... , n(k) are increasingly ordered: n(l) ::; n(2) ::; '" ::; n(k). In view of (6.1), the numbers n(I), n(2), ... ,n(k) are not all distinct. So we put n(l) = n(2) = ... = n(iI) = ml
< n(iI + 1) = n(iI + 2) = ... = m2 < ...
< n(iI + t2 + ... + tu-l + 1) = n(tl + t2 + ... + tu-l + 2) = ... = n(iI + t2 + ... + t u- l + tu) = mu' There are two distinct situations.
First Situation. One has tl ~ Qm l • In this case we take {s(I), s(2), ... ,s(Qml)} to be AmI, in lexicographical order. The remaining strings s(i) can be taken with Is(i)1 = n(i), because one has Qml
U s(i)AW = AW.
i=l
Second Situation. There exists a natural 2 ::; h .:::; u such that
and
tlQ-m l
+ t2Q-m 2 + '" + th_lQ-m h- 1 + thQ-m h ~ 1.
Multiplying by Qmh one can effectively find a natural 1 ::; t ::; th such that
We choose s(I), s(2), ... , S(tl) to be the first (in lexicographical order) strings of length ml. We have tl
U s(i)AW = UxAw, i=l
where x runs over the first iI Qmh -ml strings of length mh (in lexicographical order). The procedure continues in the same manner. Assume that we have already constructed the strings s(I), s(2), ... , S(tl) (of length ml), S(tl +
6.1 From Random Strings to Random Sequences
151
1), S(tl + 2), ... ,s(h + t2) (of length m2), ... , s(h + t2 + ... + ti-l + 1), s(h + t2 + ... + ti-l + 2), ... , s(h + t2 + ... + ti-l + ti) (of length mi), for i
< h. Suppose also that Ti
Us(j)AW= U xA j=l
w,
XEXi
where Xi consists of the first tl Qmh -ml + t2Qm h-m2 + ... + th-l Qmh -mi strings of length mi (in lexicographical order), and Ti = tl + t2 + ... + t i . In view of (6.3), the set Ami \ Xi is not empty. Let x be the first element (in lexicographical order) of the set Ami \ Xi. Then let y be the first (in lexicographical order) element of A mi+l -mi and S(Ti + 1) = xy. We construct the next strings of length mi+l (in lexicographical order):
if i
+ 1 < h,
and
s(Th- 1 + 1), S(Th-l
+ 2), ... , s(Th- 1 + t) = s(h + t2 + .,. + th-l + t),
if i = h - 1. It is seen that T
Us(j)AW = AW, j=l
where T = h +t2+" ·+th-l +t, again by virtue of (6.3). So, if k > T, the remaining strings s(i), i > k, can be taken arbitrarily with the condition Is(i)1 = h(i); the property (6.2) will hold true.
ii)
*
i) Again assume that n(l) S n(2) S '" S n(k), and put Ji
= {x
I s(i)
E An(k)
ex},
1 SiS k. Condition (6.2) implies that k
An(k)
U Ji
C
i=l
and this in turn implies the inequality k
L #Ji i=l
2: #An(k).
152
6. Random Sequences
This means that
Lk Qn(k)-n(i)
>
Qn(k),
i=l
o
which is exactly (6.1).
Definition 6.3. A p.c. function F : N ~ N is said to be small if 00
L
Q-F(n)
= 00.
n=O
Example 6.4. a) Let kEN. The constant function F : N by F(n) = k, for all n E N, is a small function.
-+
N given
b) Take a to be a strictly positive rational, a < 1 or a 2:: Q. The p. c. function F (n) = lloga n J, for n 2:: 1, is a small function. In particular, F( n) = llogQ n J is small. Lemma 6.5. Let F be a small function and let k be an integer such that F(n) + k 2:: 0, for all n E dom(F). We define the function F + k : dom(F) -+ N by (F+k)(n) = F(n)+k. Then, F+k is a small function. Lemma 6.6. Let 9 be a small function with a computable graph. Then one can effectively find another small function G with a computable domain such that: a)
The function 9 extends G.
b)
For every n E dom(G) one has G(n) S n.
c)
For every natural k there exists at most one natural n with G(n) = n-k.
Proof. We define the p.c. function G : N ~ N as follows: g(n), G(n)
=
{ 00,
if g(n) S nand m - g(m) for every natural m < n, otherwise.
=1=
n - g(n),
Since 9 has a computable graph, it follows that all conditions in the above definition are computable and G satisfies the above three requirements.
6.1 From Random Strings to Random Sequences
153
In particular, G has a computable graph. It remains to be proven that G has a computable domain and 00
L
Q-G(n)
=
00.
(6.4)
n=O To this end we define the sets
x = {n E N I g(n) :::; n}, Xk Notice that X
=
= {n E N I g(n) = n - k}, kEN.
U~O
Xk and the sets Xk are pairwise disjoint. Because
9 is small and
L
< 00
Q-g(n)
nEN\X
one has
L
Q-g(n)
= 00,
nEX which means that
L L
= 00,
Q-g(n)
(6.5)
kEYnEX" where Y
= {k
E N I Xk
=I 0}.
For every kEY we denote by nk the smallest element of X k . Then dom(G) = {nk I Xk i= 0}. So,
G(n)
< 00 iff G(n) :::; m, for some m :::; n.
Accordingly, dom( G) is computable. We put a
=
L
L
Q-g(n) ,
kEY nEX" \ {nl (n +
v'n)/2.
The above properties are asymptotic, in the sense that the infinite behaviour of a sequence x determines if x does or does not have such a property. Kolmogorov has proven a result (known as the All or Nothing Law) stating that practically any conceivable property is true or false almost everywhere with respect to f.t. It is clear that a sequence satisfying a property false almost everywhere with respect to f.t is very "particular". Accordingly, it is tempting to try to say that a sequence x is "random" iff it satisfies every property true almost everywhere with respect to f.t. Unfortunately, we may define for every sequence x the property P x as follows y satisfies Px iff for every n ~ 1 there exists a natural m ~ n
such that
Xm
i= Ym·
Every Px is an asymptotic property which is true almost everywhere with respect to f.t and x does not have property P x . Accordingly, no sequence can verify all properties true almost everywhere with respect to f.t. The above definition is vacuous! The above analysis may suggest that there is no truly lawless sequence. Indeed, a "universal" non-trivial property shared by all sequences was discovered by van der Waerden (see for example [214]): In every binary sequence at least one of the two symbols must occur in arithmetical progressions of every length. Looking at the proof of van der Waerden's result (and of a few similar ones) we notice that they are all non-constructive. To be more precise,
172
6. Random Sequences
there is no algorithm which will tell in a finite amount of time which alternative is true: 0 occurs in arithmetical progressions of every length or 1 occurs in arithmetical progressions of every length. However, there is a way to overcome the above difficulty: We consider not all asymptotic properties true almost everywhere with respect to f.t, but only a sequence of such properties. So, the important question becomes: "What sequences of properties should be considered?" Clearly, the "larger" the chosen sequence of properties is, the "more random" will be the sequences satisfying that sequence of properties. In the context of our discussion a constructive selection criterion seems to be quite natural. Accordingly, we will impose the minimal computational restriction on objects, i.e. each set of strings will be c.e., and every convergent process will be regulated by a computable function. As a result, constructive variants of open and null sets will playa crucial role. Consider the compact topological space (AW, 'T) used in the topological proof of Theorem 6.20. The basic open sets are exactly the sets xAw, with x E A *. Accordingly, an open set G c A W is of the form G = X A W , where X c A*.
Definition 6.26. a) A constructively open set G set G = XA W for which Xc A* is c.e.
c AW
is an open
b) A constructive sequence of constructively open sets, for short, c.s.c.o. sets, is a sequence (G m )m2:1 of constructively open sets G m = XmAw such that there exists a c. e. set X c A * x N with Xm
= {x E A* I (x,m) EX},
for all natural m ::::: 1.
c) A constructively null set S c.s.c.o. sets (G m )m2:1 for which
c
A W is a set such that there exists a
and
lim f.t( G m )
m-+oo
= 0,
constructively,
i. e. there exists an increasing, unbounded, computable function H : N N such that f.t(G m ) < Q-k /(Q -1) whenever m::::: H(k).
---+
6.3 Characterizations of Random Sequences
173
It is clear that J-l(S) = 0, for every constructive null set, but the converse is not true.
Our first example of a constructive null set is a strong form of the Law of Large Numbers: we will show that the set of binary sequences not satisfying the relation limn-too Sn(x)/n = ~ is not only a null set but also a constructive null set. Theorem 6.27 (Constructive Law of Large Numbers). Let A {O, I}. Then, the set y
=
{x
E AW
I
lim n-too
Xl + X2 + ... + Xn i= ~} n
2
is a constructive null set. Proof. We will use Chernoff's bound: for every non-negative integer t there exists a rational qt E (0, 1) such that for all n we have
J-l({XEAW Then
J-l ({x
E
AW
II Xl +
X2
II XI+X2:",+ + ... + Xn n
xn
-~I ~
t}) S2qr·
~I ~ ~,for some n ~ k})
-
2
t
S 2qf . 1 - qt
Given non-negative integers m, t we can effectively find the smallest k such that 2qk 1 _ _ t_ < __ I - qt - m2t' which we will denote by km,t. Hence
J-l
(U
oo {
t=l
X
E AW I
IXl +
X2
III})
+n ... + Xn -"2 ~ t' for 00
1
some n ~ km,t
1
(3n - 1)/2, then taking q = n in (6.18) we get al = Yn = Yn+k = a2, a contradiction. Case 3: 1 S k S n - 1. We consider the equality Yq as follows: 1.
for q = n
2.
for q
3.
+1-
k, giving al
= Yq+k from (6.18)
= Xl,
= n + 1, giving Xl = xk+l, for q = n + k + 1, giving xk+l = Yn+1+2k (in case of the validity of the previous two equalities).
There are two possibilities according to the relation between k and (n1)/2. i) If k > (n - 1)/2, then from Xk+1 = Yn+1+2k we deduce Xk = a2 and one of these equalities is false. ii) If k S (n - 1)/2, then we consider the natural t satisfying the inequalities k
+ 1 + tk S n, k + 1 + (t + l)k > n.
Recalling the equalities already obtained,
we take successively
189
6.4 Properties of Random Sequences q=n q=n
q=n
+ 1 + 2k, + 1 + 3k,
to get X2k+1 to get X3k+1
= X3k+l,
= X4k+l,
q = n + 1 + tk, to get Xtk+l = X(t+l)k+l, + 1 + (t + 1)k, to get (assuming all previous equalities)
o
The last equality is false.
We are now going to set a piece of new notation. Let n E N + and c E A + . We define the set
M(n,c) Of course, M(O, c)
= M(1, c)
= {x E An I C 2 one has nn < 2n(n - 1)n-l, because the last inequality may be equivalently rewritten as nn(n - 1)-n < 2n j(n - 1), and the sequence nn(n - 1)-n decreases to e. Its maximum value is 4 (for n = 2) and 4 < 2n j(n - 1), for n ~ 3. Consider the function F: R ~ R given by F(x) = x n _qx n- 1 +1. Assume n ~ 3 (the case n = 2 is obvious). The derivative F' has the roots 0, t. In case n is odd, the sign of the derivative gives three solutions for the equation F(x) = 0, lying in the intervals (-q, 0), (0, t) and (t, q). In case n is even, one has two solutions in the intervals (0, t), (t, q). Therefore, the assertion is true for real roots (we have constantly used the Intermediate Step). We now prove that every non-real root z has Izl take a non-real root z = p(cosB + i sin B), where p satisfies the equation we get
< 2. To this end we = 14 Writing that z
p = q sin(n - 1)B j sin nB
(we have sin nB
i=
°
since otherwise sin(n - 1)B
= 0,
i.e. sin B = 0).
Using this value for p and again writing that z verifies the equation we get pn = sin(n - 1)BjsinB. But, IsinkBI ::; klsinBI, for every natural k. So, assuming p ~ 2 we get the false inequality 2n ::; n - 1. Thus, p < 2 ::; q. 0
Remark. One can show that for n strictly less than 1.
> 2, the non-real roots have modulus
Fact 6.49. For every unbordered string c E A * of length a natural M such that for every m ~ M
~
3, there exists
(6.22)
192
6. Random Sequences
Proof We put L = lei 2': 3. In view of Fact 6.47 there exist L complex numbers al, ... ,aL such that for every natural n 2': 1 L
R(m, c) =
L ai A7, i=l
where AI, ... ,AL are the (simple) roots of the equation xL_Qx L - l +1 = For every 1 SiS L,
o.
IAdQI < 1, so
Accordingly, from Fact 6.48, we can find a natural M
for all m 2': M,1 SiS L, which implies (6.22).
> 0 such that
o
Theorem 6.50 (Calude-Chitescu). Every non-empty string occurs infinitely many times in every random sequence. Proof We proceed by contradiction. Let x be a sequence having the property that some string y does not occur infinitely many times in x. We shall prove that x rt rand. Deleting, if necessary, an initial string from x (using Theorem 6.40) we may assume that y does not occur in x. In view of Fact 6.46 there exists an unbordered string cof length L = Icl 2': 3 with Y Q-1, for some 1 SiS Q. Q
1 = limninf(x~
+ x~ + .. , + x~) :2: '~ " limninf x~ > 1, j=l
195
6.4 Properties of Random Sequences a contradiction.
D
Lemma 6.55. If for every 1 SiS Q,
lim inf x~ = n
Q-1,
then for all 1 SiS Q,
Proof Assume, by absurdity, that lim infn x~ i= lim sUPn x~, for some 1 SiS Q, i.e. there exists a 8 > 0 such that lim sUPn x~ = Q-l + 8. Since liminfn(-x~) = -limsuPnx~, it follows that liminf(1n
x~)
= 1+
liminf(-x~) n
=
1-limsupx~ = n
QQ-1 - 8.
On the other hand, lim inf(1 - x~) n
liminf x~ j=l,#i
n
Q-1
=
Q Q-1 ---8
>
Q
'
a contradiction.
D
First we deal with the case m = 1. For every sequence x E AW we consider the sequences ,i=1, ... ,Q ( Ni(x(n))) n n:2:1 which satisfy the conditions in Lemma 6.54 and Lemma 6.55. So, in order to prove that lim Ni(x(n)) = Q-l, n->oo
whenever x is random, it suffices to show that · . f Ni(x(n)) 1Imln n
n
> Q-l _,
196
6. Random Sequences
for every 1 :S i :S Q. Assume, by absurdity, that there exists an i, 1 :S i :S Q, such that · III . f Ni(x(n)) 11m
n
n
< Q-1 •
Elementary reasoning shows that the set
1 Ni(x(n)) } { n> 1 I >f Q n is infinite, for some rational, small enough Consider now the computable set S
c
f
> 0.
A* x N+:
(6.24) Clearly, x E SnAw, for infinitely many n (here Sn = {y E A* I (y, n) E S}). Using Theorem 6.37 now, it is clear that all that remains to show reduces to the convergence of the series \
when S comes from (6.24). A combinatorial argument (Sn C An ) shows that p'(SnAW)
=
(~) (Q _1)n-k
I:
Q-n.
{kEN I O::;k Q-(n+i+ 1 ). The sets (Oy), Y E An+i+1, are disjoint, so
U
1 = J-L(AW) > J-L(
Oy)
iyi=n+i+l
L
J-L(Oy)
iyi=n+i+l
Q-(n+i+ L iyi=n+i+l
>
1)
1, a contradiction. We now fix n E N and let C = AW
\
U 0Yn,i' i2':O
The set C is closed (but not constructively closed). Next,
1-
L Q-(n+i+
1)
1- Q-n /(Q - 1)
> 1- Q-n. Let F : Aoo ~ Aoo be a computable function. From Lemma 6.72 there exists a computable and increasing function gi : A* ~ A* such that F(x) = Gi(x), for all x E AW. Finally, C n 0Yn,i = 0 implies Gi(C) n Yn,iAw = 0, i.e. F(C) nYn,iAw = 0; this shows that F(C) =I AW. D
219
6.5 The Reducibility Theorem Next we show that the quantitative condition f.l( C)
> kOI is not necessary.
Proposition 6.91. Assume that Q > 2 and let
B = {aI, a2, ... , aQ-d c A and C = B WcAw. Then C is a constructive null set and there is a process (which can be effectively constructed) F : AW -+ A W such that F( C) = AW. Proof A straightforward computation shows that f.l(C)
1 - f.l(B*aQA W) 1 - f.l(
U BnaQAW) n:2:0
o. Next we define the computable functions G : A * -+ {aI, a2}* and 9 : {aI, a2}* -+ A* as follows: G is a monoid morphism acting on generators by G(ai) = ai,i = 1,2,G(ai) = )..,2 < is Q, and
g(x)
=
{
aiG(y), in case x = WiY, 1 SiS Q, Y E {al,a2}*, A, otherwise.
Here WI
= aI, W2 = a2al, ... , WQ-I = a~-lal' WQ = a~.
The definition of 9 is correct since the set {Wi 11 SiS Q} is prefix-free (more exactly, for every x E {aI, a2}W there exists a unique 1 SiS Q such that x E wi{al,a2}W). We define the computable function F : A * -+ A * , F (x) = g( G (x)). Clearly, F is prefix-increasing, so according to Lemma 6.71 the extension F : A 00 -+ A 00 is a process. For every
we can construct the sequence
for which the following relations hold true: F(x)
= sup{F(x(n)) I n ~ I} = sup{g(x(n)) I n
~
I} = y.
0
6. Random Sequences
220
Is it possible to replace the measure-theoretical condition in Theorem 6.73 by a more general condition not involving the measure? The answer is affirmative and a result in this sense will be presented in what follows. Let L; and r be two fixed alphabets having p and q elements, respectively. If X c L;OO and n E N, the set {y E L;n I X n yL;oo =I- 0} will be denoted by x[nl.
Definition 6.92. Let 9 : N -+ N be an increasing function and h : N -+ N be a function with h(n) ;:: 2, for all n E N. A set X C L;w is called a (g, h)-Cantor set if it is non-empty and for each n E N and each x E x[g(n)l we have
# (xL;w n x[g(n+1)l) ;:: h(n + 1). A set X C L;w is called a computably growing Cantor set if there is a computable increasing function 9 : N -+ N such that X is a (g, 2)-Cantor set; here 2 is the constant function h( n) = 2.
The main result is the following stronger form of reducibility:
Theorem 6.93 (Hertling). Let 9 : N -+ Nand h : N -+ N be two increasing computable functions with g(O) = h(O) = O. Let C c L;W be a constructively closed set which contains a (g, n 1--+ qh(n+1)-h(n))-Cantor set. Then there is a process F : L;OO -+ roo satisfying the following two conditions: 1.
F(C) = rw.
2.
For all n E N and all non-terminal strings for F, x E E* with Ixl ;:: g(n), we have If(x)1 ;:: h(n).
Before presenting the proof we will state the following important consequence:
Corollary 6.94. Let C c L;w be a constructively closed set which contains a computably growing Cantor set. Then there is a process F : L;OO -+ roo with F( C) = rw. Proof Assume that 9 : N -+ N is a computable increasing function and X c C is a (g,2)-Cantor set. Let c E N be a number with 2 c ;:: q.
6.5 The Reducibility Theorem
221
We define two functions g, h : N -+ N by g(O) = 0, g(n) g(c . n), for n > 0, and h(n) = n for all n. These functions are computable, increasing and satisfy g(O) = h(O) = O. The set X is a (g,2 C )-Cantor set, hence a (g, n f--t qh(n+I)-h(n))_Cantor set. The corollary follows from Theorem 6.93. 0 We continue with the proof of Theorem 6.93. Let wo, WI, W2, computable sequence of strings in ~* with
For tEN we define
Ct
= ~w \
...
be a
U Wk~w. k t.
Proof If x E Mr, then x E Lr. By (1) and (3) we get x E Lr+1' With x rj. M!+l we conclude x rj. Dr+1' Lemma 6.95.4 implies x rj. D~, for any s
> t.
0
Corollary 6.98. For each n E N there is atE N with M:" = MF, for
all s 2 t and m ::; n. Proof The assertion follows from Lemma 6.97 and the fact that each set M:" is a subset of the finite set L;g(m). 0 We define the function s : N
s(n)
= min{t E N
--t
I M;.n
N by
= MF for
all r 2 t and m::; n}.
Property (3) implies that If(x)1 ::; h(m) for all x E L;g(m) , mEN. Hence, the function f coincides with fs(n+1) on the sets M~n+1) and M~~~l) and If(x)1 = h(n) for x E M~n+l)' for any n E N. Applying (5) to s(n + 1) we deduce that for each x E M~n+l)' the function f maps the set xL;oo n M~~~l) bijectively onto the set f(x)r OO n r h(n+l). Note that M~n+l) = M~n)' We claim that for each n E N,
f maps L;g(n) n
n M;(m)L;w bijectively onto rh(n).
(6.30)
m~n
M2
= {.\} for all This is clear for n = 0 because g(O) = h(O) = 0 and = {A}). Assume that it is true for n. We have proved that
t (Mg(o)
for each x E L;g(n) n nm~n M;(m)L;w the function f maps xL;oo n M~~~l) bijectively onto the set f(x)r OO nrh(n+l). This gives the claim (6.30) for n+l. We define the set Y c L;w by Y = nnM~n)L;w. By (6.30), f maps Y bijectively onto rw. We claim that Y c C. Let x E Y. Then for every n, x(g(n)) E M~n) c D~(n)' Hence, x(g(n))L;W n Cs(n) -I 0, so x(g(n))L;WnC -10. Since C is constructively closed we deduce that x E C and thus Y c C. This completes the proof of the relation f (C) = r w, hence of Theorem 6.93. 0
229
6.6 The Randomness Hypothesis
Comment. Let ~ be a finite alphabet. Every constructively closed subset of ~w with positive measure contains a computably growing Cantor set. Hence, we can apply Corollary 6.94 in order to obtain for any constructively closed set C c ~w a process F with FC) = r w , i.e. Theorem 6.73 follows. A sharper constructive result appears in Exercise 6.7.21.
6.6
The Randomness Hypothesis
Some other equivalent definitions of random sequences have been proposed by various authors. In this section we will briefly review some of these characterizations and the "randomness hypothesis" will be stated. A very interesting approach to randomness, a topological one, has been proposed by Hertling and Weihrauch [235]. We present the main ideas here. A randomness space is a triple (X, B, p,), where X is a topological space, B, a map from N to the power set of X, is a total numbering of a subbase of the topology of X, and p, is a measure defined on the (jalgebra generated by the topology of X.5 Let (Wn)n be a sequence of open subsets of X; a sequence (Vn)n of open subsets of X is called Wcomputable if there is a c.e. set A c N such that Vn = U7r(n,i)EA Wi for all n E N.6 Next we define W[ = W'(i) = njED(1+i) Wj , for all i E N; here D : N ........ {E lEe N is finite} is the bijection defined by
D-1(E)
= I:2i. iEE
Note that if B is a numbering of a subbase of a topology, then B' is a numbering of a base of the same topology. A randomness test on X is a B'-computable sequence (Wn)n of open sets with p,(Wn) ~ 2- n, for all n E N. An element x E X is called random if x rf. nnEN Wn , for every randomness test (Wn)n on X. The simplest example of randomness space is (~, B, p,), where ~ = {so, ... ,sd is a finite, non-empty set, the numbering B is given by Bi = {sd for i ~ k and Bi = X for i > k, and the measure p, is given by p,( {Si}) = k~l. Notice that p, is a probability measure. Every 5Recall that a subbase of a topology is a set (3 of open sets such that the sets WEE W, for finite, non-empty sets E c (3, form a basis of the topology. 67r(n, i) is a computable bijection; for example, 7r(n, i) = (n + i)(n + i + 1)/2 + i.
n
230
6. Random Sequences
element of is at least
~
is random because the measure of any non-empty open set
k!l'
Consider now the topological space AW (where A comes equipped with the discrete topology and AW is endowed with the product topology) and the numbering B of a subbase (in fact a base) of the topology is given by Bi
= (i)AW = {x E A W I string(i)
0 such that K(x(n)) infinitely many natural n, then x is random. 16. Let
f :N
--+
~
n - c, for
N be a function such that the series 00
LQ-f(n) n=l
is convergent. Show that the set {x E A W I K(x(n))
~
n - fen), for all but finitely many n}
has measure one. 17. Let
f :N
--+
N be a computable function such that the series 00
L
Q-f(n)
n=l
is constructively convergent. If the sequence x is random, then K(x(n)) n - fen) for all but finitely many natural n.
~
18. Show that the set {x E AW I there is a natural c such that K(x(n)) > n - c, for infinitely many n} has measure one.
6.8 History of Results
233
19. A p.c. function tp : A* ~ A* is called a monotonic function (Zvonkin and Levin [455]) or a process (Schnorr [359]) if tp(x)
0 there exist a constant c and a process F : ~oo --+ roo with F( C) = rw and
IF(x)1 :2: logqp. Ixl -
(2 + c) .logqp.
for all non-terminating strings x
6.8
E ~+
Vlxl
.logp Ixl
-
c,
for F.
History of Results
Borel [40, 41] was probably the first author who systematically studied the random sequences. He was followed by von Mises who - starting in 1919 - tried to base probability theory on random sequences (Kollectives) [421, 422]. Von Mises' path has been followed by many authors, notably Church [141] and Wald [427]; see also Ville [418]. The oscillation of the complexity of strings in arbitrary sequences was discovered by Chaitin [111] and Martin-Lof [304]; for alternative proofs see Katseff [248], and Calude and Chitescu [71] (our presentation follows [71]). Various equivalent definitions of random sequences come from Martin-LM [302, 301]' Chaitin [110, 111, 113, 114, 118, 121,122, 123, 125], Solovay (quoted in [121]), Schnorr [360], Levin [277] and Gacs [200]. Independent
234
6. Random Sequences
proofs of the equivalence between Martin-Lof and Chaitin definitions have been obtained by Schnorr and Solovay, cf. [121, 133]. Martin-Lof [302] has proven that - in a constructive measure-theoretical sense - almost all sequences are random; the computational and topological properties of random sequences come from Calude and Chitescu [72, 69]. For more facts concerning the property of Borel normality see Copeland and Erdos [146], Kuipers and Niederreiter [268] and Niven and Zuckerman [320]. Chait in [111] investigated the Borel normality property for the first time for random sequences; he proved that any Omega Number is Borel normal in any base; this result was generalized for all numbers having a random sequence of digits in Calude [53]; see also Campeanu [108]. The Reducibility Theorem is due to Kucera [265] and Gacs [202]; we have followed the proof in Mandoiu [295]. Theorem 6.93 was proved by Hertling [232]. Chaitin's Omega Numbers - discovered by Chaitin in [114]- are the first "concrete" examples of numbers having a random binary expansion. Omega Numbers have received a great deal of attention; see, for instance, Barrow [15], Bennett and Gardner [32]' Casti [103, 104]' Davies [155]. We will devote most parts of Chapters 7 and 8 to Omega Numbers. Exercises 6.7.4-8 come from Calude and Chitescu [71]. We have followed Martin-Lof [304] for Exercises 6.7.15-18 and Gacs [202] for Exercise 6.7.20. Exercise 6.7.21 comes from Hertling [232]. More details can be found in Arslanov [6], Calude [51], Calude and Chitescu [69], Chaitin [110, 111, 114, 118, 121, 122, 123]' Calude, Hromkovic [86], Davie [154]' Cover [150]' Cover, Gacs and Gray [151]' Dellacherie [166]' Fine [197]' Gacs [201, 203], Gewirtz [208], Khoussainov [253], Knuth l255], Kolmogorov and Uspensky [261]' Kramosil [263], Kramosil and Sindelar [264], Levin [277, 278], Li and Vitanyi [280, 282]' Marandijan [297], Martin-Lof [301, 302]' Mendes-France [311], Schnorr [359, 361], Sipser [367], Svozil [391], van Lambalgen [411, 412], von Mises [421,422], Vereshchagin [415] and Zvonkin and Levin [455]. The randomness hypothesis has been proposed and discussed by Delahaye [164], and, independently, by Calude [59]. Interesting non-technical discussions pertaining to randomness in general and random sequences in particular, may be found in Barrow [15], Beltrami [25], Bennett and Gardner [32], Casti [103, 104]' Chown [139, 139], Davies [155], Davies and Gribbin [156]' Davis [157], Davis and Hersh [160]' Delahaye [165]' Pagels [328]' Paulos [329]' Rucker [349, 350]' Ruelle [351]'
6.8 History of Results
235
Stewart [380] and Tymoczko [406]. More references and applications will be cited in Chapter 9.
Chapter 7
Computably Enumerable Random Reals Not everything that can be counted counts, and not everything that counts can be counted. Albert Einstein
In this chapter we will introduce and study the class of c.e. random realso A key result will show that this class coincides with the class of all Chaitin's Omega Numbers.
7.1
Chaitin's Omega Number
In this section we briefly study Chaitin's random number Ou representing the halting probability of a universal Chaitin computer U)". Recall that Ou= uEdom(U;,.}
is the halting probability of a universal Chait in computer U with null-free data (= >.). In contexts in which there is no danger of confusion we will write U, M, C instead of U)", M)", C)". Let AQ
= {a, 1, 2, ... , Q-l} and f:
N+
--7
A* be an injective computable
7. C.E. Random Reals
238
function such that f(N+) = dom(U>..) and put k
Wk
=L
Q-lf(i)l.
(7.1)
i=l
It is clear that the sequence (Wk)k::::O increasingly converges to O.
Let
o = Ou = 0.0 1 0 2 ... On . .. be the non-terminating base Q expansion of 0 (at this moment we do not know that 0 is actually an irrational number!) and put
Lemma 1.1. If Wn 2: O(i), then
O(i) ::; Wn < 0 < O(i)
+ Q-i.
Proof. The inequalities follow from the following simple fact: 00
Q-i
L
2:
OjQ-j,
j=i+1
o
as OJ E {O, 1,2, ... ,Q -I}.
Theorem 1.2 (Chaitin). The sequence rQ(O) E AQ is random. Proof. We define a Chaitin computer M as follows: given x E A * we compute y = U(x) and the smallest number (if it exists) t with Wt 2: O.y. Let M(x) be the first (in qua~i-lexicographical order) string not belonging to the set {U(f(l)), U(f(2)), ... ,U(f(t))} if both y and t exist, and M(x) = 00 if U(x) = 00 or t does not exist. If M(x) < 00 and x' is a string with U(x) = U(x'), then M(x) = M(x'). Applying this to an arbitrary x with M(x) < 00 and to the canonical program x' = (U(x))* of U(x) yields
HM(M(x)) ::;
Ix'i = Hu(U(x)).
(7.2)
Furthermore, by the universality of U there is a constant c> 0 with
Hu(M(x)) ::; HM(M(x))
+c
(7.3)
7.1 Chaitin's Omega Number for all x with M(x) a string with
239
< 00. Now, we fix a number n and assume that x is
Then M(x) < 00. Let t be the smallest number (computed in the second step of M) with Wt 2:: 0.0102'" On. Using Lemma 7.1 we have 0.0 10 2 " , On
< Wt
(7.4) 00
< Wt+
Q-lf(s)1 2: s=t+1
Ou
oo
follows from limn->oo an
= a and
an
=1
< a, for all n.
o
In order to compare the information contents of c.e. reals, Solovay [375] (see also Chaitin [118]) has introduced the following definition.
7. C.E. Random Reals
274
Definition 1.52 (Solovay). The real a is said to dominate the real /3 if there are a partially computable function f : Q ~ Q and a constant c > 0 with the property that if p is a rational number less than a, then f (p) is (defined and) less than /3, and it satisfies the inequality
c· (a - p)
~
/3 -
f(p).
In this case we write a ~dom /3 or /3 5:.dom a. The relation the Solovay domination relation.
Sdom
is called
Roughly speaking, a real a dominates a real /3 if from any good approximation to a from below (say, from a rational number p < a with a - p < 2- n ) one can effectively obtain a good approximation to /3 from below (a rational number f(p) < /3 with /3 - f(p) < 2-n+constant). For c.e. reals this can also be expressed as follows.
Lemma 1.53. A c.e. real a dominates a c.e. real/3 iff there are computable, increasing (or non-decreasing) sequences (a i) and (b i ) of rationals and a constant c with lim n -+ oo an = a, lim n -+ oo bn = /3, and c(a - an) ~ /3 - bn , for all n. Proof First, we assume that a dominates /3. Let (an) and (b n ) be increasing, computable sequence of rationals converging to a and /3, respectively. Since a dominates /3 there are a constant c > 0 and an increasing, total computa~le function 9 : N -+ N with c( a - an) ~ /3 - bg(n) , for all n. We
set bn
=
bg(n)'
On the other hand, assume now that (an) and (b n ) are computable, nondecreasing sequences converging to a and to /3, respectively, and that c > 0 is a rational constant such that c( a - an) ~ /3 - bn , for all n. The sequences (an) and (b n ) defined by an = an - 2- n and bn = bn - c2- n are computable, increasing, converge to a and to /3, respectively, and satisfy c(a - an) ~ /3 - bn , for all n. We define a partially computable function f : Q ~ Q as follows. Given p E Q, compute the smallest i such that ai ~ p. If such an i has been found, set f(p) = bi . If P < a, then f(p) is defined and is smaller than /3. It is clear that this function f shows /3 5:.dom a. 0
Next we prove a few results about the structure of c.e. reals under
Sdom'
7.5 G.E. Reals, Domination and Degrees
275
Lemma 7.54. Let a, (3 and, be c. e. reals. Then the following conditions hold: 1.
The relation '2dom is reflexive and transitive.
2.
For every a, (3 one has a
3.
If, '2dom a and, '2dom (3, then, '2dom a
4.
For every non-negative a and positive (3 one has a . (3 '2dom a.
5.
If a and (3 are non-negative, and, '2dom a and, '2dom (3, then , '2dom a . (3.
+ (3 '2dom a. + (3.
Proof. The statement 1 follows from the definition. For 2 we consider a rational number P < a + (3 and we can compute two rational numbers PI,P2 such that PI < a, P2 < (3 and PI + P2 '2 P because a and (3 are c.e. reals. Now a+(3-p '2 a+(3-PI-P2 > a-Pl· Hence a+(3 '2dom a. For 3 we start with a constant c such that for each rational number P < , we can find - in an effective manner - two rational numbers PI < a and P2 < (3 satisfying cb - p) '2 a - PI and cb - p) '2 (3 - P2. Then
2c· b - p) '2 a - PI
+ (3 -
P2
=
a
+ (3 -
(PI
+ P2).
The assertion 4 is clear for a = O. Let us assume that a > O. Given a rational P < a(3 we can compute two positive rationals PI < a and P2 < (3 such that PIP2 '2 p. For c = 1/(3 we obtain c· (a(3 - p) '2 c· (a(3 - PIP2) '2 c· (a(3 - PI(3)
=
a - Pl·
The assertion 5 follows immediately from Lemma 7.53 that all c.e. reals dominate O. Therefore the assertion is true if a = 0 or (3 = O. Assume that a > 0 and (3 > 0, and that c is a constant such that, given a rational P < " we can find rationals PI < a and P2 < (3 satisfying cb-p) '2 a-PI and c(, - p) '2 (3 - P2. We can assume that PI and P2 are positive. With C= c . (a + (3) we obtain a(3 - PIP2
a((3 - P2)
+ P2(a -
PI)
< (a + P2)cb - p) < (a + (3)cb - p)
cb -
p).
o
Corollary 7.55. The sum of a random c.e. real and a c.e. real is a random c. e. real. The product of a positive random c. e. real with a positive c. e. real is a random c. e. real.
7. G.E. Random Reals
276
o
Proof This follows from Lemma 7.54 and Theorem 7.59.
Corollary 7.56. The class of random c. e. reals is closed under addition. The class of positive random c. e. reals is closed under multiplication. Remark. Corollary 7.55 contrasts with the fact that addition and multiplication do not preserve randomness. For example, if a is a random number, then 1- a is random as well, but a + (1- a) = 1 is not random. For two reals a and {3, a =dom {3 denotes the conjunction a '2dom {3 and {3 '2dom a. For a real a, let
[aJ
= {{3 E R
Ia
=dom
{3} and R c.e .
= {[aJI a
is a c.e. real}.
Theorem 7.57. The structure (R c .e .; ~dom) is an upper semi-lattice. It has a least element which is the =dom -equivalence class containing exactly all computable real numbers.
Proof By Lemma 7.54 the structure (Rc .e .; ~dom) is an upper semi-lattice. Let a be a computable real, so there exists an increasing computable sequence (an) of rationals with la - ani ~ 2- n . Clearly, if a dominates a c.e. real {3, then also {3 must be computable. Now let {3 be a c.e. real and (b n ) be an increasing computable sequence of rationals converging to (3. We define an increasing computable sequence an of rationals by an = ag(n)' where 9 : N -+ N is the total computable function defined by
g(-l) = -1 and g(n) = min{m 1m> g(n -1) and 2- m for all n E N. Then, (an) {3 dominates a.
-+
~ bn +1 -
bn },
a, and {3 - bn > a - an for all n E N. Hence, 0
Comment. Corollary 7.110 and Theorem 7.109 will show that (Rc .e .; ~dom) also has a greatest element, which is the equivalence class containing exactly all Chaitin Omega Numbers. We are now in a position to describe the relationship between the domination relation and the program-size complexity. Lemma 7.58. For every c E N there is a positive integer Nc such that for every n E N and all strings x, y E ~n with 10.x - O,yl ~ c· 2- n we have IH(y) - H(x)1 ~ N c .
7.5 G.E. Reals, Domination and Degrees
277
Proof For n ~ 1 and two strings x, y E ~n with IO.x - O.yl :S c· 2- n , one can compute y if one knows the canonical program x* of x and the integer 2n. (O.x-O.y) E [-c,c]. Consequently, there is a constant Nc > 0 depending only upon c such that H (y) :S H (x) + N c , for all n ~ 1, and all x, y E ~n with IO.x - O.yl :S c· 2- n . The lemma follows by symmetry. 0 Theorem 1.59 (Solovay). Let x, y E ~w be two infinite binary sequences such that both O.x and O.y are c.e. reals and O.x ~dom O.y. Then
H(y(n)) :S E(x(n))
+ 0(1).
Proof In view of the fact that O.x ~dom O.y, there is a constant c E N such that, for every n E N, given x(n), we can find, in an effective manner, a rational Pn < O.y satisfying 2c ~ c· ( O.x - O.x(n) - 2n+1 1 ) ~ O.y - Pn 2n+1
> O.
Let zPn be the first n + 1 digits of the binary expansion of Pn. Then
o :S O.y(n) -
2c+ 1
O,zPn:S 2n +1
.
Hence, by Lemma 7.58, we have
H(y(n)) :S H(zPn)
+ 0(1) :S H(x(n)) + 0(1).
o
Remark. If a :Sdom (3, then (3 is "more random" than a in the sense that the program-size complexity of the first n digits of a does not exceed the complexity of the first n digits of (3 by more than a constant, cf. Theorem 7.59. The more random an effective object is, the closer it is to Chaitin Omega Numbers; the less random an effective object is, the closer it is to computable reals. The converse implication is false, see Exercise 7.8.26. A slightly more general form of Theorem 7.59 is true: the hypothesis that the sequence is increasing is not necessary. Theorem 1.60. Let (ai) and (b i ) be converging sequences with O.x limi-too ai and O.y = limi-too k If (ai) dominates (bi), then
H(y(n)) :S H(x(n))
+ 0(1).
=
7. C.E. Random Reals
278
Proof For every n and large enough i we have 10.x - ail::; 2- n hence, 10.x(n) - ail::; 10.x(n) - O.xl + 10.x - ail::; Tn.
1
and
Therefore, given x(n), we can compute an index in such that
For this index in we have
Let c > 0 be a constant such that
for all i. Let Zn be the string consisting of the first n + 1 digits after the radix point of th.e binary expansion of bin (containing infinitely many ones). Then
10.y(n) - O.znl
< 'IO.y(n) - O·yl + 10.y - bini + Ib in < T n - 1 + c ·IO.x - ainl + T n- 1 < 2- n - 1 + c. 3. 2- n - 1 + 2- n - 1 (3c
- O.znl
+ 2) ·2- n - 1 .
Hence, by Lemma 7.58, we have
H(y(n)) ::; H(zn)
+ 0(1) ::; H(x(n)) + 0(1).
o
Theorem 1.61. Let (an) be a computable sequence of rationals converging to a non-random real 0, and let (b n ) be a computable sequence of rationals converging to a random real {3. Then, for every c > 0 there are infinitely many i such that
Proof For the sake of a contradiction assume that the assertion is not true and that (ai) dominates (bi). Let 0 = O.x and {3 = O.y (we can assume without loss of generality that 0 and (3lie in the interval [0,1)). Then, by Theorem 7.60, there is a constant c such that H(y(n)) ::; H(x(n)) + c, for all n. This implies that also x is random, i.e. 0 is random, a contradiction.
o
7.5 G.E. Reals, Domination and Degrees
279
We are now in a position to cast new light on Theorem 7.44. Lemma 7.62. Let (b i ) be a computable sequence of rationals which converges to a random real {3. Then for every d > 0 and almost all i, 1{3 - bil
> 2d-i.
Proof Let d > 0 be fixed. It is clear that we can assume without loss of generality that {3 and all rationals bi lie in the interval (0,1). Let O.y be the binary expansion of {3. For every i, let Zi E ~i+1 be the string consisting of the first i + 1 digits after the radix point of the binary expansion of bi (containing infinitely many ones). Then
o::; bi -
O,Zi ::; 2- i -
1.
Since the sequence (Zi) is a computable sequence of strings there exists a constant el such that for all i (7.9)
For the sake of a contradiction let us assume that there are infinitely many i with 1{3 - bi 1::; 2d-i. Then for all these i we have 10.y(i) - O.zil
< 10.y(i) - O·yl + 10.y - bil + Ibi - O.zil < T i - 1 + 2d+1 . T i - 1 + T i - 1 (2 + 2Ml) . Ti-l.
With Lemma 7.58 we conclude that there is a constant H(y(i)) ::; H(Zi) + e2 for all these i. Using (7.9) we obtain
H(y(i)) ::; 2logi + el
e2
such that
+ e2,
for infinitely many i. This contradicts the randomness of y, i.e. the randomness of the real {3. D The following result is a scholium to Theorem 7.44.
Scholium 7.63. Let (ai) be a computable sequence of rationals which converges computably to a computable real 0, and let (bi) be a computable sequence of rationals which converges monotonically to a random real {3. Then for every c > 0 there exists ad> 0 such that for all i 2 d (7.10)
280
7. G.E. Random Reals
Proof. Let (ai) and (b i ) be as in the scholium and fix a number c> O. We show that (7.10) is true for almost all i. First, we show that it is sufficient to prove this for c = 1. Indeed, since we can enlarge c, we can assume that c is a rational. Then we can prove the assertion for the sequence (cai) instead of (ai) with the constant c in (7.10) replaced by 1. The sequence (cai) is also a computable sequence of rationals and it converges computably to the computable real ca. Secondly, we show that we can restrict ourselves to the case that the sequence (ai) is of the form ai = 2- s (i) where 8 : N --+ N is a computable, non-decreasing, unbounded function with 8(0) = O. Indeed, since we will show 1,8 - bi I > la - ai I only for almost all i, we can forget finitely many terms of both sequences (ai) and (b i ) and assume that la - ail::; 1, for all i. Since the sequence (ai) converges computably to a there is a computable function 9 : N --+ N with la-a'I O. Then we observe i ::::: g( 8( i)) and hence i. Therefore, it is sufficient to prove that
la -
ai I ::; 2- S (i), for all (7.11)
holds true for almost all i. Hence, from now on we assume that. 8 : N --+ N is a computable, nondecreasing, unbounded function with 8(0) = 0 and we wish to show that (7.11) is true for almost all i. We define the computable non-decreasing function f : N --+ N by f(i) = max{j I 8(j) ::; i}, for all i. Then we have for all k ::::: 0
f(8(k)) = max{j I s(j) ::; s(k)} ::::: k. Finally we define a computable sequence (b i ) by bi = bf(i)' Since the sequence (b i ) converges monotonically there exists a constant d::::: 0 such that for all i, j with j ::::: i,
7.5 G.E. Reals, Domination and Degrees
281
By Lemma 7.62 there exists a constant el such that 1,8 - bjl > 2d-J,'
for all j ;::: el. We set e2 = f(el)
+ l.
Then s(i) > el for all i ;::: e2. Because of i ::; f(s(i)) for all i ;::: 0 we obtain for all i ;::: e2
1,8 -
bi I ;::: 2- d •
1,8 -
bf(s(i)) I = Td .
1,8 -
bS(i) I > Td . 2 d - s (i)
= Ts(i).
o
which completes the proof.
We have considered arbitrary converging and computable sequences (ai) and (b i ) and have explicitly formulated two gaps with respect to the convergence rates, one from computable to non-computable reals, and one from non-random to random reals. Both results were based on the inequality 1,8 - bil > cia - ail holding for infinitely many i. Can we claim that (b i ) converges slower than (ai)? If we compare monotonically converging sequences with computable limit and monotonically converging sequences with random limit and replace the quantifier "for infinitely many i" by the quantifier "for almost all i" , then it is justified to say that (bi) converges slower than (ai). Theorem 1.64. Let (ai) be a computable sequence of rationals which converges monotonically to a computable real (x, and let (bi) be a computable sequence of rationals which converges monotonically to a random real,8. Then for every c> 0 there exists ad> 0 such that for all i ;::: d (7.12) Proof This follows immediately from Proposition 7.44 and Scholium 7.63.
o We continue by comparing the domination relation with Turing reducibility. For every infinite sequence x E L;W such that O.x is a c.e. real, let
Ax = {v E L;* I O.v ::; O.x} and A~ = {string(n) I Xn = I}. Then, obviously, Ax is a c.e. set which is Turing equivalent to A~. 8 In the following, we establish the relationship between domination and Turing reducibility. Recall that we denote by XA the characteristic function of A 8Note that
A;t
is not necessarily a c.e. set.
c
L;*.
7. G.E. Random Reals
282
Definition 7.65. A set A c L;* is Turing reducible to a set B c L;* (we write A -:5:T B) if there is an oracle Turing machine M such that MB(x) = XA(X), for all x E L;*. Lemma 7.66. Let x, y E L;w be two infinite binary sequences such that both O.x and O.y are c.e. reals and O.x 2.dom O.y. Then Ay -:5:T Ax. Proof Without loss of generality, we may assume that
x, y tj. {xOOOO ... ,x1111 ... I x E L;*}.
(7.13)
Let f : L;* ~ L;* be a partially computable function and c E N a constant satisfying the following inequality for all n > 0: c
0< O.y - O·f(x(n - 1)) -:5: 2n . Given a string z we wish to decide whether z E A y . Using the oracle A~ we compute the least i 2. 0 such that either O·f(x(i - 1)) 2. O.z or O.z - O·f(x(i - 1))
>
c 2i
.
Such an i must exist in view of the relation y tj. {xOOOO . .. , xlIII . .. I x E L;*}. Finally, if O.f(x(i - 1)) 2. O.z, then z E Ay; otherwise z tj. A y . 0 Does the converse of Lemma 7.66 hold true? A negative answer will be given in Corollary 7.114. Let (CE; -:5:T) denote the upper semi-lattice structure of the class of c.e. sets under the Turing reducibility.
Definition 7.67. A strong homomorphism from a partially ordered set (X, -:5:) to another partially ordered set (Y, -:5:) is a mapping h : X -+ y such that 1.
For all x, x' E X, if x -:5: x', then h(x) -:5: h(x' ).
2.
For all y, y' E Y, if Y -:5: y', then there exist x, x' in X such that x -:5: x' and h(x) = y, h(x' ) = y'.
Theorem 7.68. There is a strong homomorphism from (Rc.e.; -:5:dom) onto (CE; -:5:T).
7.5 G.E. Reais, Domination and Degrees
283
Proof By Lemma 7.54 the structure (R c.e .; ~dom) is an upper semi-lattice. Every =dom-equivalence class of c.e. reals contains a c.e. real of the form O.x. Lemma 7.66 shows that by O.x f-t Ax one defines a mapping from (Rc.e.; ~dom) to (CE; ~T)' which satisfies the first condition in the definition of a strong homomorphism.
We have to show that this mapping also satisfies the second condition. Let B, C C L;* be two c.e. sets with C ~T B. To this end we will show that there are two c.e. reals O.x and O.y with the following three properties: (I) (II) (III)
O.x dominates O.y, Ax is Turing equivalent to B, and
Ay is Turing equivalent to C.
We can assume that the sets Band C are infinite and have the form B = {string(n) I nEB} and C = {string(n) I n E C}, where B is a c.e. set of odd natural numbers and C is a c.e. set of even natural numbers. Then the set D = B U C is Turing equivalent to B. We define two sequences x, y E L;W by x = XD and y = Xc. The real numbers O.x and O.y are c.e. They have the properties (II) and (III) because Ax is Turing equivalent to A~ = D, which is Turing equivalent to B, and Ay is Turing equivalent to = C. We are left to show that O.x dominates O.y. Let bo,b1 ,b2 , ••• and CO,C},C2, •..
At
be one-to-one computable enumerations of B and of C, respectively. The rational sequences
are increasing, computable, converge to O.x and to O.y, respectively, and satisfy the inequality n
O.x - 2)2- bi i=O
+ 2-C;)
n
~ O.y -
L 2-
Ci •
i=O
Hence, by Lemma 7.53, the number O.x dominates O.y.
o
Definition 7.69. Two sets A, B are Turing equivalent if A and Bare Turing reducible to each other. A n equivalence class with respect to the relation =T is called Turing degree. A c.e. Turing degree is a Turing degree containing a c. e. set.
7. C.E. Random Reals
284
We write a, b, and so on to denote the Turing degrees. We define a S b if there is some A E a and B E b such that A ST B. Turing degrees form a partial order with respect to ST which we denote by D(S). For example, 0 is the c.e. Turing degree containing all computable sets. Finally, identifying N with L;* via the computable bijection string we can talk about reducibility between sets of non-negative integers. Recall that (i.px) is a Godel numbering of all p.c. string functions. In what follows we will use a standard enumeration (Di) of the class of finite sets of strings (Do denotes 0). Definition 7.70. (a) Let
'Halt = {x E L;* I i.px(x) < (Xl}, 9 and let ~g be the class of all sets A ST 'Halt. (b) A computable approximation to a ~g set A is a sequence (Df(i)) of finite sets indexed by some computable function f such that XA(X) = limi-->oo XDf(i) (x), for all x. For q E Q n [0,1] we write q(x) = i if the xth bit of the binary representation containing infinitely many ones of q is i. Rephrasing the Shoenfield Limit Lemma (see Odifreddi [321], p. 373) we get: Proposition 7.71. For a real a E [0,1] the following two conditions are equivalent:
(1)
There exists a computable sequence (ai) of rationals converging to a.
(2)
a = O.XA, for some ~g set A.
Proof For the direct implication we can assume that all rationals ai lie in the unit interval [0,1]. We define x E A[s] if x < sand as(x) = 1. Then XA = lims-->oo XA[s] is a ~g set and a = O,XA· Conversely, suppose a = O.XA where A is a ~g set and {A[S]}SEN is a computable approximation to A. Let qs = O,XA[s]' Then clearly (qs) is a computable sequence converging to a. 0 9The standard notation K was not convenient in this context. It is well known that the decision problem for Halt - the Halting Problem - is uncomputable; an information-theoretic proof will be discussed in Section 9.2.
7.5 C.E. Reais, Domination and Degrees
285
Definition 7.72. We define the degree of a real 0;, degT(o;), to be the degree of A, where O.XA is the fractional part of 0;. Note that either there is a unique such set A or there are two, one finite and one cofinite. Theorem 7.73. Suppose 0; = O.XA, for some ~g set A. Then, for every c. e. degree b there exists a computable sequence (qi) with limit 0; such that {qi} has degree b.
Proof Let (Pi) be a computable sequence converging to 0; such that {Pi} is infinite. We can construct a computable subsequence (rj) of (Pi) such that O(rj) is strictly increasing. Let B be an arbitrary infinite c.e. set of natural numbers and ba, b1 , b2 , ••• be an effective injective enumeration of B. Then (qi) = (rbJ is a computable sequence of rationals which converges to 0;. We claim that {qi} =T B. Indeed, a natural number m is in B iff rm is in {qi}. Conversely, for an arbitrary rational number s we can decide s E {qd by first asking whether s E {ri}. This is decidable because O(ri) is strictly increasing. If the answer is positive we compute the unique 0 number b with rb = s, and ask whether bE B. So far we have considered arbitrary computable sequences of rationals that converge. It is possible for the left cut L( 0;) to be c.e. and the set A satisfying the equality 0; = O,XA not to be c.e. (see Exercise 7.8.23) Next we define the strongly w-c.e. sets and prove that if L( 0;) is c.e., then A is a strongly w-c.e. set.
Definition 7.74. Let A be a ~g set. We say that A is strongly w-c.e. if there is a computable approximation (A[s])s to A such that 1. 2.
A[O] = 0, If x E A[s] \ A[s + 1], then there exists y < x such that y E A[s +
1] \A[s]. The following theorem gives another characterization of c.e. reals. Theorem 7.75. Let are equivalent:
0;
be in [0,1]. Then, the following two conditions
7. C.E. Random Reals
286
(1)
The real
(2)
There is a strongly w-c.e. set A such that
0:
is c.e.
Proof. The implication (1)
0:
= O.XA.
'* (2) holds for 0: = O. Suppose 0: > 0 and (qi)
is an increasing computable sequence of rationals in [0,1] converging to 0:. We define XA = lims--->ooXA[sj, where A[s] = {x I x < sand qs(x) = I}. Then, 0: = O.XA and A is strongly w-c.e.
'*
For the converse implication, (2) (1), we consider a real 0: = O,XA, for some strongly w-c.e. set A. Let qs = O'XA[sj, where {A[s]} is a computable approximation to A satisfying Definition 7.74. Then L(o:) can be 0 enumerated from an enumeration of {qs I SEN}, so 0: is c.e. Corollary 7.76. If A is a strongly w-c. e. set, then A is of c. e. degree. Proof. As L(O.XA) rem 7.75.
=T
A, for A
c
N, the assertion follows from Theo0
Definition 7.77. A set B C Q of rationals is called a representation of 0: if there is an increasing computable sequence (qi) of rationals with limit 0: and {qi} = B.
To study the degrees of sets of rational numbers we will identify a set B C Q with its image under a fixed computable bijection B : Q --t Nand call B(B) a representation of 0:. Next we will look at the Thring degrees of representations of c.e. reals. Clearly, degT(O:) = degT(L(o:)). Lemma 7.78. Every c. e. degree is the degree of L( 0:), for some c. e. real 0:.
Proof. Let A be a c.e. set of degree a and let O.XA. Then it is clear that L(o:) =T A.
0:
be the c.e. real equal to 0
Definition 7.79. A splitting of a c. e. set A is a pair of disjoint c. e. sets Al and A2 such that Al U A2 = A. Then we say that Al and A2 form a splitting of A and that each of the sets Al and A2 is a half of a splitting of A.
7.5 C.E. Reals, Domination and Degrees
287
Recall that the disjoint sum of two sets A, B is defined by A EB B
= {2n I n
E A} U {2n
+ 1 In E B}.
It is not difficult to see that degT(A EB B) is the least upper bound of degT(A) and degT(B), and so D(:S) forms an upper semi-lattice. If Al and A2 form a splitting of a c.e. set A, then A =T Al EB A 2. The following two lemmata show the connection between representations of c.e. reals and splitting. Lemma 1.80. If B is a representation of a c.e. real a, then B is an infinite half of a splitting of L( a). Proof. It is clear that any representation B of a c.e. real a is an infinite c.e. subset of L( a). Hence, all we have to show is that L( a) \ B is c.e. Let (qi) be the increasing computable sequence of rationals with B = {qi}. The set L(a) is c.e. We can for each element p E L(a) wait until we find a qj with p :S qj (as rationals), and choose p iff P rj. {qQ, ... , qj}. Hence, we can enumerate L(a) \ B. 0
Lemma 1.81. Let B be a representation of a c. e. real a and let C Then the following two conditions are equivalent: 1.
C is a representation of a.
2.
C is an infinite half of a splitting of B.
c B.
Proof. The direct implication follows the proof of Lemma 7.80. For the converse, let (qi) be the increasing computable sequence of rationals with B = {qi}, let C be an infinite half of a splitting of B, and let D be the other half of this splitting. We construct an increasing rational sequence (Pi) with limit a and C = {pd by going through the list (qi), waiting for each element qi until it is enumerated either in C or in D, and finally choosing it iff it is enumerated in C. 0
Remark. From Lemma 7.80 it follows that L(a) is an upper bound for the degrees of representations of a. Corollary 1.82. If B is a representation of a c.e. real a, then B :ST
L(a).
288
7. O.E. Random Reals
For the special case of computable reals we then get the following: Corollary 7.83. If a is a computable real, then every representation of a is computable.
For a c.e. real a, let 8'(a) be the partial order (with respect to Turing reducibility) of those c.e. Turing degrees below degT(L(a)) that contain a representation of a. Proposition 7.84. For every c.e. real a, 8'(a) is an upper semi-lattice. Proof Let a be a c.e. real. Then 8'(a) is closed under the usual join operation on Turing degrees. Indeed suppose a, b E 8'( a) with A and B being representations of a in a and b, respectively. Let 0 = Au B. Then 0 is the representation of a formed by effectively enumerating the sequences of A and B in increasing order (as rationals). We claim that
degT(O) = aU b, i.e. 0(0) =T O(A) EB O(B). It is obvious that 0(0) ~T O(A) EB O(B). For the converse we use Lemma 7.81: the set A is a half of a splitting of 0, hence O(A) ~T 0(0), the same for B. 0 We further study the upper semi-lattice 8'(a). We first prove that 0 and
degT(L(a)) are in 8'(a). Proposition 7.85. For any c.e. real a there is a computable representation of a. Proof The classical result that every infinite c.e. set contains an infinite computable subset yields the assertion. 0
Theorem 7.86. Every non-computable c. e. real number a has a noncomputable representation. Proof We fix an increasing computable sequence (qi) converging to a such that {qd is computable. We construct by stages a non-computable representation B such that (Pi) is a subsequence of Band B is not the complement of any c.e. set.
289
7.5 C.E. Reals, Domination and Degrees
= 0 let bo = qo. At stage s + 1 we have already constructed B[s] = {bo, ... , bkJ, where bo < ... < bk8 (as rationals) and bk8 = qs. If there is a least e < s + 1 such that We[s] n B[s] = 0 and an x E We[s] with qs < x :S qs+l, then let bk8+1 = x, bk 8+2 = qs+l and ks+l = ks + 2. If there is no such e, then let bk8+1 = qs+l and ks+l = ks + 1. At stage s
We complete the construction by letting B =
Us B[s].
Clearly (bi) is an increasing computable sequence of rationals converging to o. It remains to show that B is not computable. Suppose B is computable. Then let e be the least index such that B = We. Let So be a stage such that for all i < e and all s ~ So we have Wds]nB[s] i= 0 or there is no x E Wi[S] with qs < x :S qs+l. We will show that for all P > qSQ (as rationals), p E L( 0) is decidable, contradicting the hypothesis of the theorem. To compute p E L( 0), we enumerate B and We until p occurs in one of them. If p E B then p E L(o). Otherwise p E We and we claim that p 1- L(o). Indeed, suppose that p E L(o). Then at some least stage t > So, qt < p :S qt+l, and the construction enumerates some p' E B for qt < p' :S qt+l and p' E We. This contradicts B n We = 0 and hence B is not computable. 0 Theorem 7.87 (Calude-Coles-Hertling-Khoussainov). Let 0 be a c. e. real. Then 0 has a rep res entation of degree L (0). Furthermore, every representation of 0 can be extended to a representation of degree L(o).
Proof Let (Pi) be an increasing computable sequence of rationals converging to o. We shall construct a new computable sequence (qi) of rationals such that {qi} is a representation of 0 with {qd =T L(o). Additionally we define li = max{O(pj) I j :S i}, for all i, and the sequence (ji) of natural numbers with qji = Pi, for all i. We start with jo = 0 and qo ji+1 > ji such that
= Po. Given ji with qji = Pi, we define
and for m = 1, ... ,ji+l - ji we define the numbers qji+ m as the rational numbers in this set in increasing order.
7. G.E. Random Reals
290
It is obvious that (qi) is an increasing computable sequence of rationals converging to a, and qji = Pi, for all i. From Corollary 7.82 we know that {qd "5.T L(a). We still have to prove that L(a) "5.T {qi}. Let P E Q. In order to decide P E L(a) we compute the minimal k with lk 2: O(p). Then we check whether P "5. qjk. If P "5. qjk' then P E L(a). If P > qjk' then P E L(a) iff P E {qi}. 0
Comment. The following alternative proof for the first assertion of Theorem 7.87 shows that we can obtain a representation of a of degree L( a) consisting only of dyadic rational numbers. We fix an increasing computable sequence (Pi) of dyadic rationals with limit a with increasing denominator
for a computable sequence (ni)i of integers and a computable, increasing sequence (ki)i of natural numbers. We shall construct a new computable sequence (qi)i of rationals such that {qd is a representation of a having Turing degree degT(L(a)). To this end we will define a sequence (ji) of natural numbers such that qji = Pi, for all i. We start with jo
= 0 and qo = PO. Given ji with % = Pi, we set m
qji+ m = qji
+ 2ki+l
Of course, (qi) is an increasing computable sequence of rationals converging to a since % = Pi, for all i. We have to show that L(a) "5.T {qi}. If a is a rational, then L(a) is computable, so "5.T {qd. So we assume that a is irrational. If the set {qi} contains a dyadic number 2~tl, then it contains all dyadic numbers in
,a)
the interval e~tl whose denominator is at most2k. But {qi} does not contain any number greater than a. Furthermore, the denominator of the dyadic number qji is at least 2ki 2: 2i. Hence, given {qi} as an oracle, for an arbitrary natural number l we can compute a dyadic rational (2n+ 1 )2- with k 2: l and such that the interval (2~tl, 2~t3) contains a.
k
291
7.5 C.E. Reals, Domination and Degrees
Using {qd, for a given rational number r, we can decide whether r < ex by computing such an interval which contains ex but not r (any sufficiently small interval containing the irrational number ex will not contain r) and checking whether r lies to the left or to the right of this interval.
Corollary 7.88. Every c. e. degree contains a representation of a c. e. real.
o
Proof By Lemma 7.78 and Theorem 7.87.
By Lemma 7.80 every representation of a c.e. real ex is a half of a splitting of L(ex). The following result shows that there is a representation of ex of the same degree as the other half.
Theorem 7.89. Suppose B is a representation of a c.e. real ex. Then there is a representation C of ex such that C =T L(ex) \ B. Proof Let (bi) be the increasing computable sequence such that B = {bi }. Let (Pi) be a representation of ex such that {pd is computable and {pd n {bd = 0. We construct a new increasing computable sequence of r~tionals (Ci) such that {Ci} =T L(ex) \ B. To this end we define li = max{O(pj) I j ~ i}, for all i, and a sequence (ji)i of natural numbers with Cji = Pi, for all i. We start with )0 = 0 and Co = Po. Let bPi denote the least rational in B which is greater than Pi. Then given ji with Cji = Pi, we define ji+I > ji such that
ji+I - ji
= #( {q
E
Q I Pi < q ~ Pi+I, O(q)
~ li+}, q
1- {bo, ... , bpi+l}})'
and for m = 1, ... ,ji+I - ji we define Cji+ m to be those rational numbers in this set in increasing order. Let C = {cd. It is clear that (Ci) is an increasing computable sequence of rationals converging to ex, since Cji = Pi, for all i. We now show that C =T L(ex) \B.
First, C ~T L(ex) \ B as follows. Let p E Q. If P 1- L(ex) \ B, then P 1- C. Otherwise, if P E L(ex) \ B, enumerate C until reaching a least Ci such that Ci 2 p. Then P E C iff p E {co, ... , Ci}. Secondly, L(ex) \B ~T C as follows. Let P E Q. Compute the least k such that lk 2 O(p) and then check whether P ~ Cjk. If P ~ Cjk' then enumerate B until reaching a least bi such that P ~ bi, and conclude p E L( ex) \ B iff P 1- {bo, ... , bd· Otherwise, P > Cjk and we can conclude that P E L(ex) \B ilipEQ
0
7. G.E. Random Reals
292
Remark. Theorem 7.89 is also a strengthening of Theorem 7.87: we can take B to be a computable representation in order to obtain the first part of Theorem 7.87. So we have established that for non-computable c.e. reals a, #( 0 such that c(a - an) ;::: {3 - bn , for all n E N, where a = limn-too an and {3 = limn-too bn · A real is called O-like if it is the limit of a universal computable, increasing sequence of rationals.
7. G.E. Random Reals
298
Theorem 7.103 (Solovay). Let U be a universal Chaitin computer. Every computable, increasing sequence of rationals converging to nu is universal.
Proof Let (an) be an increasing, computable sequence of rationals with limit nu, and let (b n ) be an increasing, computable, converging sequence of rationals. Set (3 = lim n -+ oo bn . We will show that there is a constant c > 0 with c(nu - an) 2: (3 - bn , for all n.
Let (Xi) be a one-to-one, computable enumeration of dom(U), and Wn = L:i=o 2- lxil . We define a total computable, increasing function 9 : N -+ N, where we also define g(-I) = -1, by
g(n) = min{j > g(n -1)
I Wj 2: an}.
We have already seen that the sequence (Wg(n)) is an increasing, computable sequence with limit nu. In view of the inequality nu - an 2: nu - Wg(n) , it is sufficient to prove that there is a constant c > 0 such that for all n E N,
For each i EN, let Yi be the first string (with respect to the quasilexicographical ordering) which is not in the set
{U(Xj) I j
~ g(i)} U
Furthermore, put ni = l-log(bi+1 - bi)J
{Yj I j < i}.
+ 1.
Since
00
LTni ~ (3 - bo < 1, i=O
by the Kraft-Chaitin Theorem 4.2 we can construct a Chaitin computer C such that, for every i EN, there is a string Ui E :E ni satisfying C (Ui) = Yi. Hence, there is a constant Cc such that H U (Yi) ~ ni + Cc. In view of the choice of Yi, there is a string x~ E dom(U) \ {Xj I j ~ g(i)} such that Ix~1 ~ ni + Cc and U(xD = Yi (here we have used the fact that U is surjective). For different i and j we have Yi =1= Yj, hence x~ =1= xj. Finally we obtain
7.6 A Characterization of G.E. Random Reals
nu -
299
wg(n)
o
which proves the assertion. We continue by observing that:
Lemma 7.104. Any n-like real dominates every c.e. real. Theorem 7.105 (Calude-Hertling-Khoussainov-Wang). For every n-like real a we can construct a universal Chaitin computer U such that a = nu. Hence, every n-like real is a Chaitin Omega Number. Proof. Let V be a universal Chaitin computer. Since a is n-like it dominates every c.e. real, in particular
By Theorem 7.100 there exist an infinite prefix-free c.e. set A with {L(A:L:W) = a, a computable function f : A -+ dom(V) with A = dom(f), f(A) = dom(V), and a constant c > 0 such that Ixl ::; If(x)1 + c, for all x E A. We define a Chaitin computer U by U(x) = V(f(x)). The universality of V implies the universality of U and
o In view of Lemma 7.104 and Theorem 7.105 we get: Theorem 7.106. Let a be a c.e. real. equivalent:
The following statements are
300
7. G.E. Random Reals
1.
There exists a universal computable, increasing sequence of rationals converging to a.
2.
Every computable, increasing sequence of rationals with limit a is universal.
3.
The real a dominates every c. e. real.
Random reals can be directly defined as follows: a real a is random iff for every Martin-Laf test A, art. ni>O Ai' In the context of reals, a MartinLof test A is a constructive seque-nce of constructively open sets (An) in the space :L: w such that {L(An) :::; 2- n .
Lemma 7.107 (Slaman). Let (an), (b n ) be two computable, increasing sequences of rationals converging to a and (3, respectively. One of the following two conditions holds:
A) B)
There is a Martin-Laf test A such that a E ni2:0 Ai. There is a rational constant c > 0 such that c( a - ai) 2': {3 - bi, for all i E N.
Proof. We enumerate the Martin-Lof set A by stages. Let An[s] be the union of finitely many open c.e. sets that have been enumerated into An during stages less than s. We put An [0] = 0 and An[s + 1] = An[s] U (as, as + (b s - bso )2- n ), in case as ¢ An[s] and bs =I bso ; here So is the last stage during which we enumerated a c.e. open set into An or So = 0 if there was no such stage; otherwise, An[s + 1] = An[s]. Clearly, An = Us An[s] is a disjoint union of c.e. open sets.
Let tl, t2, ... , tn, ... be the sequence of stages during which we enumerate open sets into An. Then, {L (YAn[s]) L{L(An[ti]) i2:1
1 00 2n (btl - bo) + L(b tHl j=l
1 2n ({3 - bo)
1
< 2n
-
btj )
7.6 A Characterization of G.E. Random Reals
301
If a E ni>O Ai, then A) holds. Assume that a ¢ An, for some n. We shall prove that 2i(a - ai) 2': (3 - bi, for almost all i, so B) holds. If the open set (as, as + (b s - bso )2- n ) is enumerated into An at stage s, then there is a stage t > s such that at > as + (b s - bso )2-n. We fix i > 0 and let to be the greatest stage t :::; i such that we enumerate something into An during stage t or to = 0, otherwise. Let tl, t2, . .. ,tn , . .. be the sequence of stages after to during which we enumerate open sets into An. Clearly, to :::; i :::; tl. As
a - ah > atk - atl
+ (b tk
- btk _1)2- n ,
for all k, and it follows that
so
a - ah 2': 2:Jbtk - btk_1)T n
= ((3 - bto)Tn.
k;:::l
Finally, for every i 2': max{to, tIl,
a - ai 2': a - atl 2': ((3 - bto)Tn 2': ((3 - bi)T n , because (an), (b n ) are increasing.
o
Theorem 7.108 (Slaman). Every c.e. random real is o'-like. Proof We apply Lemma 7.107: if A) holds, then a is not random; if B) holds, then (3 :::;dom a, and the theorem follows as (3 has been arbitrarily chosen. 0 The following theorem summarizes the characterization of c.e. random reals:
Theorem 7.109. Let a E (0,1). The following conditions are equivalent: 1.
The real a is c. e. and random.
2. 3.
For some universal Chaitin computer U, a = o,u. The real a is o'-like. Every computable, increasing sequence of rationals with limit a is universal.
4.
302
7.7
7. G.E. Random Reals
Degree-theoretic Properties of Computably Enumerable Random Reals
In this section we prove a few important degree-theoretic properties of c.e. random reals. We first obtain the following addendum to Theorem 7.57. Corollary 7.110. The structure (Rc.e .; "5odom! has a greatest element which is the =dom -equivalence class containing exactly all Chaitin Omega Numbers. In analogy with Corollary 7.55 we obtain: Corollary 7.111.
(1)
The fractional part of the sum of an Omega Number and a c.e. real is a Omega Number.
(2)
The fractional part of the product of an Omega Number with a positive c. e. real is an Omega Number.
(3)
The fractional parts of the sum and product of two Omega Numbers are again Omega Numbers.
Proof Use Lemma 7.54 and Theorem 7.109.
o
We continue with a classical result: Theorem 7.112 (Chaitin). Given the first n bits ofn u one can decide whether U(x) halts or not for every string x of length at most n. Proof Assume that 0, = 0.0,10,2 ... nn ... , x is an arbitrary program of length less than n and proceed by dovetailing the computations of U on all possible binary strings ordered quasi-lexicographically (considered as possible inputs). That is, we execute one step of the computation of U on the first input, then the second step of the computation of U on the first input and the first two steps of the computation of U on the second input, a.s.o., and we observe halting computations. Any halting computation of U on x improves the approximation of 0, by 2- lxl . This process eventually leads to an approximation of 0, which is better than 0.0,10,2 ... nn. At
7.7 Degree-theoretic Properties of G.E. Random Reals
303
this stage we check whether x is among the halting programs; if it is not, then x will never halt, because a new halting program x will contribute to the approximation of 0 with 2- lxl 2: 2- n , contradicting (7.4). 0 Remark~
The number Ou includes a tremendous amount of mathematical knowledge. According to Bennett [32, 206],
[Omega] embodies an enormous amount of wisdom in a very small space inasmuch as its first few thousand digits, which could be written on a small piece of paper, contain the answers to more mathematical questions than could be written down in the entire universe. Of course, the above comment is not valid for every Ou. Indeed, in view of Theorem 6.40, for every positive integer n one can construct an Omega Number whose first n bits are O. However, the claim becomes true for every Ou if we replace the bound "a few thousand" by some appropriate larger number.
It is worth noting that even if we get, by some kind of miracle, the first n digits of Ou, the task of solving the problems whose answers are embodied in these n bits is computable but unrealistically difficult: the time it takes to find all halting programs oflength less than n from 0.0102 ... On grows faster than any computable function of n. In a truly poetic description, Bennett continues: Throughout history mystics and philosophers have sought a compact key to universal wisdom, a finite formula or text which, when known and understood, would provide the answer to every question. The use of the Bible, the Koran and the I Ching for divination and the tradition of the secret books of Hermes Trismegistus, and the medieval Jewish Cabala exemplify this belief or hope. Such sources of universal wisdom are traditionally protected from casual use by being hard to find, hard to understand when found, and dangerous to use, tending to answer more questions and deeper ones than the searcher wishes to ask. The esoteric book is, like God, simple yet undescribable. It is omniscient, and transforms all who know it ... Omega is in many senses a cabalistic number. It can be known of, but not known, through human reason. To
7. C.E. Random Reals
304
know it in detail, one would have to accept its uncomputable digit sequence on faith, like words of a sacred text.
The converse implication in Theorem 7.112 is false. We shall return to this discussion in Sections 8.5 and 8.7. Corollary 7.113. The realO'X'Halt is not an Omega Number. Proof It is well known that O,X'Halt is not random.
o
Now we can answer the question raised after Lemma 7.66. Recall that the sets An and AX?-lalt are defined as before Lemma 7.66. Corollary 7.114. Let 0 be an Omega Number. Then the following statements hold: 1.
O,X'Halt 'l.dom 0,
2.
An
=T AX?-lalt =T
Halt.
Proof The first claim follows from Corollary 7.113. The relations An "5.T Halt =T AX?-lalt are clear and AX?-lalt "5.T An follows from Lemma 7.66.
o Clearly, all Omega Numbers are in ~g. Does there exist a random real in ~g which is not in the set {a, 1 - a I a is c.e. random}? The answer is positive. Proposition 7.115. There is a random sequence y with A~ E ~g such that neither O.y nor 1 - O.y is a c. e. real. Proof Let x = XIX2 •.• be an infinite binary sequence such that O.x is an Omega Number, hence O-like. We define an infinite binary sequence y = YIY2 ... by
if i = 1, if 3n < i "5. 2 . 3n , if 2 . 3n < i "5. 3n + 1 .
7.7 Degree-theoretic Properties of G.E. Random Reals
305
The sequence y is obtained by recursively reordering the digits of the sequence x. Hence, also y is a random sequence in ~g. Next we show that neither O.y nor 1 - O.y is a c.e. real. In fact, we show more:
O.x
£dom
O.y
and
O.x
£dom
1 - O.y .
(7.19)
By symmetry, it suffices to show that O.x does not dominate O.y. For the sake of a contradiction, assume that O.x "2dom O.y. Then, by Theorem 7.59, and hence, by the definition of y, we obtain
for all n EN. That is,
Since lim (3n +1
n--->oo
-
2 . 3n
-
H(string(2 . 3n ))) =
00,
the sequence x is not random by Theorem 6.99, hence we have proved 0 (7.19). We conclude that neither O.y nor 1 - O.y is a c.e. real. If one could solve the Halting Problem, then one could compute the program-size complexity. What about the converse implication: can the Halting Problem be solved if one could compute program-size complexity?
We will show that the above question has an affirmative answer. In fact, a stronger result will be proven. To this end we need some more notation and definitions. Recall that Wx is the domain of i.px'
Definition 7.116. Let A, Be 'E*. (a) We say that A is weak truth-table (wtt) reducible to B (we write A :;'wtt B) if A :;'T B via a Turing reduction which on input x only queries strings of length less than g(x), where g : 'E* -+ N is a fixed computable function. (b) We say that A is truth-table (tt) reducible to B (we write A :;'tt B) if there is a computable sequence of Boolean functions {Fx }XE~*' Fx : 'ErxH -+ 'E, such that for all x, we have
7. C.E. Random Reals
306
(c) For two infinite sequences x, y E 2:;w we write O.X :;'wtt O.y (O.x :;'tt O.y) in case A~ :;'wtt A~ (A~ :;'tt A~).
Note that in contrast with tt-reductions, a wtt-reduction may diverge.
Definition 7.117. A c.e. set A is tt (wtt)-complete if Halt :;'tt A (Halt :;'wtt A). We will use Arslanov's12 Completeness Criterion (see Odifreddi [321], p. 338 or Soare [372], p. 88) for wtt-reducibility
Theorem 7.118 (Arslanov's Completeness Criterion). A c.e. set A is wtt-complete iff there is a function f :;'wtt A without fixed-points, i.e. Wx =I- Wf(x), for all x E 2:;*. Next we show that c.e. random reals are wtt-complete, but not ttcomplete. 13
Theorem 7.119 (Arslanov-Calude-Chaitin-Nies). £H
= {(x,n)
The set
I x E 2:;*,n EN, H(x):::; n}
is wtt-complete. Proof We will use Theorem 7.118 and the formula
max H(x) = n
xE:E n
+ O(logn)
(7.20)
from Theorem 5.4. First we construct a positive integer c > 0 and a p.c. function 'IjJ ~ 2:;* such that for every x E 2:;* with Wx =I- 0,
(7.21) and
I'IjJ (x) I :::; p(x) + c. 12M. Arslanov. 13In the next result Arslanov is for A. Arslanov. son of M. Arslanov.
(7.22)
7.7 Degree-theoretic Properties of C.E. Random Reals
307
We now consider a Chaitin computer C such that C(OP(x)l) E Wx whenever Wx =1= 0. Let c' be the simulation constant of Con U in the Invariance Theorem and let () be a p.c. function satisfying the following condition: if C(u) is defined, then U(())(u) = C(u) and 1()(u)1 :::; lui + c'. We put c = c' + 1 and note that in case Wx =1= 0, C(QP(x)l) E wx , so ()(QP(x)l) is defined and belongs to W x . Finally, we put 'ljJ(x) = ()(OP(x)l) and note that Next we define the function
F(y) = min{x
E ~* I H(x)
> p(y) + c},
where the minimum is taken according to the quasi-lexicographical order and c comes from (7.22). In view of (7.20) it follows that
F(y) = min{x
E ~*
I H(x)
> p(y) + c, Ixl :::; p(y) + c}.
The function F is total, H-computable 14 and U('ljJ(y)) =1= F(y) whenever Wy =1= 0. Indeed, if Wy =1= 0 and U('ljJ(y)) = F(y), then 'ljJ(y) is defined, so U('ljJ(y)) E Wyand 1'ljJ(y) I :::; p(y) + c. But, in view of the construction of F, H(F(y)) > p(y) + c, an inequality which contradicts (7.22):
H(F(y)) :::; 1'ljJ(y) I :::; p(y)
+ c.
Let f be an H-computable function satisfying Wf(y) = {F(y)}. To compute f(y) in terms of F(y) we need to perform the test H(x) > p(y) + c only for those strings x satisfying the inequality Ixl :::; p(y) + c, so the function f is wtt-reducible to £H. We conclude by proving that for every y E ~*, Wf(y) =1= W y. If = Wy, then Wy = {F(y)}, so by (7.22), U('ljJ(y)) E W y, that is, U('ljJ(y)) = F(y). Consequently, by (7.21) H(F(y)) :::; 1'ljJ(y) I :::; p(y) + c, o which contradicts the construction of F. Wf(y)
Theorem 7.120. The set £H is wtt-reducible to
nu.
Proof Let 9 : N -+ ~* be a computable, one-to-one function which enumerates the domain of U and we put Wm = 2:i!o 2- lg (i)l. Given x and n > 0 we compute the smallest t ?': 0 such that
14That is, computable using the subroutine H.
7. G.E. Random Reals
308
From Lemma 7.1 00
O.D(n) :::; Wt < Wt
+
L
Tlg(s)1 = D < O.D(n)
+ Tn
s=t+l
we deduce that Ig( s)
1
> n, for every 8 2:: t + 1. Consequently, for all x,
x rf. {g(O),g(l), ... ,get)} iff H(x) > n. Indeed, if x rf. {g(O),g(l), ... ,get)}, then H(x) > n as H(x) = Ig(8)1, for some 8 2:: t + 1; conversely, if H(x) :::; n, then x must be produced via U by one of the elements of the set {g(O), g(l), . .. ,g(t)}. 0 As a consequence we obtain
Theorem 7.121 (Juedes-Lathrop-Lutz). not random.
If Halt
:::;tt
x, then x is
Proof Assume x is random and Halt :::;tt x; that is, there exists a computable sequence of Boolean functions {FU}UE~*' Fu : L:ru+ 1 ~ 2:, such that for all wE L:*, we have XA(W) = Fw(XOXl ... xrw). We will construct a Martin-Lof test V such that x E nn:::::O VnL: w , which will contradict the randomness of x. For every string z let
M(z) = {u E L:rz+ 1 Fz(u) = O}. 1
Consider the set
{z
E
L:* 1 p,(M(z)L:W) 2:: 1/2}
of inputs to the tt-reduction of Halt to x where at least half of the possible oracle strings give the output O. This set is c.e., so let Wzo be a name for it. From the construction it follows that
Zo E Halt iff Fzo (XOXI ... xrzo) = 1, hence if we put r
= rzo + 1 and
we ensure that V is c.e. and p,(VoL:W) :::; 1/2. because if u = xlr, then
Moreover, x E Vo L: w ,
7.7 Degree-theoretic Properties of C.E. Random Reals Assume now that Zn, Vn have been constructed such that x {L(Vn'EW) :::; 2- n- 1. Let Zn+1 ¢ {zo, Zl, ... , zn} be such that
309 E
Vn'Ew and
W Zn +1 = {u E 'E* I {L(M(u)'Ew n Vn'EW) 2: {L(Vn'EW)j2}. Then
Zn+1
E
Halt iff {L(M(u)'Ew n Vn'EW) 2: {L(Vn'EW)j2.
Finally, we put r = r Zn+1 +1 and
Vn+1
=
{u
E
'ET I ulrzn
E
Vn and ({L(M(Zn+1)'Ew n Vn'EW) 2: {L(Vn'EW)j2 iff
FZn+l (u) =
I)}
and note that Vn +1 is c.e., x E Vn +1 and
{L(Vn+1'EW) :::; {L(Vn'EW)j2 :::; Tn-2. Consequently, (Vn ) is a Martin-Lof test with x E nn2':O Vn'Ew.
D
Because Omega Numbers are the same as n-like reals, compared with a non-Omega Number, an Omega Number either contains more information or at least has its information structured in a more useful way. Indeed, we can find a good approximation from below to any c.e. real from a good approximation from below to any fixed Omega Number. Sometimes we wish to compute not just an arbitrary approximation (say, of precision 2- n ) from below to a c.e. real, but instead, a special approximation, namely the first n digits of its binary expansion. Is the information in n organized in such a way as to guarantee that for any c.e. real a there exists a total computable function 9 : N -> N (depending upon a) such that from the first g( n) digits of n we can actually compute the first n digits of a? We show that the answer to this question is negative if one demands that the computation is done by a total computable function. Theorem 7.122. The following statements hold: 1.
For every c.e. real a, a
2.
O,X'Halt
:::;tt O,X'Halt.
itt n.
Proof For the first assertion we observe that for an arbitrary c.e. real O.x the set Ax is c.e., whence Ax :::;1 Halt (i.e. there is a computable oneto-one function 9 with Ax = g-l(Halt)). Since A~ :::;tt Ax we obtain
A~ :::;tt Halt. The second assertion follows from Theorem 7.121 and the randomness of D
n.
310
7.8
7. G.E. Random Reals
Exercises and Problems
1. Let X be an infinite c.e. subset of dom(U>..). I:uEx Q-1u l is also a Chaitin's Omega Number.
Show that Q(X) =
2. (Hartmanis-Hemachandra-Kurtz) Show that a computable real function f has a Chaitin random root iff the set of roots of f has positive /-l measure. 3. (Hemaspaandra) Is
X2
a random number provided x E (0,1) is random?
4. Let A, B be two alphabets and t : A* ~ B* be a p.c., prefix-increasing function. Let /-lA, /-lB be the product measures on A W , B W , respectively. We denote by T the natural extension of t to A 00, i.e. T : A 00 ~ Boo, T (x) = t(x), for every x E dom(t), and T(x) = limn->oo t(x(n)), for every x E AW. Call the transformation T measure-bounded in case there exists a natural M 2: 1 such that
for every c.e. subset S
c B*.
a) Show that the base transformation bounded.
r
(see Section 7.2) is measure-
b) Show that every measure-bounded transformation T preserves random sequences, i.e. if x E AW is a random sequence (over A) and T(x) E BW, then T(x) is a random sequence (over B). 5. Show that the computable transformations x I--t y, X I--t Z mapping every binary sequence x = XIX2 ... X n .•• into the sequences y = OXIOX2 •.• 0Xn .•• and z = XIXIX2X2 .•. XnX n .•• do not preserve randomness. 6. To each binary sequence x = Xl X2 ••• Xn •• , E {O, l}W we associate, following Szilard [396], the binary sequence z = ZIZ2." Zn ••• where Zl = Xl, Zj = Xj EEl Xj-l, for j = 2,3, ... and EEl is the modulo-2 addition. a) Show that y is random provided x is random. b) Compare this result with von Mises' sequence y in Example 6.43. c) Show that each of the sequences x, y, z can be obtained from the other two by computable transformations. 7. Let x E AW be a random sequence over the alphabet A containing at least three letters and let a E A. Delete from x consistently all occurrences of the letter a. Show that the new sequence is random over the alphabet A \ {a}. 8. Let p : N ----t N be a computable permutation of the naturals. Show that a sequence XIX2 •.. Xn .•. is random iff the sequence xp(1)Xp(2) •.• Xp(n) •.• is random.
7.8 Exercises and Problems
311
9. Show that no sequence x E
A~
is random over the alphabet AQ in case
Q > q::::: 2. 10. (Dragomir) Let x be a random sequence over the alphabet {O, 1, 2}, y a random sequence over the alphabet {O, I}, and z a random sequence over the alphabet {3,4}. Construct a new sequence w over the alphabet {O, 1,2,3, 4} by inserting in x elements from z as follows: if Yi = 1, then insert on the ith position of x the letter Zi. All elements in x remain unchanged; they are just shifted to the right by accepting new elements from the disjoint alphabet {3,4}. For example, ify = 000101100 ... , then w = XlX2X3X4Z4X5X6Z6X7Z7XSX9 .... Is w random? 11. (Staiger) Let 0; E [0,1] be a real number, and let x E AQ and y E A~ be its base Q and base q expansions, respectively. Prove that there is a constant c > 0 such that for every lEN the following equations hold true:
IKQ(x(ll.logQ bJ)) -logQ q. Kq(y(l))1 ::; c, IHQ(x(ll.logQ qJ)) -logQ q. Hq(y(l))1 ::; c. 12. Deduce from the above relations the invariance of randomness under the change of base. 13. (Hertling-Weihrauch) Use the topological definition of random reals to prove the invariance under the change of base. (Hint: Consider the set of reals R with the usual Lebesgue measure jj and B the numbering of a base of the real line topology defined by B 1r (i,j) = {x E R Ilx - vD(i)1 < 2- j }, where VD( < k, l, m » = (k - l)2-m is defined on the set of dyadic reals D = {x E R I x = (i - j)2- k , for some i,j, k}. For the unit interval (0,1) we work with the restriction of the Lebesgue measure and Bi n [0,1].) 14. The lower and upper limits of the relative complexity of a sequence x E are defined by 1:£(x)
A~
. . Hr(x(n)) (). Hr(x(n)) = hmmf and R x = hmsup . n
n-HXl
a) Prove that every x E
A~
n-HXl
n
with 1:£(x) = 1 is Borel normal.
b) Prove that every computable sequence x E A~ has R(X)
= O.
15. (Staiger) In view of Exercise 7.8.14, we can define the lower and upper limits of the relative complexity of a real number by 1:£( vr(x)) = 1:£(x) and R(vr(x)) = R(x). Prove that every Liouville number 0; E [0,1] has 1:£(0;) = O. Deduce that no Liouville number is random. 16. Prove that there are uncountably many Liouville numbers 0; such that for every bEN, b::::: 2, the sequence x E Ab' with Vb(X) = 0; is disjunctive. 17. Show that the class of computable reals forms a real closed field.
312
7. G.E. Random Reals
18. Show that there is an algorithm to determine for every computable reals a -I /3 whether a < /3 or a > /3. 19. Show that there is no algorithm to determine for every computable reals a, /3 whether a = /3 or a -I /3. 20. Show that there exist two infinite prefix-free c.e. sets A and B such that J.t(A2: W ) = J.t(B2: W ) = 1 but A 1.88 Band B 1.88 A. Hence, the mapping in Corollary 7.101 cannot be one-to-one.
21. Show that for every universal Chaitin computer U we can effectively construct two universal Chaitin computers VI and V2 such that DVl = ~ . Du and DV2 = ~(1 + Du). 22. Let U be a universal Chaitin computer, Du = O.Wl ... , and let S = SI ... Sm be a binary string. Show that we can effectively construct a universal Chaitin computer W such that Dw = O.SI ... SmWl ....
23. (Soare) We associate to every subset A c N the real a = 0.XA(1)XA(2) ... , where XA(i) = 1 if i E A and A(i) = 0 if i A, and we write a = O.XA. Construct a set A which is not c.e. but L(a) is a c.e. set.
rt
24. Let D be a total standard notation of all finite sets of words in 2:;*. Let A, B c 2:*. Show that A :::;tt B iff there are two total computable functions f: 2:* -} Nand g: 2:* -} 2:* such that x E A iff XB(f(X)) E Dg(x). 25. (Soare) Show that A :::;tt L(a) but L(a) is not necessarily truth-table reducible to A, although L(a) :::;T A. 26. (Calude-Coles) Show that there are c.e. reals O.x and O.y such that H(x(n)) :::; H(y(n)) + 0(1) and O.y does not dominate O.x,.
27. With reference to Corollary 7.92, construct directly a low non-computable representation B avoiding the upper cone of a c.e. D.
28. Show that O.x :::;tt O.y iff there are two total computable functions 9 : N -} Nand F: 2:* -} 2:* with x(n) = F(y(g(n))), for all n. 29. The preorder :::;tt has a maximum among the c.e. reals, but this maximum is not D, as no random c.e. real is maximal. 30. Show that for every c.e. real O.x there exist a total computable function g: N -} N and a p.c. function F : 2:* ~ 2:* with x(n) = F(D(g(n))), for all n. (Hint: use A~ :::;tt Ax.)
31. (Slaman) Let (Vn ) be a universal Martin-Li:if test. Prove that for every n ~ 1, v(Vn2:W) is c.e. and random. 32. (Downey) Prove that the following conditions are equivalent: a) b is the m-degree of a splitting of L( a), b) b is the wtt-degree of a representation of a.
7.9 History of Results
313
33. (Downey-Hirschfeldt-Nies) Show that for c.e. reals, a -:::;dom f3 iff there exists an integer c > 0 and a c.e. real '"Y such that cf3 = a + '"Y. (Hint: let (an) be a computable increasing sequence with limit a; then, by speedingup the enumeration, we can construct a computable, increasing sequence f3n with limit f3 such that for all n, f3n+ I - f3n < c . (a n+I - an); at each stage one part of c(an+l - an) makes f3n+1 - f3n and the other part makes '"Yn+l - '"Yn.) 34. (Downey-LaForte) Show the existence of an uncomputable c.e. real a such that every prefix-free set A such that a = OA is computable. 35. (Arslanov) We say that a set X is (n + l)-c.e. if X = Xl \ X 2 , for some c.e. set Xl and n-c.e. set X 2 ; c.e. sets are called l-c.e. sets. Show that for every positive integer n every sequence of n-c.e. degree strictly below 0' is not random. 36. (Arslanov) A sequence x is w-c.e. if there exist two computable functions f,g such that Xk = lims-oo f(s,k), f(O,k) = 0, and #({s EN I f(s,k) =1= f (s + 1, k)}) -:::; g( k). Show that there exist w-c.e. random sequences x such that x =T 0'. Give a direct construction of a non-computable c.e. real that does not realize the cone. (Hint: try a finite injury priority argument with strategies that resemble those needed to construct sets without the USP together with a technique to deal with computable sequences of rationals.) 37. (Kucera-Terwijn) Show the existence of a c.e. set A such that rand A rand. Here rand A is the relativization of rand to oracle A.
=
38. (Kummer) Construct a set A such that there is a constant c with K(XA(l) ... XA(n)) ~ 2logn - c, for infinitely many i.
7.9
History of Results
Theorem 7.2 was proved by Chaitin [114]; see also [122, 118, 121]. Section 7.2 follows Calude and Jurgensen [89]; other proofs of invariance can be found in Hertling and Weihrauch [235] and Staiger [383]. The material presented in Section 7.3 comes from Calude and Zamfirescu [100, 101]. Definition 7.22 comes from Jurgensen and Thierrin [245]. For disjunctive sequences see Staiger [384, 388]. The equivalence of the statements 1 and 3 in Theorem 7.106 comes from Chaitin [118]. The analysis of the convergence of computable sequences of rationals was developed in Calude and Hertling [82]; see a1so Ho [238] . The definition of c.e. reals was given in Soare [371]; we direct the reader to [371] for related work on the relative computability of cuts of arbitrary reals. Solovay's manuscript [375] contains the definition of the domination relation
7. G.E. Random Reals
314
and its basic properties. The paper Calude, Hertling, Khoussainov and Wang contains the first detailed analysis of the Solovay domination relation. It has been followed by many papers, including Hertling and Wang [234], Hertling and Weihrauch [235], Slaman[369], Kucera and Slaman [266J, Downey, Hirschfeldt and Nies [184], Downey and LaForte [185]' Downey, Hirschfeldt and LaForte [183J, WU [442], Zheng [450], Rettinger, Zheng, Gengler and von Braunmiihl [344], Downey [181]' Downey and Hirschfeldt [182J. See also Calude [60, 66, 61J Theorem 7.108 was proved in Slaman; the final paper, which has appeared as Kucera and Slaman [266], also contains a discussion of early results in the area of random reals published by Demuth [168, 169, 170, 171J. Kucera [265J and Kautz [250J were among the first studies of c.e. degrees of random reals. For example, they observed that 0' is the only c.e. degree which contains random reals. Kucera [265J has used Arslanov's Completeness Criterion to show that all random sets of c.e. T-degree are T-complete. Hence, every Chaitin Omega Number is T-complete. Theorem 7.119 is a stronger result; it summarizes results obtained in Arslanov, Calude [7J, Chaitin [129], and Calude and Nies [95J Theorem 7.96 and other facts regarding the universal splitting property come from Lerman and Remmel [274, 275J. Tadaki [397J has introduced and studied the following generalization of
n: nuD
= "2:;xEdom(U)
real function D
f--+
J D 2 -~ D, where D E (0, 1. The numbers nu and the have very interesting randomness properties.
nS
Exercise 7.8.2 comes from Hartmanis, Hemachandra and Kurtz [226J. Exercise 7.8.4 generalizes Proposition 6.5 in Schnorr [359J. Exercise 7.8.9 comes from Calude and Campeanu [64J. Exercise 7.8.10 was communicated to us by S. Dragomir [186J. Exercises 7.8.23, 25 come from Soare [371 J. Exercise 7.8.26 was proved in Calude and Coles [75J; a simpler proof was discovered by Vereshchagin [416J. Exercise 7.8.31 comes from A. Kucera and Slaman [266J. Exercise 7.8.32 was proved in Downey [181], Exercise 7.8.33 was proved in Downey, Hirschfeldt and Nies [184J and Exercise 7.8.34 comes from Downey and LaForte [185J. Exercises 7.8.35, 36 come from Arslanov [6, 5, 4J. Exercise 7.8.37 comes from Kucera and Terwijn [267J. Kummer [269, 270J is the author of Exercise 7.8.37.
Chapter 8
Randomness and Incompleteness All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident. Arthur Schopenhauer
8.1
The Incompleteness Phenomenon
Godel's Incompleteness Theorem (GIT) has the same scientific status as Einstein's principle of relativity, Heisenberg's uncertainty principle, and Watson and Crick's double helix model of DNA. Incompleteness has captured the interest of many. Many books and thousands of technical papers discuss it and its implications. The March 29, 1999 issue of TIME magazine has included Godel and Turing in its list of the 20 greatest twenty scientists and thinkers of the twentieth century. Interest in incompleteness dates from early times. Incompleteness was an important issue for Aristotle, Kant, Gauss, Kronecker, but it did not have a fully explicit, precise meaning before the works of Hilbert and Ackermann, Whitehead and Russell, Godel and Turing.
In a famous lecture before the International Congress of Mathematicians (Paris, 1900), David Hilbert expressed his conviction of the solvability of
8. Randomness and Incompleteness
316
every mathematical problem:
Wir miissen wissen. Wir werden wissen. 1 Hilbert highlighted the need to clarify the methods of mathematical reasoning, using a formal system of explicit assumptions, or axioms. Hilbert's vision was the culmination of 2000 years of mathematics going back to Euclidean geometry. He stipulated that such a formal axiomatic system should be both 'consistent' (free of contradictions) and 'complete' (in that it represents all the truth). In their monumental Principia Mathematica (1925-1927), Whitehead and Russell developed the first coherent and precise formal system aimed to describe the whole of mathematics. Although Principia Mathematica held great promise for Hilbert's demand, it fell short of actually proving its completeness. After proving the completeness of the system of predicate logic in his doctoral dissertation (1929), Godel continued the investigation of the completeness problem for more comprehensive formal systems, especially systems encompassing all known methods of mathematical proof. In 1931 Godel proved his famous first incompleteness result, 2 which reads: Theorem 8.1 (Godel's Incompleteness Theorem). Every very axiomatic formal system which is (1) finitely specified, (2) rich enough to include the arithmetic, and (3) sound, is incomplete; that is, there exists (and can be effectively constructed) an arithmetical statement which (A) can be expressed in the formal system, (B) is true, but (C) is unprovable within the formal system. Our main example of an axiomatic formal system is the Zermelo-Frankael set theory with choice, Z FC. We fix an interpretation of Peano Arithmetic (PA) in ZFC. Each sentence of the language of PA has a translation into a sentence of the language of Z FC, determined by the interpretation of PAin ZFC. A "sentence of arithmetic" indicates a sentence lWe must know. We will know. 2The second incompleteness result states that consistency cannot be proved within the system.
8.1 The Incompleteness Phenomenon
317
of the language of Z FC that is the translation of some sentence of P A. We will assume that Z FC is arithmetically sound: that is, any sentence of arithmetic which is a theorem of ZFC is true (in the standard model of PA).3 All conditions are necessary. Condition (1) says that there is an algorithm listing all axioms and inference rules (which could be infinite): the axioms and inference rules form a c.e. set. Taking as axioms all true arithmetical statements will not do, as this set is not c.e. But what does it mean to be a "true arithmetical statement"? It is a statement about non-negative integers which cannot be invalidated by finding any combination of nonnegative integers that contradicts it. In Connes' terminology (see [145], p. 6), a true arithmetical statement is a "primordial mathematical reality" . Condition (2) says that the formal system has all the symbols and axioms used in arithmetic, the symbols for 0 (zero), S (successor), + (plus), x (times), = (equality) and the axioms making them work (as, for example, x + S(y) = S(x + V»~. Condition (2) cannot be satisfied if you do not have individual terms for 0, 1,2, .... For example, Tarski proved that the Euclidean geometry, which refers to points, circles and lines, is complete. Finally (3) means that the formal system is free of contradictions. The essence of GIT is to distinguish between truth and provability. A closer analogy in real life is the distinction between truths and judicial decisions, between what is true and what can be proved in court. 4 The essence of the original formulation of GIT involves the set Arith of true arithmetical sentences in which we use the usual operations of successor, addition and multiplication. 5 It reads as follows: Theorem 8.2 (Incompleteness of Arith). There is no formal axiomatic system satisfying all properties (1)-(2) in Theorem 8.1 and proving all true statements of Arith.
Proof Assume by contradiction that Arith is c.e., so there exists a computable function enumerating all elements of Arith. Let F( i) be an arithmetical formula saying that the ith p.c. function 'Pi halts in i, 3The metatheory is ZFC itself; that is, "we know" that P A itself is arithmetically sound. 4The Scottish judicial system which admits three forms of verdict, guilty, not-guilty and not-proven, comes closer to the picture described by GIT. 5 Actually, Godel has investigated the more powerful system constructed in the Russell and Whitehead Principia Mathematica.
318
8. Randomness and Incompleteness
i.e. CPi (i) < 00. It is clear that Arith is capable of expressing F( i). But deciding whether F( i) is true or false is equivalent to solving the Halting Problem. If there is no mechanical procedure for deciding the Halting Problem,6 then there is no complete set of underlying axioms either. Indeed, if there were, they would provide a (tremendously long) procedure for running through all possible proofs to show which programs halt! 0 The above reasoning is important not only for justifying the GIT for Arith, but also because it shows that the details of the formal axiomatic system are not relevant for GIT! Indeed, we can ignore anything regarding the inner mechanism of the system, what the axioms are or what logic is used. What is important is the fact that there should be a proof-checking algorithm, an algorithm which may help to run through all possible proofs in size order, see which ones are correct and then print out all and only all theorems. This is impractical, but conceptually important:
the essence of a formal axiomatic system is the fact that its theorems form a c. e. set (under a suitable codification). So, we are now in a position to reformulate the GIT for Arith as: Theorem B.3. The set Arith is not c. e.
As Chaitin has observed, there is more information in the above argument than in the original proof due to G6del. Following G6del we know that the axiomatic formal system is incomplete; however,
there still might be a mechanical procedure to decide if a given assertion is true or false! This possibility was ruled out by the above argument. GIT ended a hundred years of attempts to establish axioms to put mathematics on an axiomatic basis. GIT does not destroy the fundamental idea of formalism, but suggests that a) mathematics will be described by many formal systems as opposed to a universal one, b) a more sophisticated and comprehensive form of formal system than that envisaged by Hilbert is required (see also Post [337]). 6 An information-theoretic proof of the undecidability of the Halting Problem will be presented in Section 9.2.
8.1 The Incompleteness Phenomenon
319
Anticipating resistance to his results, Godel wrote his papers very carefully. He took pains to convince various people about the validity of his assertions and results, but he avoided any public debate and considered his results to have been accepted by those whose opinion mattered to him; see Dawson [161]. Unlike the other critics, Post expressed "the greatest admiration" for Godel's work, conceding that after all it is not ideas but the execution of ideas that constitute{s} . .. greatness. Godel's result provoked Hilbert's anger, but he apparently accepted its correctness (cf. [161]). Hilbert never cited Godel's work. There is a variety of reactions in interpreting GIT, ranging from pessimism to optimism or simple dismissal (as irrelevant for the practice of mathematics). For pessimists, this result can be interpreted as the final, definite failure of any attempt to formalize the whole of mathematics. For example, H. We yl acknowledged that GIT has exercised a "constant drain on the enthusiasm" with which he engaged himself in mathematics, and for S. Jaki, GIT is a fundamental barrier in understanding the Universe. In contrast, scientists like F. Dyson acknowledge the limit placed by GIT on our ability to discover the truth in mathematics, but interpret this in an optimistic way, as a guarantee that mathematics will go on forever (see Barrow [16], pp. 218-221). A lucid analysis of the impact of GIT in physics is presented in Barrow [17]. The reactions of two great philosophers are also of interest. Wittgenstein's negative comments (dated 1938 and posthumously published in "Remarks on the foundations of mathematics" in [436]) are now generally considered an embarrassment in the work of a great philosopher. Russell realized the importance of Godel's work, but expressed his continuous puzzlement in a rather ambiguous way in a letter from 1 April 1963 (addressed to L. Henkin; see [161]): Are we to think that 2+2 is not 4, but 4.001? Following the same source, Godel remarked (in a letter addressed to A. Robinson) that "Russell evidently misinterprets my result; however he does so in a very interesting manner ... ". In the long run Godel's own interpretations of incompleteness prevailed: GIT neither rejected the notion of formal system (quite the opposite) nor caused despair over the imposed limitations. It reaffirms the creative power of human reason. In Post's celebrated words: mathematical proof is {an} essentially creative (activity).
320
8. Randomness and Incompleteness
How large is the set of true and unprovable statements? If we fix a formal system satisfying all three conditions (1)-(3) in Theorem 8.1, then the set of true and unprovable statements is topologically "large" (constructively, a set of second Baire category, and in some cases even "larger"), cf. Calude, Jurgensen and Zimand [91]. No probabilistic similar result has been (yet?) proven. As we shall see later in this chapter (e.g. in Corollary 8.8), AIT forms of GIT suggest reinforcement of the above results: incompleteness is not an accident, it is a pervasive phenomenon. This raises the natural question (see Chaitin [135]): "How come that in spite of incompleteness, mathematicians are making so much progress?"
8.2
Information-theoretic Incompleteness (1)
This section presents the first information-theoretic approach to incompleteness. Incompleteness asserts a coding impossibility: an axiomatic system satisfying properties (1)-(3) in Theorem 8.1 does not have enough resources to "code" all true statements which it can express. Is it possible to get a more quantitative form of this fact? AIT is able to shed more light on GIT by analysing, following Chaitin [113, 115, 120, 122, 123, 125]1 the reason for this phenomenon. The main result can be informally stated as: An axiomatic formal system of complexity N cannot yield a theorem that asserts that a specific object is of complexity substantially greater than N. We consider an axiomatic formal system F whose rules of inference form a c.e. set of ordered pairs of the form
< a,T > indicating that the theorem T is deductible from the axiom a:
7See van Lambalgen [412] or Raatikainen [340] for critical discussions. 80 ne often writes a I- F T instead of < a, T >.
8.2 Information-theoretic Incompleteness (1)
321
So, F is fixed and a - which is a string via some standard encoding varies. The first information-theoretic version of GIT (see [123, 122, 125, 131, 136]) reads: Theorem 8.4. (Chaitin Information-theoretic Incompleteness (I». We consider an axiomatic formal system Fa consisting of all theorems derived from an axiom a using the rules of inference F. There exists a constant CF - depending upon the formal system Fa - such that if
a r-F "H(x) > n" only if H(x) > n, then a
only if n < H(a)
r-F "H(x) > n"
+ CF.
Proof. We shall present three proofs. Information-theoretic direct proof. We consider the following Chaitin computer C: for u, v E :L:* such that U(u) = string(k) and U(v) = a we put the first string s that can be shown in Fa
C(uv)
to have complexity greater than k
+ Ivl.
Note that in the above definition "first" refers to the quasi-lexicographical order. To understand how C actually works just notice that the set
Fa = {T I a r- F T} = {T
I
}
is c.e. Among the admissible inputs for C we may find the minimal self-delimiting descriptions for string( k) and a, i.e.
u = (string(k))*, v = a*, having complexity H(string(k)), H(a), respectively. If C(uv)
= s, then Hc(s) ::::::
luvl : : :
l(string(k))*a*l·
322
8. Randomness and Incompleteness
On the other hand, using the Invariance Theorem for U and C we get a constant d such that
k + la*1 < H(s) :::; l(string(k))*a*1
+ d.
We therefore get the following inequalities:
k + H(a) < H(s) :::; H(string(k))
+ H(a) + d,
hence
k < H(string(k))
+ d = o (log k),
which can be true only for finitely many values of the natural k. We now pick CF = k, where k is a value that violates the above inequality. We have proven that s cannot exist for k = CF, i.e. the theorem is proved.
Recursion-theoretic proof. Recall that d( x) is a self-delimiting version of the string x. Let (Ce)eE~* be a c.e. enumeration of all Chaitin computers. We construct the Chaitin computer
Cw(d(x))
=
y, if y is the first string such that a statement
of the form "Cx(d(x))
Fa and z
=1=
z" is provable in
= y.
We prove first that
Cw(d(w))
= 00.
Indeed, if Cw(d(w)) =1= 00, then Cw(d(W),A) = y, for some string y E L;*; we admit that y is the first such string. On the other hand one has
a r-F "Cw(d(w), A)
=1=
y",
and, in view of the soundness of the formal system,
Cw(d(w), A)
=1= y.
We thus have a contradiction. The set of axioms a augmented with the axiom
{Cw(d(w), A)
=
y}
is consistent, for every string y. Otherwise we would have
a r-F "Cw(d(w), A)
=1=
y",
8.2 Information-theoretic Incompleteness (1)
323
for some string y, a false relation. Finally, the set of axioms a augmented with the axiom
{H(y) ::; Id(w)1 + c} (c comes from the Invariance Theorem applied to C w and U) is also consistent, showing that in the formal system Fa one cannot deduce any statement of the form "H(y) > Id(w)1 + c".
Information-theoretic indirect proo]. We delete from the list of theorems all statements which are not of the form "H(y) > m" - this operation can be effectively performed, so it may increase the complexity by at most a constant factor - and identify the set of theorems with a c.e. subset with Godel number e of the set on
{< w,m >E
L;* x N I H(w)
> m}.
In view of Theorem 5.33 all codes of theorems are bounded in the second argument by a constant (not depending on e), thus completing the proof.
o Remark. A false reading of Theorem 8.4 might say that the complexity of theorems proven by Fa is bounded by H(a) + CF. Indeed, if the set of theorems proven by Fa is infinite, then their program-size complexities will be arbitrarily large. How does Theorem 8.4 compare with Theorem 8.2? To answer this question we need need a result of the type Theorem 8.3 for Theorem 8.4. This is Theorem 5.31 (more precisely, in its proof we showed that the set C = {< w,m > E L;* x N I H(w) > m} is immune). Of course, every immune set is not c.e. and the converse implication is not generally true. Is Arith immune? The answer is negative as it is clear that Arith has infinite c.e. subsets. To understand better that immunity is a stronger form of non-computability than non-c.e., let us stop for a moment and describe a set which is not immune. Following Delahaye [164] such a set A may be called "approximable" as it is either finite or contains a c.e. set B, so A = Un::::l An, where
An = (A n {x E L;* i.e. A is a union of c.e. sets.
In 2': Ixl}) U B,
324
8. Randomness and Incompleteness
To conclude, Theorem 8.4 is stronger than Theorem 8.2. Recognizing high complexity is a difficult task even for ZFC. The difficulty depends upon the choice of U: some U's are worse than others. Raatikainen [340] has shown that there exists a universal Chaitin computer U so that Z FC, if arithmetically sound, can prove no statement of the form "Hu(x) > n". It follows that ZFC, if arithmetically sound, can prove no (obviously, true) statement of the form "Hu(x) > 0".
8.3
Information-theoretic Incompleteness (2)
Consider now a Diophantine equation, i.e. an equation of the form
P(n, x, YI, Y2,···, Ym) = 0, where P is a polynomial with integer coefficients. The variable n plays an important role as it is considered to be a parameter; for each value of n we define the set Dn
= {x
E N I P(n,
x, YI, Y2, ... , Ym) = 0, for some YI, Y2, ... , Ym
E Z}.
It is clear that for every polynomial P of m + 2 arguments the associated set Dn is c.e. By Matiyasevich's Theorem, every c.e. set is of the form Dn-
In particular, there exists a universal polynomial P such that the corresponding set Dn encodes all c.e. sets. So, P(n, x, YI, Y2, ... , Ym)
= 0,
(8.1)
iff the nth computer program outputs x at "time" (YI, Y2, .. . ,Ym). The diagonal set is not c.e., so there is no mechanical procedure for deciding whether equation (8.1) has a solution. In other words, no system of axioms and rules of deduction can permit one to prove whether equation (8.1) has a solution or not. Accordingly, we have obtained the following: Theorem 8.5 (Diophantine Form of Incompleteness). No formal axiomatic formal system with properties (1)-(3) in Theorem 8.1 can decide whether a Diophantine equation has a solution or not.
8.3 Information-theoretic Incompleteness (2)
325
Is there any relation between randomness and the sets of solutions of Diophantine equations? The answer is affirmative. For technical reasons we shall deal with exponential Diophantine equations, the larger class of equations which are built with addition, multiplication and exponentiation of non-negative integers and variables. Consider also an Omega Number Ou. First we prove the following technical result:
Theorem 8.6 (Chaitin). Given a universal Chaitin computer U one can effectively construct an exponential Diophantine equation
P(n, x, Yl, Y2, ... , Ym) = 0
(8.2)
such that for every natural fixed k the equation P(k, x, Yl, Y2, ... , Ym) = 0
has an infinity of solutions iff the kth bit of the binary expansion Ou is 1.
Proof Consider the sequence of rationals (7.1) defining Ou and note that the predicate "the nth bit of Ou(k) is I" is computable. Using now Jones and Matiyasevich's Theorem 9 one gets an equation of the form (8.2). This equation has exactly one solution Yl, Y2, ... , Ym if the nth bit of Ou(k) is 1, and it has no solution Yl, Y2, ... , Ym if the nth bit of Ou(k) is O. The number of different m-tuples Yl, Y2, ... , Ym of natural numbers which are solutions of the equation (8.2) is therefore infinite iff the nth bit of the base 2 expansion of Ou is 1. 0 It is interesting to remark on the sharp difference between the following two questions:
1. Does the exponential Diophantine equation P
= 0 have a solution?
2. Does the exponential Diophantine equation P of solutions?
= 0 have an infinity
The first question never leads to randomness. If one considers such an equation with a parameter n, and asks whether or not there is a solution 9S ee Theorem 1.3.
8. Randomness and Incompleteness
326
for n = 0,1,2, ... , N -1, then the N answers to these N questions contain only log2 N bits of information. Indeed, we can determine which equation has a solution if we know how many of them are solvable. The second question may sometimes lead to randomness, as in Theorem 8.6. It is remarkable that Chaitin [121 J has effectively constructed such an equation; the result is a huge equation.lO We are now in a position to prove the second information-theoretic version of GIT (see [123, 122, 125, 131]):
Theorem 8.7. (Chaitin Information-theoretic Incompleteness (II». Assume that the set of theorems of a formal axiomatic system T is c. e. If T has the property that any statement of the form "the nth bit of nu is a 0", "the nth bit of nu is a 1 ", can be represented in T and such a statement is a theorem of T only if it is true, then T can enable us to determine the positions and values of at most finitely many scattered bits of nu. Proof We will present two proofs. First proof. If T provides k different bits of nu, then it gives us a covering Coverk of measure 2- k which includes nu. Indeed, we enumerate T until k bits of nu are determined, and put
IXll = i l -1, IX21 = i2 - i l -1, ... , IXkl = ik - ik-l -1} C {a, 1}* (i l < i2 by T).
< ... < ik
are the positions where the right 0/1 choice was given
Accordingly, p,(CoverdO, 1}W)
= 2ik - k /2 ik = 2- k ,
and T yields infinitely many different bits of randomness of nu.
nu, which contradicts the
10 A 900,000-character 17,000-variable universal exponential Diophantine equation. See also the recent software in [130J.
8.3 Information-theoretic Incompleteness (2)
327
Second proof. Assume that T may give us an infinity of positions and corresponding values of n. Then we can get an increasing function i N ~ N such that the set
{(i(k), ni(k)) I k :2: o} is computable. Then, by virtue of Theorem 6.41, the sequence r2(nU) is not random, a contradiction. 0
In fact one can give a bound on the number of bits of nu which ZFC can determine; this bound can· be explicitly formulated, but it is not computable. For example, in [130] Chaitin has described, in a dialect of Lisp, a universal Chaitin computer U and a formal axiomatic system T satisfying properties (1)-(3) in Theorem 8.1 such that T can determine the value of at most H(T) + 15,328 bits of nu (an uncomputable number). Consider now all statements of the form "The nth binary digit of the expansion of for all n :2: 0, k
nu is k",
(8.3)
= 0, 1.
Theorem 8.7 can be restated in the following form which shows the pervasive nature of incompleteness:
Corollary 8.8 (Chaitin). If ZFC is arithmetically sound and U is a universal Chaitin computer, then almost all true statements of the form (8.3) are unprovable in T. To compare Theorem 8.4 with Theorem 8.7 we need the following:
Definition 8.9. A set of non-negative integers A is called random if sequence x = XIX2 ••• X n . .. defined by Xi
= { 1,
0,
if i E A, if i rf- A,
is random. Random sets are immune, but the converse is not necessarily true. In particular, the immune set C in Theorem 5.31 is not random, hence Theorem 8.7 is stronger than Theorem 8.4. Indeed, the analogue of Theorem 5.31 is:
328
8. Randomness and Incompleteness
Theorem 8.10. The set A of non-negative integers n such that ZFC proves a theorem of the form (8.3) is random. Remark. Of course, stronger and stronger forms of incompleteness can be imagined just following, for example, the arithmetical hierarchy. As noted by Delahaye [164], the beauty of the information-theoretic forms of incompleteness is given by the natural and simple constructions.
8.4
Information-theoretic Incompleteness (3)
In this section we fix T = ZFC. Note that each statement of the form (8.3) can be formalized in P A. Moreover, if U is a Chaitin computer which P A can prove universal and ZFC proves the assertion (8.3), then this assertion is true. By tuning the construction of the universal Chaitin computer, Solovay [377] has obtained a dramatic improvement of Corollary 8.8: Theorem 8.11 (Solovay). We can effectively construct a universal Chaitin computer U such that ZFC, if arithmetically sound, cannot determine any single bit of nu. Note that Corollary 8.8 holds true for every universal Chaitin computer U (it is easy to see that the finite set of (true) statements of the form (8.3) which can be proven in ZFC can be arbitrarily large) while Theorem 8.11 constructs a specific U. We will first obtain a stronger result Theorem 8.12 - from which Theorem 8.11 follows.
In what follows, if j is one of 0 or 1, the string of length 1 whose sole component is j will be denoted by (j). Theorem 8.12 (Calude). Assume ZFC is arithmetically sound. Let i 2: 0 and consider the c. e. random real
Then, we can effectively construct a universal Chaitin computer, U (depending upon ZFC and a.), such that the following three conditions are satisfied:
8.4 Information-theoretic Incompleteness (3)
329
a)
P A proves the universality of U.
b)
Z FC can determine at most i initial bits of 0, u.
c)
et=nu.
Proof We start by fixing a universal Chaitin computer V such that the universality of V is provable in PA and nv = et. We use Theorem 7.109 ~d Exercise 7.8.22 to effectively construct a universal Chaitin computer V such that
nv =
0.~eti+1eti+2"" i Os
if i ~ 1, and a universal Chaitin computer
V such that
nv = 0. et l et 2"·' in case i = O. Next we construct, by cases, a p.c. function W(l, s) (l is a non-negative integer and s E I:*) as follows: Step 1:
Set W(l,A) to be undefined.
Step 2:
If i = 0, then go to Step 6.
W(l, (1))
Otherwise, set
= W(l, 10) = ... = W(l,~O) =
A.
i Is
If s = OOt, for some tEI:* , then set
Step 3:
W(l, s)
= V(t),
and stop. Step 4:
If s = Oli, for some tEI:* , then go to Step 5.
Step 5: List all theorems of Z FC, in some def ini te order, not depending on t, and search for a theorem of the form (8.3). If no such theorem is found, then W(l,s) is undefined, and stop. If such a theorem is found, then let n, l, k be its parameters .
It I =1= n, It I
then W(l,s) is undefined, and stop.
•
If
•
If n, then let r be the unique dyadic rational, in [0,1), whose binary expansion is t(k) and set r' = r + 2-(n+1). Search for the least integer m such that ndm] E (r, r') . If this search fails, or s E Dz[m], then W(l,s) is undefined, and stop. In the opposite case set W(l, s) = A, and stop.
330 Step 6:
8. Randomness and Incompleteness If
8
= (O)t,
for some string t, then set
111(1,8) = lI(t), and stop. Step 7:
If
8
= (l)t,
for some string t, then go to Step 5.
The Recursion Theorem 1.1 provides a j such that !.pj(8) = 111(j,8). We fix such a j and set U = !.pj. We will show that U is a universal Chaitin computer which satisfies conditions a)-c). First we prove that U is a Chaitin computer. Let i = O. Suppose that and 82 are in the domain of U and 81
[0,1), b(x) = 2x (mod 1). Here "mod I" means "ignore the integer part". If we choose the infinite binary representation for reals x E [0,1), then b can be regarded as the (computable) map b: {O,I}W --> {O,I}W,
Consider a system in which a state is an element of {O,I}W and the evolution is given by Baker's map. We assume that time is discrete. So, starting from each state x E {O, I}W we obtain the trajectory
bO(x)
x,
b1 (x)
b(x),
2For a more general discussion see Moore [315].
9.5 Randomness and Cellular Automata
b2 (x) b3 (x)
b(b(x)), b(b(b(x))),
bn(x)
b( ... b(b(x)) .. .),
367
Given an initial part of the sequence
we would like to compute the next state bn(x) and if this is not possible, then we would like to compute a "prediction" for bn(x) which should be a better approximation of the true value than a random coin toss. If the initial state x is "randomly drawn" from {O, 1}W, then according
to Corollary 6.32 with probability 1 x is random, hence due to Theorem 6.40 each element of the trajectory (bn(x))n is random. So, the behaviour of the system cannot be predicted better than a coin toss, a conclusion argued by Ford in [193J. See also Wolfram [438, 439], White [433J, Batterman, White [21 J, Brudno [50J, Calude and Dumitrescu [80J, Fouche [194J.
9.5
Randomness and Cellular Automata
Cellular automata have been introduced by Ulam and von Neumann [424J as models for natural complex systems, especially self-reproducing biological systems. Since then they have been analysed in many other contexts, e.g. for the simulation of physical phenomena, for computability questions (cellular automata form a universal model of computation), for random number generation, in the framework of formal language theory, in symbolic dynamics, and many more. See, for example, Wolfram [437J and other papers in the same volume [438, 439], Toffoli and Margolus [401J and Lind and Marcus [285J. Cellular automata show a uniform behaviour in a certain region of the space. They operate on configurations which consist of a discrete lattice of cells each of which is in one of finitely many states. Time is discrete; at each time step the value of each cell is updated uniformly according to a finite set of rules. The new value of a cell depends only on the current
368
9. Applications
values of finitely many cells in its neighbourhood. Although cellular automata can be described easily by a finite set of rules (the local function) they exhibit a rich and complicated global behaviour which often seems chaotic or random (see Wolfram [438]). In what follows we will follow Calude, Hertling, Jurgensen and Weihrauch [83] to give several rigorous mathematical characterizations of random configurations and analyse the behaviour of cellular automata on random and non-random configurations. We fix an alphabet I.; with Q 2: 2 elements, and a positive integer d 2: l. Then Zd is the d-dimensionallattice over the integers Z. The space I.;Zd is called a full shift space. We call the elements of I.; the states, the number d the dimension, and the elements c E I.;Zd the configurations of the full shift space. For a configuration c E I.;Zd and a E Zd we write Ca instead of c(a); elements of Zd will be sometimes called cells and Ca will then be the content of cell a. For r EN, let [-r, r] denote the set {-r, ... ,0, ... ,r}. On the spaces I.;Zd we use the product topology induced by infinitely many copies of the discrete topology on the finite space I.;. The space I.;Zd is compact because it is a countable product of compact spaces (Tychonoff's Theorem). This space is in fact a metric space. One can, for example, use the metric dist defined by dist( c, c') = 2- m (c,c'), where
m(c, c') = min{r EN for c, c' E
I.;Zd;
I Ca i- c~,
for some a E [-r, r]d},
here min 0 = 00. The sets
{c E
I.;Zd
I Cz = s},
S
E '" LJ,
z E Zd
form a subbase of the topology on I.;Zd. Cellular automata operate on full shift spaces. The name shift spaces comes from the fact that the shift mappings on the space I.;Zd play an important role. Each integer vector a = (al, ... , ad) E Zd induces a bijection O"id) : I.;Zd -> I.;Zd defined by O"id) (ch = cb+a, for every b E Zd; it is called the shift map associated with a. The superscript (d) will be omitted when the dimension is clear from the context. The shift map 0" ei associated with the unit vector ei = (0, ... ,0,1,0, ... ,0) E Zd having a one in position i and zeros in all other positions is also written O"i. The shift mapping 0"1 is the usual left shift in the one-dimensional case. Next we define a random configuration of a full shift space. First let us look at the simplest case, when the dimension d is equal to 1. The
369
9.5 Randomness and Cellular Automata
simplest way to define randomness for two-way infinite sequences over ~, that is, for elements of ~z, is to use a standard computable bijection from Z to N, e.g. the bijection (... ) : Z ---+ N defined by
() {2Z' Z
=
if 2( -z) - 1, if
Z:2 0, Z
< O.
This bijection induces a bijection from ~w to ~z in an obvious way: one maps an element x = XIX2 ••• E ~w to the two-way sequence q = (qz)z E ~z defined by qz = x(z), for all Z E Z.
Definition 9.6. A two-way infinite sequence q E ~z is called random if the corresponding one-way infinite sequence x E ~w is random. This procedure can also be carried out in the case of any dimension d :2 1. To this end we use a computable bijection from Zd onto N. For example, the mapping 7f : N 2 ---+ N defined by 7f(i,j) = ~(i + j)(i + j + 1) + i is a bijection. For d :2 2 we define (... ) : Zd ---+ N recursively by (Zl,"" Zd) = 7f((Zl), (Z2,'" ,Zd))' This is a computable bijection for each d:2 1. If Ll and L2 are countable sets, then a total mapping f a mapping l : ~L2 ---+ ~Ll via
: Ll
---+
L2 induces
for all p E ~L2 and hELl. If f is a bijection, then also l is a bijection. Hence, for each d :2 1, the induced mapping ~w ---+ ~Zd is a bijection. It is clear that it is even a homeomorphism that induces a bijection of the following subbases of the respective topologies: the preimage under of the cylinder {c E ~Zd I Cz = s} C ~Zd, for s E ~ and Z E Zd, is the cylinder {c E ~w I c(z) = s}, and these sets form a subbase of the product topology on ~w.
n :
n
Furthermore, if we consider the product measure ji, on ~w and ji, on ~Zd of the uniform measure fJ, on ~, then is also measure-preserving, i.e.
n
d
---
for all open U c ~z . Thus, the mapping ( ... ) shows that the spaces ~w and ~Zd are identical with respect to topology and measure. Using these considerations we can give the following:
9. Applications
370
Definition 9.7 (Calude-Hertling-Jiirgensen-Weihrauch). A two-way sequence c E L;Zd is called random if the one-way infinite se---1 quence ( ... ) (c) E L;w is random. Does the construction above depend upon the bijection (- .. ) : Zd - t N? Does the choice ofthe bijection influence the definition? Certainly it does, because the notion of randomness for elements of L;w is not invariant under an arbitrary permutation of its entries.
Example 9.8. For every sequence c E L;w J there exists a bijection 'lj; N - t N such that the sequence C?j;(1)C?j;(2) .•. E L;w is not random. Proof If the sequence C1 C2 • .. is not random we can take 'lj; to be the identity. Otherwise we can assume, without loss of generality, that L; = {O, 1, ... ,Q - I}, for some q 2: 2. Some element of L; appears in the sequence infinitely many times, say Ci = 0, for infinitely many i. Let f : N - t N be the unique increasing function such that cf(i) is the (i + 1)st zero in C1 C2 •.• , for all i. We define 'lj; by f(2j
'lj;(i) = { f(2j) i,
+ 1),
+ 1,
if i if i if i
= =
f(2j) + 1, f(2j + 1),
1- UjEN {f(2j) + 1, f(2j + I)}.
Then the sequence C?j;(l) C?j;(2) •.. does not contain an isolated zero, hence it does not contain the string 101. Consequently, it is not random by Theorem 6.50. D
Remark. In view of Exercise 7.8.8, if 'lj; : N - t N is a computable bijection, then a sequence CI C2 .• , E L;w is random iff the sequence C?j;(1)C?j;(2) •.. E L;w is random. Hence, if we choose a bijection b : Zd - t N such that (- .. ) 0 b- 1 is computable, then we obtain via b the same randomness notion on L;Zd as via the bijection There is another more direct way to define randomness on full shift spaces, without reference to random one-way infinite sequences: we will use the Hertling-Weihrauch topological approach discussed in Section 6.6. In order to view the full shift space L;Zd as a randomness ~pace (L;Zd, B, ji,) we have to describe the measure ji, and the numbering B of a subbase of the topology. The measure ji, is given by
n.
ji,({c E L;Zd I Cz = s}) = I/Q,
9.5 Randomness and Cellular Automata for s E ~ and z E Zd. The numbering -
B·J+ Q·\Zl,··.,Zd I )
= {c E
~
371
H is defined by Zd
I c(Zl,···,Zd ) = s'} J,
for l"'5:j"'5: Q and (Zl, ... ,Zd) E Zd. If (... ) is the bijection from Zd to N defined above, then we obtain the same randomness notion as in Definition 9.7. In fact a more general result is true. Before stating and proving it we will give another characterization for computable sequences of open sets in ~Zd. For an arbitrary finite set A C Zd and v E ~A we set
[VJ={CE~Zd Icz=v z , for all zEA}. The set
Cubes (~, d)
=
U ~[-r,rld r2:0
is countable. The sets [vJ for elements v E Cubes (~, d) form a base of the topology on ~Zd. We define the "length-lexicographical" bijection Cube: N -> Cubes (~, d) in the following way. For fixed r 2:: 0 we define an ordering between the cells in [-r, rJd by Z < if (z) < (Z) for z, E [-r, rJd. With respect to this ordering on [-r, rJd and a fixed ordering on ~ we consider the lexicographical ordering on ~[-r,rld. Finally we define Cube in such a way that first Cube lists all elements in ~[O,Old according to their lexicographical order, then all elements in ~[-l,lld according to their lexicographical order, then all elements in ~[-2,2ld according to their lexicographical order, and so on. The following result is easy, but useful.
z
z
Lemma 9.9. Consider a sequence (Ui)i of open subsets of ~Zd. Then, the following conditions are equivalent: 1.
The sequence (Ui) is H'-computable.
2.
The sequence (Ui) is Cube-computable.
3.
The sequence computable.
C00- 1(Ui))i
of open subsets of ~N is (v(j)~N)r
Theorem 9.10. Let d 2 1 be a positive integer. For a two-way sequence c E ~Zd the following conditions are equivalent: 1.
The infinite one-way sequence n-1(c) E ~N is random.
9. Applications
372
2.
The two-way sequence c zs a random element of the randomness Zd space (~ ,B, ji,).
Proof The equivalence follows from Lemma 9.9 and from the fact that the homeomorphism R : ~w - t ~Zd is measure-preserving. 0 Remark. If (Ui)i is a universal Martin-Lof test on ~w, then (R(Ui))i is a universal Martin-Lof test on ~Zd. In the case of dimension d = 1 the first of the conditions in Theorem 9.10 says that a two-way infinite sequence c = ... C-3C-2 C-lCOCI C2 C3 ... E ~z is random iff the one-way infinite sequence C-lClC-2 C2 C-3 C3 ... E ~w is random. This is also equivalent to the following condition: 3. The one-way sequences co, Cl, C2, ... and dom.
C-l, C-2, C-3, ...
are ran-
Comment. This last condition is often expressed by saying that the sequences (co, Cl, C2, ... ) and (C-l' C-2, C-3,"') are "independently random". Next we will use Martin-Lof tests to get more insight into the nature of randomness of two-way sequences. One must distinguish between MartinLof tests for two-way infinite sequences and for one-way infinite sequences. Let (Ui)i be a universal Martin-Loftest on the space (~W, B, ji,) of one-way infinite sequences, and let A c N be a c.e. set such that
u
iEN,1f(n,i)EA for all n (here v : N - t ~* is the standard computable bijection). Let An = {v(i) I 11"(n, i) E A}, for all n. We assume without loss of generality that all sets An are suffix-closed, i.e. if a prefix of a string w is contained in An, then also w itself is in An- Then a two-way infinite sequence c = ... C-3C-2C-lCoCIC2C3 ... E ~z is non-random iff for each n E N there is an mEN with COC-lClC-2C2 ... C-mCm E An. But
notice
that
we
cannot
replace
COC-lClC-2C2 ... C-mCm
C- m · .. C-lCOCl ... Cm:
Proposition 9.11. Every random two-way infinite sequence _ ... C-2C-lCOClC2 ... E "Z L..J
c -
by
9.5 Randomness and Cellular Automata
373
has the property that for every n E N there is an mEN with C- m ... C-ICOCI ... Cm
E
An.
Proof Let us fix a number n and an arbitrary string w = WI ... WI E An. For every random sequence C = ... C-2C-ICOCIC2 ... E I.;z there exists an m > l such that C- m ... C-m+l-I = w, hence the string w is a prefix of C- m ... C-ICOCI .. , em. Because An is assumed to be suffix-closed we conclude that C- m ... C-ICOCI ... Cm E An. 0
Next we observe that the shift mappings preserve randomness.
Proposition 9.12. Let d 2:: 1 and a E Zd an integer vector. If C E is random, then also O"a(c) is random.
I.;Zd
Proof. If (Ui) is a Martin-Lof test on I.;Zd, then also ((O"a)-I(Ui))i is a Martin-Lof test on I.;Zd, for arbitrary a E Zd. Assume that O"a(c) is non-random. Then there is a Martin-Lof test (Ui) on I.;Zd with O"a(c) E niEN Ui. Then also C E niEN(O"a)-I(Ui). We conclude that c is nonrandom as well. 0
Definition 9.13. Two configurations c(1), c(2) E I.;Zd are called equivalent (we write: c(1) =Shift c(2)) if one of them can be obtained by shifting the other one appropriately, i. e. if there exists an integer vector a E Z d with c(2) = O"~d)(c(1)). This defines an equivalence relation on the space I.;Zd, and often instead of the space I.;Zd one considers the quotient space I.;Zd / =Shift obtained by identifying equivalent configurations. Proposition 9.12 tells us that the randomness notion on I.;Zd induces a natural randomness notion on this quotient space. Is it also possible to obtain this randomness notion directly by applying the definition of a randomness space to the quotient space? It is interesting that this is not the case, at least not by using the quotient topology on the quotient space. We give the reason for the one-dimensional case. We need first to define a new notion, namely that of the rich two-way sequence, the two-way analogue of the disjunctive one-way sequence.
Definition 9.14. Let A, B C Zd be two finite sets and an integer vector a E Zd. The sets A, B are called a-equivalent if A = a + B. Two
9. Applications
374
elements v E ~A and w E ~B are called equivalent if there exist an integer vector a and two a-equivalent finite sets A, B such that va+b = Wb, for all b E B. The equivalence classes of elements of ~A for finite subsets A c Zd are called patterns (of dimension d over ~). The equivalence classes of elements of ~{1,2, ... ,n}d for any positive integer n are called cube patterns.
The number n is called the side length of the cube pattern. Definition 9.15. We say that a pattern, given by a representative W E ~A for some finite set A C Zd, occurs in c E ~Zd if there exists an integer vector b E Zd such that cb+a = Wa for all a E A. A two-way sequence c E ~Zd dimension d occurs in c.
is
called rich if every pattern over ~ and of
It is clear that a configuration is rich iff every cube pattern (over ~, of dimension d) occurs in c.
Remark. In contrast to randomness richness is very fragile even under the computable rearrangement of sequences. Indeed, if a one-way infinite sequence C = COCI C2 . " E ~w is rich, then also the two-way infinite sequence H(c) = ... C3CICoC2C4 ... E ~z is rich, but the converse is not true. To see this let c = COCIC2 . " be a one-way rich sequence and define another one-way sequence c by C2i = Ci and C2i+l = s for all i where s is a fixed element of~. Then c is not rich, but the corresponding two-way sequence
H(c)
= ... sSCoC2C4···
is rich. By choosing a different bijection from Z to N one can achieve the equivalence of the richness notions on ~w and ~z. It is not difficult to check that a two-way sequence c = ... C-2C-ICOCIC2 ... is rich iff the following one-way sequence is rich:
... C-15 Cll ... C15 ....
Finally, note also that in contrast to randomness, richness is not base invariant.
9.5 Randomness and Cellular Automata
375
Lemma 9.16. Every random configuration is rich. Proof We fix an arbitrary cube pattern. By a simple counting argument one can easily prove in an effective way that the set of all configurations which do not contain this pattern has measure zero. Therefore all such configurations are non-random. Since this is true for all cube patterns, it follows that all random configurations are rich. 0
Remark. In fact, much more is true. One can define in a natural way normal configurations, in which all patterns occur with the expected frequency. In the same way as one proves that every random real number has a normal binary expansion, one can also prove that every random configuration is normal. It is clear that every normal configuration is rich. We can now come back to the problem of randomness on the quotient space. A base of the quotient topology on ~z / =Shift is given by the sets {[C]=Shift ICE
~Zd and c contains the string w},
for arbitrary w E ~*. But any of these basic open sets contains the =shift-equivalence classes of all rich sequences! Hence, any open set in the quotient space contains the =shift-equivalence classes of all rich sequences. Especially, for any sequence (Ui)i of open subsets Ui of the quotient space, the =shift-equivalence classes of all rich sequences lie in the intersection nEN Ui . Therefore, any Martin-Lof test on the quotient space would show that these classes are non-random. Hence, the direct approach via Martin-Lof tests cannot give the "most natural" randomness notion on the quotient space ~ Zd / =Shift. Cellular automata are continuous functions which operate on a full shift space ~Zd and commute with the shift mappings O"a, for a E Zd. Definition 9.17. A cellular automaton (in short, CAY is a triple (~, d, F) consisting of a finite set ~ containing at least two elements, called the set of states, a positive integer d, called the dimension, and a continuous function
which commutes with the shift mappings O"i for i F is called the global map of the CA.
= 1, ... ,d.
The function
9. Applications
376
The usual definition of CA involves the so-called local function. Since the space I.;Zd is a compact metric space any continuous function F : I.;Zd - t I.;Zd is uniformly continuous. Hence, if F is continuous and commutes with the shift mappings, then there exist a finite set A C Zd and a function f : I.;A - t I.; such that
and bE Zd, where Cb+A E I.;A is defined in an obvious way: for all a E A. The function f is called a local function for F which is induced by f. for all C E
I.;Zd
(Cb+A)a = Cb+a,
Obviously, one could choose A to be the d-dimensional cube [-r, rjd, for some sufficiently large r. On the other hand it is clear that any function F induced by a local function f is the global map of a CA. Whenever we consider a local function for some CA we will assume that there is a natural number r such that f maps I.;[-r,rjd to I.;. The number r will be called the radius of f. Let f : I.;[-r,rjd - t I.; be a local function with radius r. It induces a function f* mapping any v E I.;[-k,kjd for arbitrary k :2 2r + 1 to an element f*(v) E I.;[-k+r,k-rjd in an obvious way. This function induces a mapping fpattern which maps any cube pattern of side length k for any k :2 2r + 1 to a cube pattern of side length k - 2r in an obvious way.
= 2 and a local function f : I.;[-r,rj2 - t I.; with radius r. We take a square pattern P with k·k cells, for some k :2 2r + 1. For simplicity let us assume that the indices of the cells are running from 1 to k, in both dimensions. We define the image pattern Q, which is a square pattern with (k - 2r) . (k - 2r) cells, in the following way. The indices of the cells of the image pattern Q are running from 1 + r to k - r, in both dimensions. For any index (i, j) of a cell in the image pattern Q, hence, with 1 + r ::; i ::; k - rand 1 + r ::; j ::; k - r, the value of the cell with index (i, j) in the image pattern Q is defined to be the value of the local function f, applied to the square subpattern of P with side length 2r + 1 and centre (i, j), hence, to the square subpattern of P with the cells running from i - r to i + r in the first dimension and from j - r to j + r in the second dimension. Example 9.18. Let us consider the dimension d
Definition 9.19. A CA
(I.;, d, F) is finitely injective if for all configurations c(1), c(2) E I.;Zd with c(1) # c(2) and c~l) = c~2), for almost all a E Zd we have F(c(1)) # F(C(2)).
9.5 Randomness and Cellular Automata
377
Definition 9.20. A continuous function F : L;Zd --+ L;Zd is measurepreserving if i1(F- 1(U)) = i1(U), for all open U C L;Zd. Theorem 9.21. (Moore-Maruoka-Kimura-Calude-HertlingJiirgensen-Weihrauch). Let (L;, d, F) be a CA, and f : L;[-r,rjd --+ L; be a local function inducing F. The following conditions are equivalent: 1.
F is surjective.
2.
For every finite pattern w there exists a configuration c such that w occurs in F(c).
3.
F is finitely injective.
4. For every n 2:: 2r + 1 and every cube pattern w of side length n we have
(9.1) 5.
F is measure-preserving.
6.
For all configurations c, if c is rich, then also F( c) is a rich configuration.
7.
For all configurations c, if c is random, then also F(c) is a random configuration.
Proof. The implication 1 =} 2 is trivial.
For 2 =} 1 let c E L;Zd be an arbitrary configuration. By 2, for each n there exists a configuration c(n) such that
The sequence (c(n))n has an accumulation point c in the compact space L;Zd. By continuity of F we conclude that F(c) = c. For 4 =} 2 it is sufficient to deduce that for every cube pattern w there exists a configuration c such that w occurs in F(c). This is the case iff #( (jpattern) -1 {w}) 2:: 1. Therefore, 2 follows immediately from 4. For the implication 2 =} 33 we assume that 3 is not true. Let c(l), c(2) E L;Zd be two different configurations with C~l) = c~2) , for almost all a E Zd, and with F(c(l)) = F(c(2)). Let 3A
strengthening of the Garden of Eden Theorem [316].
9. Applications
378
and k = 4r + 2l + 1, where lal = max{lall, ... , ladl} for a = (al, ... ,ad) E
Zd. We introduce an equivalence relation between cube patterns of side length k by calling two cube patterns v and w of side length k interchangeable if they are equal to each other or if each of them is equal to the pattern (1) d (2) represente d by c[-2r-I,2r+ll d or to the pattern represente by c[-2r-I,2r+ll d ' Obviously, if v and ware interchangeable, then jPattern(v) and jPattern(w) are equivalent. Let us fix a positive integer i and extend this relation to cube patterns of side length ik in the following way. Each cube pattern of side length ik can be viewed as consisting of i d non-overlapping cube patterns of side length k. Two cube patterns v and w of side length ik are called interchangeable if each of these i d cube subpatterns of v of side length k is interchangeable with the cube subpattern of w of side length k at the corresponding position. Since the outer 2r layers of any two interchangeable cube patterns of side length k are identical (this is especially true for (1)
(2))
the two cube patterns represented by c[-2r-I,2r+ll d and by c[-2r-I,2r+ll d we conclude that jPattern (v) = jPattern (W ),
,
for any two interchangeable cube patterns of side length ik. With respect to the "interchangeable" equivalence relation the set of all cube patterns of side length ik splits into exactly (Qk d - 1 )i d equivalence classes. 4 Hence, the set jpattern(cube patterns of side length ik) contains at most (Qk d - 1)i d cube patterns. They have side length ik - 2r, of course. But there are altogether Q(ik-2r)d cube patterns of side length ik - 2r. We claim that for sufficiently large i
(9.2) In order to prove the claim we choose i so large that
2r)d
kd - ( k - -
i
< 10gQ
Qkd
(Qk d
Raising Q to these powers and rearranging gives
4Recall that
I;
has Q elements.
-
1)
.
9.5 Randomness and Cellular Automata
379
and raising both sides to the power i d finally gives (9.2). We can now finish the argument. According to (9.2), for sufficiently large i there exists a cube pattern of side length ik - 2r which is not in the set fpattern(cube patterns of side length ik). This cube pattern cannot occur in F(c), for any configuration c, a contradiction. For the implication 3 :::} 45 we assume that 4 is not true. If there exists a cube pattern w of side length n such that equation (9.1) is not true then there must be a pattern v of side length n such that (9.3)
We set M = # ((jpattern) -1 { V }) and k = n + 2r. Let us fix a state s E L; and let r = (r, r, ... ,r) E Zd be the integer vector with constant value r. We fix a positive integer i and consider the set 8 of all configurations c E L;Zd such that each of the i d cube patterns represented by Cr+ka+{l, ... ,k}d,
for some a E {O, ... ,i - l}d is one of the patterns in (jpattern)-l{ v} such that Cb = s, for all bE Zd \ {r + 1, ... ,r + ik }d. There are exactly Mid such configurations, i.e. #(8) = Mid. The images F(c(l)) and F(c(2)) of any two configurations c(l) E 8 and c(2) E 8 are identical outside the cube {I, ... ,2r + ik }d, i.e.
for all a E Zd \ {I, ... ,2r + ik}d. Furthermore the i d cube subpatterns
F(c(1)) 2r+ka+{1, ... ,n} d , for a E {O, ... ,i - l}d are all equal to v. Hence, the set F(8) contains at most Q(2r+ik)d- i dn d configurations. We claim that for sufficiently large i
(9.4) In order to prove the claim we choose i so large that
5A
strengthening of a result by Maruoka and Kimura [306].
380
9. Applications
(remember M > Qkd_n d ). Raising Q to these powers and rearranging we obtain Q kd_n d . Q(k+k)d_kd , -_ Q(k+k)d_nd , <M, and raising both sides to the power i d finally gives (9.4). We now finish the argument. According to (9.4), for sufficiently large i there exist two different configurations C(l) and c(2) with C~l) = s = C~2), for all a E Zd \ {r + 1, ... , r + ik}d and F(c(l)) = F(C(2)). This shows that P is not finitely injective, a contradiction. We now prove 4 ~ 5. For a vector a E Zd, a positive number n, and a cube pattern w of side length n, the set Ca,w
= {c E ~Zd
I ca+{l, ... ,n}d is a representative for
w}
has measure l/Qn d , and its pre-image p-l(Ca,w)
= {c E ~Zd I f*(c-r+a+{l, ... ,n+r}d) is a representative for w}
has measure
#( (Jpattern)-l (v))/ Q(n+2r)d. Therefore, if F is measure-preserving, then 4 is true. On the other hand, if 4 is true, then each set Ca,w has the same measure as its pre-image p-l(Ca,w). Since every open set can be written as the disjoint union of sets Ca,w, we conclude that 4 implies 5. The implication 2 Lemma 9.16.
~
6 is trivial; the implication 7 =? 2 follows by
Finally, for 5 =? 7 we assume that c is a configuration such that P( c) is non-random. Then there is a Martin-Lof test (Ui)i such that
F(c)
E
n Ui. iEN
The sequence of open sets (F-1(Ui))i is also a Martin-LOf test. In view of 5 we have
!i(p-l(Ui )) = !i(Ui ) :; Ti. The facts that F is induced by a local function f and that the sequence (Ui)i of open sets is B'-computable, imply that the sequence (F-1(Ui))i of open sets is B'-computable. We have c E
n
F-1(Ui ).
iEN
Hence, c is non-random.
o
9.5 Randomness and Cellular Automata
381
First, from Condition 2 in Theorem 9.21 it follows that if F is not surjective, then there is no configuration c such that F( c) is rich or random. Hence, a non-surjective CA "destroys" both richness and randomness. Secondly, we ask what happens when one applies CA to a non-random configuration or to a non-rich configuration. 6 Example 9.22. The function F : I.;z
---+
I.;z defined by
is computable and measure-preserving. If all odd entries to one fixed element s E I.;, then the sequence
C2i+l
are equal
is non-random. But its image under F, the sequence
can still be random. It is not clear a priori whether the same phenomenon can occur when one
considers CA. One-dimensional CA preserve non-randomness, i.e. they transform non-random two-way infinite sequences into non-random twoway infinite sequences. Theorem 9.23 (Calude-Hertling-J iirgensen-Weihrauch). (I.;, d, F) be a CA.
Let
1.
If a configuration c E I.;Zd is not rich, then also F(c) is not rich.
2.
If d = 1 and a configuration c E I.;Zd is non-random, then also F(c) is non-random.
Proof. Let
f : I.;[-r,rjd
---+
I.; be a local function inducing F.
1. Let us fix a non-rich configuration c and a cube pattern of side length k which does not occur in c. Hence, at most Qkd - 1 cube patterns of side length k can occur in c. Let us consider cube patterns of side length ik, for an arbitrary positive integer i. Since cube patterns of side length 6We have seen examples of very simple computable functions on the space of oneway infinite sequences which transform some non-random sequences into random ones.
382
9. Applications
ik can be viewed as consisting of i d non-overlapping cube patterns of side kd ·d length k, we conclude that at most (Q - 1)~ different cube patterns of side length ik can occur in c. Let Pik denote the set of all cube patterns of side length ik which occur in c. We have just proved
Hence, also the set jPattern(Pik) contains at most (Qk d - 1)i d different cube patterns. These cube patterns have side length ik - 2r, of course. But there are altogether Q(ik-2r)d cube patterns of side length ik - 2r. By exactly the same counting argument as in the proof of the implication 2 ::::} 3 of Theorem 9.21 we conclude that for sufficiently large i there exists a cube pattern of side length ik - 2r which is not in the set jPattern(Pik). This cube pattern cannot occur in F(c). Hence, F(c) is not rich.
2. For the second assertion we assume that the dimension d of the cellular automaton is 1. We fix a non-random configuration c and a Martin-Lof test (Ui)i on ~Zd such that c E niEN Ui. We show that there is a MartinLof test CVi)i on ~Zd such that F(c) E niEN Vi. By Lemma 9.9 and a compactness argument we deduce from the fact that the sequence (Ui)i of open sets is H'-computable that the set
{7r(i,j) EN I [Cube(j)] CUi} is c.e. We set l
Vi
=
(9.5)
= flog2 (Q2r) l, and define
U{[J*(v)] I v E Cubes (~, 1), side length(v) ~ 2r + 1, [v] c Uz+d.
We claim that the sequence (Vi)i is a Martin-Lof test with F(c) E n iEN Vi. lt is clear that it is a sequence of open sets which is H'-computable (we use the fact that the set in (9.5) is c.e. and Lemma 9.9). For arbitrary i we have c E Uz+ i . Hence, there is an element v E Cubes(~, 1) of side length ~ 2r + 1 with c E [v] and [v] C Uz+ i . This shows F(c) E Vi. Finally we have to show that p,(Vi) ::; 2- i , for all i. We fix an i. There exists a set W
such that
c {v
E
Cubes (I.:, d) I side length( v) ~ 2r
+ 1, [v] c
Uz+ i }
U [v] = UZ+i vEW
and for any two v, w E W, the sets [v] and [w] are disjoint. If v, w E Cubes (I.:, d) and [v] C [w], then also [J*(v)] C [J*(w)].
9.6 Randomness and Riemann's Zeta-function Hence,
Vi
=
383
U [J*(v)]. vEW
Since for arbitrary v E Cubes (~, 1) with side length( v) 2: 2r + 1 we have
j1([J*(v)]) = Q2r . j1([v]) , we obtain
j1(Vi)
=
U [J*(V)])
j1 (
VEW
N be an arbitrary computable bijection and b ~ 2 be an arbitrary base. Then any sequence (an) of real numbers is f -random to base b iff it is 7r-random to base b. Proof. We can assume that all numbers an lie in the interval [0,1). The bijection f 07r- 1 is computable. We fix a sequence q E ~b' The sequence q = q1q2 ... is random iff the sequence p = qfor1(1)qfor1(2)qfo ... r1(n)'" is random (see Lemma 9.27 below). Furthermore, Qf(i,n) = P7r(i,n) , for all i,n, hence an = vb(Qf(1,n)Qf(2,n)"') iff an = Vb(P7r(1,n)P7r(2,n)" .), for all n. This proves the assertion. D
Lemma 9.25 justifies the following definition. Definition 9.26. Let b ~ 2 be an integer. A sequence (an) of real numbers an is called random to base b if there exists a random sequence q = Q1Q2··· E ~b such that an = Vb(Q7r(1,n)Q7r(2,n) ... ). This notion of randomness has natural properties, as Proposition 9.28 and Theorem 9.29 show. We will use the following simple fact (see, for example, Lemma 3.4 in Book, Lutz and Wagner [39]). Lemma 9.27. Let f : N -> N be a computable one-to-one function. If 0"10"2 ... E ~w is a random sequence, then the sequence 0" f(l)O" f(2) .. , is random as well. Proof. Consider the computable function m : N -> N defined for every i > by m(i) = max{n I f(n) ~ i} .. The function F : ~* -> ~* defined by F(X1X2 ... xn) = Xf(l)X f(2) ... Xf(m(n)) is computable and prefixincreasing (if x
T. Let k = kN. We shall prove that for all n 2:: N, f(n,x) = g(n) provided i) there exists a string y with h(lyl) = nand Iyl 2:: in, and ii) x is a Chaitin i-random string such that h(lxl) = n. We proceed by reductio ad absurdum. Suppose x to be a Chaitin random string, n = h(lxl) 2:: Nand
f(n,x) =J g(n). It is not difficult to see that
#{z E A*
Ilzl = lxi, f(n, z) = g(n)} 2::
(1 - vn)Q1x 1.
9. Applications
392 But, (x, mw(x)
#{z E A*
+ 1) ¢ W,
so
Ilzl = Ixl,f(n,z) = g(n)}
~ (1- Q-mw(x)-l/(Q _l))Qlxl.
Combining the last inequalities, we get Vn
2:: Q-mw(x)-l/(Q - 1),
or, equivalently,
so
Finally, we use the Martin-Lof asymptotical formula
K(x)
< Ixl- m(x) + q
< Ixl + (q + i + 1) -llogQ
vn(d -l)J
Ixl- k n < Ixl-T, since kn > T. In view of Corollary 5.8, x ¢ RAND?, thus contradicting the hypothesis. 0 We have obtained the main result in [137J: Corollary 9.33 (Chaitin-Schwartz). For almost all inputs n, the probabilistic algorithms of Solovay-Strassen and Miller-Rabin are errorfree in case they use a long enough Chaitin i-random input. Proof Consider
h(n) = n + 1, Vn = T Ln/3J, in = max{n - 1, a}.
o
An analysis of the proof of Theorem 9.32 reveals the number of potential witnesses of compositeness which must be visited to ensure the primality
9.8 Structural Complexity
393
of numbers of some special form correctly with high probability (in fact with certainty - if some "oracle" gives us a long Chaitin random stringS). For instance, a number of the following simple form N
requires O(1og n)
+ o (log m)
=
lOn
+m
potential witnesses.
Mersenne numbers
N require 0 (log n)
= 2n-1
= 0 (log log N) potential witnesses.
Fermat numbers N
require 0 (log log n)
= 22n + 1
= 0 (log log log log N) potential witnesses.
Finally, Eisenstein-Bell numbers 22 ...
N= n 2'S
need O(1ogn)
9.8
= O(logk N)
2 +1 '-v-' altogether
witnesses, for every natural k.
Structural Complexity
There is no general agreement as to what defines the structural complexity, 9 but there is a more common view as concerns the position of this area inside theoretical computer science - a leading role. We are not going to describe this fascinating subject; instead we shall give the reader an idea about the impact of AIT in structural complexity. See more details in Barthelemy, Cohen and Lobstein [19], Downey [180], Garey and Johnson [207], Balcazar, Diaz and Gabarro [14], Hemachandra, and Ogihara [229], Li and Vitanyi [280,282]' Longpre [286], Wagner and Wechsung [426] and Watanabe [428]. Perhaps the most known and discussed problem of structural complexity is the (in)famous problem P =? NP. Here is a very common illustration. 8We know, by virtue of results proven in Section 5.5, that, in spite of the fact the almost all strings are Chaitin t-random, no algorithm can produce an infinity of such strings. 9 As usual, a criterion like I know it when I see it works very well.
394
9. Applications
Given an undirected graph G we recall that a Hamiltonian path in G is a path through each of the vertices of G, passing through each vertex exactly once. 10 The main problem connected to Hamiltonian paths is to find such a path if it does exist: construct an algorithm such that for every graph G it computes a Hamiltonian path in G, or tells us one does not exist.ll A lot of work has been invested in this problem. One way to solve it is to proceed by trial and error. The resulting algorithm may run - in the worst case - more than O(2n) steps, where n is the number of edges of G. For a size> 103 the performance is pretty bad! What would be very desirable is a "polynomial-time algorithm", i.e. an algorithm running in time bounded by a low degree polynomial, say of order 3 or 4. Nobody at the time being knows such an algorithm! There is also a sense in which the above problem may be considered typical for a large class of similar problems 12 which are all equally difficult: if we can solve any of these problems by a fast algorithm - fast, in structural complexity, means in polynomial-time - then we can solve all of them fast. It is important to note the difference between two important measures of complexity: time and space. With respect to the space complexity, the above problem is tractable, i.e. it may be solved in polynomial-space (write: PSPACE) since space is reusable. We do not know if this problem is in P, i.e. if it can be solved in polynomial-time. Actually, most people think that the answer is negative! On the other hand, finding a Hamiltonian path is a problem that can be solved non-deterministically in polynomial-time, i.e. it lies in the class NP.
The problem P =? NP is really meta-mathematical! Indeed, assume an appropriate coding and measure of the size of proofs. For example, a Hamiltonian path is a proof that the graph has a Hamiltonian proof; moreover, the validity of this proof can be checked in polynomial-time. As we hinted in the above discussion, the difference between P and NP if any - may be seen as a difference between constructing a polynomialsize proof and verifying a polynomial-size proof. If P = NP, then the two tasks have the same degree of difficulty. lOThis problem is extremely useful in many practical situations. Just choose, at random, a book in operations research and you will be convinced. 11 Technically, NP problems are decision problems that give a YES/NO answer; in this case the output would be "YES, there is a Hamiltonian path", or "NO, there is no Hamiltonian path". 12Most of these problems have a strong practical significance.
9.8 Structural Complexity
395
Two more mathematical problems are quite relevant for our discussion. Both of them belong to number theory and are currently open. The prime number problem asks for a polynomial-time algorithm to check whether an arbitrary number n is prime. It should be emphasized that the interest is in a polynomial-time algorithm in the number of digits representing the number n (not in n, a trivial problem). It is plain that this problem is in co_Np 13 as determining whether a number is composite is in NP, a proof being just a prime factor; Pratt [339] has shown that it is also in NP. Miller [314] has proven that this problem is in P if one assumes the extended Riemann Hypothesis. The other problem, the factorization problem, asks for non-trivial factors of the natural number n, if n is composite. It is basic for many public-key crypto-systems ("trapdoor ones") and it is widely believed to be intractable. See more in Salomaa [356]. But, we do not even know if this problem is NP-complete, i.e. we do not know if the Hamiltonian path problem can be solved fast given a routine for factorization. We may ask: "Why is the problem P =? NP so hard?" To answer this question we have to rely on a technique from computability theory known as relativization. Roughly speaking, this means the introduction of the so-called oracles - devices able to perform even "non-algorithmic tasks" . Most statements true for oracle-free machines remain true for machines with oracles. An important step in this direction has been made by Baker, Gill and Solovay [13]: they have shown that the P = ? NP problem cannot be settled by arguments that relativize. Theorem 9.34. There exist two computable oracles B, A such that P(A) -=I NP(A) and P(B)
= NP(B).
Hartmanis and Hopcroft [227] have proven the following independence result: Theorem 9.35. There exist two computable sets A, B with P(A) -=I NP(A) and P(B) = NP(B), but neither result is provable in ZFC.
More light has been shed on this problem by the Bennett and Gill [33] result: 13 CO _NP
is the class of sets X such that the predicate x
if. X
is in NP.
396
9. Applications
Theorem 9.36. If A is a random oracle, then P(A) =J NP(A), i.e. with probability 1, P(A) =J NP(A). Hemaspaandra and Zimand [230] have obtained the following stronger result:
Theorem 9.31. Relative to a random oracle, there is a language in NP, on which each polynomial-time algorithm is correct on half of the inputs at each sufficiently large lengths, and is wrong on the other half. A modification of the central idea in ArT has been developed by Hartmanis [224]: consider not only the length of a computer outputting a string, but also, simultaneously, the running time of the computer. Given a universal computer 'ljJ and two computable functions G, g, a string x of length n is in the "generalized Kolmogorov class"
K1j>[g(n), G(n)], ifthere is a string y of length at most g(n) with the property that 'ljJ will generate x on input y in at most G(n) steps. A set X of strings has small generalized Kolmogorov complexity if there exist constants c, k such that for almost all x, one has
This class is usually denoted by K[log, poly]. For any set X we denote by enumx the function that for every natural n has as value a string encoding the set of all strings in X of length at most n. The set X is self-p-printable if there is a (deterministic) oracle computer that computes the function enumx relative to X and that runs in polynomial-time. Every self-p-printable set is sparse, i.e. there is a polynomial P such that for every natural n, the number of strings x E X of length less than n is bounded by P(lxl). An easy characterization follows: P
= NP iJJfor every self-p-printable set X, P(X) = NP(X).
Hartmanis and Hemachandra [225] have proven that the class of self-pprintable sets can be viewed as a relativized version of K[log, poly]:
9.8 Structural Complexity
397
Theorem 9.38. A set X is self-p-printable iff X E K[log, poly]. A very interesting approach has been inaugurated by Book, Lutz and Wagner [39] (see also Book [38]). They have studied the algorithmically random languages (RAND) in a framework which is very close to the main stream of Chapter 6. Motivated by Theorem 9.34 of Bennett and Gill, they have designed a new way to gain information about the complexity of a language L. Here is a typical result:
Theorem 9.39. a) Let L c AW be a union of constructively closed sets 14 that is closed under finite variation. Then fJ,(L)
= 1 iff
X
n RAND
=1=
0.
b) Let L be an intersection of constructively open sets that is closed under finite variation. Then fJ,(L)
=1
iff RAND
c
L.
Finally, consider the exponential complexity classes
E = DT I M E ( 2linear) , and E2 = DT I M E ( 2Polynomial) . There are several reasons for considering these classes (Lutz [289, 290]): 1.
Both classes E, E2 have rich internal structures.
2.
E2 is the smallest deterministic time complexity class known to contain NP and PSPACE.
c E 2, E
E 2, and E contains many NP-complete problems.
3.
PeE
4.
Both classes E, E2 have been proven to contain intractable problems.
=1=
In view of the property 2 there may well be a natural "notion of smallness" for subsets of E2 such that P is a small subset of E 2, but NP is not. Similarly, it may be that P is a small subset of E, but that NP n E is not! In the language of constructive measure theory smallness can be 14That is, L is a union of a family of sets each of which is the complement of a constructively open set.
9. Applications
398
translated by "measure zero" (with respect to the induced spaces E or E2). One can prove that indeed P has constructive measure zero in E and E2, Lutz [289]. This motivates Lutz [290] to adopt the following quantitative hypothesis: The set NP has not measure zero.
This is a strong hypothesis, as it implies P =J NP. It is consistent with Zimand's [453] topological analysis (with respect to a natural, constructive topology, if NP \ P is non-empty, then it is a second Baire category set, while NP-complete sets form a first category class) and appears to have more explanatory power than traditional, qualitative hypotheses. As currently we are unable to prove or disprove this conjecture, the best strategy seems to be to investigate it as a scientific hypothesis; its importance will be evaluated in terms of the extent and credibility of its consequences. Some interesting results have been obtained by Lutz [289] and Lutz and Mayordomo [291]. For instance, they have proven the following result: Theorem 9.40. For every real 0 < a < 1, only a subset of measure zero of the languages decidable in exponential time are ~~"'-trreducible to languages that are not exponentially dense. Here the truth-table ~~a_tt-reducibility is "truth-table reducibility with n cx queries on inputs of length n".
9.9
What Is Life?
The idea that the Universe is a living organism is very old. Aristotle thought that the entire Universe "resembles a gigantic organism, and it is directed towards some final cosmic goal" .15 But, "What is life?" "When must life arise and evolve?" Or, maybe better, "How likely is life to appear and evolve?" "How common is life in the Universe?" The evolution of life on Earth is seen as a deterministic affair, but a somewhat creative element is introduced through random variations and 15Teleology is the idea that physical processes can be determined by, or drawn towards, an a priori determined end-state.
9.9 What Is Life?
399
natural selection. Essentially, there are two views as regards the origins of life. The first one claims that the precise physical processes leading to the first living organism are exceedingly improbable, and life is in a way intimately linked to planet Earth (the events preceding the appearance of the first living organism would be very unlikely to have been repeated elsewhere). The second one puts no sharp division between living and non-living organisms. So, the origin of life is only one step, maybe a major one, along the long path of the progressive complexification and organization of matter. To be able to analyse these views we need some coherent concept of life! Do we have it? It is not difficult to recognize life when we see it, but it looks tremendously difficult to set up a list of distinct features shared in common by all and only all living organisms. The ability to reproduce, the response to external stimuli, and growth are among the most frequently cited properties. But, unfortunately, none of these properties "defines" life. Just consider an example: a virus does not satisfy any of the above criteria of life though viral diseases clearly imply biological activity. A very important step towards understanding life was taken by Stanley Miller and Harold Urey; their classical experiment led to amino acids, which are not living organisms or molecules, but the building blocks of proteins. Life is ultimately based on these two groups of chemicals: nucleic acids and proteins. Both are made from carbon, hydrogen, oxygen, nitrogen and small quantities of other elements (sulphur, phosphorus). Nucleic acids are responsible for storing and transmitting all the information required to build the organism and make it work - the genetic code. The role of proteins is twofold: structural and catalytic. Little is known about the crucial jump from amino acids to proteins and even less about the origins of nucleic acids. Along the line of reasoning suggested by the Miller and Urey primeval soup and Darwinian evolution it appears that the spontaneous generation of life from simple inanimate chemicals occurs far more easily than its deep complexity would suggest. In other words, life appears to be a rather common feature in the Universe! Von Neumann wished to isolate the mathematical essence of life 16 as it evolves from the above physics and biochemistry. In [424J he made the first step by showing that the exact reproduction of universal Turing machines is possible in a particular deterministic model Universe. 16In Chaitin's words: If mathematics can be made out of Darwin, then we will have added something basic to mathematics; while if it cannot, then Darwin must be wrong, and life remains a miracle ...
400
9. Applications
Following this path of thought it may be possible to formulate a way to differentiate between dead and living matter: by the degree of organization. According to Chaitin [122] an organism is a highly interdependent region, one for whieh the complexity of the whole is much less than the sum of the complexities of its parts. Life means unity. Dead versus living can be summarized as the whole versus the sum of its parts. Charles Bennett's thesis is that a structure is deep if it is superficially random but subtly redundant, in other words, if almost all its algorithmic probability is contributed by slow-running programs. To model this idea Bennett has introduced the notion of "logical depth": a string's logical depth reflects the amount of computational work required to expose its "buried redundancy" :17 A typical sequence of coin tosses has high information content, but little message value. ... The value of a message thus appears to reside. .. in what might be called its buried redundancy - parts predictable only with difficulty, things the receiver could in principle have figured out without being told, but only at considerable cost in time, money and computation. In other words, the value of a message is the amount of mathematical or other work plausibly done by its originator, which its receiver is saved from having to repeat.
We arrive at a point when the question Is the Universe a computer?
becomes inevitable. Maybe Douglas Adams' story ([1], pp. 134-137) is after all not science fiction: the answer to the Great Question of Life, the Universe and Everything, the Ultimate answer searched for in seven and a half million years of work, is -"Fm:'ty~1wo-#~said~f}eep-'fb:ought;- with-infinite
majesty and
calm. IS 17See Bennett [27], p. 297 and for more details [26, 28,31]. 18 "I checked it very thoroughly", said the computer, "and that quite definitely is the answer. I think the problem, to be quite honest with you, is that you've never actually known what the question is." ...
9.9 What Is Life?
401
For John Wheeler the Universe is a gigantic information processing system in which the output is as yet undetermined. He coined the slogan: It from bit! That is, it - every force, particle, etc. - is ultimately present through bits of information. And Wheeler is not unique in this view. Ed Fredkin and Tom Toffoli emphatically say yes: the Universe is a gigantic cellular automaton. No doubt! The only problem is that somebody else is using it. All we have to do is "hitch a ride" on his huge ongoing computation, and try to discover which parts of it happen to go near where we want - says Toffoli [400]. For the physicist Frank Tipler the Universe can be equated with its own simulation viewed very abstractly. Feynman [196] considered the ... possibility that there is to be an exact simulation, that the computer will do exactly the same as nature, ... that everything that happens in a finite volume of space and time would have to be exactly analyzable with a finite number of logical operations. He concludes: The present theory of physics is not that way, apparently. It allows space to go down to infinitesimal distances. This is a strong objection, but perhaps not a fatal one. As Paul Davies argues, the continuity of time and of space are only assumptions about the world, they are merely our working hypotheses. They cannot be proven! Here is his argument: ... we can never be sure that at some small scale of size, well below what can be observed, space and time might not be discrete. What would this mean? For one thing it would mean that time advanced in little hops, as in a cellular automaton, rather than smoothly. The situation would resemble a movie film which advances one frame at a time. The film appears to us as continuous, because we cannot resolve the short time intervals between frames. Similarly, in physics, our current experiments can measure intervals of time as short as 10- 26 "The Ultimate Question?" "Yes!" "Of Life, the Universe and Everything?" "Yes!" "But can you do it?" cried Loonquawl. Deep Thought pondered this for another long moment. Finally: "No", he said firmly. '" "But I'll tell you who can," said Deep Thought. '" "I speak of none but the computer that is to come after me," ... "A computer which can calculate the Question to the Ultimate Answer, a computer of such infinite and subtle complexity that organic life itself shall form a part of its operational matrix .... Yes! I shall design this computer for you. And I shall name it also unto you. And it shall be called ... The Earth."
402
9. Applications seconds; there are no sign of any jumps at that level. But, however fine our resolution becomes, there is still the possibility that the little hops are yet smaller. Similar remarks apply to the assumed continuity of space.
And, we may add, the results proved by methods of non-standard analysis reinforce the duality between the continuous and the discrete. A computer simulation is usually regarded as a model, as a (simplified) representation, as an image of the reality. Is it possible to realistically claim that the activity going inside a computer could ever create a real Universe? Can a computer simulate consciousness? Roger Penrose dedicated a fascinating book to this problem [331].19 His conclusion is strong: a brain's physical action evokes awareness, but physical action cannot, even in principle, be simulated computationally. It may even be possible that awareness cannot be explained in any scientific terms. 20 (An account of these matters was presented in [333].) Tipler distinguishes two "worlds": one inside the computer and the other outside. The key question is this: Do the simulated people exist? As far as the simulated people can tell, they do. By assumption, any action which real people can and do carry out to determine if they exist - reflecting on the fact that they think, interacting with the environment - the simulated people also can do, and in fact do do. There is simply no way for the simulated people to tell that they are "really" inside the computer, that they are merely simulated, and not real. They can't get at the real substance, the physical computer, from where they are, inside the program.
How do we know that we ourselves are real and not "simulated" by a gigantic computer 21 ? "Obviously, we can't know" says Tipler. But this is irrelevant. The existence of the Universe itself is irrelevant: Such a physically real universe would be equivalent to a K anti an thing-in-itself. As 19It will be soon followed by another one. 2°In his own words [332]: I... suggest that the outward manifestations of conscious mental activity cannot even be properly simulated by calculation. 21 Following Ilya Prigogine, God is reduced to a mere archivist turning pages of a cosmic history book already written; according to Paul Erdos, God has a large book containing all mathematics - and every mathematician is allowed to look into it only once, maybe twice, the rest being his job to discover.
9.9 What Is Life?
403
empiricists, we are forced to dispense with such an inherently unknowable object: the universe must be an abstract progam. The "world view from within" and "from the outside" have been suggested by other authors as well. Svozil has dedicated a chapter of his book [391] to a detailed presentation of his own views. Here are the main facts summarized in Svozil [392]22: Epistemologically, the intrinsic/extrinsic concept, or, by another naming, the endophysics/exophysics concept, is related to the question of how a mathematical or a logical or an algorithmic universe is perceived from within/from the outside. The physical universe (in Rossler's dictum, the "Cartesian prison"), by definition, can be perceived from within only. Extrinsic or exophysical perception can be conceived as a hierarchical process, in which the system under observation and the experimenter form a two-level hierarchy. The system is laid out and the experimenter peeps at every relevant feature of it without changing it. The restricted entanglement between the system and the experimenter can be represented by a one-way information flow from the system to the experimenter; the system is not affected by the experimenter's actions. Intrinsic or endophysical perception can be conceived as a nonhierarchical effort. The experimenter is part of the universe under observation. Experiments use devices and procedures which are realisable by internal resources, i.e., from within the universe. The total integration of the experimenter in the observed system can be represented by a two-way information flow, where "measurement apparatus" and "observed entity" are interchangeable and any distinction between them is merely a matter of intent and convention. Endophysics is limited by the self-referential character of any measurement. An intrinsic measurement can often be related to the paradoxical attempt to obtain the "true" value of an observable while - through interaction - it causes "disturbances" of the entity to be measured, thereby changing its state. 22 Historically, Archimedes conceived points outside the world, from which one could move the earth. Archimedes' use of "points outside the world" was in a mechanical rather than in a metatheoretical context: he claimed to be able to move any given weight by any given force, however small.
404
9. Applications Among other questions one may ask, "what kind of experiments are intrinsically operational and what type of theories will be intrinsically reasonable?" Imagine, for example, some artificial intelligence living in a (hermetic) cyberspace. This agent might develop a "natural science" by performing experiments and developing theories. It is tempting to speculate that also a figure in a novel, imagined by the poet and the reader, is such an agent. Intrinsic phenomenologically, the virtual backfiow could manifest itself by some violation of a "superselection rule;" i.e., by some virtual phenomenon which violates the fundamental laws of a virtual reality, such as symmetry and conservation principles.
The whole story is fascinating. Most facts are currently at the stage of hypotheses, beliefs .... Here are some relevant references for the interested reader: Akin [2], Barrow [15], Barrow and Tipler [18], Bennett [26, 28], Calude and Salomaa [98], Chaitin [119, 122, 125]' Davies [155], Davies and Gribbin [156]' Feynman [196], Levy [279]' Penrose [331, 332], Svozil [391]' Tymoczko [406] and von Neumann [423, 424]. As a bridge to the next section we quote the conclusion reached by Deutsch [176], p. 101:
The reason why we find it possible to construct, say, electronic calculators, and indeed why we can perform mental arithmetic, cannot be found in mathematics or logic. The reason is that the laws of physics· "happen" to permit the existence of physical models for the operations of arithmetic such as addition, subtraction and multiplication. If they did not, these familiar operations would be non-computable functions. We might still know of them and invoke them in mathematical proofs (which would be presumably called "non-constructive") but we could not perform them.
9.10 Randomness in Physics
9.10
405
Randomness in Physics
All science is founded on the assumption that the physical Universe is ordered and rational. The most powerful expression of this state of affairs is found in the successful application of mathematics to make predictions expressed by means of the laws of physics. Where do these laws come from? Why do they operate universally and unfailingly? Nobody seems to have reasonable answers to these questions. The most we can do is to explain that the hypothesis of order is supported by our daily observations: the rhythm of day and night, the pattern of planetary motion, the regular ticking of clocks. However, there is a limit to this perceived order: the vagaries of weather, the devastation of earthquakes, or the fall of meteorites are (perceived) as fortuitous. How are we to reconcile these seemingly random processes with the supposed order? There are at least two ways. The most common one starts by observing that even if the individual chance events may give the impression of lawlessness, disorderly processes may still have deep (statistical) regularities. This is the case for most interpretations of quantum mechanics - to which we shall return later. It is not too hard to notice some limits to this kind of explanation. It is common sense to say that "casino managers put as much faith in the laws of chance as engineers put in the laws of physics". We may ask: "How can the same physical process obey two contradictory laws, the laws of chance and the laws of physics?" As an example consider the spin of a roulette wheel.
There is a second, "symmetric" approach, which is mainly suggested by AlT. As our direct information refers to finite experiments, it is not out of question to discover local rules functioning on large, but finite, scales, even if the global behaviour of the process is truly random. 23 But, to percei ve this global randomness we have to have access to infinity! It is important to notice that, consistently with our common experience, 23Recall that in a random sequence every string - of any length - appears infinitely many times. So, in such a random sequence the first billion digits may be exactly the first digits of the expansion of 7r!
9. Applications
406
facing global randomness does not imply the impossibility of making predictions. Space scientists can pinpoint and predict planetary locations and velocities "well enough" to plan missions years in advance. Astronomers can predict solar or lunar eclipses centuries before their occurrence. We have to be aware that all these results - as superb as they may be - are only true within a certain degree of precision. Of course, in the process of solving equations, say of motion, small errors accumulate, making the predictions less reliable as the time gets longer. We face the limits of our methods! Why are our tools so imperfect? The reason may be found in some facts proved in Chapter 6: a random sequence cannot be "computed" , it is only possible to approximate it very crudely. AIT gives researchers an appreciation of how little complexity in a system is needed to produce extremely complicated phenomena and how difficult it is to describe the Universe. We shall return to this point of view in Section 9.11. It is important to note the main conclusions of Svozil (for a detailed and convincing argumentation see [391]):
• Chaos in physics corresponds to randomness in mathematics. • Randomness in physics may correspond to uncomputability in mathematics. Where do we stand with regard to computability in physics? The most striking results have been obtained by Pour-EI and Richards [338] (for an ample discussion see Penrose's book [331]) for the wave equation. They have proven that even though solutions of the wave equation behave deterministically, in the most common sense, there exist computable initial data24 with the strange property that for a later computable time the determined value of the field is non-computable. Thus, we get a certain possibility that the equations - of a possible field theory - give rise to a non-computable evolution. In the same spirit, da Costa and Doria [149] have proven that the problem whether a given Hamiltonian can 24More precisely, the initial condition is 0 1 (Le. continuous, with continuous deriva,tive), but not twice differentiable. Penrose [331] p. 243-244 appreciates that the initial data vary in a non-smooth way one would "normally" require for a physical sensible field. Of course, one may ask whether the physical Universe is really "normal". Once again, note the indirect way we are using the hypothesis of order! See also Weihrauch and Zhong [432].
9.10 Randomness in Physics
407
be integrated by quadratures is undecidable; their approach led to an incompleteness theorem for Hamiltonian mechanics. Perhaps the most important relation between randomness and the Universe is provided by quantum mechanics. Let us examine it very briefly. This theory pertains to events involving atoms and particles smaller than atoms, events such as collisions or the emission of radiation. In all these situations the theory is able to tell what will probably happen, not what will certainly happen. The classical idea of causality (i.e. the idea that the present state is the effect of a previous state and cause of the state which is to follow) implies that in order to predict the future we must know the present, with enough precision. 25 Not so in quantum mechanics! For quantum events this is impossible in view of Heisenberg's Uncertainty Principle. According to this principle it is impossible to measure both the position and the momentum of a particle accurately at the same time. Worse than this, there exists an absolute limit on the product of these inaccuracies expressed by the formula 6.p.6.q ~ h, where q,p refer, respectively, to the position and momentum and 6.p,6.q to the corresponding inaccuracies. In other words, the more accurately the position q is measured, the less accurately can the momentum p be determined, and vice versa. The measurement with an infinity of precision is ruled out: if the position were measured to infinite precision, then the momentum would become completely uncertain and if the momentum is measured exactly, then the particle's location is uncertain. To get some concrete feeling let us assume that the position of an electron is measured within to an accuracy of 10- 9 m; then the momentum would become so uncertain that one could not expect that, 1 second later, the electron would be closer than 100 kilometres away (see Penrose [331], p. 248). Borel [42] proved that if a mass of 1 gram is displaced through a distance of 1 centimetre on a star at the distance of Sirius it would influence the magnitude of gravitation on the Earth by a factor of only 10- 10 More recently, it has been proven that the presence/absence of an electron at a distance of 1010 light years would affect the gravitational force at the Earth by an amount that could change the angles of molecular trajectories by as much as 1 radian after about 56 collisions.
°.
But, what is the point of view of the main "actors"? 25In company with Laplace: a thing cannot occur without a cause which produces it.
408
9. Applications Heisenberg: In experiments about atomic events we have to do with things and facts, with phenomena that are just as real as any phenomena in daily life. But the atoms or the elementary particles themselves are not as real: they form a world of potentialities or possibilities rather than one of things or facts. Bohr: Physics is not about how the world is, it is about what we can say about this world. Dirac: The only object of theoretical physics is to calculate results that can be compared with experiment, and it is quite unnecessary that any satisfying description of the whole course of the phenomenon should be given.
Einstein was very upset about this situation! His opposition to the probabilistic aspect of quantum mechanics 26 is very well known: Quantum mechanics is very impressive. But an inner voice tells me that it is not yet the real thing. The theory produces a good deal but hardly brings us closer to the secret of the Old One. I am at all events convinced that He does not play dice. 27 It is important to note that Einstein was not questioning the use of probabilities in quantum theory (as a measure of temporary ignorance or error), but the implication that the individual microscopic events are themselves indeterminate, unpredictable, random.
Quantum randomness is precisely the kind of randomness usually considered in probability theory. It is a "global" randomness, in the sense that it addresses processes (e.g. measuring the diagonal polarization of a horizontally polarized photon) and not individuals (it does not allow one to call a particular measurement random). ArT succeeds in formalizing the notion of individual random sequence using a self-delimiting universal computer. However, we have to pay a price: if a more powerful computer 26Recall that Einstein put forward the concept of the photon in 1905 - out of which the idea of wave-particle duality was developed! 27From his reply to one of Niels Bohr's letters in 1926, quoted from Penrose [331], p. 280.
9.11 Metaphysical Themes
409
is used - for instance, a computer supplied with an oracle for the Halting Problem - then the definition changes. Moreover, there is no hope of obtaining a "completely invariant" definition of random sequences because of Berry's paradox. In Bennett's words [29]: The only escape is to conclude that the notion of definability or nameability cannot be completely formalized, while retaining its usual meaning. Here are some more references: Barrow [15], Barrow and Tipler [18], Brown, Calude and Doran [79], Chaitin [118, 120, 121, 122], Davies [155], Davies and Gribbin [156], Davis and Hersh [160], Denbigh and Denbigh [172], Hawking [228], Levin [278], Li and Vitanyi [282]' Mendes-France [312], Penrose [331, 332]' Peterson [334] and Svozil [391].
9.11
Metaphysical Themes
After physics, metaphysics .... Metaphysics is a branch of philosophy which studies the ultimate nature and structure of the world. Kant considered that the three fundamental concepts of metaphysics were the self, the world and God. The nature of God involves the problem of the infinity of God. This remark generated many important scholastic studies about the relation between the finite and the infinite. 28 In this context one can formulate one of the most intriguing questions: 29 Is the existence of God an axiom or a theorem?
Following the discussion in the preceding section we would like to suggest replacing the hypothesis of order by its opposite: The Universe is Lawless. 3D First let us note that the ancient Greeks and Romans would not have objected to the idea that the Universe is essentially governed by chancein fact they made their gods play dice quite literally, by throwing dice in 28The work of Scotus [362J has to be specifically mentioned [389J. 29 A very interesting point of view is discussed in Odifreddi [323, 324J; see also Calude, Marcus and ~tefanescu [93J. 30For a more elaborate discussion see Calude, and Meyerstein [94J; for an original presentation of scientific knowledge from the perspective of ArT see Brisson and Meyerstein [48J.
410
9. Applications
their temples, to see the will of gods; the Emperor Claudius even wrote a book on the art of winning at dice. 31 Poincare may have suspected and even understood the chaotic nature of our living Universe. More than 90 years ago he wrote:
If we knew exactly the laws of nature and the situation of the universe at the initial moment, we could predict exactly the situation of that universe at a succeeding moment. But even if it were the case that the natural law no longer had any secret for us, we could still only know the initial situation approximately. If that enabled us to predict the succeeding situation with the same approximation, that is all we require, that [it] is governed by the laws. But it is not always so; it may happen that small differences in the initial conditions produce very great ones in the final phenomena. A small error in the former will produce an enormous error in the latter. Prediction becomes impossible, and we have the fortuitous phenomenon. Of course, one may discuss this hypothesis and appreciated its value (if any) by its fruitfulness. We may observe, following Davies [155],
apparently random events in nature may not be random at all . .. Chaitin's theorem ensures we can never prove that the outcome of a sequence of quantum-mechanical measurements is actually random. It certainly appears random, but so do the digits of 7r. Unless you have the "code" or algorithm that reveals the underlying order, you might as well be dealing with something that is truly random. ... Might there be a "message" in this code that contains some profound secrets of the universe? This type of argument - which is very appealing - has been used to reconcile "acts of God" with physical reality. Most of those discussions have been focused on quantum indeterminism, which in the light of AIT is a severe limitation. Randomness is omnipresent in the Universe, and by no means is it a mark of the microscopic Universe! 31 However, from the point of view of Christianity, playing dice with God was definitely a pagan practice - it violates the first commandment. St Augustine is reported to have said that nothing happens by chance, because everything is controlled by the will of God.
9.11 Metaphysical Themes
411
A famous parable due to John Wheeler and discussed by Davies [155] may illuminate our point. One day Wheeler was the subject in the game of 20 questions. 32 Wheeler started asking simple questions: Is it big? Is it living? Eventually he guessed. Is it a cloud? And the answer came back "Yes" .in a general burst of laugh. The players revealed their strategy: no word had been chosen, but they tried to answer his questions randomly, only keeping consistent with their previous answers. In the end an answer came out. The answer was not a priori determined - as a fair play of the game would require - but neither was it arbitrary: it resulted from Wheeler's questions and players' binary answers, i.e. to a large extent by pure chance. Going on to a more serious argument we mention Godel [210], who discusses the essence of time. Under the influence of Einstein - during their stay at the Institute of Advanced Study in Princeton 33 - Godel produced some new solutions for Einstein's gravitational field equations. His main conclusion is that the lapse of time might be unreal and illusory.34 In his own words:
It seems that one obtains an unequivocal proof for the view of those philosophers who, like Parmenides and Kant, and the modem idealists, deny the objectivity of change and consider change as an illusion or an appearance due to our special perception. His model describes a rotating Universe giving rise to space-time trajectories that loop back upon themselves. Time is not a straight linear sequence of events - as is commonly suggested by the arrow - but a curving line. There is no absolute space; matter has inertia only relative to other matter in the Universe.
By making a round trip on a rocket ship in a sufficiently wide curve, it is possible in these worlds to travel into any region of the past, present, and future, and back again. 32Players agree on a word and the subject tries to guess that word by asking at most 20 questions. Only binary yes-no answers are allowed.
33See the nice book by Regis [343]. 34Karl Svozil pointed out in [392] that "Godel himself looked into celestial data for support of his solutions to the Einstein equations; physicists today tend to believe that the matter distribution of the universe rules out these solutions, but one never knows
412
9. Applications
It is to be remarked that the hypothesis of lawless offers a simpler way to deal with questions like: Does God exist? Is God omnipotent? Is God rational? Do the laws of physics contradict the laws of chance?
Finally, let us go back to the widely based conviction that the future is determined by the present, and therefore a careful study of the present allows us to unveil the future. As is clear, we do not subscribe to the first part of the statement, but we claim that our working hypothesis is consistent with the second part of it. We hope that the results presented in this book contribute to this assertion. The above results support Chaitin's claim that randomness has pervaded the inner structure of mathematics! It is important to note that the above assertion does not mean a "mandate for revolution, anarchy, and license" . It means that our notion of proof should be accordingly "modified". This point of view is consistent with the opinion expressed (30 years ago) by G6del [212, 213]: ... besides mathematical intuition there exists another (though only probable) criterion of truth of mathematical axioms, namely their fruitfulness in mathematics, and one may add, possibly also in physics . .. The simplest case of an application of the criterion under discussion arises when some . .. axiom has number-theoretical consequences verifiable by computation up to any given integer. . .. axioms need not be evident in themselves, but rather their justification lies (exactly as in physics) in the fact that they make it possible for these "sense perceptions" to be deduced. I think that. .. this view has been largely justified by subsequent developments, and it is to be expected that it will be still more so in the future. It has turned out that the solution of certain arithmetical problems requires the use of assumptions essentially transcending arithmetic. .. Of course, under these circumstances mathematics may lose a good deal of its "absolute certainty"; but, under the influence of the modern criticism of the foundations, this has already happened to a large extent
We end with an impressive remark made by Bridges [46]. Consider the following function f, defined on the set N of natural numbers:
9.11 Metapbysical Tbemes
f(n)
={
~:
413
if the Continuum Hypothesis is true, if the Continuum Hypothesis is false.
Deep work by Godel [211] and Cohen [144] shows that neither the Continuum Hypothesis nor its negation can be proven within Z FC. According to classical logic, f is computable because there exists an algorithm that computes it: that algorithm is either the one which always produces 0, or else the one which always produces 1. The trouble is we cannot know the correct one! And, as the Continuum Hypothesis is independent of the axioms of ZFC - the standard framework for mathematics - we will never know which of the two algorithms actually computes f. As the most recent developments show, the blend of logical and em piricalexperimental arguments ("quasi-empirical mathematics" for Tymoczko [405, 406], Chaitin [132, 135] or "experimental mathematics" for Bailey and Borwein [9], Borwein [45]; see also Bailey, Borwein and Devlin [10]) may lead to a new way to understand (and practise) mathematics; see also Chaitin [126]' Jaffe and Quinn [240],35, Zeilberger [449] and Horgan [239].
350ne distinguishes between "theoretical mathematics" (referring to the speculative and intuitive work) and "rigorous mathematics" (the proof-oriented phase) in an attempt to build a framework assuring a positive role for speculation and experiment.
Chapter 10
Open Problems It's kind of fun to do the impossible. Walt Disney AIT raises a large number of challenging open problems; they are motivated both from the inner structure of the theory and from the interreaction of the theory with other subjects.
1. We start with a group of problems communicated to us by Greg Chaitin: a) Further develop AIT for enumeration computers; see Chaitin [116, 125], Solovay [376] and Becher, Daicz and Chaitin [23] and Becher, Chaitin [22]. b) Discover interesting instances of randomness in other areas of mathematics, e.g. algebra, calculus or geometry. c) Prove that a famous mathematical conjecture is unsolvable in the usual formalizations of number theory. d) Develop formal definitions for intelligence and measures of its various components. Apply the AIT to AI. e) Develop measures of self-organization and proofs that life must evolve. More precisely, set up a non-deterministic model universe, . .. formally define what it means for a region of spacetime in that Universe to be an organism and what is its degree of organization, and .. , rigorously demonstrate that, starting from simple initial conditions, organisms will appear and
416
10. Open Problems evolve in degree of organization in a reasonable amount of time and with high probability. See more in von Neumann [424], Chaitin [122, 125], Levy [279].
2. Study the class offunctions f : A* -> A* such that f(x) is a random string whenever x is random string. 3. Study the class of reals which can be approximated by computable sequences of rationals converging monotonically. 4. How large is the class of finitely refutable mathematical problems? 5. We have seen that the program-size complexity can be used to study the rate of convergence of computable sequences of rationals. It would be interesting to apply these ideas to questions of physical interest (as in Pour-El and Richards [338] and Weihrauch and Zhong [432]). For example, is it possible to construct problems which on computable and low program-size complexity inputs have noncomputable solutions with high complexity, perhaps even random solutions? 6. Extend the invariance of randomness with respect to natural positional representations to other types of representations. 7. (Conjecture) In the context of GIT, the class oft rue but unprovable statement is "large" in probabilistic terms. 8. Define and study the symmetry of random strings and sequences. Is the absence of symmetry related to randomness? See in this respect Marcus [299]. 9. Do arbitrary CA of higher dimension preserve non-randomness? 10. Analyse the behaviour of CA with respect to the complexity of finite patterns. 11. We have seen that surjective CA are measure-preserving with respect to the uniform measure, hence they are dynamical systems in the sense of ergodic theory. For non-surjective CA one has to consider other measures in order to apply results from ergodic theory. For an application of ergodic theory to CA see Lind [284]' Cervelle, Durand and Formenti [109], Dubacq, Durand and Formenti [175], Galato [204] and V'yugin [425]. It seems to be very interesting to combine AIT and ergodic theory to study CA and other dynamical
10. Open Problems
417
systems; see, for example, Brudno [50], White [433] and Batterman and White [21]. 12. Construct a simpler Diophantine equation satisfying Theorem 8.6. 13. Find an appropriate notion of "pseudo-random sequence of reals" such that the zeros of Riemann's zeta-function form a pseudorandom sequence. A meaningful definition should be base invariant and a "pseudo-random sequence of reals" should be uniformly distributed modulo 1. For other open problems see Chaitin [122, 132]' Uspensky [407], Downey [181] and Downey and Hirschfeldt [182].
~ibliography
[1] D. Adams. The Hitch Hiker's Guide to the Galaxy, Pan Books, London, 1979. [2] E. Akin. The spiteful computer: a determinism paradox, Math. Intelligencer 14 (1992),45-47. [3] P. Andreasen. Universal Source Coding, Masters Thesis, University of Copenhagen, Denmark, 2001. [4] A. Arslanov. On a conjecture of M. Van Lambalgen, EATCS Bull. 62 (1997), 195-198. [5] A. Arslanov. Contributions to Algorithmic Information Theory, Ph.D. Thesis, University of Auckland, New Zealand, 1998 [6] A. Arslanov. On the phenomenon of auto computability, Electron. Notes Theor. Comput. Sci., 31 (2001), 1-14. [7] A. Arslanov, C. Calude. Program-size complexity computes the halting problem II, EATCS Bull. 57 (1995), 199-200. [8] J. P. Azra, B. Jaulin. Recursivite, Gauthier-Villars, Paris, 1973. [9] D. H. Bailey, J. M. Borwein. Experimental mathematics: recent developments and future outlook, in World Mathematical Year 2000 Book, Springer-Verlag, Berlin, to appear; see also http://www.cecm.sfu.ca/proj ects/lntegerRelations /2001/future.html.
[10] D. H. Bailey, J. Borwein, K. Devlin. The Experimental Mathematician, A K Peters, Wellesley, MA, to appear. [11] D. H. Bailey, R. E. Crandall. On the random character of fundamental constant expansions, Exp. Math. 10, 2 (2001), 175-190.
420
Bibliography
[12] D. H. Bailey, R. E. Crandall. Random generators and normal numbers, Manuscript, March 2002. http://www . nersc. gov / -dhbailey/dhbpapers/bcnormal.pdf. [13] T. Baker, J. Gill, R. Solovay. Relativizations of the problem P = ?NP question, SIAM J. Comput. 4 (1975),431-442. [14] J. B alcazar , J. Diaz, J. Gabarro. Structural Complexity I, Springer-Verlag, Heidelberg, 1995. [15] J. Barrow. Pi in the Sky, Clarendon Press, Oxford, 1992. [16] J. Barrow. Impossibility - The Limits of Science and the Science of Limits, Oxford University Press, Oxford, 1998. [17] J. Barrow. Mathematical jujitsu: some informal thoughts about Godel and physics, Complexity 5 (2000), 28-34. [18] J. Barrow, F. J. Tipler. The Anthropic Cosmological Principle, Oxford University Press, Oxford, 1986. [19] J.-P. Barthelemy, G. Cohen, A. Lobstein. Complexite algorithmique et problemes de communications, Masson, Paris, 1992. [20] S. Bassein. A sampler of randomness Amer. Math. Monthly 103, 6 (1996),483-490. [21] R. W. Batterman, H. S. White. Chaos and algorithmic complexity, Found. Phys. 26 (1996), 307-336. [22] V. Becher, G. Chaitin. Another example of higher order randomness, CDMTCS Research Report 187, 2002, 16pp. [23] V. Becher, S. Daicz, G. Chaitin. A highly random number, in C. S. Calude, M. J. Dinneen, S. Sburlan (eds.). Combinatorics, Computability and Logic, Proceedings of DMTCS'Ol, SpringerVerlag, London, 2001, 55-68. [24] V. Becher, S. Figueira. An example of a computable absolutely normal number, Theoret. Comput. Sci. 270 (2002), 947-958. [25] E. Beltrami. What Is Random? Chance and Order in Mathematics and Life, Copernicus, New York, 1999. [26] C. H. Bennett. The thermodynamics of computation - a review, Int. J. Theoret. Physics 21 (1982), 905-940.
Bibliography
421
[27] C. H. Bennett. Dissipation, information, computational complexity and the definition of organization, in D. Pines (ed.). Emerging Syntheses in Science, Proc. Workshop, Santa Fe Institute, 1985, 297-313. [28] C. H. Bennett. Logical depth and physical complexity, in R. Herken (ed.). The Universal Turing Machine. A Half-Century Survey, Oxford University Press, Oxford, 1988, 2270-258. [29] C. H. Bennett. E-mail to C. Calude, 25 April 1993. [30] C. H. Bennett. Chaitin's Omega, in M. Gardner (ed.). Fractal Music, Hypercards, and More ... , W. H. Freeman, New York, 1992, 307-319. [31] C. H. Bennett, P. Gacs, M. Li, P. M. Vitanyi, W. H. Zurek. Thermodynamics of computation and information distance, Proc. STOC'93, 21-30. [32] C. H. Bennett, M. Gardner. The random number omega bids fair to hold the mysteries of the universe, Scientific American 241 (1979),20-34. [33] C. H. Bennett, J. Gill. Relative to a random oracle A, pA =lN pA =I- co - N pA, with probability one, SIAM 1. Comput. 10 (1981),96-113. [34] J. L. Bentley, A. C. Yao. An almost optimal algorithm for unbounded search, Inf. Proc. Lett. 5 (1976), 82-87. [35] J. Bernoulli. The law of the large numbers, in J. R. Newman (ed.). The World of Mathematics, Vol. 3, Simon and Schuster, New York, 1956, 1452-1455. [36] J. Berstel, D. Perrin. Theory of Codes, Academic Press, New York, 1985. [37] M. Blum. On the size of machines, Inform. and Control 11 (1967), 257-265. [38] R. V. Book. On languages reducible to algorithmically random languages, SIAM J. Comput. 23 (1994), 1275-1282.
422
Bibliography [39] R. V. Book, J. Lutz, K. Wagner. On complexity classes and algorithmically random languages, Proc. STACS-92, Lecture Notes Comput. Sci. Springer-Verlag, Berlin, 577, 1992, 319-328. [40]
E.
[41]
E.
[42]
E.
Borel. Le hasard, Alcan, Paris, 1928.
[43]
E.
Borel. Les paradoxes de l'injini, Gallimard, Paris, 1946.
Borel. Les probabilites denombrables et leurs applications arithmetiques, Rend. Circ. Mat. Palermo 27 (1909), 247-271.
Borel. Ler;ons sur la theorie des fonctions, Gauthier-Villars, Paris, 1914 (2nd ed).
[44] E. Borger. Computability, Complexity, Logic, North-Holland, Amsterdam, 1989. [45] J. M. Borwein. The experimental mathematician: the pleasure of discovery and the role of proof, Int. J. Comput. Math. Learning, to appear. [46] D. S. Bridges. Computability: Springer-Verlag, Berlin, 1994.
A Mathematical Sketchbook,
[47] D. S. Bridges, F. Richman. Varieties of Constructive Mathematics, Cambridge University Press, Cambridge, 1987. [48] L. Brisson, F. W. Meyerstein. Inventing the Universe, State University of New York Press, New York, 1995. [49] R. H. Brown. Does God play dice?, in P. A. P. Moran (ed.). Chance in Nature, Australian Academy of Science, Sydney, 1979, 29-34. [50] A. A. Brudno. Entropy and the complexity of the trajectories of a dynamical system, Trans. Moscow Math. Soc. 2 (1983), 127-151. [51] C. Calude. Theories of Computational Complexity, Holland, Amsterdam, 1988.
North-
[52] C. Calude. Meanings and texts: an algorithmic metaphor, in M. Balat, J. Deledalle-Rhodes (eds.). Signs of Humanity, Mouton de Gruyter, Berlin, 1992, 95-97.
Bibliography
423
[53] C. Calude. Borel normality and algorithmic randomness, in G. Rozenberg, A. Salomaa (eds.). Developments in Language Theory, World Scientific, Singapore, 1994, 113-129 (with a note by G. J. Chaitin). [54] C. Calude. What is a random string? J. UCS 1 (1995), 48-66. [55] C. Calude. What is a random string? - Extended Abstract, in W. Depauli-Schimanovich, E. Koehler, F. Stadler (eds.). The Foundational Debate, Complexity and Constructivity in Mathematics and Physics, Kluwer, Dordrecht, 1995, 101-113. [56] C. Calude. Algorithmic information theory: open problems, J. UCS 2 (1996), 439-441. [57] C. Cahide. Computability and information, in E. Craig (ed.). Routledge Encyclopedia of Philosophy, Routledge, London, Vol. 2 (1998), 477-482. [58] C. S. Calude. Who is afraid of randomness?, in E. von Collani (ed.). Millennial Symposium 'Defining the Science of Stochastics', Wuerzburg University, 2000, 99-122. [59] C. S. Calude. A glimpse into algorithmic information theory, in L. Cavedon, P. Blackburn, N. Braisby, A. Shimojima (eds.). Logic, Language and Computation, Vol. 3, CSLI Series, CSLI Lectures Notes 111, Stanford, 2000, 67-83. [60] C. S. Calude. A characterization of c.e. random reals, Theoret. Comput. Sci. 217 (2002), 3-14. [61] C. S. Calude. Incompleteness, complexity, randomness and beyond, Minds and Machines, to appear; see also CDMTCS Research Report 166, 2001, 11pp. [62] C. S. Calude. Chaitin n numbers, Solovay machines and incompleteness, in K.-I. Ko, A. Nerode, K. Weihrauch (eds.). "Computability and Complexity in Analysis", Theoret. Comput. Sci. 284 (2002), 269-277. [63] C. Calude, C. Campeanu. Note on the topological structure of random strings, Theoret. Comput. Sci. 112 (1993), 383-390. [64] C. Calude, C. Campeanu. Are binary codings universal?, Complexity 1 (1996), 47-50.
424
Bibliography
[65] C. S. Calude, J. L. Casti. Parallel thinking, Nature 392 (1998), 549-551. [66] C. S. Calude, G. J. Chaitin. Randomness everywhere, Nature 400, 22 july (1999), 319-320. [67] C. Calude, 1. Chi~escu. Strong noncomputability of random strings, Int. J. Comput. Math. 11 (1982), 43-45. [68] C. Calude, 1. Chi~escu. Probabilities on the space of sequences, Technical Report 103, 1994, Computer Science Department, University of Auckland, New Zealand, 10pp. [69] C. Calude, 1. Chi~escu. Random sequences: some topological and measure-theoretical properties, An. Univ. Bucure§ti Mat. -Inf. 2 (1988), 27-32. [70] C. Calude, 1. Chi~escu. A combinatorial characterization of P. Martin-Lof tests, Int. 1. Comput. Math. 17 (1988), 53-64. [71] C. Calude, 1. Chi~escu. Upper limitation of Kolmogorov complexity and universal P. Martin-Lof tests, J. Comput. Math. 7 (1989), 61-70. [72] C. Calude, 1. Chi~escu. Qualitative properties of P. Martin-LM random sequences, Boll. Unione Mat. Ital. (7) 3-B (1989), 229240. [73] C. Calude, 1. Chi~escu, L. Staiger. P. Martin-Lof tests: represent ability and embeddability, Rev. Roumaine Math. Pures Appl. 30 (1985), 719-732. [74] C. S. Calude, R. J. Coles. On atheorem of Solovay, CDMTCS Research Report 94, 1999, 14pp. [75] C. S. Calude, R. J. Coles. Program-size complexity of initial segments and domination relation reducibility, in J. Karhumaki, H. A. Maurer, G. Paun, G. Rozenberg (eds.). Jewels Are Forever, Springer-Verlag, Berlin, 1999, 225-237. [76] C. S. Calude, R. Coles, P. H. Hertling, B. Khoussainov. Degreetheoretic aspects of computably enumerable reals, in S. B. Cooper, J. K. Truss (eds.). Models and Computability, Cambridge University Press, Cambridge, 1999, 23-39.
Bibliography
425
[77] C. S. Calude, M. J. Dinneen, C.-K. Shu. Computing a glimpse of randomness, Exp. Math., to appear in 2002; see also CDMTCS Research Report 167, 2001, 12pp. [78] C. S. Calude, M. J. Dinneen, K. Svozil. Reflections on quantum computing, Complexity 6, 1 (2000), 35-37. [79] C. Calude, R. W. Doran. Does God play dice?, EATCS Bull. 50 (1993), 338-341. [80] C. S. Calude, Monica Dumitrescu. Entropic measures, Markov information sources and complexity, Appl. Math. Comput., to appear in 2002; see also CDMTCS Research Report 150, 2001, 12pp. [81] C. Calude, C. Grozea. Kraft-Chaitin inequality revisited, J. UCS 2 (1996), 306-310. [82] C. S. Calude, P. Hertling. Computable approximations of reals: an information-theoretic analysis, Fundam. Informaticae 33 (1998), 1-16. [83] C. S. Calude, P. Hertling, H. Jurgensen, K. Weihrauch. Randomness on full shift spaces, Chaos, Solitons, Fractals, 12/3 (2001), 491-503. [84] C. Calude, P. Hertling, B. Khoussainov. Do the zeros of Riemann's zeta-function form a random sequence?, EATCS Bull. 62 (1997), 199-207. [85] C. S. Calude, P. Hertling, B. Khoussainov, Y. Wang. Recursively enumerable reals and Chaitin n numbers, Theoret. Comput. Sci. 255 (2001), 125-149. Also in M. Morvan, C. Meinel, D. Krob (eds.). STACS'98, Paris, 1998, Lecture Notes Comput. Sci. 1373, Springer-Verlag, Berlin, 1998, 596-606. [86] C. Calude, J. Hromkovic. Complexity: a language-theoretic point of view, in G. Rozenberg, A. Salomaa (eds.). Handbook of Formal Languages, Vol. II, Springer-Verlag, Berlin, 1997, 1-60. [87] C. S. Calude, H. Ishihara, T. Yamaguchi. Minimal programs are almost optimal, Int. J. Found. Comput. Sci. 12, 4 (2001), 479489.
426
Bibliography [88] C. Calude, G. Istrate. Determining and stationary sets for some classes of partial recursive functions, Theoret. Comput. Sci. 82 (1991), 151-155. [89] C. Calude, H. Jurgensen. Randomness as an invariant for number representations, in H. Maurer, J. Karhumiiki, G. Rozenberg (eds.). Results and Trends in Theoretical Computer Science, Springer-Verlag, Berlin, 1994, 44-66. [90] C. S. Calude, H. Jurgensen, S. Legg. Solving finitely refutable mathematical problems, in C. S. Calude, G. Paun (eds.). Finite Versus Infinite. Contributions to an Eternal Dilemma, SpringerVerlag, London, 2000, 39-52. [91] C. Calude, H. Jurgensen, M. Zimand. Is independence an exception?, Appl. Math. Comput. 66 (1994), 63-76. [92] C. Calude, E. Kurta. On Kraft-Chaitin inequality, Rev. Roumaine Math. Pures Appl. 35 (1990), 597-604. [93] C. Calude, S. Marcus, D. gtefanescu. The Creator versus its creation: from Scotus to Godel, Collegium Logicum. Annals of the Kurt-Godel-Society, Vol. 3, Institute of Computer Science, AS CR Prague, Vienna, 1999, 1-10.
[94] C. S. Calude, F. W. Meyerstein. Is the universe lawful?, Chaos, Solitons, Fractals 10 (1999), 1075-1084. [95] C. S. Calude, A. Nies. Chaitin n numbers and strong reducibilities, 1. UCS 3 (1997), 1161-1166. [96] C. S. Calude, G. Paun. Computing with Cells and Atoms, Taylor & Francis, London, 2001. [97] C. S. Calude, B. Pavlov. Coins, quantum measurements, and Turing's barrier, Quantum Inf. Process. 1, 1-2 (2002), 107-127.; see also CDMTCS Research Report 170, 2001, 15pp. [98] C. Calude, A. Salomaa. Algorithmically coding the universe, in G. Rozenberg, A. Salomaa (eds.). Developments in Language Theory, World Scientific, Singapore, 1994, 472-492. [99] C. S. Calude, 1. Tomescu. Optimum extendible prefix codes, 1. UCS 3 (1997), 1167-1179.
Bibliography
427
[100] C. Calude, T. Zamfirescu. The typical number is a lexicon, New Zealand 1. Math. 27 (1998), 7-13. [101] C. S. Calude, T. Zamfirescu. Most numbers obey no probability laws, Publ. Mathematicae Debrecen 54 Supplement (1999), 619623. [102] C. Calude, M. Zimand. A relation between correctness and randomness in the computation of probabilistic algorithms, Int. 1. Comput. Math. 16 (1984),47-53.
[103] J.1. Casti. Paradigms Lost, Avon Books, New York, 1990. [104] J. L. Casti. Searching for Certainty, William Morrow, New York, 1990.
[105] J. L. Casti. Computing the uncomputable, New Scientist, 154/2082, 17 May (1997), 34. [106] J. Casti. Five More Golden Rules-Knots, Codes, Chaos, and Other Great Theories of 20th-Century Mathematics, John Wiley & Sons, New York, 2000. [107] C. Campeanu. Topological Methods in Complexity Theory, Ph.D. Thesis Bucharest University, Romania, 1995. [108] C. Campeanu. Random numbers are Borel normal, EATCS Bull. 58 (1996), 155-158. [109] J. Cervelle, B. Durand, E. Formenti. Algorithmic information theory and cellular automata dynamics, in J. Sgall, A. Pultr, P. Kolman (eds.). Proc. MFCS'2001, Lecture Notes Comput. Sci. 2136, Springer-Verlag, Heidelberg, 2001, 248-260. [110] G. J. Chaitin. On the length of programs for computing finite binary sequences, 1. Assoc. Comput. Mach. 13 (1966), 547-569. (Reprinted in: [122], 219-244.) [111] G. J. Chaitin. On the length of programs for computing finite binary sequences: statistical considerations, J. Assoc. Comput. Mach. 16 (1969), 145-159. (Reprinted in: [122], 245-260.) [112] G. J. Chaitin. Computational complexity and Godel's incompleteness theorem, Notices Amer. Math. Soc. 17 (1970), 672. (Reprinted in: [122], 284.)
428
Bibliography
[113] G. J. Chaitin. Information-theoretic limitations of formal systems, 1. Assoc. Comput. Mach. 21 (1974), 403-424. (Reprinted in: [122], 113-128.) [114] G. J. Chaitin. A theory of program size formally identical to information theory, J. Assoc. Comput. Mach. 22 (1975), 329-340. (Reprinted in: [122], 197-223.) [115] G. J. Chaitin. Randomness and mathematical proof, Scientific American 232 (1975), 47-52. (Reprinted in: [122], 3-13.) [116] G. J. Chaitin. Algorithmic entropy of sets, Comput. Math. Appl. 2 (1976),233-245. (Reprinted in: [122], 153-168.) [117] G. J. Chaitin. Information-theoretic characterizations of recursive infinite strings, Theoret. Comput. Sci. 2 (1976), 45-48. (Reprinted in: [122], 203-206.) [118] G. J. Chaitin. Algorithmic information theory, IBM J. Res. Dev. 21 (1977), 350-359, 496. (Reprinted in: [122], 44-58.) [119] G. J. Chaitin. Toward a mathematical definition of "life", in R. D. Levine, M. Tribus (eds.). The Maximum Entropy Formalism, MIT Press, Cambridge, MA, 1979,477-498. (Reprinted in: [122], 92-110.) [120] G. J. Chaitin. Godel's theorem and information, Int. 1. Theoret. Physics 21 (1982), 941-954. (Reprinted in: [122], 61-71.) [121] G. J. Chaitin. Algorithmic Information Theory, Cambridge University Press, Cambridge, 1987 (3rd printing 1990). [122] G. J. Chaitin. Information, Randomness and Incompleteness: Papers on Algorithmic Information Theory, World Scientific, Singapore, 1987 (2nd ed, 1990). [123] G. J. Chaitin. Incompleteness theorems for random reals, Adv. Appl. Math. 8 (1987),119-146. (Reprinted in: [122], 129-152.) [124] G. J. Chaitin. Randomness in arithmetic, Scientific American 259 (1988), 8(}-85. (Reprinted in: [122], 14-19.) [125] G. J. Chaitin. Information-Theoretic Incompleteness, World Scientific, Singapore, 1992.
Bibliography
429
[126] G. J. Chaitin. Randomness in arithmetic and the decline and fall of reductionism in pure mathematics, EATCS Bull. 50 (1993), 314-328. [127] G. J. Chaitin. On the number of N-bit strings with maximum complexity, Appl. Math. Comput. 59 (1993), 97-100. [128] G. J. Chaitin. The Berry paradox, Complexity 1 (1995), 26-30. [129] G. J. Chaitin. Program-size complexity computes the halting problem I, EATCS Bull. 57 (1995), 199-200. [130] G. J. Chaitin. The Limits of Mathematics, Springer-Verlag, Singapore, 1998. [131] G. J. Chaitin. The Unknowable, Springer-Verlag, Singapore, 2000. [132] G. J. Chaitin. Exploring Randomness, Springer-Verlag, London, 2001. [133] G. J. Chaitin. Personal communication to C. S. Calude, December 2001. [134] G. J. Chaitin. Conversations with a Mathematician, SpringerVerlag, London, 2002. [135] G. J. Chaitin. Computers, paradoxes and the foundations of mathematics, American Sci. 90, March-April (2002), 164-171. [136] G. J. Chaitin. Meta-mathematics and the foundations of mathematics, CDMTCS Research Report 182, 2002, 14pp. [137] G. J. Chaitin, J. T. Schwartz. A note on Monte-Carlo primality tests and algorithmic information theory, Comm. Pure Appl. Math. 31 (1978), 521-527. (Reprinted in: [122], 197-202.) [138] D. G. Champernowne. The construction of decimals normal in the scale of ten, J. London Math. Soc. 8 (1933), 254-260. [139] M. Chown. The Omega man, New Scientist 10 March (2001), 29-31. bibitemmc2 M. Chown. Smash and grab, New Scientist 6 April (2002), 24-28. [140] K. L. Chung. Elementary Probability Theory with Stochastic Processes, Springer-Verlag, New York (3rd ed, 1979).
430
Bibliography
[141] A. Church. On the concept of a random sequence, Bull. Amer. Math. Soc. 46 (1940), 130-135. [142] B. Cipra. Prime formula weds number theory and quantum physics, Science 274 (20 December) (1996), 2014-2015. [143] D. E. Cohen. Computability and Logic, Ellis Horwood, John Wiley & Sons, New York, 1987. [144] P. J. Cohen. Set Theory and the Continuum Hypothesis, Benjamin, New York, 1966. [145] A. Connes, A. Linchnerowicz, M. P. Schiitzenberger. Triangle of Thoughts, American Mathematical Society, Providence, RI, 2001. [146] A. H. Copeland, P. Erdos. Note on normal numbers, Bull. Amer. Math. Soc. 52 (1946), 857-860. [147] J. Copeland. The modern history of computing, in Edward N. Zalta (ed.). The Stanford Encyclopedia of Philosophy (Fall 1999 Edition), http://plato . stanford. edu/entries/ computing-history/. [148] J. Copeland. Narrow versus wide mechanism: Including a reexamination of Turing's views on the mind-machine issue, J. Philos. XCVI, 1 (2000), 5-32. [149] N. C. A. da Costa, F. A. Doria. Undecidability and incompleteness in classical mechanics, Int. J. Theoret. Physics 30 (1991), 1041-1073. [150] T. M. Cover. Universal gambling schemes and the complexity measures of Kolmogorov and Chaitin, Technical Report 12, 1974, Stanford University, CA, 29pp. [151] T. M. Cover, P. Gacs, R. M. Gray. Kolmogorov's contributions to information theory and algorithmic complexity, Ann. Probab. 17 (1989), 840-865. [152] T. M. Cover, J. Y. Thomas. Elements of Information Theory, John Wiley & Sons, New York, 1991. [153] I. Csiszar, J. Korner. Information Theory, Academic Press, New York, 1981.
Bibliography
431
[154] G. Davie. Recursive events in random sequences, Arch. Math. Logic, 40, 8 (2001), 629-638. [155] P. Davies. The Mind of God, Science and the Search for Ultimate Meaning, Penguin Books, London, 1992. [156] P. Davies, J. Gribbin. The Matter Myth. Beyond Chaos and Complexity, Penguin Books, London, 1992. [157] M. Davis. What is a computation?, in L. A. Steen (ed.). Mathematics Today: Twelve Informal Essays, Springer-Verlag, New York, 1978, 241-267. [158] M. Davis, H. Putnam, J. Robinson. The decision problem for exponential diophantine equations, Ann. Math. 74 (1961), 425436. [159] M. Davis, Yu. V. Mat iyasevich , J. Robinson. Hilbert's tenth problem. Diophantine equations: positive aspects of a negative solution, in F. E. Browder (ed.). Mathematical Developments Arising from Hilbert Problems, American Mathematical Society, Providence, RI, 1976, 323-378. [160] P. J. Davis, R. Hersh. The Mathematical Experience, Birkhauser, Boston, 1981. [161] J. W. Dawson, Jr. Logical Dilemmas. The Life and Work of Kurt Gadel, A K Peters, Wellesley, MA, 1997. [162] J. W. Dawson, Jr. E-mail to C. Calude, 21 May 1997. [163] K. De Leeuw, E. F. Moore, C. E. Shannon, N. Shapiro. Computability by probabilistic machines, in C. E. Shannon, J. McCarthy (eds.). Automata Studies, Princeton University Press, Princeton, NJ, 1956, 183-212. [164] J.-P. Delahaye. Information, Complexite et Hasard, Hermes, Paris, 1994. [165] J.-P. Delahaye. Les nombres omega, Pour la Science, 292 May (2002),98-103. [166] C. Dellacherie. Nombres au hazard. De Borel Math., Soc. Math. France 11 (1978), 23-58.
a Martin-Loef,
Gaz.
432
Bibliography
[167] W. A. Dembski. Randomness, in E. Craig (ed.). Routledge Encyclopedia of Philosophy, Routledge, London, Vol. 8 (1998), 56-59. [168] O. Demuth. On constructive pseudorandomnumbers, Comment. Math. Univ. Carolin. 16 (1975), 315-331 (in Russian). [169] O. Demuth. On some classes of arithmetical real numbers, Comment. Math. Univ. Carolin. 23 (1982), 453-465 (in Russian). [170] O. Demuth. Reducibilities of sets based on constructive functions of a real variable, Comment. Math. Univ. Carolin. 26 (1988), 143156.
[l71J O. Demuth. Remarks on the structure of tt-degrees based on the construction of measure theory, Comment. Math. Univ. Carolin. 29 (1988), 233-247. [172] K. G. Denbigh, J. S. Denbigh. Entropy in Relation to Incomplete Knowledge, Cambridge University Press, Cambridge, 1985. [173J M. Denker, M. W. Woyczynski, B. Y cart. Introductory Statistics and Random Phenomena: Uncertainty, Complexity, and Chaotic Behavior in Engineering and Science, Birkhauser, Boston, 1998. [174] M. Detlefsen. Godel's theorems, in E. Craig (ed.). Routledge Encyclopedia of Philosophy, Routledge, London, Vol. 4 (1998), 106119. [175] J.-C. Dubacq, B. Durand, E. Formenti. Kolmogorov complexity and cellular automata classification, Theoret. Comput. Sci. 259 (2001), 271-285. [176] D. Deutsch. Quantum theory, the Church-Turing principle and the universal quantum computer, Proc. R. Soc. London A400 (1985), 97-117. [177] A. K. Dewdney. A computer trap for the busy beaver, the hardest-working Turing machine, Scientific American 251 (1984), 10-17. [178] L. E. Dickson. History of the Theory of Numbers, 3 volumes, Carnegie Institute, Washington, DC, 1919, 1920, 1923. [179] P. A. M. Dirac. The Principles of Quantum Mechanics, Oxford University Press, Oxford, 1930.
Bibliography
433
[180] R. Downey. An invitation to structural complexity, New Zealand 1. Math. 21 (1992), 33-89. [181] R. G. Downey. Some computability-theoretical aspects of reals and randomness, CDMTCS Research Report 173, 2002, 42pp. [182] R. Downey, D. Hirschfeldt. Algorithmic Randomness and Complexity, Springer-Verlag, Berlin, in preparation. [183] R. Downey, D. R. Hirschfeldt, G. 1. LaForte. Randomness and reducibility, in J. Sgall, A. Pultr, P. Kolman (eds.). Proc. MFCS'2001, Lecture Notes Comput. Sci. 2136, Springer-Verlag, Heidelberg, 2001, 316-327. [184] R. Downey, D. R. Hirschfeldt, A. Nies. Randomness, computability and density, in A. Ferreira, H. Reichel (eds.). Proc. STACS 2001, Springer-Verlag, Berlin, 2001, 195-205; full paper to appear in SIAM 1. Comput. [185] R. Downey, G. L. LaForte. Presentations of computably enumerable reals, in K.-1. Ko, A. Nerode, K. Weihrauch (eds.). "Computability and Complexity in Analysis", Theoret. Comput. Sci. 284 (2002), 539-555. [186] S. Dragomir. E-mail to C. Calude, 5 December 200l. [187] R. M. Dudley. Real Analysis and Probability, Wadsworth & Brooks/Cole, Pacific Grove, CA, 1989. [188] N. Duta. Representability and embeddability of P. Martin-Lof tests, Stud. Cercet. Mat. 47 (1995), 245-262. [189] G. Etesi, 1. Nemeti. Non-Turing computations via MalamentHogarth space-times, Int. 1. Theoretical Physics 41 (2002), 341370. [190] S. Feferman, J. Dawson, Jr., S. C. Kleene, G. H. Moore, R. M. Solovay, J. van Heijenoort (eds.) . Kurt Godel Collected Works, Vol. I, Oxford University Press, New York, 1986. [191] S. Feferman, J. Dawson, Jr., S. C. Kleene, G. H. Moore, R. M. Solovay, J. van Heijenoort (eds.). Kurt Godel Collected Works, Vol. II, Oxford University Press, New York, 1990.
434
Bibliography
[192] W. Feller. An Introduction to Probability Theory and Its Applications, Vol. 1, Chapman & Hall, London; John Wiley & Sons, New York (3rd ed, 1968). [193] J. Ford. How random is a random coin toss?, Phys. Today 36 (1983),40-47. [194] W. L. Fouche. Descriptive complexity and reflective properties of combinatorial configurations, J. London Math. Soc. 54 (1996), 199-208. [195] M. Ferbus-Zanda, S. Grigorieff. Is randomness "native" to computer science?, EATCS Bull. 74 (2001), 78-118. [196] R. Feynman. Simulating physics with computers, Int. J. Theoret. Physics 21 (1982), 467-488. [197] T. L. Fine. Theories of Probability. An Examination of Foundations, Academic Press, New York, 1973. [198] E. Fredkin, T. Toffoli. Conservative logic, Int. J. Theoret. Physics 21 (1982), 219-255. [199] P. Gacs. On the symmetry of algorithmic information, Sov. Math. Dokl. 15 (1974), 1477-1480; correction, ibid. 15 (1974), 1480. [200] P. Gacs. Exact expressions for some randomness tests, Z. Math. Logik Grundlag. Math. 26 (1980), 385-394. [201] P. Gacs. On the relation between descriptional complexity and algorithmic probability, Theoret. Comput. Sci. 22 (1983), 71-93. [202] P. Gacs. Every sequence is reducible to a random one, Inform. and Control 70 (1986), 186-192. [203] P. Gacs. Lecture Notes on Descriptional Complexity and Randomness, Boston University, 1988, 62pp. [204] S. Galato. A proof of the Beyer-Stein-Ulam relation between complexity and entropy, Discrete Mathematics 223 (2000), 367-372. [205] M. Gardner. A collection of tantalizing fallacies of mathematics, Scientific American 198 (1958), 92. [206] M. Gardner. Fractal Music, Hypercards, and More . .. , W. H. Freeman, New York, 1992, 307-319.
Bibliography
435
[207] M. Garey, D. Johnson. Computers and Intractability: A Guide to the Theory of N P-Completeness, W. H. Freeman, New York, 1979. [208] W. L. Gewirtz. Investigations in the theory of descriptive complexity, Report NSO-5, Courant Institute of Mathematical Sciences, New York University, 1974, 60pp. [209] J. Gill. Computational complexity of probabilistic Turing machines, SIAM 1. Comput. 6 (1976), 675-695. [210] K. Godel. An example of a new type of cosmological solutions of Einstein's field equations of gravitation, Rev. Modern Physics 21 (1949),447-450. (Reprinted in: [191]' 190-198.) [211] K. Godel. The Consistency of the Continuum Hypothesis, Princeton University Press, Princeton, NJ, 1940. [212] K. Godel. Russell's mathematical logic, in P. Benacerref, H. Putnam (eds.). Philosophy of Mathematics, Prentice-Hall, Englewood Cliffs, NJ, 1964,211-232. (Reprinted in [191]' 119-141.) [213] K. Godel. What is Cantor's continuum problem?, in P. Benacerref, H. Putnam (eds.). Philosophy of Mathematics, Prentice-Hall, Englewood Cliffs, NJ, 1964, 258-273. (Reprinted in: [191]' 176187.) [214] R. L. Graham, B. L. Rothschiled, J. H. Spencer. Ramsey Theory, John Wiley & Sons, New York (2nd ed, 1990). [215] E. Grosswald (ed.). Collected papers of Hans Rademacher, MIT Press, Cambridge, MA, 1974, 454-455. [216] C. Grozea. Free-extendible prefix-free sets and an extension of the Kraft-Chaitin theorem, 1. UCS 6 (2000), 130-135. [217] J. Gruska. Foundations of Computing, Thomson International Computer Press, Boston, 1997. [218] J. Gruska. Quantum Computing, McGraw-Hill, London, 1999. [219] S. Guia§u. Information Theory and Applications, McGraw-Hill, New York, 1977.
436
Bibliography
[220] P. R. Halmos. Measure Theory, Van Nostrand, Princeton, NJ, 1950. (Reprinted: Springer-Verlag, Berlin, 1974.) [221] D. Hammer. Complexity Inequalities, Wissenschaft & Technik Verlag, Berlin, 1998. [222] G. H. Hardy. Goldbach's theorem, Mat. Tid. B 1 (1922), 1-16. (Reprinted in: Collected Papers of G. H. Hardy, Vol. 1, Oxford University Press, Oxford, 1966, 545-560.) [223] G. H. Hardy, E. M. Wright. An Introduction to the Theory of Numbers, Clarendon Press, Oxford (5th ed, 1979). [224] J. Hartmanis. Generalized Kolmogorov complexity and the structure of feasible computations, in Proc. 24th IEEE Symp. Foundations of Computer Science, 1983, 439-445. [225] J. Hartmanis, L. Hemachandra. On sparse oracles separating feasible complexity classes, Inf. Process. Lett. 28 (1988), 291-295. [226] J. Hartmanis, L. Hemachandra, S. A. Kurtz. Complexity corresponds to topology, Technical Report 88-17, University of Chicago, 1988, 12pp. [227] J. Hartmanis, J. E. Hopcroft. Independence results in computer science, SIGACT News 8 (1976), 13-24. [228] S. W. Hawking. A Brief History of Time: From the Big Bang to Black Holes, Bantam Press, London, 1988. [229] L. Hemachandra, M. Ogihara. The Complexity Theory Companion, Springer-Verlag, Heidelberg, 2002. [230] L. Hemaspaandra, M. Zimand. Strong self-reducibility precludes strong immunity, Math. Syst. Theory 29 (1996), 535-548. [231] P. Hertling. Disjunctive w-words and real numbers, J. UCS 2 (1996), 549-568. [232] P. Hertling. Surjective functions on computably growing Cantor sets, 1. UCS 3 (1997), 1226-1240. [233] P. Hertling. Simply normal numbers to different bases, J. UCS 2 (2002), 235-242.
Bibliography
437
[234] P. Hertling, Y. Wang. lnvariance properties of random sequences, J. UCS 12 (1997), 1241-1249. [235] P. Hertling, K. Weihrauch. Randomness spaces, in K. G. Larsen, S. Skyum, G. Winskel (eds.). Automata, Languages and Programming, Proc. 25th Int. Coll., ICALP'98, Aalborg, Denmark, july 1998, Springer-Verlag, Berlin, 1998, 796-807. [236] J. G. Hey (ed.). Feynman and Computation. Exploring the Limits of Computers, Perseus Books, Reading, MA, 1999. [237] E. Hlawka. The Theory of Uniform Distribution, A B Academic Publishers, Zurich, 1984. [238] C.-K. Ho. Relatively recursive reals and real functions, Theoret. Comput. Sci. 219 (1999), 99-120. [239] J. Horgan. The death of proof, Scientific American 269 (1993), 74-82. [240] A. Jaffe, F. Quinn. "Theoretical mathematics": toward a cultural synthesis of mathematics and theoretical physics, Bull. Amer. Math. Soc. 29 (1993), 1-13. [241] D. S. Jones. Elementary Information Theory, Clarendon Press, Oxford, 1979. [242] J. P. Jones, Yu. V. Matiyasevich. Register machine proof of the theorem on exponential diophantine representation of enumerable sets, J. Symb. Logic 49 (1984), 818-829. [243] D. Juedes, J. Lathrop, J. Lutz. Computational depth and reducibility, Theoret. Comput. Sci. 132 (1994), 37-70. [244] H. Jurgensen, J. Duske. Codierungstheorie, Bl, Mannheim, 1977. [245] H. Jurgensen, G. Thierrin. Some structural properties of wlanguages, 13th Natl. School with Int. Participation "Applications of Mathematics in Technology", Sofia, 1988, 56-63. [246] H. Jurgensen, H. J. Shyr, G. Thierrin. Disjunctive w-languages. ElK 19 (1983), 267-278. [247] T. Kamae. On Kolmogorov's complexity and information, Osaka J. Math. 10 (1973), 305-307.
438
Bi bliography
[248] H. P. Katseff. Complexity dips in random infinite binary sequences, Inform. and Control 38 (1978), 258-263. [249] H. P. Katseff, M. Sipser. Several results in program size complexity, Theoret. Comput. Sci. 15 (1981), 291-309. [250] S. Kautz. Degrees of Random Sets, Ph.D. Thesis, Cornell University, Ithaca, NY, 1991. [251] J. L. Kelley. General Topology, Van Nostrand, Princeton, NJ, 1955. [252] A. I. Khinchin. Mathematical Foundations of Information Theory, Dover, New York, 1957. [253] B. Khoussainov. Randomness, computability, and algebraic specifications, Ann. Pure Appl. Logic 91 (1998), 1-15. [254] T. D. Kieu. Hilbert's incompleteness, Chaitin's n number and quantum physics, Los Alamos preprint archive http://arXiv:quant-ph/0111062,vl, 10 November 2001. [255] D. E. Knuth. The Art of Computer Programming, Vol. 2, Seminumerical Algorithms, Addison-Wesley, Reading, MA (2nd ed, 1981). [256] D. E. Knuth. Supernatural numbers, in D. A. Klamer (ed.). The Mathematical Gardner, Prindle, Weber & Schmidt, Wadsworth, Boston, MA, 1981, 310-325. [257] D. E. Knuth. Theory and practice, EATCS Bull. 27 (1985), 1421. [258] Ker-I Ko. Complexity of Real Functions, Birkha,user, Berlin, 1991. [259] A. N. Kolmogorov. Three approaches for defining the concept of "information quantity", Prabl. Inf. Transm. 1 (1965), 3-11. [260] A. N. Kolmogorov. Logical basis for information theory and probability theory, IEEE Trans. Inf. Theory 14 (1968), 662-664. [261] A. N. Kolmogorov, V. A. Uspensky. Algorithms and randomness, Theory Prabab Appl. 32 (1988), 389-412. (Two corrections in: Uspensky [407]' p.l02.)
Bibliography
439
[262] L. G. Kraft. A Device for Quantizing Grouping and Coding Amplitude Modulated Pulses, MS Thesis, MIT, Cambridge, MA, 1949. [263]
r. Kramosil. Recursive classification of pseudo-random sequences, Kybemetika (Prague) 20 (1984), 1-34 (supplement).
[264]
r. Kramosil, J. Sindelar. Infinite pseudo-random sequences of high algorithmic complexity, Kybemetika (Prague) 20 (1984), 429437.
[265] A. Kucera. Measure, II~-classes and complete extensions of PA, in H.-D. Ebbinghaus, G. H. Milller, G. E. Sacks (eds.). Recursion Theory Week, Proceedings, Oberwolfach 1984, Lecture Notes Math. 1141, Springer-Verlag, Berlin, 1985, 245-259. [266] A. Kucera, T. A. Slaman. Randomness and recursive enumerability, SIAM J. Comput. 31 (2001), 199-211. [267] A. Kucera, S. Terwijn. Lowness for the class of random sets, J. Symb. Logic 64 (1999), 1396-1402. [268] L. Kuipers, H. Niederreiter. Uniform Distribution of Sequences, John Wiley & Sons, New York, 1974. [269] M. Kummer. On the complexity of random strings, in C. Puech, R. Reischuk (eds.). Proceedings of STACS'96, Lecture Notes Comput. Sci. 1046, Springer-Verlag, Berlin, 1996, 25-38. [270] M. Kummer. Kolmogorov complexity and instance complexity of recursively enumerable sets, SIAM J. Comput. 25 (1996), 11231143. [271] R. Landauer. Uncertainty principle and minimal energy dissipation in the computer, Int. 1. Theoret. Physics 21 (1982), 283-297. [272] R. Landauer. Computation: a fundamental physical view, Physica Scripta 35 (1987), 88-95. [273] P. S. Laplace. A Philosophical Essay on Probability Theories, Dover, New York, 1951. [274] M. Lerman, J. B. Remmel. The universal splitting property, I, in D. van Dalen, D. Lascar, T. J. Smiley (eds.). Logic Colloquium '80, North-Holland, Amsterdam, 1982, 181-208.
440
Bibliography
[275] M. Lerman, J. B. Remmel. The universal splitting property, II, J. Symb. Logic 49 (1984), 137-150. [276] S. K. Leung-Yan-Cheong, T. M. Cover. Some equivalences between Shannon entropy and Kolmogorov complexity, IEEE Trans. Info. Theory 24 (1978), 331-338. [277] L. A. Levin. On the notion of random sequence, Sov. Math. Dokl. 14 (1973), 1413-1416. [278] L. A. Levin. Randomness conservation inequalities: information and independence in mathematical theories, Probl. Inf. Transm. 10 (1974), 206-210. [279] S. Levy. Artificial Life, Pantheon Books, New York, 1992. [280] M. Li, P. M. Vitanyi. Kolmogorov complexity and its applications, in J. van Leeuwen (ed.). Handbook of Theoretical Computer Science, Vol. A, North-Holland, Amsterdam, MIT Press, Boston, 1990, 187-254. [281] M. Li, P. M. Vitanyi. Combinatorics and Kolmogorov complexity, Proc. 6th IEEE Structure in Complexity Theory Conj., 1991, 154163. [282] M. Li, P. M. Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications, Springer-Verlag, Berlin, 1993 (2nd ed, 1997). [283] X. Li. Effective immune sets, program index sets and effectively simple sets: generalizations and applications of the recursion theorem, in C.-T. Chong, M. J. Wicks (eds.). South-East Asian Conference on Logic, Elsevier, Amsterdam, 1983, 97-106. [284] D. Lind. Applications of ergodic theory and sofic systems to cellular automata, Physica D 10 (1984), 36-44. [285] D. Lind, B. Marcus. An Introduction to Symbolic Dynamics and Coding, Cambridge University Press, Cambridge, 1995. [286] L. Longpre. Resource bounded Kolmogorov complexity: a link between computational complexity and information theory, Technical Report 86-776, Cornell University, August 1986, 101pp.
Bibliograpby
441
[287] L. Longpre and V. Kreinovich. Zeros of Riemann's zeta function are uniformly distributed, but not random: an answer to Calude's open problem, EATCS Bull. 59 (1996), 163~164. [288] J. H. Loxton. A method of Mahler in transcendence theory and some of its applications, Bull. Austral. Math. Soc. 29 (1984), 127~ 136. [289] J. H. Lutz. Almost everywhere high nonuniform complexity, J. Comput. Syst. Sci. 44 (1992), 220~258. [290] J. H. Lutz. The quantitative structure of exponential time, Proc. Eighth Annual Structure in Complexity Theory Conj., San Diego, CA, May 18-21, 1993, IEEE Computer Society Press, 1993, 158~ 175. [291] J. H. Lutz, E. Mayordomo. Measure, stochasticity, and the density of hard languages, SIAM J. Comput. 23 (1994), 762-779. [292] A. r. Mal'cev. Algorithms and Recursive Functions, WoltersNoordhoff, Groningen, 1970. [293] V. Manca. Logica Matematica, Bollati Boringhieri, 2001. [294]
r.
[295]
r.
Mandoiu. Kraft-Chaitin's theorem for free-extendable codes, St. Cerc. Mat. 44 (1992), 497~501.
Mandoiu. On a theorem of Gacs, Int. J. Comput. Math. 48 (1993), 157~169.
[296] I. Mandoiu. Optimum extensions of prefix codes, Inj. Process. Lett. 66 (1998), 35-40. [297] H. B. Marandijan. Selected topics in recursive function theory in computer science, ID - TR 75, Technical University of Denmark, Lyngby, 1990, 93pp. [298] S. Marcus (ed.). Contextual Ambiguities in Natural 8 Artificial Languages, Comm. & Cognition, Vol. 2, Ghent, Belgium, 1983. [299] S. Marcus. Symmetry in the simplest case: the real line, Computers Math. Applic. 17 (1989), 103-115. [300] G. Markowsky. An introduction to algorithmic information theory: its history and some examples, Complexity 2 (1997), 14-22.
442
Bibliography
[301] P. Martin-Lof. Algorithms and Random Sequences, Erlangen University, Nlirnberg, Erlangen, 1966. [302] P. Martin-Lof. The definition of random sequences, Inform. and Control 9 (1966), 602~619. [303] P. Martin-Lof. Notes on Constructive Mathematics, Almqvist & Wiksell, Stockholm, 1970. [304] P. Martin-Lof. Complexity oscillations in infinite binary sequences, Z. Wahrscheinlichkeitstheorie Verw. Geb. 19 (1971), 225~230.
[305] P. Martin-LOf. On the notion of randomness, in A. Kino, J. Myhill, R. E. Vesley (eds.). Intuitionism and Proof Theory, NorthHolland, Amsterdam, 1970, 73~ 78. [306] A. Maruoka, M. Kimura. Conditions for injectivity of global maps for tessellation automata, Information and Control 32 (1976), 158~162. [307] A. Maruoka, M. Kimura. Injectivity and surjectivity of parallel maps for cellular automata, J. Comput. Syst. Sci. 18 (1979), 47~ 64. [308] H. Marxen, J. Buntrock. Attacking the busy beaver 5, EATCS Bull. 40 (1990), 247~251. [309] Yu. V. Matiyasevich. Hilbert's Tenth Problem, MIT Press, Cambridge, MA, 1993. [310] M. Mendes-France. Nombres normaux. Applications aux fonctions pseudo~aleatoires, 1. Anal. Math. Jerusalem 20 (1967), 1~ 56. [311] M. Mendes-France. Suites de nombres au hasard (d'apres Knuth), Seminaire de Theorie des N ombres, Expose 6, 1974, 1~ 11. [312] M. Mendes-France. The Planck constant of a curve, in J. Belair, S. Dubuc (eds.). Fractal Geometry and Analysis, Kluwer Academic, Boston, 1991, 325~366. [313] A. R. Meyer. Program size in restricted programming languages, Inform. and Control 21 (1972), 322~394.
Bibliography
443
[314] G. L. Miller. Riemann's hypothesis and tests ofprimality, 1. Comput. Syst. Sci. 13 (1976), 300-317. [315] C. Moore. Generalized shifts: unpredictability and undecidability in dynamical systems, Nonlinearity 4 (1991), 199-230. [316] E. F. Moore. Machine models of self reproduction, Proc. Symp. Appl. Math., American Mathematical Society, 14 (1962), 17-33. [317] A. Muchnik, A. Semenov, , V. A. Uspensky. Mathematical metaphysics of randomness, Theoret. Comput. Sci. 207 (1998), 263317. [318] I. P. Natanson. Theory of Functions of A Real Variable, Frederick Ungar, New York, 1955. [319] M. Nivat. Infinite words, infinite trees, infinite computations, in J. W. De Bakker, J. van Leeuwen (eds.). Foundations of Computer Science III, Mathematical Centre 'lracts 109, Amsterdam, 1979, 3-52. [320] I. Niven, H. S. Zuckerman. On the definition of normal numbers, Pacific J. Math. 1 (1951), 103-110. [321] P. Odifreddi. Classical Recursion Theory, Vol. 1, North-Holland, Amsterdam, 1989. [322] P. Odifreddi. Classical Recursion Theory, Vol. 2, North-Holland, Amsterdam, 1999. [323] P. Odifreddi. La prova di dio, Manuscript, January 1994, 8pp. [324] P. Odifreddi. Ultrafilters, dictators, and Gods, in C. S. Calude, G. Paun (eds.). Finite Versus Infinite. Contributions to an Eternal Dilemma, Springer-Verlag, London, 2000, 255-262. [325] V. P. Orevkov. A new proof of the uniqueness theorem for constructive differentiable functions of a complex variable, Zap. Nauchn. Sem. LOMI40 (1974), 119-126 (in Russian); English translation in 1. Sov. Math. 8 (1977), 329-334. [326] J. C. Oxtoby. Measure and Category, Springer-Verlag, Berlin, 1971.
444
Bibliography
[327] J. C. Oxtoby, S. M. Ulam. Measure-preserving homeomorphisms and metrical transitivity, Ann. Math. 42 (1941), 874-925. [328] H. R. Pagels. The Dreams of Reason, Bantam Books, New York, 1989. [329] J. A. Paulos. Beyond Numeracy, Vintage Books, Random House, New York, 1992. [330] 1. Percival. Chaos: a science for the real world, in N. Hall (ed.). New Scientist Guide to Chaos, Penguin Books, London, 1991, 11-21. [331] R. Penrose. The Emperor's New Mind. Concerning Computers, Minds, and the Laws of Physics, Oxford University Press, Oxford, 1989. [332] R. Penrose. Precis of The Emperor's New Mind. Concerning Computers, Minds, and the Laws of Physics (together with responses by critics and a reply by the author), Behav. Brain Sci. 13 (1990), 643-705. [333] R. Penrose. Computability and the Mind, 1993 Forder Lecture, Auckland University, 30 April 1993. [334] 1. Peterson. Islands of Truth: A Mathematical Mystery Cruise, W. H. Freeman, New York, 1990. [335] S. Porrot, M. Dauchet, B. Durand, N. K. Vereshchagin. Deterministic rational transducers and random sequences, Theoret. Comput. Sci. 378 (1998), 258-272. [336] E. L. Post. Recursively enumerable sets of positive integers and their decision problems, Bull. Amer. Math. Soc. (New Series) 50 (1944), 284-316. [337] E. L. Post. Absolutely unsolvable problems and relatively undecidable propositions: account of an anticipation, in M. Davis (ed.). The Undecidable, Raven Press, New York, 1965, 340-433. [338] M. Pour-El, 1. Richards. Computability in Analysis and Physics, Springer-Verlag, Berlin, 1989. [339] V. Pratt. Every prime has a succinct certificate, SIAM J. Comput. 4 (1975), 214-220.
Bibliography
445
[340] P. Raatikainen. On interpreting Chaitin's incompleteness theorem, J. Philos. Logic 27 (1998), 569-586. [341] M. O. Rabin. Probabilistic algorithms, in J. F. Traub (ed.). Algorithms and Complexity: New Directions and Recent Results, Academic Press, New York, 1976, 21-39. [342] D. L. Renfro. A Study of Porous and Sigma-Porous Sets, Longman, 2002, to appear. [343] E. Regis. Who Got Einstein's Office? Eccentricity and Genius at the Institute for Advanced Study, Penguin Books, New York, 1989. [344] R. Rettinger, X. Zheng, R. Gengler, B. von Braunmiihl. Monotonically computable real numbers, in C. S. Calude, M. J. Dinneen, S. Sburlan (eds.). Combinatorics, Computability and Logic, Proc. DMTCS'Ol, Springer-Verlag, London, 2001, 187-202. [345] H. Rice. Recursive reals, Proc. Amer. Math. Soc. 5 (1954), 784791. [346] B. Riemann. Uber die Anzahl der Primzahlen unter einer gegebenen Grosse, in Gesammelte mathematische Werke und wissenchajtlicher N achlass, Springer-Ver lag, Berlin, 1990, 177185. [347] H. Rogers. Theory of Recursive Functions and Effective Computability, McGraw-Hill, New York, 1967. [348] G. Rozenberg, A. Salomaa. Cornerstones of Undecidability, Prentice-Hall, Englewood Cliffs, NJ, 1994. [349] R. Rucker. Infinity and the Mind, Bantam Books, New York, 1983. [350] R. Rucker. Mind Tools, Houghton Mifflin, Boston, 1987. [351] D. Ruelle. Chance and Chaos, Princeton University Press, Princeton, NJ, 1991. [352] B. Russell. Mathematical logic as based on the theory of types, Amer. J. Math. 30 (1908), 222. (Reprinted in: [410]' 153.)
446
Bibliography
[353] G. E. Sacks. A simple set which is not effectively simple, Proc. Amer. Math. Soc. 15 (1964), 51-55. [354] A. M. Salagean-Mandache. A geometrical proof of Kraft-Chaitin theorem, An. Univ. Bucure§ti Mat. 39/40 (1990/91), 3 Matematica-Informatica, 90-97. [355] A. Salomaa. Computation and Automata, Cambridge University Press, Cambridge, 1985. [356] A. Salomaa. Public-Key Cryptography, Springer-Verlag, Berlin, 1990 (2nd ed, 1996). [357] R. Schack. Algorithmic information and simplicity in statistical physics, Int. 1. Theor. Physics, 36 (1997) 209-226. [358] J. Schmidhuber. Algorithmic theories of everything, Los Alamos preprint archive http://arXi v: quant-ph/OOl1122, 30 November 2000. [359] C. P. Schnorr. Zujiilligkeit und Wahrscheinlichkeit: Eine algorithmische B ehandlung der Wahrscheinlichkeitstheorie, Lecture Notes Math. 218, Springer-Verlag, Berlin, 1971. [360] C. P. Schnorr. Process complexity and effective random tests, J. Comput. Syst. Sci. 7 (1973), 376-388. [361] C. P. Schnorr. A survey of the theory of random sequences, in R. E. Butts, J. Hintikka (eds.). Basic Problems in Methodology and Linguistics, Reidel, Dordrecht, 1977, 193-210. [362] D. Scotus. Philosophical Writings, Nelson, New York, 1962. [363] G. Segre. The definition of a random sequence of qubits: from noncommutative algorithmic probability theory to quantum algorithmic information theory and back, Los Alamos preprint archive http://arXiv : quant-ph/0009009 v3, 7 November 2000. [364] C. E. Shannon. A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948), 379-423, 623-656. [365] A. Shen. A strange application of Kolmogorov complexity, Manuscript, 1993, 4pp.
Bibliography
447
[366] H. Siegelmann. Computation beyond the Turing limit, Science 268 (1995), 545-548. [367] M. Sipser. A complexity-theoretic approach to randomness, Proc. 15th Annual ACM Symp. Theory of Computing, 1983, 330-335. [368] M. Sipser. Introduction to the Theory of Computation, PWS Publishing, Boston, 1997. [369] T. A. Slaman. Random implies D-like, Manuscript, 14 December 1998,2pp. [370] R. M. Smullyan. Effectively simple sets, Proc. Amer. Math. Soc. 15 (1964), 893-895. [371] R. I. Soare. Recursion theory and Dedekind cuts, Trans. Amer. Math. Soc. 140 (1969), 271-294. [372] R. I. Soare. Recursively Enumerable Sets and Degrees. A Study of Computable Functions and Computably Generated Sets, SpringerVerlag, Berlin, 1987. [373] R. J. Solomonoff. A formal theory of inductive inference, Part 1 and Part 2, Inform. and Control 7 (1964), 1-22, 224-254. [374] R. J. Solomonoff. Complexity-based induction systems: comparisons and convergence theorems, IEEE Trans. Inf. Theory 24 (1978),422-432. [375] R. M. Solovay. Draft of a paper (or series of papers) on Chaitin's work ... done for the most part during the period of Sept. Dec. 1974, Manuscript, IBM Thomas J. Watson Research Center, Yorktown Heights, New York, May 1975, 215pp. [376] R. M. Solovay. On random r.e. sets, in A. I. Arruda, N. C. A. Da Costa, R. Chuaqui (eds.). Non-Classical Logics, Model Theory and Computability, North-Holland, Amsterdam, 1977, 283-307. [377] R. M. Solovay. A version of D for which Z FC can not predict a single bit, in C. S. Calude, G. Paun (eds.). Finite Versus Infinite. Contributions to an Eternal Dilemma, Springer-Verlag, London, 2000, 323-334. [378] R. Solovay, V. Strassen. A fast Monte Carlo test for primality, SIAM J. Comput. 6 (1977), 84-85. Erratum: 7 (1978), 118.
448
Bibliograpby
[379] E. Specker. Nicht konstruktiv beweisbare Satze der Analysis, J. Symb. Logic 14 (1949), 145-158. [380] 1. Stewart. The Problems of Mathematics, Oxford University Press, Oxford, New York, 1992. [381] 1. Stewart. Deciding the undecidable, Nature 352 (1991), 664-665. [382] L. Staiger. Representable Martin-Lof tests, (1985), 235-243.
Kybernetika 21
[383] L. Staiger. Kolmogorov complexity and Hausdorff dimension, Inform. and Comput. 103 (1993), 159-194. [384] L. Staiger. w-languages, in G. Rozenberg, A. Salomaa (eds.). Handbook of Formal Languages, Vol. III, Springer-Verlag, Berlin, 1997 339-387. [385] L. Staiger. A tight upper bound on Kolmogorov complexity by Hausdorff dimension and uniformly optimal prediction, Theory Comput. Syst. 31 (1998), 215-229. [386] L. Staiger. The Kolmogorov complexity of real numbers, in G. Ciobanu, Gh. Paun (eds.). Proc. Fundamentals of Computation Theory, Lecture Notes Comput. Sci. 1684, Springer-Verlag, Berlin, 1999, 536-546. [387] L. Staiger. The Kolmogorov complexity of Liouville numbers, CDMTCS Research Report 096, 1999, llpp. [388] L. Staiger. How large is the set of disjunctive sequences? J. UCS 8 (2002), 348-362. [389] D.
~tefanescu.
Scotus, E-mail to C. Calude, 12 May 1993.
[390] K. Svozil. The quantum coin toss-testing microphysical undecidability, Phys. Lett. A143 (1990), 433-437. [391] K. Svozil. Rando~ness f3 Undecidability in Physics, World Scientific, Singapore, 1993. [392] K. Svozil. E-mail to C. Calude, 14 June 1993. [393] K. Svozil. Halting probability amplitude of quantum computers, J. UCS 1 (1995), 201-203.
Bibliography
449
[394] K. Svozil. Quantum information theory, J. UCS 5 (1996), 311346. [395] K. Svozil. The Church-Turing thesis as a guiding principle for physics, in C. S. Calude, J. Casti, M. J. Dinneen (eds.). Unconventional Models of Computation, Springer-Verlag, Singapore, 1998, 371-385. [396] A. Szilard. Private communication to C. Calude, 10 November 1993. [397] K. Tadaki. A generalization of Chaitin's halting probability 0 and halting self-similar sets, Hokkaido Math. J. 31 (2002), 219-253. [398] F. J. Tipler. The Omega point as Eschaton: answers to Pannenberg's questions for scientists, Zygon 24 (1989), 241-242. [399] M. R. Titchener. Construction and properties of the augmented and binary-depletion codes, lEE Pmc. 132 (1984), 163-169. [400] T. Toffoli. Physics and computation, Int. 1. Theoret. Physics 21 (1982), 165-175. [401] T. Toffoli, N. Margolus. Invertible cellular automata: a review, Physica D 45 (1990), 229-253. [402] J. F. Traub, G. W. Wasilkowski, H. Wozniakowski. InformationBased Complexity, Academic Press, New York, 1988. [403] J. F. Traub, A. G. Werschulz. Complexity and Information, Cambridge University Press, Cambridge, 1998. [404] A. M. Turing. On computable numbers with an application to the Entscheidungsproblem, Pmc. Amer. Math. Soc. 42 (1936-7), 230-265; a correction, ibid. 43 (1937), 544-546. [405] T. Tymoczko. The four-colour problem and its philosophical significance, J. Philos. 2 (1979), 57-83. [406] T. Tymoczko. New Directions in the Philosophy of Mathematics, Birkhiiuser, Boston, 1986 (2nd ed, 1998). [407] V. A. Uspensky. Complexity and entropy: an introduction to the theory of Kolmogorov complexity, in [428]' 86-102.
450
Bibliography
[408] V. A. Uspensky. Kolmogorov and mathematical logic, J. Symb. Logic 57 (1992), 385-412. [409] V. A. Uspensky, A. Shen. Relations between varieties of Kolmogorov complexities, Math. Syst. Theory 29 (1996), 271-292. [410] J. van Heijenoort (ed.). From Frege to Cadel. A Source Book in Mathematical Logic, 1879-1931, Harvard University Press, Cambridge, MA, 1967. [411] M. van Lambalgen. Von Mises' definition of random sequences reconsidered, J. Symb. Logic 52 (1987), 725-755. [412] M. van Lambalgen. Algorithmic information theory, J. Symb. Logic 54 (1989), 1389-1400. [413] M. van Lambalgen. The axiomatization of randomness, J. Symb. L09ic 55 (1990), 1143-1167. [414] A. van der Poorten. Notes on Fermat's Last Theorem Wiley Interscience, New York, 1996. [415] N. K. Vereshchagin. Kolmogorov Complexity, Universitat Wiirzburg, 1998, 116pp, http://www-info4 . informatik. uniwuerzburg.de/veranstalt/. [416] N. K. Vereshchagin. An enumerable undecidable set with low prex complexity: a simplied proof, http://lpcs.math.msu. ru/-ver/papers/calude.ps. [417] B. Vidakovic. Algorithmic complexity, universal priors and Ockham's Razor, Resenhas do Instituto de Matematica e Estatistica da Universidade de Sao Paolo 3, 4 (1998), 359-390. [418] J. Ville. Etude critique de la notion de collectij, Gauthier-Villars, Paris, 1939. [419] P. M. Vitanyi. Quantum Kolmogorov complexity based on classical descriptions, IEEE Trans. Inf. Theory 47, 6 (2001), 24642479. [420] S. B. Volchan. The algorithmic theory of randomness, Amer. Math. Monthly 1 (2002), 46-63.
Bibliography
451
[421] R. von Mises. Probability, Statistics and Truth, G. Allen and Unwin, London; Macmillan, New York (2nd revised English edition prepared by Hilda Geiringer), 1961. [422] R. von Mises. Mathematical Theory of Probability and Statistics, edited and complemented by Hilda Geiringer, Academic Press, New York, 1974. [423] J. von Neumann. The Computer and the Brain, Silliman Lectures Series, Yale University Press, New Haven, CT, 1958. [424] J. von Neumann. Theory of Self-Reproducing Automata, edited and complemented by A. W. Burks, University of Illinois Press, Urbana, 1966. [425] V. V. V'yugin. Ergodic theorems for individual random sequences, Theoret. Comput. Sci. 207 (1998), 343-361. [426] K. Wagner, G. Wechsung. Computational Complexity, D. Reidel, Dordrecht, 1986. [427] A. Wald. Die Widerspruchsfreiheit des Kollectivbegriffes, Ergeb. math. Kolloq. 8 (1937), 38-72. [428] O. Watanabe (ed.). Kolmogorov Complexity and Computational Complexity, Springer-Verlag, Berlin, 1992. [429] K. Weihrauch. Computability, Springer-Verlag, Berlin, 1987. [430] K. Weihrauch. The degrees of discontinuity of some translators between representations of the real numbers, Inf.-Ber. 129, Fern Universitat Hagen, 1992. [431] K. Weihrauch. Computable Analysis, Springer-Verlag, Berlin, 2000. [432] K. Weihrauch, N. Zhong. Is the linear Schrodinger propagator 'lUring computable?, in J. Blank, V. Brattka, P. Hertling (eds.). Computability and Complexity in Analysis, Lecture Notes Comput. Sci. 2064, Springer-Verlag, Heidelberg, 2000, 248-260. [433] H. S. White. Algorithmic complexity of points in dynamical systems, Ergodic Theory Dyn. Syst. 13 (1993), 807-830.
452
Bibliography
[434] C. P. Williams, S. H. Clearwater. Ultimate Zero and One, Copernicus, New York, 2000. [435] D. G. Willis. Computational complexity and probability constructions, J. Assoc. Comput. Mach. 17 (1970), 241-259. [436] L. Wittgenstein. Selections from "Remarks on the Foundations of Mathematics" , in P. Benacerref, H. Putnam (eds.). Philosophy of Mathematics: Selected Readings, Prentice-Hall, Englewood Cliffs, NJ, 1964,421-480. [437] S. Wolfram. Universality and complexity in cellular automata, Physica D 10 (1984), 1-35. [438] S. Wolfram. Origins of randomness in physical systems, Physical Rev. Lett. 55 (1985), 298-301. [439] S. Wolfram. A New Kind of Science, Wolfram Media, 2002. [440] D. Wood. Theory of Computation, Harper & Row, New York, 1987. [441] J. B. Wright, E. G. Wagner, J. W. Thatcher. A uniform approach to inductive posets and inductive closure, Theoret. Comput. Sci. 7 (1978), 57-77. [442] G. Wu. Prefix-free languages and initial segments of computably enumerable degrees, in J. Wang (ed.). COCOON 2001, Lecture Notes Comput. Sci. 2108, Springer-Verlag, Heidelberg, 2001, 576585. [443] http://www.informatik.unigiessen.de/staff/richstein/ ca/Goldbach.html.
[444] http://www.hipilib.de/zeta. [445] E. H. Yang, S. Y. Shen. Chaitin complexity, Shannon-information content of a single event, and infinite random sequences 1, Sci. China Ser. A 34, 10 (1991), 1183-1193. [446] E. H. Yang, S. Y. Shen. Chaitin complexity, Shannon-information content of a single event, and infinite random sequences 2, Sci. China Ser. A 34, 11 (1991), 1307-1319.
Bibliography
453
[447] S. Yi-Ting. A "natural" enumeration of non~negative rational numbers: an informal discussion, Amer. Math. Monthly 87 (1980), 25~29. [448] T. Zamfirescu. Porosity in convexity, Real Anal. Exch. 15 (1989/90), 424~436. [449] D. Zeilberger. Theorems for a price: tomorrow's semi-rigorous mathematical culture, Notices Amer. Math. Soc. 40 (1993), 978~ 981. [450] X. Zheng. Closure properties of real number classes under limits and computable operators, in D. Z. Du (eds.). Proc. COCOON 2000, Lecture Notes Comput. Sci. 1858, Springer-Verlag, Heidelberg, 2000, 170~ 179. [451] M. Zimand. On the topological size of random strings, Z. Math. Logik Grundlag. Math. 32 (1986), 81~88. [452] M. Zimand. Positive Relativizations and Baire Classification, Ph.D. Thesis, Bucharest University, Romania, 1991. [453] M. Zimand. If not empty, N P \ P is topologically large, Theoret. Comput. Sci. 119 (1993), 293~310. [454] W. H. Zurek. Thermodynamic cost of computation, algorithmic complexity and the information metric, Nature 341 (1989), 119~ 124. [455] A. Zvonkin, L. A. Levin. The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms, Usp. Mat. Nauk 156 (1970), 85~127.
Notation Index N,l
x,3
N+,l
dom(cp) , 3
Q,l
° (1),3
R,l
1::;g+0(1),3
R+,l
graph(cp), 3
I, 1
range(cp) , 3
lCYJ, 1 iCY l, 1
xAW, 3
log = llog2J, 1
string(n),l Ixl,l logQ' 1
