Adaptive Processes in Economic Systems (Mathematics in science and engineering series;no.20)

ADAPTIVE PROCESSES IN ECONOMIC SYSTEMS

Roy E. Murphy, Jr. DEPARTMENT OF ECONOMICS STANFORD UNIVERSITY STANFORD, CALIFORNIA

1965

~

ACADEMIC PRESS

New York and London

COPYRIGHT

© 1965,

BY ACADEMIC PRESS INC.

ALL RIGHTS RESERVED. NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.

ACADEMIC PRESS INC.

111 Fifth Avenue, New York, New York 10003

United Kingdom Edition published by ACADEMIC PRESS INC. (LONDON) LTD.

Berkeley Square House, London W.1

LrBRARY OF CONGRESS CATALOG CARD NUMBER:

PRINTED IN THE UNITED STATES OF AMERICA

65-25003

To the Late Russell Varian, Inventor, Scientist, and Humanitarian

This page intentionally left blank

ADAPTIVE PROCESSES IN ECONOMIC SYSTEMS

Roy E. Murphy, Jr. DEPARTMENT OF ECONOMICS STANFORD UNIVERSITY STANFORD, CALIFORNIA

1965

ACADEMIC PRESS

New York and London

COPYRIGHT

© 1965,

BY ACADEMIC PRESS INC.

ALL RIGHTS RESERVED. NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.

ACADEMIC PRESS INC.

111 Fifth Avenue, New York, New York 10003

United Kingdom Edition published by ACADEMIC PRESS INC. (LONDON) LTD.

Berkeley Square House, London W.l

LIBRARY OF CONGRESS CATALOG CARD NUMBER:

PRINTED IN THE UNITED STATES OF AMERICA

65-25003

To the Late Russell Varian, Inventor, Scientist, and Humanitarian

This page intentionally left blank

PREFACE During my study of economic theory I have been frequently disturbed by what I consider serious deficiencies in economic theory. These deficiencies are as follows: (a) In current economic theory, decision makers are generally assumed to possess full information or full knowledge of the parameters of the economic system in which they play an active role. However, in fact, few decision makers ever have full knowledge and still they make decisions that carry economic significance. (b) The nature of the adaptive economic process, where decision makers increase their knowledge by the cumulative experience of "doing while learning," has not been explored in any generality. (c) The theory of communications although implicit in economic theory does not play an explicit role. Actually the transmission of information is both time consuming and subject to 'equivocation. (d) The analogy between the activities of an economic market undergoing stochastic exchanges of goods and services and the stochastic collision of gas molecules in a constant energy system have never been exploited to any great degree. This analogy could permit the application of a large body of statistical thermodynamic theory to many stochastic economic problems. The assumption that decision makers possess full information about the parameters of an economic system is valid when the economist wishes to study the basic structure of an economic system. The basic structure must not be obscured by supposed irrational decisions from decision makers with less than full information. The assumption that decision makers possess full information is generally not valid when economic theory is used to formulate economic policy. In most actual situations the decision makers have only a vague idea about the system's parameters. However, because of the adaptive process, there is some hope that the decision makers will improve their knowledge of the IX

x

PREFACE

system's parameters. Three basic questions arise in the study of adaptive behavior: (1) Under what conditions does the adaptive process always improve the behavior of the decision makers? (2) What controls the rate at which the expected improvements in behavior occur ? (3) Can the adaptive process explain the diversity of observed behavior of supposed rational decision makers without appeal to the existence of individual utility functions? Restricted answers to these questions will be found in this book: Recently, a substantial amount of work has been accomplished in the application of dynamic programming techniques to statistical communications problems and self-adaptive control systems. This effort led to my recognition that this approach has important economic significance as well. In fact, the deficiencies, listed as (a)-(c), were found to be greatly reduced by this approach. Furthermore, I found that this approach actually strengthened the analogy between stochastic economic processes and thermodynamic processes, deficiency (d). Although I have demonstrated the usefulness of communications theory, self-adaptive control theory, and thermodynamic theory to certain economic processes herein, I cannot say that I have presented the complete story. Since time and space are finite, the reader will find only a beginning to the solution of these problems here. I hope, however, he will be induced to consider imaginative and useful applications and extensions of this theoretical approach. At this early stage in the development of an adaptive economic theory there is insufficient conceptual stability to warrant a rigorous mathematical treatment. Although I have been forced to employ mathematical arguments to logically extend a simple intuitive view of adaptive processes, I do not, as an engineer and economist, pretend to advance a rigorous, axiomatic theory of these processes. This task I leave to those who are better equipped in the arts of mathematics. My apology is made at the onset for the difficult notational problems which will soon confront the reader. As far as I know, a flexible and reasonable notational system for adaptive processes has not yet come into being, although progress is being made by many workers. This research would not have been possible without the vision and support of the late Dr. Russell Varian, who also searched for some useful analogy between current economic theory and the theories of electrical

PREFACE

Xl

engineering and physics. * Furthermore, without the continuing support of the Varian Foundation-in particular, Mrs. Russell Varian, Dr.Edward Ginzton (Chairman of the Board, Varian Associates), and Professor Leonard Schiff (Physics Department, Stanford University)-this work would have stopped in midstream. To these people I express my respectful gratitude. I have also received much help and encouragement from Professor Kenneth J. Arrow (Economics Department, Stanford University), who initially drew my interest to adaptive economic problems, and from Professor Norman Abramson (Electrical Engineering Department, Stanford University) whose interest in information theory extends far beyond its use in electrical engineering problems. My appreciation is extended to Dr. Richard Bellman (RAND Corporation) whose interest in my work provided the opportunity to write this book. I wish also to acknowledge the comments and suggestions of Dr. Charles W. McClelland (Operations Research, Varian Associates) and Mr. Allen H. Norris (Head, Applied Mathematics, Varian Associates). I especially wish to acknowledge the influence on my thinking of an imaginative paper written by J. L. Kelly, J r./ who first drew the connection between economic investment problems and information theory. My thanks are extended to Laura Staggers, who struggled through my obscure notes to produce a useful manuscript, and to the staff of Academic Press whose patience and understanding has made the task less of a traumatic experience. Finally, I express my gratitude to my wife, Joyce, who philosophically accepted the presence of piles of books and notes in our home during the writing of this book and who shared my many moments of triumph and despair. Roy E. MURPHY, JR. Stanford, California August, 1965

* For an interesting example, Dr. Varian was intrigued by the analogy of the principles of the bistable electronic circuit and the business cycle in the early 1930's. This theory thus actually preceded the work of Nicholas Ka1dor who, in 1940, introduced a mathematical model of the business cycle which, unknown to Kaldor, was similar to the bistable circuit theory. See: N. Ka1dor, A Model of the Trade Cycle, Economic J. 50, No. 197, 78-92 (1940). t J. L. Kelly, j-; A New Interpretation of Information Rate, Bell System Tech. J. 35, No.4, 917-926 (1956).

This page intentionally left blank

Contents PREFACE

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

••

IX

CHAPTER 1

Introduction 1.1 The Historical and Current Development of Adaptive Theory 1.2 Common Properties of Adaptive Processes References. . . . . . . . . . . . . . . . . . . . . . .

I 8

9

CHAPTER 2

The Mathematical Model 2.1 Introduction. . . . . . . . . . 2.2 2.3

Discrete Processes. . . . . . . . Some Discrete Sequential Processes 2.4 The Role of the Decision Maker . 2.5 The Optimal Type 2 (Adaptive) Process 2.6 The Constrained, Optimal Type 2 (Adaptive) Process 2.7 Causality and Markov Processes 2.8 The Dynamic Programming Model References. . . . . . . . . . .

12 12 16

20 22

24 25 28

30

CHAPTER 3

The Primitive Adaptive Process 3.1 The Notion of Learning and Adaptation 3.2 Some Hypothetical Experiments . . . . 3.3 An Adaptive Process of the First Kind 3.4 An Adaptive Process of the Second Kind 3.5 Mixed Adaptive Processes of the First and Second Kind 3.6 Summary . References. . . . . . . . . . . xiii

32 33 36 39

42 47 51

XIV

CONTENTS

CHAPTER 4

Subjective Probability 4.1 Introduction . . . . . . . . . 4.2 A Heuristic Example-The Three Investors 4.3 Economic Environmental Processes . 4.4 A Digression on Statistical Estimators . . . 4.5 Multinomial Subjective Probabilities 4.6 Properties of the Subjective Probability Vector References. . . . . . . . . . . . . . . .

52 53 60 63

65 68

69

CHAPTER 5

The Role of Entropy in Economic Processes 5.1 5.2 5.3

The Concept of Entropy Time Entropy and Information. The Entropy Paradox

71 73 82

References. • . . . • .

85

CHAPTER 6

Adaptive Economic Processes 6.1 Introduction. . . . . . . . . . . . . . . . . 6.2 General Deterministic Dynamic Economic Process 6.3 The Stochastic Dynamic Economic Process . . . . 6.4 The Adaptive Stochastic Dynamic Economic Process 6.5 Stochastic Growth Process .

86 87 88

89 91

CHAPTER 7

An Adaptive Investment Model 7.1 7.2 7.3 7.4 7.5 7.6

The Objective Function . An Example-The Three Investors Again An Investment Model with Full Liquidity An Adaptive Investment Process with Full Liquidity Consideration of the Special Constraint . . . . . . The Multivalued Payoff Adaptive Investment Process References. . . . . . . . . . . . . . . . . . .

95

96 104 109 121 125 128

xv

CONTENTS

CHAPTER 8

Multiactivity Capital Allocation Processes 8.1 8.2 8.3

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . The Adaptive Capital Allocation Process. . . . . . . . • . . . . . The Properties of the Adaptive Capital Allocation Process in the Limit .

129 130 139

References. . . . . . . . . . . . . . . . . . . . . . . . . . .

142

CHAPTER 9

Economic State Space 9.1 9.2 9.3 9.4 9.5 9.6 9.7

Introduction. . . . . . . . . . . . . The Concept of an Economic State Space Statistical Equilibrium and Enlightenment The Subjective Entropy Trajectory . . . The Dynamics of the Subjective Entropy Trajectory. The Economic State Space Representation . . . . . A Digression-A Likely Value for the Environmental Entropy References. . . . . . . . . . . . . . . . . . . . . . .

~

143 144 146 149 152 155 162 168

CHAPTER 10

Interactions between Decision Makers in State Space 10.1 10.2 10.3 10.4

Introduction . The State Space Probability Function Stochastic Equilibrium in the Market. Interperson Trading in the Investment Market

169 170

177 183

CHAPTER 11

The Conclusions 11.1 11.2

Individual Adaptive Behavior Collective Adaptive Behavior

189 193

Bibliography 1. General Background on Adaptive and Evolutionary Behavior 2. Sources on the Concept of Subjective Probability. . 3. Sources on the Mathematical Theory of Intelligence . . . .

197 197 198

CONTENTS

XVI

4. 5. 6. 7. 8. 9.

199

Dynamic Programming and Adaptive Control . . . . . . . . . . Sources on Investment Theory; mainly on Multi-Project Investment under Uncertainty. . . . . . . . . . . . . . . . . Sources on Information Theory . . . . . . . . . . . . Sources on the Concept of Entropy and Entropy-Gradient Sources on Statistical Mechanics and Thermodynamics Sources on Certain Techniques Used in This Book. . . .

SUBJECT INDEX

•

.

.

•

•

•

.

.

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

200

201 202 202 202

•

.

•

•

•

205

CHAPTER 1

INTRODUCTION For of beasts, some are gregarious, others are solitary; they live in the way which is best adapted to sustain them ... and their habits are determined for them by nature in such a manner that they may obtain with greater facility the food of their choice. Aristotle Politics

1.1 The Historical and Current Development of Adaptive Theory Economic science can be broadly defined as the study of mechanisms which allocate limited resources to productive uses so as to improve the welfare of the members of a society. We shall call such a mechanism an economic process, and such a society an economic system. A great deal of interest centers around the study of dynamic economic processes. Generally, the effects of the rate of change resources, particularly capital resources, are considered to be the essence of a dynamic economic process. It is recognized that a stream of actions taken by a number of decision makers in the economic system guides the rate of change of the system's resources. However, it is not clear what controls the rate of the stream of actions taken by the decision makers. In this book we shall consider the hypothesis that the rate of generation of information in the economic system controls the rate of decision making, and thus indirectly the rate of growth of the system's resources. Processes which are governed by the flow of information are often called adaptive processes. The study of adaptive processes in both social and biological evolution is very old, stemming from the Greeks who used evolutionary principles in their search for truth. This adaptive process, which they called the dialectic, was applied later to social systems during the rise of the Hegelian concept of history. The Hegelian historical dialectic contains the essential components that lead to an adaptive decision-making system. We shall consider the dialectic nature of social systems to identify these essential components, and we shall 1

2

1.

INTRODUCTION

note the key component especially for adaptive decision processes, i.e., the role of history in the formulation of the current decisions. Since Hegel's period it has been recognized by many that social dynamics is an adaptive phenomenon in which constant experimental adjustments are being made, shifting the social structure to best suit the economic and physical restraints in which the society finds itself. These biological concepts of social dynamics are basic not only to the Hegelian dialectic but to the philosophies of the founders of the American concept of federal government, Jefferson and Madison, and still later, to the philosophy of Engels. Hegel replaced a concept of a society in which history was simply a unique record of the past with a concept of society in which history was the engine of progress. In the Hegelian world, History's role is to produce an array of current problems to be resolved by society. Society, remembering the past triumphs and failures, adapts to the conditions of the present, and produces the decisions which may lead to future triumphs and failures. Engels grasped the underlying stochastic nature of the adaptive decision process when he wrote: In nature . . . there are only blind unconscious agencies acting upon one another and out of whose interplay the general law comes into operation. . . . In the history of society, on the other hand, the actors are all endowed with consciousness, are men acting with deliberation. . . . That which is willed happens but rarely; in the majority of instances the numerous desired ends cross and conflict with one another. ... Thus the conflict of innumerable individual wills and individual actions in the domain of history produces a state of affairs entirely analogous to that in the realm of unconscious nature . . . . Historical events thus appear on the whole to be likewise governed by chance. But where on the surface accident holds sway, there actually it is always governed by inner, hidden laws and it is only a matter of discovering these laws.1 The dialectic process illustrates certain fundamental aspects of adaptive processes which will aid our study of this phenomenon. First, there is a set of alternative views about the problems of the present. Second, there is some objective which is to be optimized by the selection of a subset of possible views. The crucial aspect that makes a stochastic decision process an adaptive stochastic decision process is the role of historical information or experience in the formulation of this

1.1

ADAPTIVE THEORY, HISTORICAL AND CURRENT DEVELOPMENT

3

optimum current decision. Free discussion and information about past experiences are necessary to evaluate the possible effect of current actions on the objective and to discover the limits on the set of possible actions. Third, a choice mechanism is necessary to determine which subset of views is optimal to society. Fourth, a police system is necessary to impose the optimal choice on the holders of other subsets of ideas. The society then experiences the consequences of its decision, and formulates the dialectic process all over again. The crucial link is the choice mechanism. An essential concept of adaptation is the irreversibility of the process. Once a choice is made, it becomes recorded as part of the past's unalterable history. Its effect, good or bad, is always felt in the making of current decisions. Most political thought centers about the qualities of one choice mechanism over the other. In the view of Engels, the choice mechanism was an irreversible social revolution. In an attempt to reduce the dangers of social rigidity without the associated dangers of revolution, the original principles of American federal government included a choice mechanism in the indirect republican representation system. Madison, in the Federalist Paper No. 10, outlines the necessity of providing social mobility in U.S. institutions by a choice mechanism among social "factions." He said:

As long as the reason of man continues fallible, and he is at liberty to exercise it, different opinions will be formed." . .. The regulation of these various and interfering interests forms the principal task of modern legislation, and involves the spirit of party and faction in the necessary and ordinary operations of the gooernments ... If a faction consists of less than a majority, relief is supplied by the republican principle, which enables the majority to defeat [the faction's] sinister views by regular vote. 4 Thus Madison outlined the decision process of American government and provided for the irreversible feature of sequential voting which is characteristic of adaptive processes. de Tocqueville pointed out the irreversible nature of American government when he said:

The American democracy ... is allowed to follow, in the formulation of laws, the natural instability of its desires.t ... In America, as long as the majority is still undecided, discussion is carried on; but as soon as its decision is irrevocably pronounced, everyone is silent, and the friends as well as the opponents of the measure unite in assenting to its propriety." ... The very essence

4

1.

INTRODUCTION

of democratic government consists in the absolute sovereignty of the majority. . . . Most of the American constitutions have sought to increase this natural strength of the majority of artificial means,'

One aspect of the general decision process in government and one which we shall deal with herein is the role of the expectations of the future in the evaluation of the set of possible alternatives of the present. Until recently, this aspect has been neglected in consideration of governmental functions as an aid to the nation's decision makers. Lippmann, in his book "The Public Philosophy," brought this out when he said: For besides the happiness and security of the individuals of whom a community is at any moment composed, there are also the happiness and security of the individuals of whom generation after generation it will be composed. If we think of it in terms of individual persons, the corporate body of the People is for the most part invisible and inaudible.i

The biological aspect of adaptation which emerged as the study of evolution was introduced by Malthus" in 1789; but it was Darwin who clearly saw the mechanism by which adaptation takes place. In the closing chapter of "The Origin of Species," Darwin said: As natural selection acts solely by accumulating slight, successive, favourable variations, it can produce no great or sudden modifications; it can act only by short and slow steps.I° ... [Modification of the species] has been effected chiefly through the natural selection of numerous successive, slight, favourable variations, aided in an important manner by the inherited effects of the use and disuse of parts; and in an unimportant manner, that is in relation to adaptive structures, whether past or present, by the direct action of external conditions, and by variations which seem to us in our ignorance to arise spontaneously.u

It is clear that Darwin recognized the possibility of "peaceful" transformations in processes where natural conditions permitted the full play of the biological dialectic. He recognized the need for a selection from a set of possibilities so as to optimize an objective function, given a state of the external conditions. Darwin recognized the importance of the history of past inheritances and a stochastic mechanism which

1.1

ADAPTIVE THEORY, HISTORICAL AND CURRENT DEVELOPMENT

5

appears to produce spontaneous variations. In the year 1859 Darwin's contribution to the understanding of adaptive processes was remarkable. Volterra, a mathematician, as early as 1901 devoted much of his time to the formulation and solution of a series of deterministic mathematical models of a two-species biological adaptive system which showed for the first time that interaction between parts of the same closed system could produce population cycles.P Lotka in 1924 published a study of mathematical biology in which, in recognition of the probabilistic nature of the adaptive process, he defined evolution as the history of a system undergoing irreversible changes. He said:

Having analyzed the submerged implications of the term evolution as commonly used, so as to bring them into the focus of our consciousness, and having recognized that evolution, so understood, is a history of a system in the course of irreversible transformations, we at once recognize also that the law of evolution is the law of irreversible transformations that the direction of evolution .. , is the direction of irreversible transformations. And this direction the physicist can define or describe in exact terms. For an isolated system, it is the direction of increasing entropy. 13 Thus, the connection of the concept of adaptive processes and the concept of entropy (a measure of uncertainty in an isolated system) was established. Lotka went on to establish in a verbal way the relation of thermodynamic systems to biological systems:

We may have little doubt that the principles of thermodynamics or of statistical mechanics do actually control the processes occurring in systems in the course of organic euolution.r: ... What is needed, in brief, is something of the nature of what has been termed" Allgemeine Zustandslehre," a general method of Theory of Stater" It is toward such a Theory of State that our efforts have been applied in the adaptation of Boltzmann's H theorem to economic processes. Lotka failed to develop the stochastic theory he had envisioned, but the seeds of the application of statistical methanics were sowed by Rashevsky who, in his book on mathematical biology, used a thermodynamic occupancy model to illustrate the formation of Pareto wealth distributions for a closed social systern.l" The work of the mathematical

6

1.

INTRODUCTION

biologists has been taken up by a new science, the discipline of cybernetics. This has been the result of the realization of certain fundamental similarities in biology, automatic control, psychology, and economic theories which lend to common treatment. Psychologists, particularly Pavlov'? and Thorndyke'" began to study learning phenomena as a type of adaptive process at the turn of the century. Tolman'? in 1937 considered a mathematical description of the learning behavior of animals in "T" mazes. His "behavior ratio" is closely related to the frequency concept of the subjective probability theory. Recently these statistical learning models have been improved by Estes.f" Bush and Mosteller.s! Flood,22 and others. t Tolman's white and black sensing "sowbug" is an ingenious example of the design of a light-sensitive adaptive mechanism which can be used to test learning theories. Mcf.ulloch'" has studied the resemblance between the function of neuron networks in the visual cortex and electronic computer logic circuits. This similarity led Rosenblattw to consider an adaptive device which he calls a perceptron. Perceptrons, which are similar to biological neurons, can be connected into devices which learn to recognize complex patterns. Many other similar "neuronic" adaptive devices have been developed. Widrow'" has developed an electrochemical simulated neuron which he uses to construct adaptive pattern recognition devices. These devices can learn to recognize human speech, and perform difficult balancing tasks. A theoretical structure for the design of a learning machine which maximizes the rate of growth of its intelligence has been developed by the author.s" This type of adaptive machine has the capacity to discriminate between what the machine considers relevant and irrelevant environmental information. This ability has been called adaptive concentration. Turing'" in 1936, during the study of certain propositions in mathematical logic, introduced the fundamental notion of a simple hypothetical machine or universal automaton, now known as a Turing machine. If a Turing machine is shown a mathematical series, the machine can adjust its operation so as to generate the continuing terms of this series. Turing showed that this is true for a series generated by any other machine, no matter how complicated this other machine may be. Thus, the simple Turing machine, given sufficient time, could duplicate the performance of a more complex machine. If we view the input series to the Turing machine as a sequence of event outcomes generated by t For example, see text and references in Duncan Luce.··

1.1

ADAPTIVE THEORY, HISTORICAL AND CURRENT DEVELOPMENT

7

the machine's environment, we could say that the Turing machine learns to cope with its environment. The theory of the Turing machine was extended in 1948 by von Neumann 29 to even greater significance. He showed the conditions under which a Turing machine could construct other Turing machines. Furthermore, under certain conditions he showed that a Turing machine could construct other machines far more complex than itself. This result has far-reaching biological implications; but for the theory of adaptive processes in general, it provides a mathematical framework of very fundamental nature. It appears reasonable to assume that a complex adaptive process can be decomposed into a hierarchy of simple Turing machine processes. Turing machines and other adaptive devices require an input of information from the outside world. The role of information and a measure of the rate of flow of information was considered by Norbert Wiener. In his books, "Cybernetics" and "The Human Use of Human Beings," Wiener introduces the concept of information (which is directly related to the concept of entropy) in the study of adaptive processes. He said:

Information is a name for the context of what is exchanged with the outer world as we adjust to it, and make our adjustment felt on it. The process of receiving and of using information is the process of our adjusting to the contingencies of the outer environment, and of our living effectively within that environment. 30 Although the idea of devising a measure of information occurred probably first to Hartley" and later to Fisher,32 Shannon.P Kolmogorov.s! and Wiener.s" the main influence of the role of information stems from the two papers by Shannon in which he formalized the concept. Adaptive machines which simulate animals have actually been constructed by several experimenters. Walter's electromechanical "Tortoise" or machina speculatrix is a well-known example of a machine which receives and responds to information from its environment." It may be argued, however, that this machine is not adaptive since it does not change its own behavior patterns. On the other hand, Ashby has constructed an electronic machine, the homeostat.P? which is an adaptive mechanism. When an essential part of the machine is torn out, an adaptive process is initiated and the machine, itself, finds a replacement so that it may survive. The study of such machines has sharpened the

1.

8

INTRODUCTION

concept of adaptation. Ashby has clearly shown some of the necessary conditions for adaptation to take place. 38 ,39 Ashby emphasizes the role of finite step transformations in the theory of adaptation and thus sets the stage for the use of dynamic programming techniques for the solution of multistage adaptive decision processes. Bellman's extension of dynamic programming to the study of adaptive control processes-? has provided a flexible mathematical tool which is itself well adapted for the analysis and simulation of adaptive processes in general. A great body of literature now exists on the theory and application of adaptive control systems. t Although the adaptive economic problem differs in some fundamental ways from the adaptive control problem, much of this work is useful in the study of adaptive economic processes. We shall employ the dynamic programming technique in our study of adaptive economic processes. First we must distill from the historical and current development of adaptive theory a notion of a general adaptive process.

1.2 Common Properties of Adaptive Processes If we examine various social and biological processes, we see certain common properties which may lead to an understanding of these processes. First, in taking an action, we see that at any point in time an adaptive system can move off in anyone of many directions. Second, we find that a move in any of these directions will cause a change to the system and perhaps its environment. This change becomes part of a record, the historical record of the system which is unchangeable. Third, there is in each adaptive process a choice or decision-making function. With knowledge of the historical record and an evaluation of the effect of each action on the present and future states of the system, the decision-making function chooses one of the alternative actions. Fourth, following the taking of an action, the system mayor may not achieve the anticipated result. Uncertainties in the environment or the effect of the action may cause the system to move in a direction which was unintended. t

For example see reference 41.

REFERENCES

9

Fifth, if the result was unfortunate, there is no recourse. The only corrective measure is to re-establish the above sequence of events all over again. Certain fundamental notions will be required to formulate adaptive processes. (I) A model of an adaptive process must be a sequential time process since uncertainties on the environment make success difficult to obtain and hold. (2) We must find a way of treating the information contained in the historical record. Thus, a concept of information will be an important notion. (3) We must have a stochastic representation of the effects of the system's environment which, as we see above, plays a vital role. (4) The role of the decision-making function must be made explicity. (5) We note that only one action is to be selected out of many possible alternatives. How this one action is selected is a fundamental notion with which we must deal. (6) We must specify the ways in which the structure of the system is changed by each of the possible actions and the possible effects of the environment. A mathematical model is the best way to provide answers to these fundamental requirements for the study of adaptive processes. In the next chapter some of these notions will take mathematical form. References 1. Engels, F., "Ludwig Feuerbach," p. 48. International Publishers, New York, 1941. 2. Madison, J., Federalist Papers No. 10, p. 55. National Home Library, Washington, D.C., 1937. 3. Madison, J., Federalist Papers No. 10, p. 56. Natl. Home Library, Washington D.C., 1937. 4. Madison, J., Federalist Papers No. 10, p. 57. Natl. Home Library, Washington D.C., 1937. 5. de Tocqueville, A., "Democracy in America," Vol. 1, p. 267. Vintage Books, New York, 1954. 6. de Tocqueville, A., "Democracy in America," Vol. 1, p. 273. Vintage Books, New York, 1954. 7. de Tocqueville, A., "Democracy in America," Vol. 1, p. 264. Vintage Books, New York, 1954.

10

1.

INTRODUCTION

8. Lippmann, W., "The Public Philosophy," p. 35. Mentor, New York, 1956. 9. Malthus, T., "Population: The First Essay," pp. 7-18. Michigan Univ. Press, Ann Arbor, 1959. 10. Darwin, C., "The Origin of Species," p. 413. Appleton, New York, 1883. 11. Darwin, C., "The Origin of Species," p. 421. Appleton, New York, 1883. 12. Volterra, V. "Fluctuations dans la Lutte pour la Vie." Gauthier-Villars, ImprimeurLibraire, Paris, 1938. 13. Lotka, A., "Elements of Mathematical Biology," pp. 24, 26. Dover, New York, 1956. 14. Lotka, A., "Elements of Mathematical Biology," p. 36. Dover, New York, 1956. 15. Lotka, A., "Elements of Mathematical Biology," p. 40. Dover, New York, 1956. 16. Rashevsky, N., "Mathematical Biology of Social Behavior," pp. 72-75. Chicago Univ. Press, Chicago, 1951. 17. Pavlov, I. P., "Conditioned Reflexes, an Investigation of the Physiological Activity of the Cerebral Cortex" (Dover ed.). Dover, New York, 1960. 18. Thorndyke, E. L., "The Fundamentals of Learning." Columbia Univ., Teachers Col1ege, New York, 1932. 19. Tolman, E. C., "Behavior and Psychological Man, Essays in Motivation and Learning." California Univ, Press, California, 1958. 20. Estes, W. K., Toward a statistical theory of learning, Psychol. Rev. 57, No.2, 94107 (1950). 21. Bush, R. R, and Mosteller, F., "Stochastic Models for Learning." Wiley (Interscience), New York, 1955. 22. Flood, M., Stochastic learning theory applied to choice experiments with rats, dogs, and men," Behavioral Sci. 7, No.3, 289 (1962). 23. Luce, D. R., "Individual Choice Behavior," Wiley, New York, 1959. 24. McCul1och, W. S., The brain as a computing machine, Elect. Eng. 68,492-497 (1949). 25. Rosenblatt, F., Approaches to the study of brain models, in "Principles of SelfOrganization" (H. von Foerster and G. W. Zopf, Ir., eds.), pp. 385-402. Macmillan, (Pergamon), New York, 1962. 26. Widrow, B., Generalization and information storage in networks of adaline "neurons," in "Self-Organizing Systems 1962" (M. C. Yovits, G. T. Jacobi, and G. D. Goldstein, eds.) pp. 435-461. Spartan Books, Washington, D.C., 1962. 27. Murphy, R E., Relations between self-adaptive control theory and artificially intel1igent behavior in a stationary stochastic environment, in "Artificial Intel1igence," Publication S-142, Winter General Meeting, pp. 45-63. Inst, Elect. and Electron. Eng., 1963. 28. Turing, A. M., On computable numbers with an application to the entscheidungsproblem, Proc. London Math. Soc. Ser. 2, 42, 230-265 (1937). 29. Von Neumann, J" The general and logical theory of automata, in "John von Neumann -Col1ected Works" (A. H. Taub, ed.), Vol. 5, pp. 288-328. Macmillan (Pergamon), New York, 1963. 30. Wiener, N., "The Human Use of Human Beings," pp. 17-18. Doubleday, New York, 1954. 31. Hartley, R V. L., Transmission of Information, Bell System Tech. J. 7, 535 (1928). 32. Fisher, R A., "Contributions to Mathematical Statistics," pp. 26-47. Wiley, New York, 1950. 33. Shannon, C. E., A mathematical theory of communication, in Bell System Tech. J. 27, 379-423 (1948); ibid. 27, 623-656 (1948).

REFERENCES

11

34. Kolmogorov, A., Interpolation und Extrapolation von stationiiren Zufiilligen Folgen, in Bull. Achad. Sci. USSR Sec. Math. 5, 3-14 (1941). 35. Wiener, N., "Cybernetics," pp. 75-77. Wiley, New York, 1948. 36. Walter, W. Grey, An Electromechanical "Animal," in Discovery, Vol. 11, No, 3, 90-93 (1950). 37. Ashby, W. R., "Design for a Brain," pp. 93-102. Wiley, New York, 1954. 38. Ashby, W. R., "Design for a Brain," pp. 238-240. Wiley, New York, 1954. 39. Ashby, W. Ross, "An Introduction to Cybernetics," pp. 195-259. Wiley, New York, 1956. 40. Bellman, R., "Adaptive Control Processes: A Guided Tour." Princeton Univ. Press, Princeton, New Jersey, 1961. 41. Proc. First Internat. Symp, on Optimizing and Adaptive Control, Rome, April, 1962. Instrum. Soc. Am., Pittsburgh, Pennsylvania, 1962.

CHAPTER 2

THE MATHEMATICAL MODEL •.. mathematics is the art of giving the same name to different things. Poincare

2.1

Science and Method

Introduction

Casual examination of adaptation in social and biological systems provides a framework for the development of a mathematical model of adaptive processes. A mathematical model is a formal assembly of logical interrelations which facilitates the determination of the essential elements of some physical, biological, or social process. Furthermore, a mathematical model permits one to analyze the effects of external and internal parametric changes on the system's variables. The adaptive process can be studied by use of the mathematical theory of discrete processes. A discrete process is defined on a set of real positive integers, usually called the stages of the process. This type of process is different but closely related to a continuous process which is defined on a continuous line usually called time. Although physical time appears to advance as a continuous process, events occur discretely. We are primarily concerned with the effect of events on adaptive decision makers, the passage of continuous time being an indirectly related notion. Due to this concern over events, particularly economic events, we shall find that the discrete process is a natural model for the adaptive process.

2.2

Discrete Processes

There are three fundamental notions in the theory of discrete processes. These are the notions of (1) (2) (3)

stage, state, and transition. 12

2.2

DISCRETE PROCESSES

13

Consider two sets of positive real integers: S

=

{I, 2, ..., s, ...}

(2.2.1 )

T

= {I, 2, ..., t, ...},

(2.2.2)

and

the elements of which are known as the state indexes and the stage indexes, respectively, of a discrete process. There is a set of numbers (or more generally, vectors) defined on the ordered pairs of S X T, which we will call the value function. In some explicit sense this function will represent the value of the process at each of the state and stage indexes. In deterministic processes, the value function at a given stage index is single valued. Thus, in deterministic processes we always know that there is a unique state for each stage of the process. Stochastic processes are characterized by value functions which are multivalued. It is possible for these processes to be in many different states for a given stage; thus, we are never certain what th~ value of the process will be at a given stage. We shall be dealing with stochastic processes in this book and it is fortunate that certain tricks have been developed which make these processes mathematically tractable. Figure 2.1 shows graphic examples of these discrete deterministic and stochastic processes.

"" 5 III '" 4 ~ 3

o

Vi

2 1

/

/

I

/"-...

\

\

r-,

12345678

Stages: fliT

"" 5 I--II--I---!----.--+--+--+---+III

'" 4 f---'HHH-+-+--+--+~ 3 HHHH-+--+--+--+-

o

Vi 2 f--HI--I----.H--+--+--+1 f-...-...-I--- ( - - )

!f => (---)

0.4

0.3 02 0.1

/

/

/

/

/

I

/

/

/

10

I

..-//

O'--:'---:-'---'-"--=---'------JL.---'--------'L.---'-------J o 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10 Probability, PI or (1- pz)

FIG. 7.2. Asymmetrical case. - - the optimum portfolio ratio. - - - - - the maximum rate of growth.

by his volition, maintain a constant optimum portfolio ratio over the duration of the process.

7.3

An Investment Model with Full Liquidity

We now consider the multistage investment process but we do not assume that the portfolio ratio is constant over the duration of the process. We find an optimum portfolio ratio for each stage which is optimal for the current stage, and all those that follow. In other words, we make use of Bellman's principle of optimality. From the economic view this means that there is sufficient liquidity so that funds invested need only be committed for one stage. We start by redefining the portfolio ratios. For each investor, say the kth, we have a vector A k • The elements of these vectors for aT-stage process are given by such that

o ~ akt ~

I}.

(7.3.1)

The maximization problem can now be stated as (7.3.2)

7.3

105

AN INVESTMENT MODEL WITH FULL LIQUIDITY

Equation (7.3.2) can be written as a sequence of terms, one for each period. We have gkT

= ~ 1E [In :kI-] + E [In KKkT-1] + .., + E [In kT-l

kT-2

KKk!

kO

]II'

(7.3.3)

Using Bellman's principle of optimality, we may maximize Eq. (7.3.3) one stage at a time. For a process containing T stages (the horizon is the Tth stage), optimization of the last stage is independent of the future because the last stage has no future. In the previous section, we optimized the portion of invested funds by maximizing the expected rate of capital growth. We can obtain the same results by maximizing the log of the capital at the end of the process, given the initial capital available for investment. It has been found that it is more instructive to maximize with respect to the expected log of capital at the end of each stage, and then show that this sequence leads to the maximization of the expected rate of capital growth for the whole duration of the process. With this in mind, the optimum portfolio ratio at the beginning of the Tth stage is found by maximization of F1(KkT-1)

= max E(ln akT

K kT)

= akTEA max

k

{In K kT- 1

+ P1ln(1 + akTr1) (7.3.4)

where K kT is the kth investor's capital funds available for investment at the end of the Tth stage, akT is the portfolio ratio for the kth investor at the beginning of the Tth stage, and F1(KkT- 1) is the expected log of the capital, K kT , for a one-stage optimal process, given that the capital at the beginning of the stage is K kT - 1 • We have shown previously that this maximum of Eq. (7.3.4) occurs when

and at this value of a kT , Eq. (7.3.4) reduces to F1(K kT-1)

= In K kT- 1 + H* - H

+ Y.

(7.3.5)

Furthermore, the optimum portfolio ratio for a two-stage process is given by the maximum of F 2(KkT-2)

= akT·akT max

-1

{In K kT} = max {PIFl(KkT-lleventt) a k T_1 P2Fl(KkT-llevent2)}'

+

(7.3.6)

106

7.

AN ADAPTIVE INVESTMENT MODEL

Using Eq. (7.3.5) we define Fl(KkT-lleventI) = In K kT_2(1 + akT-lrl)

+ H* - H + Y; akT-lT2) + H* -,- H + Y.

(7.3.7)

Fl(KkT-1Ievent2) = In K kT_2(1 -

(7.3.8)

Substituting Eqs. (7.3.7) and (7.3.8) into Eq. (7.3.6), we find F 2(KkT-2) = max {Plln(l akT-l

+ akT_lrl) + P2ln(1 - akT_lT2)} + In K kT- + H* - H + Y. 2

Note that max {Pl ln(1 + akT-lTl) + P21n(1 - akT-lT2)}

akT-l

=

H* - H

+ Y.

Therefore, we find that (7.3.9)

The above development can be repeated for T stages but the solution to the tth state can be given in a general statement. If it is true that: (1)

process,

r l and r 2 are constant over the duration of a T-stage investment

PI and P2 are constant over the duration of the process, (3) PI' P2 ~ 0, PI + P2 = 1,

(2)

(4) (5)

r lr 2 ~ PlT l - P2T2 ~ 0, and the principle of optimality holds, then we have

(i) the solution of a (T - t)-stage process for the kth decision maker at stage (t - 1) is given by FT-t(Kkt)

=

In K kt + (T - t)(H* - H

+ Y),

(7.3.10)

and (ii)

the optimum portfolio ratio is given by

for

Proof.

t

= 1,2, ..., T.

(7.3.11)

For a process at the tth period we can write

FT-t+l(Kkt-l) = max{PlFT-t(KktteventI) a • k

+ P2FT-t(Kkttevent2)}.

(7.3.12)

7.3

107

AN INVESTMENT MODEL WITH FULL LIQUIDITY

With the help of Eq. (7.3.10) in the statement for a process in the tth stage, we can write

and FT-t(Kktlevent2)

= In K k t-l (1 -

aktT2)

+ (T -

t)(H* - H

+ Y).

Substitution of these equations into Eq. (7.3.12) gives F T-t+l(Kkt- l)

= max {In K k t-l Ukt

+ (T -

+P

lln(1

t)(H* - H

+ aktTl) + P21n(1

-

aktT2)}

+ Y).

(7.3.13)

But we again note that

Therefore, Eq. (7.3.13) becomes FT-t+l(Kkt-l)

=

In K

kt-l

+ [T - t + 1][H* -

H

+ Y],

(7.3.14)

which is the solution to a process at the tth stage, as given by Eq. (7.3.10). Thus, part (i) of the statement is proved by mathematical induction on t. Since

where the conditional functions are defined in the manner of Eqs. (7.3.7)· and (7.3.8). Equation (7.3.15) then reduces to FT_tCK kt) •

=

max {Plln(1 Uk t

+1

+ (T -

t-

+ akt+lTl) + P21n(1 - akt+lT2)} 1)(H* - H + Y) + In K kt •

(7.3.16)

. We have already noted that the maximum of the bracketed term occurs when (7.3.17)

Since t is arbitrarily chosen and the p's and T'S are constants, all the a's are equal for all t. This proves part (ii) of the statement.

108

7.

AN ADAPTIVE INVESTMENT MODEL

It remains to be shown how the maximization of In K k t leads to the maximization of the expected rate of capital growth. From Eq. (7.3.10), with t = 0, we have . (7.3.18)

Dividing by T and subtracting In K k O , we get

mJ.x ~ E [In ~::]

=

H* - H

+ Y.

From previous work, we know that

t:

T

= H* - H

+ Y;

therefore, we can write -* = gkT

mJ.x r1 E [1n KkTJ K • kO

(7.3.19)

Economically, then, this means that under restrictions set forth in the statement of the general solution, even when the investor has full liquidity and can vary the portfolio ratio at each stage of a multistage investment process, he does not. Thus, the maximum expected rate of capital growth is the same as in the illiquid case where the portfolio ratios are forced to be equal throughout the process. If the constants, the p's and r's, become variables, the liquidity postulate would result in nonequal elements in the optimum portfolio ratio vector. The effect of the constant p's will be relaxed in the next section. Examination of Eq. (7.3.17) shows that the portfolio of each investor (assuming that every investor knows' the true probability density function of the payoff process) is the same. Furthermore, because of the assumption of the logarithmic objective (maximization of expected rate of growth) as initial holding of each investor's capital, K k t does not affect the choice of an optimal portfolio ratio. If this is true, then all investors will hold the same portion of their funds in securities. Of course special objective functions could be postulated for each investor which could explain why actual investors might hold different portions of their funds in the same kind of security. However, this is really no solution since then we would have to show why the investors have different objective functions. There is a more appealing approach. Investors do not have full knowledge of the probability function of the payoff process. The perfect foresight investment service simply does not exist.

7.4

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY

109

7.4 An Adaptive Investment Process with Full Liquidity So far, the concept of adaptation has not been brought into the investment problem. In the adaptive process we go one step farther than the introduction of risk (stochastic payoff structure). In an adaptive process, the payoff probabilities are unknown parameters. Thus, in an adaptive process, we have risk and uncertainty. In the adaptive investment process we have a probability function which plays a major role in the determination of the optimum portion of available funds to be invested each period, and the maximum expected rate of capital growth. The values of the probabilities making up the probability function of the environmental stochastic process are in reality unknown to the decision maker. On what ground are we able to say anything definite about the rate of capital growth? As in the previous adaptive processes we can substitute subjective probabilities based on a scheme which is, in some sense, the best for a given quantity of historical information. In Chapter 4 we derived such subjective probabilities and we shall employ them here. The binomial environmental process is characterized by the occurrence of certain kinds of random payoff events. We have two possible payoff events for each stage. For the number of payoffs of 7 1 , we denote the number nlt , and for the number of payoffs of 7 2 , we denote the number n2 1 • We assume here that the decision maker has experienced t stages or t events. The kth decision maker's subjective probabilities for binomial environment are given by for

i = 1,2.

(7.4.1)

We have gone about as far as we can with the simplified notational system for subjective expectations. Because of the complexity of a multistage adaptive process, we shall now be forced to introduce the notational scheme used in Chapter 5. This scheme is not as instructive as the previous scheme, but it is far more flexible. NOTATION FOR SUBJECTIVE EXPECTATIONS. Suppose we identify the finite sequence of payoff events that have occured up to the tth stage of an investment process by the vector (7.4.2)

110

7.

AN ADAPTIVE INVESTMENT MODEL

At any particular stage, say the tth, the event anyone of a finite set of possible values:

Z

t

can be associated with (7.4.3)

For the binomial case (m = 2), if Zt = T1, then the payment of T1 occurred at the the tth stage; if z , = T2, then the payment of T2 occurred at the tth stage. In our case, R, is the same for all t, and m = 2. We have assumed that the actual probability function for the payoff events is independent of the stage index, and is denoted by for any

t

=

1,2, ..., T.

(7.4.4)

For T stages, suppose we wish to know the actual probability that some particular sequence of payoff events would occur. For example, say T1 is paid at t = 1, Tj is paid at t = 2, ..., and Tk is paid at t = T. Since the events are independent of the stage index, the joint probability of these payoffs, given by the product of the probabilities of each of these payoffs, would be (7.4.5)

On the other hand, the subjective probabilities are not independent of the stage index numbers. The joint subjective probability of the sequence of payoffs denoted above for the kth decision maker would be Pk(Zl , Z2' ... , ZT)

=

{Prk(Zl

=

Ti)' Pr k(Z2

=

T;

I Zl)' ... , (7.4.6)

Let us denote the function, Pk(Zt

I Z1 , Z2 , ... , Zt-1) =

{Prk(Zt

=

Ti

I Zl

i E M} t = 1,2, ..., T, (7.4.7)

, Z2 , ... , Zt-l):

for any

as the conditional subjective probability function of an event at the tth stage for the kth decision maker. Note that (7.4.8)

In our old notation, we see from Eq. (7.4.7) that Pk(Zt

I Zl

, Z2 , ... , Zt-l)

== {Pkit : i E M}

for any

t

= 1,2, ..., T.

7.4

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY Present

Future

I

I

I

I I I II

------f-----.......,

I I

I

1'2' r,

I 1'1'

111

r,

I

1'2' r,

Stage I

FIG. 7.3.

Slage 2

Subjective probabilities for a three-stage process.

Fig. 7.3 may clarify the notion of the conditional subjective probability function. Suppose we desire to find the marginal subjective probability of the events at the tth and (t - I)th stage. This probability is given by Pk(Zt , Zt-l)

!-

=

Pk(Zl , Z2 , ... , Zt-l • Zt)·

(7.4.9)

R,Rz· .. R,_z

Also, the marginal subjective probability of just the (t - I)th event is PiZH)

=

!-

PiZl' Z2' •••• Zt-2' Zt-l)'

(7.4.10)

R,Rz···Rt-.

Using Eqs. (7.4.9) and (7.4.10), we can evaluate the subjective conditional probability of the tth event, given knowledge of the (t - I)th event, by p( k Zt

I Zt-l ) -_

P k( Z t , Z t- l ) P ( ) . k Zt-l

(7411) • .

112

7. AN ADAPTIVE INVESTMENT MODEL

We now have at our disposal a notational scheme which is sufficiently compact to greatly simplify the equations that follow. OPTIMIZING THE T -STAGE BINOMIAL ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY: Returning to the investment problem and starting with the equations developed for a nonadaptive process, we have for the the tth stage and the kth decision maker, kt 10 KK = nit 10(1 ko

+ aktTl) + n2t 10(1 -

aktT2)'

(7.4.12)

Suppose we are presently at the end of the(T - l)th stage of aT-stage process. We have, as a consequence of the information gained from the T - I stages of history just experienced, estimates of the probabilities of receiving one of a set of possible payoffs. Substituting T for t in Eq. (7.4.12) taking the initial state to be the state of the (T - l)th stage, and taking the subjective expectation of Eq. (7.4.12) from the point of view of the kth decision maker, we have

2k (-)

indicates that the expectation is with respect to the subjective probabilities known to the kth decision maker. The factor "one" is emphasized by the parenthesis to indicate that Eq. (7.4.13) is for one stage, and the final stage of the process. From the market point of view (the actual expected rate of capital growth resulting from the decisions of the kth decision maker), we would have E (10 KKkI-.) kT-l

= (I)P 1 lo(1 + akTTl) + (l)p210(l - akTT2)'

(7.4.14)

This expectation is taken with respect to the actual probability function of the environmental stochastic process. Since the decision maker knows only the subjective expectation of the result of his decisions when he set a value for akT' he maximizes not Eq. (7.4.14) but Eq. (7.4.13), the ex ante equation, to obtain the optimum portion of funds to be invested in stage T. Maximization of Eq. (7.4.13) leads to an optimum portfolio ratio for the Tth stage given by (7.4.15)

704

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY

113

subject to the condition that (7.4.16)

i.e., (7.4.17)

Because the Pkir'S are themselves random variables, we take note here that there is a finite probability that Eq. (704.17) will be violated even if it is true that (7.4.18)

The probability of this violation occuring will be derived later, but at this point let us condition our next results on the chance that such a violation does not occur during the stage under study. The actual expected rate of capital growth can be determined from the decision maker's optimal portfolio ratio. It is a function related to Eq. (704.13), but different in an important way. Equation (704.13) is an expectation with respect to the subjective probability function since it is viewed from the decision maker's standpoint at the beginning of a stage. The actual expected rate of capital growth is an expectation with respect to the actual probability function of the payoff. It is the expectation of capital growth which is actually seen on the market at the end of a stage. This is given by (7.4.19)

where we define (7.4.20)

From the point of view of the decision maker the expected growth of capital is given by (7.4.21)

where we define (7.4.22)

7.

114

AN ADAPTIVE INVESTMENT MODEL

and

In the dynamic programming notation equation (7.4.21) is given as fl(K kT-1 )

= In K kT-1 + H* + Yk(R T I R T-1 ••• R 1)

-

Hh(RT I R T-1 ••• R 1) · (7.4.24)

For a two-stage process starting with K kT- 2 , we have from the decision maker's point of view, f2(KkT-2) = max {PklT-dl(KkT-1 I ZT-l = a.T_l

T1)

+ Pk2T-dl(KkT-

1 [

ZT-l = T2)}·

(7.4.25)

We define fl(K kT- 1 I ZT-l

=

T1)

= In K kT- 2(1 + akT-ITl)

+ Yk(R T I R T-

and, fl(K kT-1 I ZT-l

+ H*

1

R 1)

IZT_1-rl

- Hk(R T I R T-1

R 1)

[zT_l-rl

(7.4.26)

= T2) = In K kT_2(I - akT-IT2) + H*

+ Yk(R T I R T_1 ... R 1) IzT_1-r.

In these equations we define for i = 1, 2, Hk(R T I R T-1

•••

R 1)

IzT_1-r,

= - ~ Pk(ZT I ZT-l = T, , ..., Zl) RT

In Pk(ZT I ZT-l = T, , ..., Zl)' (7.4.28)

Note that Hk(R T I R T-1

.. ,

R 1)

= ~

RT_l

Pk(ZT-l I ZT-2 , ... , zl)Hk(R T [ R T-1

...

R 1)

IzT_1-r,·

(7.4.29)

7.4

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY

Also we define for i Yk(R T I R T-1 ... R 1)

115

= I, 2,

IZT_1-rj

= In [r1 ~ r2] I ZT-l = ri' ..•, Zl) In r2

- Pk(ZT

=

r1

- Pk(ZT

=

r 2 1 ZT-l = r;, ..., Zl) In r 1 • (7.4.30)

Again note that Yk(R T I R T-1 ... R 1) = ~ PiZT-l I ZT-2 , ..., Zl)Yk(R T I R T-1 ... R 1) RT_l

IzT_1=rj •

(7.4.31)

Substitution of Eqs. (7.4.26) and (7.4.27) into Eq. (7.4.25) results in f2(K kT-2)

=

max {PklT-1ln(l

a.T_l

+ akT-lrl) + A2T-lln(1

+ H* + Yk(R T I R T-1 ... R 1) -

- akT-1r2)}

lIk(R T I R T-1

•••

R 1) ·

(7.4.32)

Maximization of the bracketed term in Eq. (7.4.32) leads to max{PklT_1ln(1 a.T_l

+ akT-1r1) + Pk2T-1ln(1

- akT-lr2)}

If an interior solution exists, then 0< atT_l

< I,

(7.4.34)

or what is the same thing, (7.4.35)

Again under the condition that Eq. (7.4.35) is not violated, we may write f2(K kT-2)

= In K kT-2 + 2H* + Yk(R T-1 I R T-2 ..· R 1)

+ Yk(R T I R T-1 ... R 1) - lIk(R T_ 1 I R T-2 ... R 1) - Hk(RT I R T-1

...

R 1)·

(7.4.36)

The above process may be repeated for T stages, but a general solution may be given for any tth stage of the process by the following statement.

116

7.

AN ADAPTIVE INVESTMENT MODEL

If r1 and r 2 are constant over the T-stage adaptive investment process, the principle of optimality holds, and if at no stage in the process the condition

if

for is violated, then

(i) the maximum value of the (t of the kth decision maker is given by

+ 1)th

fr-tCK kt) = In K kt + (T - t)H*

t

= 1,2, ..., T

(7.4.37)

stage from the point of view

+k T

Yk(R I R T

T-

1 •••

R1)

T~t+l

(7.4.38)

where (7.4.39)

and

(ii)

The optimal portfolio ratio for the kth decision maker is given by t = 1,2, ... , T.

(7.4.41)

(iii) The maximum expected rate of capital growth for the kth decision maker from the market point of view is given by (7.4.42)

where g~t is the maximum expected rate of capital growth for the kth decision maker at the tth stage of the process, and where we define (7.4.43) Proof.

For an adaptive process at the tth stage, we can write

7.4

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY

117

With the help of Eq. (7.4.38) of the solution statement, we can say fT-t(K kt I Zt = r1) = In K kt_1(1 + aktr1)

+ I. T

T=1+1 T

+ (T -

Yk(R T I R T-1 ...

t)H*

R1) I't~rl

- T=1+1 I. Hk(R T I R T- 1 ... R 1) Iz,~rl A

and

+ I. T

T=1+1 T

Yk(R T I R T- 1 ••• R 1) I.,~r.

- T=1+1 I. Hk(R T I R T- 1 ... R 1) I.,=r•. A

In view of the definitions (7.4.26) and (7.4.27), we substitute these equations into Eq, (7.4.44) and get fT-t+l(Kkt- 1)

= max {Pklt In(l a kt

+ aktY1) + Pkzt In(l

+ In K kt-1 + (T -

+ I. T

t)H*

T=1+1

- akt1"Z)}

Yk(R T I R T-1 ... R 1) (7.4.45)

We note again that max {Pklt In(I °kt

+ aktr1) + PkZt In(l

- akt1"Z)}

= H* + Yk(R t I R t-1 ... R 1) - Hk(R t I R t- 1 ... R 1)· Therefore, Eq. (7.4.45) becomes fT-1+1(K kt-1) = In Kkt-l

+ (T - t + I)H* + I. Yk(R TI R T- 1 •.. R 1) T

T=t

- I. Hk(R T I R T-1 ... R 1) T

T=t

A

(7.4.46)

7.

118

AN ADAPTIVE INVESTMENT MODEL

which is the solution to a process at the tth stage as given by Eq. (7.4.38) in the statement. Thus, the first part of the solution statement is proved by induction on t. When the bracketed term of Eq. (7.4.45) is maximized, the maximum occurs at Pkltrl - Pk2t r2 r 1T 2

t = 1,2, ..., T.

for any

(7.4.47)

This proves part (ii) of the solution statement. Extending Eq. (7.4.19), we can write the sequence E*[In K kt] E*[In Kkt-l]

+ H* + Y E*[In K kt- Z] + H* + Y -

= E*[In Kkt-l]

Hk(R t I R t- 1

=

H k(Rt_1 I R t- 2

...

R1 )

•••

R1)

Repeated substitution of these equations, one into the other, gives us E*(In K kt)

= In K kO+ t(H*

+ Y) -

!- Hk(R t

T

IR

T-

1 •••

R 1) .

7=1

Subtracting In K k O from both sides and dividing by t gives

This proves part (iii) of the solution statement. The general solution statement shows that the optimal portfolio ratio for the kth adaptive decision maker is a function of the magnitudes of the possible payoffs, the historical information concerning the frequency of the payoffs, and the a priori conviction of the decision maker. Contrary to the optimal portfolio ratio for the nonadaptive decision maker we see that there is good reason to believe that each adaptive decision maker behaves differently, even if all adaptive decision makers face the same payoff structure and history. This is because there is little possibility of all adaptive decision makers having the same a priori convictions, the o:'s.

7.4

AN ADAPTIVE INVESTMENT PROCESS WITH FULL LIQUIDITY

119

For example, let us examine the implications of the general solution on our three investors, Mr. Holmes, Mr. Stone, and Mr. Wise. Using the data provided in Section 4.2, Table 4.1, we see that the three investors will follow quite different investment plans and end up with different amounts of capital, even having started with the same sum. TABLE 7.1 PORTFOLIO RATIOS FOR THE THREE INVESTORS

Decision maker

Mr. Wise Mr. Holmes Mr. Stone

Conviction vector "kl

"k2

30 2 20

10 2 20 T1

Subj. prob.

Observations

100 100 100 =

nl

n.

Pklt

Pw

70 70 70

30 30 30

.714 .692 .643

.286 .308 .357

1 point rise

T2

Port. ratios Ok'

.428 .384 .286

1 point fall

=

Table 7.1 shows the results for the three investors following a number of entropy periods (stages). Now let us examine the adaptive investment process as the length of the process increases; i.e., the number of entropy stages increases without limit. The nature of such an adaptive process is tied up ill the effect of the limiting values for the subjective conditional entropy of each decision maker. In our particular process, where the payoff events are not dependent on the stage index, it can be shown that (7.4.49)

and we denote for

T

= 1,2, ..., T.

(7.4.50)

Therefore, we note that the average entropy is given by (7.4.51)

The average subjective entropy is not the same as Eq. (7.4.51) since the subjective probabilities are not independent of the stage index. The

120

7.

AN ADAPTIVE INVESTMENT MODEL

limiting value of the average subjective entropy is of use in determining the probable limit of the adaptive process. Since Pk(ZT I ZT-l' "0' Zl) is a consistent estimator (see Section 4.4), we have for the kth decision maker.

(7.4.52)

Since also H(R r)

= ~ P(zr) In P(zr) Rr

and Bk(Rr I R r-l ... R1 )

= -

P(zr)p(Zr-l) ... P(ZI) In Pk(zr I Zr-l , ..., Zl),

~ RrRr_,oo.R,

we have in probability as

T

-+ 00,

= - ~ P(zr) In P(zr) Rr

Because of Eq. (7.4.50), we have for the probability limit of the kth decision maker's average subjective entropy,

=

H(R).

(7.4.53)

Under condition (7.4.35), we can then say Rlimg* = g*. T->oo kT

(7.4.54)

We see then that the adaptive maximum expected rate of capital growth, in probability, approaches the full knowledge stochastic rate of growth as the process extends in time. Given the same initial capital for both processes, and using the full knowledge stochastic process as a

7.5

CONSIDERATION OF THE SPECIAL CONSTRAINT

121

comparative base, we can define the relative efficiency of the kth decision maker in an adaptive binomial investment process as (7.4.55)

and since we have Eq. (7.4.55) at the limit, in probability (7.4.56)

Thus the relative efficiency of this adaptive process approaches unity in probability as time extends to infinity. However, one should note that during this period of adaptation, the kth decision maker has accumulated a definite loss in capital relative to the full knowledge process that is irretrievable. He can minimize this loss by rhe use of a "best" subjective estimator, but he can never eliminate this loss since it is the cost of insufficient information. This irretrievable loss is characteristic of irreversible processes.

7.5 Consideration of the Special Constraint We are ready now to return to the consideration of the constraint (7.4.35) and the finite probability that it will be violated. We can say in addition to Eq. (7.4.41) that if

(7.5.1)

if

(7.5.2)

and

We define three probabilities: CPkl = Pr[O CPk2

=


r+-r . 1

2

(7.5.12)

124

7.

AN ADAPTIVE INVESTMENT MODEL

Because of these facts we can write in the limit that limPr [

n

lt

-

t-ocr:>

t

nklt

< 0] =

0,

or lim CfJkZ = O. t-ecc

(7.5.13)

rZ PI0].

(7.5.18)

Now from Eq. (7.5.10) we see that

A possible condition for PI is that

PI
0]

=

0,

or lim t-+oo

epk3

= O.

(7.5.20)

On the other, hand, suppose that (7.5.21 )

then we find epk3 = 1. lim t-+oo

(7.5.22)

Condition (7.5.19) is equivalent to saying (7.5.23)

This condition is one which was previously made and shown to be a characteristic of a real market situation. If we assume condition (7.5.19), we also assume that the limit (7.5.20) is valid. As a result, as t increases, the upper bound pulls away faster from the region of probable deviation of nIt than that region can expand. Provided that the probability function of the environmental stochastic process does not violate Eqs. (7.5.12) and (7.5.19), it is quite likely that the subjective probabilities will not violate Eq. (7.4.35) in the later periods of the process. In earlier periods of the process the probability of a violation of Eq. (7.4.35) is governed in part by the accuracy of the initial convictions (logical width) and the closeness of a violation of Eqs. (7.5.12) and (7.5.19) by the probability function of the environmental stochastic process.

7.6 The Multivalued Payoff Adaptive Investment Process The model introduced in the last section could be generalized in a number of ways. We would like to make it possible to introduce a multivalued payoff structure. This could be done in several ways, but by the extension of the entropy time concept we can introduce a multivalued

126

7.

AN ADAPTIVE INVESTMENT MODEL

payoff structure while pointing out some interesting relationships between decision processes in entropy time and in physical time. First let us define the payoff structure in a special way by the set R

= {rl = lr,

where

1= ±l, ±2, ...},

(7.6.1)

where r is come constant minimum incremental payoff. For example, in the context of the stock market, r could be related to a price change of -lth of a point. Now suppose the stages, the entropy time increments, are made very short so that a rise of several T'S of magnitude in physical time could be represented in entropy time as a sequence of several stages. In other words, if a stock price rose I point in a day's trading, in entropy time eight stages could have passed during this day's trading (assuming that r = -lth point). The process in entropy time could be at the least a sequence of eight increases of -lth point each. An investment process formulated in this way can be seen as a form of the classical random walk process. Bacheller" introduced the notion that investment price statistics were probably the result of a random walk process. He also noticed the connection between random walk processes and the diffusion of heat in matter. This observation led to the modern thermodynamic theory and to the study of the EinsteinWiener processes. Unfortunately, the economic aspect of Bachelier's work has been largely unexploited. Recently modern statistical methods have been applied in the analysis of investment price fluctuations. These studies have uncovered evidence that the investment price mechanism is the result of a random walk process.v l" In the classical random walk model a particle is permitted to advance or fall back by a unit distance which is a random variable generated by a ,{ quence of independent Bernoulli trials. Using this approach, the complex payoff process can be treated in the same manner as in the previous section except for a tighter definition of entropy time which we shall call the decomposition postulate. (Note: we let both T 1 and r 2 in the previous process equal r in this process. Thus, if r = -l, then T 1 = -l and r 2 = -l) Suppose that the physical time process is a Poisson process. The probability of '\ rise in the price of a security of I points in one time period would be given by Pr(R

= rl) =

where A is the Poisson parameter.

e-~

,\I

IT

(7.6.2)

7.6

MULTIVALUED PAYOFF ADAPTIVE INVESTMENT PROCESS

127

Setting 1 equal to 1, we have Pr(R = r1) = Pr(R

= r) =

r

A,\.

(7.6.3)

Now we know that the probability of one payoff or r points, event 1 in the entropy based process, has been given as P1' Substitution of this fact into Eq. (7.6.3) and then Eq. (7.6.3) into Eq. (7.6.2) gives us the relation between the probabilities of both time bases. We have Pr(R = r ) = Z

z

rAU-1)

P1 l! '

1=1,2, ....

(7.6.4)

This probability function tells us that there is a chance that many entropy time periods (and thus 1 decision stages) may occur in one unit of clock time. One might ask why should not decisions come at a constant clock time rate instead of a constant entropy time rate? Actually, the logic behind such an assumption of a one-to-one correspondence between clock time and the decision rate is not realistic. Consider an investor who is trading a speculative, widely fluctuating stock. If he is to increase his capital gains over, say, a month of clock time, he will have to observe the price fluctuations of this stock on an hour-to-hour basis. Thus, there may be about 160 potential decision stages (entropy time periods) for this investor in one month of clock time. On the other hand, suppose another investor purchases a conservative blue chip stock. He may only have to check its price once a week to assure the safety of his investment. During the month the first investor has required 160 potential decision stages and the second investor has only required 4 stages. From this example we see that the rate of passage of decision stages, and thus entropy time periods, actually increases and decreases with respect to clock time in realistic investment processes, and depends on the rate of transmission of information. What are the limitations of the decomposition postulate? The essential problem which we must face is illustrated by the above example. Suppose that the investor who trades the speculative stock finds it necessary to stay each day at the stock ticker, watching minute-to-minute changes in the price of the stock. Now suppose that the general market is in a panic and all stocks are falling rapidly. The stock ticker is the only information transmission medium, and if it falls behind the quotations in New York our investor will find himself subjected to considerable risk. Events will be occuring at a faster rate than the stock ticker's maximum information transmission rate. In this case the decomposition

128

7.

AN ADAPTIVE INVESTMENT MODEL

postulate will not hold since a one-to-one relation between decisions and events cannot be maintained. If we can be assured that the rate of information transmission within the economic system is not constrained, we could by the use of the decomposition postulate reduce the multivalue payoff processes to the simple investment model with a positive and a negative payoff by a suitable time transformation relation. Furthermore, if the probability relations between the clock time process and the entropy time process are known, they can be used to determine the maximum expected decision rate with respect to clock time. It is possible then to design an optimal economic information transmission system which will not limit the decision rates of the decision makers. The importance of adequate communication facilities in a modern economy must not be underestimated. References 1. Bernoulli, D., Exposition of a new theory on the measurement of risk (Trans!. by Louise Sommer), Econometrica 22, No.1, pp. 23-36 (1954). 2. Arrow, K., Alternative approaches to the theory of choice in risk-taking situations, Econometrica 19, No.4, 404-435 (1951). 3. Tobin, J., Liquidity preference as a behavior toward risk, Rev. Economic Studies 25 (2), No. 67, 65-86 (1958). 4. Markowitz, H. M., "Portfolio Selection." Wiley, New York, 1959. 5. Kaldor, N., The equilibrium of the firm, Economic J. 4, 60-76 (1934). 6. Wald, A., "Sequential Analysis," pp. 93-94. Wiley, New York, 1947. 7. Bachelier, L., "Le jeu, la Chance, et Ie Hasard." Flammarion, Paris, 1914. 8. Osborn, M. F. M., Brownian movement in the stock market, Operations Res. 7, No.2, 145-173 (1959). 9. Borch, K., "Prime Movements in the Stock Market," Economic Research Program No.7, April 30, 1963. Princeton Univ., Princeton, New Jersey. 10. Granger, C. W. J., and Morgenstern, 0., Spectral Analysis of New York Stock Market Prices, KYKLOS 16, No.1, 1-27 (1963).

CHAPTER 8

MULTIACTIVITY CAPITAL ALLOCATION PROCESSES The machines of nature . . . are still machines in their smallest parts ad infinitum.

Leibniz

8.1

Monadology

Introduction

In the investment process considered in the last section, "investment" was considered as an activity in which there was a possibility of capital gains or capital losses. The constant possibility of a loss created the necessity for the decision maker to hold some of his assets in the form of cash. Actually, however, this single investment activity is composed of a whole array of investment opportunities or activities. The decision maker has some choice over his participation in this array of activities; some he rejects outright; others he finds attractive and profitable. We would like to extend our analysis to this kind of multiactivity investment process. This task is not as easy as might be expected. In multi-activity processes there is a possibility of interaction between the payoff probabilities of the activities. Interactions between the payoff probabilities of the investment activities could be handled by the assumption of a Markovian environmental stochastic process which generates the probability of the payoff events. For example, in such a model the probability payoff of anyone activity falling (risirtg), given that in the previous stages many of the other activities fell (rose) would be very high. In an adaptive version of such a process the decision makers would have to estimate the Markovian probability transition matrix to determine their optimal policies. Because of the budget constraint (the allocations of capital to the investment activities and cash must equal the whole amount of capital at each stage) and the positivity of the portfolio ratios, a difficult problem in nonlinear programming would 129

130

8.

MULTIACTIVITY CAPITAL ALLOCATION PROCESSES

result, and we would be limited to numerical solutions. Such problems are beyond the scope of this book. Instead of tackling the general problem, we shall consider a simplified version of the multiactivity capital investment process which has theoretical significance. We shall make the investment activities noninteracting, and we shall limit the set of possible activities to those having only positive payoff events. Thus, each investment activity will have a positive expected payoff. This kind of multi-activity process will be called a capital allocation process.

8.2 The Adaptive Capital Allocation Process Suppose we have an ordered set, M, consisting of iii possible payoffs, one fo"r each of iii investment activities denoted by the index numbers i E M. Suppose further that these payoffs are all positive. Based on entropy time where one payoff event for each state of the process, we can write the kth decision maker's subjective expected rate of capital growth at stage T as

gkT = ~t [In T

K kT

K kO

]

= ~t [iln~] = ~ T

t=l

K k t- 1

T

it [In~] t=l

K k t- 1

(8.2.1)

where gkT is the kth decision maker's subjective expected rate of capital growth for a T-stage process, and K k l is the kth decision maker's capital at stage t. The decision maker must maximize gkT by determining a sequence of vectors A k l , one for each stage of the process, such that for each for each

iEM} t = 1,2, ..., T.

(8.2.2)

The vectors A k l are the policy vectors which consist of the capital allocation ratios (portfolio ratios) for each stage of the process. Now the decision maker must maximize gkT stage by stage. This is because the information he needs to form the subjective probability function is revealed stage by stage as the payoff events occur. Nevertheless, at each stage the decision maker must consider the future effect of each current decision. For this reason we shall use again the dynamic programming technique. We shall maximize gkT stage by stage, starting at the Tth stage, the last stage of the process.

8.2

THE ADAPTIVE CAPITAL ALLOCATION PROCESS

131

We have, for the subjective expectation of the rate capital growth for the kth decision maker and for the last Tth stage of the process, E'[ln K kT ] - In K kT- l

=

k qkiT In(1 + akiTYi)

(8.2.3)

ieM

where Yi is the positive payoff generated by the ith event, and qikT is the subjective probability of the ith event for the kth decision maker at the beginning of stage T. We have that and for each i E Iff. qkiT = I

k

ieM

Putting Eq, (8.2.3) in the dynamic programming form, we have fl(K kT- l)

= max lin K kT- l akiTeAkT I .

+ «u kqkiTln(1 + akiTYi) I, \

(8.2.4)

subject to the constraints (8.2.2). We find this maximum value by use of the Kuhn-Tucker theorem.' First, from the ordered set N! we form two subsets. The first, designated as MtT' contains all the indexes of the portfolio ratios which are positive. The second subset, designated as MZT , contains the indexes of the rest of the portfolio ratios, which are zero. The particular elements, or numbers of elements, in subset MtT and M2T have yet to be determined, so we shall assign an arbitrary number, mkT, of the iii ratios to be in the subset MtT' Since the function gkT is continuous, possesses continuous partial derivatives and is concave, the Kuhn-Tucker theorem in this application recognizes the two following conditions (i) The condition ~ie1i7 atiT cash) which implies


0 for

i

E

133

MitT' we can write Eqs. (8.2.12) and (8.2.13) as

~

qkiT

-ieM+kT

qkiTri > 1

1

i E MtT'

for

+ ieM+ ~rs

(8.2.14)

- kT

and

(8.2.15)

for

We now order the qkiTr/S of subset MtT in order of decending magnitude, starting with qklTr1 and ending with qkmTrm' Also the subset of events M2T is ordered such that qkm+lTrm+l is largest and qkmTrm is the smallest of the qkiTr/S in this subset. We take in particular the smallest qkiTri in subset MtT' i.e., qkmTrm' and the largest qkiTri in subset M2T' that is qkm+lTrm+l' For these qkiTr/s we have from Eqs. (8.2.14) and (8.2.15) that (8.2.16)

We may substitute, one at a time, all the permissible values of m (m = 1,2, ..., M), into Eq. (8.2.16) until the inequality holds. This value of m which satisfies Eq. (8.2.16) is denoted mkTand is the lower limit of the ordered subset MtT' Using this method to find mkT we may rewrite Eq. (8.2.12) as

* _'hiT [1 +

a kiT -

""" 1] - ~ 1 £.< ~

ieM+!.

We define , qkiT PkiT = -,-, UkT

kT

for

"t

where and

(8.2.18)

134

8.

MULTIACTIVITY CAPITAL ALLOCATION PROCESSES

The conditional probability, PkiT, is defined on the set of those events which actually contribute to the growth of capital. Because of the maximizing behavior of the decision maker, the remaining events contribute nothing to capital growth and are classed along with the "no change" events. We excluded these "no change" events in previous processes when the assumption of an entropy time base was made. Excluding the "no change" events, and basing the expectation of the rate of capital growth only on the subset of events which leads to capital growth for the decision maker, we can determine the maximum expected rate of growth of capital for the Tth stage. Since we have an expression for the optimal allocation vector, At'T' we can substitute this expression back into Eq. (8.2.3) to find the optimal expected rate of capital growth for the Tth stage from the decision maker's point of view. We have 2*[ln Kd

=

In K kT - 1

+ ~ qkiT 1n(1 + a0.Tri)' «u

Since the log terms in Eq. (8.2.19) are zero for i (8.2.19) as

E

(8.2.19)

M2T' we can write (8.2.20)

Substituting Eq. (8.2.12) for 2*[ln K kT ]

= In K k T- 1 + ~

atiT

in Eq. (8.2.20) gives us

tlkiTln qkiT

ieM+ kT

+

~

qkiTln ri

ieM+ kT

(8.2.21 )

Using Eq. (8.2.18), we can simplify Eq. (8.2.21) to 2*[ln K kT ]

=

In K kT- 1

I

+ UkT ~ ieM: T

PkiTlnpkiT

+ In UkT

8.2

135

THE ADAPTIVE CAPITAL ALLOCATION PROCESS

Cancelling terms, introducing notation similar to that used in Section 7.4 and writing in dynamic programming context, we have fl(K kT-1) = In K kT _ 1 + Gk(R T I R T-1 ... R1){Hk*(R T I R T-1 '" R 1) - Hk(R T I R T-1 ... R 1)

+ l\(R T I R T-1 ... R 1)}

(8.2.23)

where we define Gk(R T I R T-1 ... R 1) =

~ qkiT, ieM+kT

(8.2.24) (8.2.25) (8.2.26)

and l\(R T I R T-1

...

R 1)

= ~

PkiT In Ti

+ In (1 +

ieM~T

~ I/Ti )

(8.2.27)

!.eM~T

-In mtT' We have now prepared the way for the consideration of a two-stage process starting with stage T - 1. We have from the decision maker's point of view (8.2.28) where the last terms in the summation are the expectations of growth in the next stage (stage T) from the decision maker's point of view and conditioned on the effect of the current event, i.e., ZT-l' on the next stage. We have for these terms

+ Gk(R T I R T-

R 1 ), IZT~l=r;{Hk*(RT I R T-1

- Hk(R T I R T-1

R1)

1

IZT_1-rj

...

+ Yk(R T I R T-

R 1 ) !zT_l=r;

1 ...

R 1)

IzT_1> 0,

j

E

J,

k E K,

(10.2.13)

elsewhere.

=0,

We can maximize the log of Eq. (10.2.13) by the variation of ijjkt' under the constraints given above, by the Lagrangian technique. Assuming Q and each ijjkl to be large numbers, and using Sterling's formula, we have for the log of Eq. (10.2.13), In Pr[{qikt}] ~ -Q

+ Q InQ -k

iEJ,kEK

qikt In qikt +

k

qikt - Q In Q.

iEJ,kEK

By reduction, we find that In Pr[{gikt}] ~ -

k

jEJ,kEK

qjkt In gild·

(10.2.14)

10.2

175

THE STATE SPACE PROBABILITY FUNCTION

The Lagrangian is given by L

= -

~

qjkt

In qjkt

A ~

-

jEJ,keK

- QfL

qjkt

jEJ,kEK

~

Piqjkt

i£M,l EJ

In(l

+ AQ

+ ai;tbkri) + fLGj

(10.2.15)

UK

where Aand fL are Lagrangian multipliers. Noting that the a'tj/s are independent of the variations of qjkl, we take the partial differential of z with respect to the qjk/S, A, and fL. The partials are evaluated at qf,., where they equal zero. We have

for each

k

E

K, j

E

J; (10.2.17)

and

I

T8L _ = - Q1 ~ ~ fL q;kl iEM, jEJ

-*

Piqjkt

1n(l

* bkri ) + G*t + a,;t+l

= O.

(10.2.18)

kEK

This result can be written as exp [-fLIQ ~ Pi In(l tEM

+ atJtb~j)]

This equation can be put in terms of the entropy corresponding to the jth information state for the a'tj/s We have that exp[ -(fLIQ)Hjt]

~ exp[ -(fLIQ)Ht t]

LEJ

C

(10.2.20) k

176

10.

INTERACTIONS BETWEEN DECISION MAKERS IN STATE SPACE

where

c k -

exp [-(p.IQ) In

(b + ~ +)] k

'EM'

~ exp [ _(p.IQ) In (bJs + ~ _1)]

~EK

!EM

(10.2.21)

r,

Equation (10.2.20) or (10.2.19) is a member of the Maxwell-Boltzmann class of probability functions. They show us that it is highly unlikely that a large number of decision makers can be found realizing large capital gains and that it is unlikely that just a few decision makers can be found realizing small capital gains. We note that the differences in capital growth for the same amount of capital initially are caused by the differences in the information state of the decision makers. The adaptive process will work to reduce these differences. The equations show us that if the adaptive process could be completed (the state of enlightenment), the differences in expected capital growth would be only due to the initial capital possessed by each decision maker. Further, note that the parameter p. is the same for all members of the economic system. This is because the constraint (10.2.12) is for the whole system. An economic system, then, could be defined as being composed of those decision makers who face the same collective constraints. The parameter fL is so far undetermined. In Section 9.7, a method was presented for the determination of a similar parameter. We find that, given the system's expected rate of capital growth, and by virtue of Khinchin's theorem (see Section 9.7), there exists a unique value of fL which is compatible with constraint (l 0.2. 12). We denote this value of fL as fL *. This value of fL is found from the economic characteristic function for the system, ,p(fL) (see Section 9.7). When the partial of !f;(fL) with respect to fL equals the negative of the system's expected rate of capital growth, the value of fL at this point is fL *. With the most likely value of ijjkt now on hand, the growth of the system in the stage can be found. We have for this growth, (10.2.22)

Before turning to the market trading process and statistical equilibrium, we will find the contribution to the above growth by a single decision

10.3

STOCHASTIC EQUILIBRIUM IN THE MARKET

177

maker in the jkth cell. This contribution would be the variation of Eq, (10.2.22) with respect to the number ijfkt. We have then

SCi

IjEJ.kt'K

= Q1

~ Pi 1n(l + atitbkTi)'

(10.2.23)

iEM

We will use this equation in Section 10.4 to develop the notion of a market trading process.

10.3 Stochastic Equilibrium in the Market Each decision maker in the economic system is in a specified capital state and a specified information state. We have seen that each information state is associated with a particular capital allocation vector (a portfolio). Suppose that the payoff events generated by the environment change the subjective probabilities and thus change the information states of the decision makers. Are we sure that the necessary securities can be traded to assure that the new optimum portfolio can be obtained for each decision maker? There appears to be a possibility that in a disorderly market the two or more decision makers needed to consummate a transaction could not find each other. Ultimately, the price mechanism would be pressed into action to bring the market into equilibrium. But if all the decision makers could find the necessary partners to consummate their transactions, there would be no price changes, since the market would be cleared. We see that the probability of one decision maker finding another is an essential part of the market clearing mechanism. Examples of the need for order are clearly seen in the security exchanges where specialists in particular securities are fixed in location to facilitate the trading operation. Most markets are not so well organized, and disorder is quite pronounced. For example, consider the market for used automobiles. The probability that there are more Ford/Chevrolet deals is higher than the probability of EMF/Wasp deals simply because there are far more Ford and Chevrolet owners than there are EMF and Wasp owners. The probability of a transaction is likely to be proportional to the number of pairs of decision makers seeking that transaction. In order to determine these probabilities, let us assign an optimum portiolio to each point in state space. Let the jth state, and kth capital state, be associated with a particular portfolio, given by (10.3.1)

178

10.

INTERACTIONS BETWEEN DECISION MAKERS IN STATE SPACE

where w~i means that W~i capital units of the ith kind of investment are possessed by a decision maker with portfolio ~. Let us suppose that there is a finite probability that the quantity of any investment will be a particular magnitude. This probability is defined by for each

iEM.

(10.3.2)

Since the ~th portfolio is made up of certain quantities of the m investments in the magnitudes {W'i}' the probability of finding a decision maker with the ~th portfolio is the product of the probabilities that the portfolio will contain the specified quantities, {wd. This probability is

I, = IIpr[w,;] iEM

(10.3.3)

Suppose the ~th portfolio is one-step communicable with some other portfolio, say the gth, by the exchange of a specified quantity of security u for security v. Suppose also the 1]th portfolio can be transformed into the pth portfolio by the transfer or the same quantity of-security v for u. Then, if a transaction were completed between the two decision makers, one with the ~th portfolio and the other with the 1]th portfolio, the decision maker who had ~ would now have g; and, the decision maker who had 1] would now have p, We will assume that the probability of such a deal is proportional to the product of the probabilities of there being decision makers with the ~th and gth kinds of portfolios. We have Pr[decision makers shifting from the {th to the portfolios] =

1.1'1 .

~th

and 7Jth to pth (10.3.4)

Furthermore, we assume that the greater the probability of this event the greater is the probable change in the expected number of decision makers holding these portfolios. Thus we have di}, ~dt

=rxld'f/Q

(10.3.5)

~ ~'f/

=rxld'f/Q

(10.3.6)

10.3

179

STOCHASTIC EQUILIBRIUM IN THE MARKET

(10.3.7) (10.3.8)

where fie, for example, is the expected number of decision makers holding the ~th kind of portfolio and ex is a constant of proportionality. Such a market has a statistical parameter, which we shall call the market entropy. The market entropy, H m , is a measure of the uncertainty in the distribution of decision makers holding each kinds of portfolio. Starting at some initial condition, for example, where all Q decision makers hold certain portfolios with probability one (market entropy equal to zero), the decision makers carry out transactions on a probabilistic basis. After the first transaction the probability for finding a decision maker holding in a particular portfolio ceases to be one or zero and the market entropy increases. The problem is to prove entropy will increase and to show when it will stop increasing and become a constant at its maximum. We define a system possessing a constant entropy as being in statistical equilibrium. The market entropy is given by Hm

= -

2.f, , In / ,

(10.3.9)

where ~ is the portfolio index number. Taking the differential of H with respect to time we find that dH m = dt

Now

-2., dl,dt In/, - 2., dhdt .

2.1, , = 1,

and so

therefore dH m dt

=

2.,dt df, In/, .

(10.3.10)

Assume that we let only one transaction of the type described above be made per stage. For- this transaction the change in market entropy is given by only dH m

&

I

~E

'1... p

= -

dh In', _ dl" Inf, _ diE InlF

&

J.

&

"

&

.

_

dip Int.

&

p

(10.3.11)

180

10.

INTERACTIONS BETWEEN DECISION MAKERS IN STATE SPACE

the rest of the sum (10.3.10) being zero. We denote as representing a transaction where decision maker A moves from portfolio ~ to portfolio g, and decision maker B moves from portfolio TJ to portfolio p.

Furthermore, we know that the expected number of decision makers holding each portfolio is given by ijr"

= Qfr",

(10.3.12)

ij~

=

(10.3.13)

ij,

= Qf"

ijp

=

Qf~,

(10.3.14)

and

Qfp.

(10.3.15)

Taking the differential of the above four equations using Eqs. (10.3.5)(10.3.8), and substituting the result into Eq. (10.3.11), we have

d~m /c. e = + (xjd~ Inf~ + ctld~ Inf~ - ctld~ Infp ~

p

-

ctfd~ In Ie

'

which may be rewritten dHm I I Id~ T r".... e = ct)Jj, r" ~ n f, I" • ,)p

(10.3.16)

~~p

Since the market is free (all portfolios are communicable), there is a finite probability of a reverse transaction occuring; i.e., a decision maker with portfolio g making a transaction with decision makers with portfolios p and obtaining portfolios ~ and 'Y), respectively. By the same argument as before, we find

ar; Ip ....~ = T

I. I fdp

< p n f j, .

,I"

ct)