Fundamentals of Stochastic Networks

FUNDAMENTALS OF STOCHASTIC NETWORKS

FUNDAMENTALS OF STOCHASTIC NETWORKS OLIVER C. IBE University of Massachusetts Lowell, Massachusetts

A JOHN WILEY & SONS, INC., PUBLICATION

Copyright © 2011 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Ibe, Oliver C. (Oliver Chukwudi), 1947– â•… Fundamentals of stochastic networks / Oliver C. Ibe. â•…â•…â•… p. cm. â•… Includes bibliographical references and index. â•… ISBN 978-1-118-06567-9 (cloth) â•… 1.╇ Queuing theory.â•… 2.╇ Stochastic analysis.â•… I.╇ Title. â•… QA274.8.I24 2011 â•… 519.2'2–dc22 2011007713 Printed in the United States of America oBook ISBN: 9781118092972 ePDF ISBN: 9781118092996 ePub ISBN: 9781118092989 10╇ 9╇ 8╇ 7╇ 6╇ iv5╇ 4╇ 3╇ 2╇ 1

CONTENTS

Preface

xi

Acknowledgments,  xii 1â•… Basic Concepts in Probability

1

1.1 Introduction,  1 1.2 Random Variables,  1 1.2.1 Distribution Functions,  2 1.2.2 Discrete Random Variables,  3 1.2.3 Continuous Random Variables,  3 1.2.4 Expectations,  4 1.2.5 Moments of Random Variables and the Variance,  4 1.3 Transform Methods,  5 1.3.1 The s-Transform,  5 1.3.2 Moment-Generating Property of the s-Transform,  5 1.3.3 The z-Transform,  6 1.3.4 Moment-Generating Property of the z-Transform,  7 1.4 Covariance and Correlation Coefficient,  8 1.5 Sums of Independent Random Variables,  8 1.6 Random Sum of Random Variables,  9 1.7 Some Probability Distributions,  11 1.7.1 The Bernoulli Distribution,  11 1.7.2 The Binomial Distribution,  12 1.7.3 The Geometric Distribution,  13 1.7.4 The Pascal Distribution,  13 1.7.5 The Poisson Distribution,  14 1.7.6 The Exponential Distribution,  14 1.7.7 The Erlang Distribution,  15 1.7.8 The Uniform Distribution,  15 1.7.9 The Hyperexponential Distribution,  16 1.7.10  The Coxian Distribution,  17 v

vi

CONTENTS

1.7.11 The General Phase-Type Distribution,  19 1.7.12 Normal Distribution,  19 1.8 Limit Theorems,  21 1.8.1 Markov Inequality,  21 1.8.2 Chebyshev Inequality,  22 1.8.3 Law of Large Numbers,  22 1.8.4 The Central Limit Theorem,  23 2â•… Overview of Stochastic Processes

26

2.1 Introduction,  26 2.2 Classification of Stochastic Processes,  27 2.3 Stationary Random Processes,  27 2.3.1 Strict-Sense Stationary Processes,  27 2.3.2 WSS Processes,  28 2.4 Counting Processes,  28 2.5 Independent Increment Processes,  29 2.6 Stationary Increment Process,  29 2.7 Poisson Processes,  30 2.8 Renewal Processes,  32 2.8.1 The Renewal Equation,  33 2.8.2 The Elementary Renewal Theorem,  34 2.8.3 Random Incidence and Residual Time,  35 2.9 Markov Processes,  37 2.9.1 Discrete-Time Markov Chains,  38 2.9.1.1  State Transition Probability Matrix,  39 2.9.1.2 The n-Step State Transition Probability,  39 2.9.1.3 State Transition Diagrams,  41 2.9.1.4 Classification of States,  42 2.9.1.5 Limiting State Probabilities,  44 2.9.1.6 Doubly Stochastic Matrix,  47 2.9.2 Continuous-Time Markov Chains,  48 2.9.2.1 Birth and Death Processes, 51 2.9.2.2 Local Balance Equations, 54 2.10 Gaussian Processes,  56 3â•… Elementary Queueing Theory 3.1 3.2 3.3 3.4 3.5

Introduction,  61 Description of a Queueing System,  61 The Kendall Notation,  64 The Little’s Formula,  65 The M/M/1 Queueing System,  66 3.5.1 Stochastic Balance,  69 3.5.2 Total Time and Waiting Time Distributions of the   M/M/1 Queueing System,  70

61

CONTENTS

vii

3.6 Examples of Other M/M Queueing Systems,  71 3.6.1 The M/M/c Queue: The c-Server System,  72 3.6.2 The M/M/1/K Queue: The Single-Server   Finite-Capacity System,  74 3.6.3 The M/M/c/c Queue: The c-Server Loss System,  76 3.6.4 The M/M/1//K Queue: The Single Server Finite   Customer Population System,  77 3.7 M/G/1 Queue,  79 3.7.1 Waiting Time Distribution of the M/G/1 Queue,  81 3.7.2 The M/Ek/1 Queue,  84 3.7.3 The M/D/1 Queue,  85 3.7.4 The M/M/1 Queue,  86 3.7.5 The M/Hk/1 Queue,  86 4â•… Advanced Queueing Theory

93

4.1 Introduction,  93 4.2 M/G/1 Queue with Priority,  93 4.2.1,  Nonpreemptive Priority,  94 4.2.2 Preemptive-Resume Priority,  96 4.2.3 Preemptive-Repeat Priority,  97 4.3 G/M/1 Queue,  99 4.3.1 The Ek/M/1 Queue,  103 4.3.2 The D/M/1 Queue,  104 4.3.3 The H2/M/1 Queue,  104 4.4 The G/G/1 Queue,  105 4.4.1 Lindley’s Integral Equation,  106 4.4.2 Laplace Transform of FW(w),  107 4.4.3 Bounds of Mean Waiting Time,  109 4.5 Special Queueing Systems,  109 4.5.1 M/M/1 Vacation Queueing Systems with   Exceptional First Vacation,  109 4.5.2 M/M/1 Threshold Queueing Systems,  115 5â•… Queueing Networks 5.1 5.2 5.3 5.4 5.5

Introduction,  124 Burke’s Output Theorem and Tandem Queues,  126 Jackson or Open Queueing Networks,  128 Closed Queueing Networks,  130 BCMP Networks,  132 5.5.1 Routing Behavior,  132 5.5.2 Service Time Distributions,  133 5.5.3 Service Disciplines,  134 5.5.4 The BCMP Theorem,  134

124

viii

CONTENTS

5.6 Algorithms for Product-Form Queueing Networks,  138 5.6.1 The Convolution Algorithm,  138 5.6.2 The MVA,  142 5.7 Networks with Positive and Negative Customers,  144 5.7.1 G-Networks with Signals,  145 5.7.2 Extensions of the G-Network,  146 6â•… Approximations of Queueing Systems and Networks

150

6.1 Introduction,  150 6.2 Fluid Approximation,  151 6.2.1 Fluid Approximation of a G/G/1 Queue,  151 6.2.2 Fluid Approximation of a Queueing Network,  155 6.3 Diffusion Approximations,  155 6.3.1 Diffusion Approximation of a G/G/1 Queue,  156 6.3.2 Brownian Approximation for a G/G/1 Queue,  159 6.3.2.1 Brownian Motion with Drift,  161 6.3.2.2 Reflected Brownian Motion,  161 6.3.2.3 Scaled Brownian Motion,  162 6.3.2.4 Functional Central Limit Theorem,  163 6.3.2.5 Brownian Motion Approximation of the   G/G/1 Queue,  163 6.3.3 Diffusion Approximation of Open Queueing   Networks,  165 6.3.4 Diffusion Approximation of Closed Queueing Networks,  168 7â•… Elements of Graph Theory 7.1 Introduction,  172 7.2 Basic Concepts,  172 7.2.1 Subgraphs and Cliques,  174 7.2.2 Adjacency Matrix,  175 7.2.3 Directed Graphs,  175 7.2.4 Weighted Graphs,  176 7.3 Connected Graphs,  177 7.4 Cut Sets, Bridges, and Cut Vertices,  177 7.5 Euler Graphs,  178 7.6 Hamiltonian Graphs,  178 7.7 Trees and Forests,  179 7.8 Minimum Weight Spanning Trees,  181 7.9 Bipartite Graphs and Matchings,  182 7.9.1 The Hungarian Algorithm,  184 7.10 Independent Set, Domination, and Covering,  186 7.11 Complement of a Graph,  188 7.12 Isomorphic Graphs,  188

172

CONTENTS

ix

7.13 Planar Graphs,  189 7.13.1 Euler’s Formula for Planar Graphs,  190 7.14 Graph Coloring,  191 7.14.1 Edge Coloring,  191 7.14.2 The Four-Color Problem,  192 7.15 Random Graphs,  192 7.15.1 Bernoulli Random Graphs,  192 7.15.1.1 Phase Transition,  193 7.15.2 Geometric Random Graphs,  193 7.15.3 Markov Random Graph,  194 7.16 Matrix Algebra of Graphs,  195 7.16.1 Adjacency Matrix,  196 7.16.2 Connection Matrix,  197 7.16.3 Path Matrix,  197 7.16.4 Laplacian Matrix,  198 7.17 Spectral Properties of Graphs,  198 7.17.1 Spectral Radius,  200 7.17.2 Spectral Gap,  200 7.18 Graph Entropy,  201 7.19 Directed Acyclic Graphs,  201 7.20 Moral Graphs,  202 7.21 Triangulated Graphs,  202 7.22 Chain Graphs,  203 7.23 Factor Graphs,  204 8â•… Bayesian Networks

209

8.1 8.2 8.3 8.4 8.5

Introduction,  209 Bayesian Networks,  210 Classification of BNs,  214 General Conditional Independence and d-Separation,  215 Probabilistic Inference in BNs,  215 8.5.1 The Sum-Product Algorithm,  217 8.5.2 The Junction Tree Algorithm,  221 8.5.2.1 Belief Propagation in a Junction Tree,  225 8.6 Learning BNs,  227 8.6.1 Parameter Learning,  227 8.6.1.1 Maximum Likelihood Estimation  228 8.6.1.2 Maximum A Posteriori Estimation,  230 8.6.2 Structure Learning,  231 8.7 Dynamic Bayesian Networks,  231 9â•… Boolean Networks 9.1 Introduction,  235 9.2 Introduction to GRNs,  236

235

x

CONTENTS

9.3 9.4 9.5 9.6 9.7

Boolean Network Basics,  236 Random Boolean Networks,  238 State Transition Diagram,  239 Behavior of Boolean Networks,  240 Petri Net Analysis of Boolean Networks,  242 9.7.1 Introduction to PNs,  242 9.7.2 Behavioral Properties of PNs,  245 9.7.3 PN Model of Boolean Networks,  246 9.8 Probabilistic Boolean Networks,  250 9.9 Dynamics of a PBN,  251 9.10 Advantages and Disadvantages of Boolean Networks,  252 10â•… Random Networks

255

10.1 Introduction,  255 10.2 Characterization of Complex Networks,  256 10.2.1 Degree Distribution,  256 10.2.2 Geodesic Distances,  257 10.2.3 Centrality Measures,  257 10.2.4 Clustering,  258 10.2.5 Network Entropy,  259 10.2.6 Percolation and the Emergence of Giant   Component,  259 10.3 Models of Complex Networks,  261 10.3.1 The Small-World Network,  261 10.3.2 Scale-Free Networks,  263 10.4 Random Networks,  265 10.4.1 Degree Distribution,  265 10.4.2 Emergence of Giant Component,  266 10.4.3 Connectedness and Diameter,  266 10.4.4 Clustering Coefficient,  267 10.4.5 Scale-Free Properties,  267 10.5 Random Regular Networks,  267 10.6 Consensus over Random Networks,  268 10.6.1 Consensus over Fixed Networks,  270 10.6.1.1 Time to Convergence in a Fixed Network,  272 10.6.2 Consensus over Random Networks,  273 10.7 Summary,  274 References

276

Index

280

PREFACE

This book brings into one volume two network models that can be broadly classified as queueing network models and graphical network models. Queueing networks are systems where customers move among service stations where they receive service. Usually, the service times and the order in which customers visit the service stations are random. The order in which service is received at the service stations is governed by a probabilistic routing schedule. Queueing networks are popularly used in traffic modeling in computer and telecommunications networks, transportation systems, and manufacturing networks. Graphical models are systems that use graphs to model different types of problems. They include Bayesian networks, which are also called directed graphical models, Boolean networks, and random networks. Graphical models are used in statistics, data mining, and social networks. The need for a book of this nature arises from the fact that we live in an era of interdisciplinary studies and research activities when both networks are becoming important in areas that they were not originally used. Thus, any person involved in such interdisciplinary studies or research activities needs to have a good understanding of both types of networks. This book is intended to meet this need. The book is organized into three parts. The first part, Chapters 1 and 2, deals with the basic concepts of probability (Chapter 1) and stochastic processes (Chapter 2). The second part, Chapters 3–6, deals with queueing systems. Specifically, Chapter 3 deals with basic queueing theory, particularly a class of queueing systems that we refer to as Markovian queueing systems. Chapter 4 deals with advanced queueing systems, particularly the non-Markovian queueing systems. Chapter 5 deals with queueing networks, and Chapter 6 deals with approximations of queueing networks. The third part, Chapters 7–10, deals xi

xii

Preface Traffic Engineering Transportation Networks Manufacturing Networks

1

3

4

5

6 10

2 Expert Systems Statistics Social Sciences

7

8

9

Figure 1â•… Precedence relations of chapters.

with graphical models. Chapter 7 deals with an introduction to graph theory, Chapter 8 deals with Bayesian networks, Chapter 9 deals with Boolean networks, and Chapter 10 deals with random networks. The book is self-contained and is written with a view to circumventing the proof–theorem format that is traditionally used in stochastic systems modeling books. It is intended to be an introductory graduate text on stochastic networks and presents the basic results without much emphasis on proving theorems. Thus, it is designed for science and engineering applications. Students who have an interest in traffic engineering, transportation, and manufacturing networks will need to cover parts 1 and 2 as well as Chapter 10 in part 3, while students with an interest in expert systems, statistics, and social sciences will need to cover parts 1 and 3. The precedence relations among the chapters are shown in Figure 1. ACKNOWLEDGMENTS I would like to express my sincere gratitude to my wife, Christie, for bearing with my writing yet another book. She has been my greatest fan when it comes to my writing books. I would also like to acknowledge the encouraging words from our children Chidinma, Ogechi, Amanze, and Ugonna. I would like to express my sincere gratitude to my editor, Susanne Steitz-Filler, for her encouragement and for checking on me regularly to make sure that we met the deadlines. Finally, I would like to thank the anonymous reviewers for their useful comments and suggestions that helped to improve the quality of the book. Oliver C. Ibe Lowell, Massachusetts January 2011

1 BASIC CONCEPTS IN PROBABILITY

1.1  INTRODUCTION The concepts of experiments and events are very important in the study of probability. In probability, an experiment is any process of trial and observation. An experiment whose outcome is uncertain before it is performed is called a random experiment. When we perform a random experiment, the collection of possible elementary outcomes is called the sample space of the experiment, which is usually denoted by Ω. We define these outcomes as elementary outcomes because exactly one of the outcomes occurs when the experiment is performed. The elementary outcomes of an experiment are called the sample points of the sample space and are denoted by wi, i╯=╯1, 2,╯.╯.╯.╯If there are n possible outcomes of an experiment, then the sample space is Ω╯=╯{w1, w2,╯.╯.╯.╯, wn}. An event is the occurrence of either a prescribed outcome or any one of a number of possible outcomes of an experiment. Thus, an event is a subset of the sample space. 1.2  RANDOM VARIABLES Consider a random experiment with sample space Ω. Let w be a sample point in Ω. We are interested in assigning a real number to each w╯∈╯Ω. A random variable, X(w), is a single-valued real function that assigns a real number, Fundamentals of Stochastic Networks, First Edition. Oliver C. Ibe. © 2011 John Wiley & Sons, Inc. Published 2011 by John Wiley & Sons, Inc.

1

2

Basic Concepts in Probability

called the value of X(w), to each sample point w╯∈╯Ω. That is, it is a mapping of the sample space onto the real line. Generally a random variable is represented by a single letter X instead of the function X(w). Therefore, in the remainder of the book we use X to denote a random variable. The sample space Ω is called the domain of the random variable X. Also, the collection of all numbers that are values of X is called the range of the random variable X. Let X be a random variable and x a fixed real value. Let the event Ax define the subset of Ω that consists of all real sample points to which the random variable X assigns the number x. That is,

Ax = {w X (w ) = x} = [ X = x ] .

Since Ax is an event, it will have a probability, which we define as follows: p = P [ Ax ] .



We can define other types of events in terms of a random variable. For fixed numbers x, a, and b, we can define the following:

[ X ≤ x ] = {w X (w ) ≤ x} , [ X > x ] = {w X (w ) > x} , < a [ X < b] = {w a < X (w ) < b} .

These events have probabilities that are denoted by • P[X╯≤╯x] is the probability that X takes a value less than or equal to x. • P[X╯>╯x] is the probability that X takes a value greater than x; this is equal to 1╯–╯P[X╯≤╯x]. • P[a╯