本书叙述深入浅出,涉及面广。主要内容有随机变量、条件概率及条件期望、离散及连续马尔可夫链、指数分布、泊松过程、布朗运动及平稳过程、更新理论及排队论等;也包括了随机过程在物理、生物、运筹、网络、遗传、经济、保险、金融及可靠性中的应用。特别是有关随机模拟的内容,给随机系统运行的模拟计算提供了有力的工具。除正文外,本书有约700道习题,其中带星号的习题还提供了解答。
本书可作为概率论与统计、计算机科学、保险学、物理学、社会科学、生命科学、管理科学与工程学等专业随机过程基础课教材。

1. Introduction to Probability Theory 1
1.1. Introduction 1
1.2. Sample Space and Events 1
1.3. Probabilities Defined on Events 4
1.4. Conditional Probabilities 7
1.5. Independent Events 10
1.6. Bayes' Formula 12
Exercises 15
References 21
2. Random Variables 23
2.1. Random Variables 23
2.2. Discrete Random Variables 27
2.2.1. The Bernoulli Random Variable 28
2.2.2. The Binomial Random Variable 29
2.2.3. The Geometric Random Variable 31
2.2.4. The Poisson Random Variable 32
2.3. Continuous Random Variables 34
2.3.1. The Uniform Random Variable 35
2.3.2. Exponential Random Variables 36
2.3.3. Gamma Random Variables 37
2.3.4. Normal Random Variables 37
2.4. Expectation of a Random Variable 38
2.4.1. The Discrete Case 38
2.4.2. The Continuous Case 41
2.4.3. Expectation of a Function of a Random Variable 43
2.5. Jointly Distributed Random Variables 47
2.5.1. Joint Distribution Functions 47
2.5.2. Independent Random Variables 51
2.5.3. Covariance and Variance of Sums of Random Variables 53
2.5.4. Joint Probability Distribution of Functions of Random Variables 61
2.6. Moment Generating Functions 64
2.6.1. The Joint Distribution of the Sample Mean and Sample Variance from a Normal Population 74
2.7. Limit Theorems 77
2.8. Stochastic Processes 83
Exercises 85
References 96
3. Conditional Probability and Conditional Expectation 97
3.1. Introduction 97
3.2. The Discrete Case 97
3.3. The Continuous Case 102
3.4. Computing Expectations by Conditioning 105
3.4.1. Computing Variances by Conditioning 117
3.5. Computing Probabilities by Conditioning 120
3.6. Some Applications 137
3.6.1. A List Model 137
3.6.2. A Random Graph 139
3.6.3. Uniform Priors, Polya's Urn Model, and Bose-Einstein Statistics 147
3.6.4. Mean Time for Patterns 151
3.6.5. The k-Record Values of Discrete Random Variables 155
3.7. An Identity for Compound Random Variables 158

3.7.1. Poisson Compounding Distribution 161
3.7.2. Binomial Compounding Distribution 163
3.7.3. A Compounding Distribution Related to the Negative Binomial 164
Exercises 165
4. Markov Chains 185
4.1. Introduction 185
4.2. Chapman-Kolmogorov Equations 189
4.3. Classification of States 193
4.4. Limiting Probabilities 204
4.5. Some Applications 217
4.5.1. The Gambler's Ruin Problem 217
4.5.2. A Model for Algorithmic Efficiency 221
4.5.3. Using a Random Walk to Analyze a Probabilistic Algorithm for the Satisfiability Problem 224
4.6. Mean Time Spent in Transient States 230
4.7. Branching Processes 233
4.8. Time Reversible Markov Chains 236
4.9. Markov Chain Monte Carlo Methods 247
4.10. Markov Decision Processes 252
4.11. Hidden Markov Chains 256
4.11.1. Predicting the States 261
Exercises 263
References 280
5. The Exponential Distribution and the Poisson Process 281
5.1. Introduction 281
5.2. The Exponential Distribution 282
5.2.1. Definition 282
5.2.2. Properties of the Exponential Distribution 284
5.2.3. Further Properties of the Exponential Distribution 291
5.2.4. Convolutions of Exponential Random Variables 298
5.3. The Poisson Process 302
5.3.1. Counting Processes 302
5.3.2. Definition of the Poisson Process 304
5.3.3. Interarrival and Waiting Time Distributions 307
5.3.4. Further Properties of Poisson Processes 310
5.3.5. Conditional Distribution of the Arrival Times 316
5.3.6. Estimating Software Reliability 328
5.4. Generalizations of the Poisson Process 330
5.4.1. Nonhomogeneous PoissonProcess 330
5.4.2. Compound Poisson Process 337
5.4.3. Conditional or Mixed Poisson Processes 343
Exercises 346
References 364
6. Continuous-Time Markov Chains 365
6.1. Introduction 365
6.2. Continuous-Time Markov Chains 366
6.3. Birth and Death Processes 368
6.4. The Transition Probability Function eij (t) 375
6.5. Limiting Probabilities 384
6.6. Time Reversibility 392
6.7. Uniformization 401

6.8. Computing the Transition Probabilities 404
Exercises 407
References 415
7. Renewal Theory and Its Applications 417
7.1. Introduction 417
7.2. Distribution of N(t) 419
7.3. Limit Theorems and Their Applications 423
7.4. RenewalReward Processes 433
7.5. Regenerative Processes 442
7.5.1. Alternating Renewal Processes 445
7.6. Semi-Markov Processes 452
7.7. The Inspection Paradox 455
7.8. Computing the Renewal Function 458
7.9. Applications to Patterns 461
7.9.1. Patterns of Discrete Random Variables 462
7.9.2. The Expected Time to a Maximal Run of Distinct Values 469
7.9.3. Increasing Runs of Continuous Random Variables 471
7.10. The Insurance Ruin Problem 473
Exercises 479
References 492
8. Queueing Theory 493
8.1. Introduction 493
8.2. Preliminaries 494
8.2.1. Cost Equations 495
8.2.2. Steady-State Probabilities 496
8.3. Exponential Models 499
8.3.1. A Single-Server Exponential Queueing System 499
8.3.2. A Single-Server Exponential Queueing System Having Finite Capacity 508
8.3.3. A Shoeshine Shop 511
8.3.4. A Queueing System with Bulk Service 514
8.4. Network of Queues 517
8.4.1. Open Systems 517
8.4.2. Closed Systems 522
8.5. The System M/G/1 528
8.5.1. Preliminaries: Work and Another Cost Identity 528
8.5.2. Application of Work to M/G/1 529
8.5.3. Busy Periods 530
8.6. Variations on the M/G/1 531
8.6.1. The M/G/1 with Random-Sized Batch Arrivals 531
8.6.2. Priority Queues 533
8.6.3. An M/G/10ptimizationExample 536
8.6.4. The M/G/1 Queue with Server Breakdown 540
8.7. The Model G/M/1 543
8.7.1. The G/M/1 Busy and Idle Periods 548
8.8. A Finite Source Model 549
8.9. Multiserver Queues 552
8.9.1. Erlang's Loss System 553
8.9.2. The M/M/k Queue 554
8.9.3. The G/M/k Queue 554
8.9.4. The M/G/k Queue 556

Exercises 558
References 570
9. Reliability Theory 571
9.1. Introduction 571
9.2. Structure Functions 571
9.2.1. Minimal Path and Minimal Cut Sets 574
9.3. Reliability of Systems of Independent Components 578
9.4. Bounds on the Reliability Function 583
9.4.1. Method of Inclusion and Exclusion 584
9.4.2. Second Method for Obtaining Bounds on r(p) 593
9.5. System Life as a Function of Component Lives 595
9.6. Expected System Lifetime 604
9.6.1. An Upper Bound.on the Expected Life of a Parallel System 608
9.7. Systems with Repair 610
9.7.1. A Series Model with Suspended Animation 615
Exercises 617
References 624
10. Brownian Motion and Stationary Processes 625
10.1. Brownian Motion 625
10.2. Hitting Times, Maximum Variable, and the Gambler's Ruin Problem 629
10.3. Variations on Brownian Motion 631
10.3.1. Brownian Motion with Drift 631
10.3.2. Geometric Brownian Motion 631
10.4. Pricing Stock Options 632
10.4.1. An Example in Options Pricing 632
10.4.2. The Arbitrage Theorem 635
10.4.3. The Black-Scholes Option Pricing Formula 638
10.5. White Noise 644
10.6. GaussianProcesses 646
10.7. Stationary and Weakly Stationary Processes 649
10.8. Harmonic Analysis of Weakly Stationary Processes 654
Exercises 657
References 662
11. Simulation 663
11.1. Introduction 663
11.2. General Techniques for Simulating Continuous Random Variables 668
11.2.1. The Inverse Transformation Method 668
11.2.2. The Rejection Method 669
11.2.3. The Hazard Rate Method 673
11.3. Special Techniques for Simulating Continuous Random Variables 677
11.3.1. The Normal Distribution 677
11.3.2. The Gamma Distribution 680
11.3.3. The Chi-Squared Distribution 681
11.3.4. The Beta (n,m) Distribution 681
11.3.5. The Exponential Distribution The Von Neumann Algorithm 682
11.4. Simulating from Discrete Distributions 685
11.4.1. The Alias Method 688
11.5. Stochastic Processes 692
11.5.1. Simulating a Nonhomogeneous Poisson Process 693
11.5.2. Simulating a Two-Dimensional Poisson Process 700

11.6. Variance Reduction Techniques 703
11.6.1. Use of Antithetic Variables 704
11.6.2. Variance Reduction by Conditioning 708
11.6.3. Control Variates 712
11.6.4. Importance Sampling 714
11.7. Determining the Number of Runs 720
11.8. Coupling from the Past 720
Exercises 723
References 731
Appendix: Solutions to Starred Exercises 733
Index 775