Stochastic Processes: Theory, Examples & Exercises is an introductory textbook to a central and active field of Mathematics with applications in Engineering, Economics, and Management Science.
It starts with an overview of the field and a detailed account of the concept of stochastic process. Following this, the text delves into Poisson processes, renewal processes, and Markov chains in discrete and continuous time.
The author aims to explain the theory very clearly and show how it can be applied to solve concrete problems.
A critical and unique feature of this well-written book is the structure of its exercises. They are embedded in the main text and have spaces to fill. This feature helps the students to stop and reflect on what they have just read and encourages them to become active during the educational process.
This “workbook” is mathematically accurate yet avoids unnecessary mathematical sophistication, making it suitable for undergraduate and
graduate students in Mathematics, Computer Science, Engineering, and Management.
0 INTRODUCTION TO STOCHASTIC PROCESSES 1
0.1 Stochastic processes and their characterization 4
0.2 A pivotal characteristic of some stochastic processes 13
0.3 A few examples of stochastic processes 21
1 POISSON PROCESSES 39
1.1 Properties of the exponential distribution 42
1.2 Poisson process: three definitions 56
1.3 Event times in Poisson processes 68
1.4 Merging and splitting Poisson processes 81
1.5 Non-homogeneous Poisson process 95
1.6 Conditional Poisson process 112
1.7 Compound Poisson process 120
2 RENEWAL PROCESSES 129
2.1 Introduction 132
2.2 Properties of the number of renewals 134
2.3 Renewal function 140
2.4 Renewal-type equations 150
2.5 Key renewal theorem and some other limit theorems 161
2.6 Recurrence times; the inspection paradox 182
2.7 Renewal reward processes 201
2.8 Alternating renewal processes 214
2.9 Delayed renewal processes 221
2.10 Regenerative processes 2313 DISCRETE-TIME MARKOV CHAINS 239
3.1 Definitions and examples 242
3.2 Chapman-Kolmogorov equations; marginal and joint distributions 247
3.3 Classification of states; recurrent and transient states 258
3.4 Asymptotic behaviour of irreducible Markov chains 280
3.5 Asymptotic behaviour of reducible Markov chains 292
3.6 Markov chains with costs/rewards 295
3.7 Reversible Markov chains 304
3.8 Branching processes 314
3.9 First passage times; absorption probabilities 334
4 CONTINUOUS-TIME MARKOV CHAINS 347
4.1 Definitions and examples 351
4.2 Properties of the transition matrix; Chapman-Kolmogorov equations 354
4.3 Computing the transition matrix: finite state space 367
4.4 Birth and death processes 369
4.5 Classification of states 387
4.6 Asymptotic behaviour 393
4.7 Birth and death queueing systems in equilibrium 405
4.7.1 Description of a queueing system 405
4.7.2 Performance measures 409
4.7.3 M/M/1, the single server queueing system 416
4.7.4 M/M/8, the queueing system with responsive servers 430
4.7.5 M/M/m, the m–server queueing system 439
4.7.6 M/M/m/m, the m–server loss queueing system 452
BIBLIOGRAPHY 463
HYPERLINKS 473
ACRONYMS 477
INDEX 479
FORMULAE 485
Antonis Economou
Professor, Mathematics
National and Kapodistrian University of Athens | Greece