Markov Processes for Stochastic Modeling

This book presents an algebraic development of the theory of countable state space Markov chains with discrete- and continuous-time parameters. A Markov chain is a stochastic process characterized by the Markov prop­ erty that the distribution of future depends only on the current state, not on the whole history. Despite its simple form of dependency, the Markov property has enabled us to develop a rich system of concepts and theorems and to derive many results that are useful in applications. In fact, the areas that can be modeled, with varying degrees of success, by Markov chains are vast and are still expanding. The aim of this book is a discussion of the time-dependent behavior, called the transient behavior, of Markov chains. From the practical point of view, when modeling a stochastic system by a Markov chain, there are many instances in which time-limiting results such as stationary distributions have no meaning. Or, even when the stationary distribution is of some importance, it is often dangerous to use the stationary result alone without knowing the transient behavior of the Markov chain. Not many books have paid much attention to this topic, despite its obvious importance. "Markov Processes for Stochastic Modeling presents a review of the author's more recent work in this active area of applied probability, together with an indication of where it links to established research. The book presents an algebraic development of the theory of countable state space Markov chains with discrete and continuous time parameters."--BOOK JACKET. "The emphasis is on time-dependent behavior, including first passage times of Markov chains. The book discusses measures of the speed of convergence, an algebraic discussion of monotone Markov chains and recent developments of quasi-stationary distributions. These features are complemented by numerous examples drawn from queueing, reliability and other models."--BOOK JACKET. "The book will be of particular interest to researchers in applied probability, mathematics, telecommunications, econometrics, genetics, epidemiology and electronic engineering, and will prove invaluable as a course text for graduates studying stochastic processes and stochastic modeling."--Jacket Front Matter....Pages i-x Introduction....Pages 1-23 Discrete-time Markov chains....Pages 25-100 Monotone Markov chains....Pages 101-165 Continuous-time Markov chains....Pages 167-241 Birth—death processes....Pages 243-293 Review of matrix theory....Pages 295-301 Generating functions and Laplace transforms....Pages 303-312 Total positivity....Pages 313-318 Back Matter....Pages 319-341