Markov Chains And Stochastic Stability (communications And Control Engineering)

**Markov Chains and Stochastic Stability** is part of the __Communications and Control Engineering Series__ (__CCES__) edited by Professors B.W. Dickinson, E.D. Sontag, M. Thoma, A. Fettweis, J.L. Massey and J.W. Modestino. The area of Markov chain theory and application has matured over the past 20 years into something more accessible and complete. It is of increasing interest and importance. This publication deals with the action of Markov chains on general state spaces. It discusses the theories and the use to be gained, concentrating on the areas of engineering, operations research and control theory. Throughout, the theme of stochastic stability and the search for practical methods of verifying such stability, provide a new and powerful technique. This does not only affect applications but also the development of the theory itself. The impact of the theory on specific models is discussed in detail, in order to provide examples as well as to demonstrate the importance of these models. **Markov Chains and Stochastic Stability** can be used as a textbook on applied Markov chain theory, provided that one concentrates on the main aspects only. It is also of benefit to graduate students with a standard background in countable space stochastic models. Finally, the book can serve as a research resource and active tool for practitioners. Front Matter....Pages i-xvi Front Matter....Pages 1-1 Heuristics....Pages 3-22 Markov Models....Pages 23-53 Transition Probabilities....Pages 54-79 Irreducibility....Pages 80-99 Pseudo-Atoms....Pages 100-125 Topology and Continuity....Pages 126-148 The Nonlinear State Space Model....Pages 149-169 Front Matter....Pages 171-171 Transience and Recurrence....Pages 173-199 Harris and Topological Recurrence....Pages 200-228 The Existence of π....Pages 229-254 Drift and Regularity....Pages 255-284 Invariance and Tightness....Pages 285-305 Front Matter....Pages 307-307 Ergodicity....Pages 309-329 f -Ergodicity and f -Regularity....Pages 330-353 Geometric Ergodicity....Pages 354-381 V -Uniform Ergodicity....Pages 382-409 Sample Paths and Limit Theorems....Pages 410-445 Positivity....Pages 446-464 Generalized Classification Criteria....Pages 465-491 Back Matter....Pages 493-552 Contents: Communication and Regeneration: Heuristics Markov Models Transition Probabilities Irreducibility Pseudo-atoms Topology and Continuity Nonlinear State Space Model Stability Structures: Transience and Recurrence Harris and Topological Recurrence The Existence of + Drift and Regularity Invariance and Tightness. Convergence: Ergodicity f-Ergodicity and f-Regularity Geometric Ergodicity V-Uniform Ergodicity Sample Paths and Limit Theorems Positivity Generalised Classification Criteria. Appendices: Mud Maps Testing for Stability Glossary of Model Assumptions Mathematical Background.