Often, real-world problems modeled by Markov decision processes (MDPs) are difficult to solve in practise because of the curse of dimensionality. In others, explicit specification of the MDP model parameters is not feasible, but simulation samples are available. For these settings, various sampling and population-based numerical algorithms for computing an optimal solution in terms of a policy and/or value function have been developed recently. Here, this state-of-the-art research is brought together in a way that makes it accessible to researchers of varying interests and backgrounds. Many specific algorithms, illustrative numerical examples and rigorous theoretical convergence results are provided. The algorithms differ from the successful computational methods for solving MDPs based on neuro-dynamic programming or reinforcement learning. The algorithms can be combined with approximate dynamic programming methods that reduce the size of the state space and ameliorate the effects of dimensionality. Simulation-based Algorithms For Markov Decision Processes Brings, State-of-the-art Research Together For The First Time And Presents It In A Manner That Makes It Accessible To Researchers With Varying Interests And Backgrounds. In Addition To Providing Numerous Specific Algorithms, The Exposition Includes Both Illustrative Numerical Examples And Rigorous Theoretical Convergence Results. The Algorithms Developed And Analyzed Differ From The Successful Computational Methods For Solving Mdps Based On Neuro Dynamic Programming Or Reinforcement Learning And Will Complement Work In Those Areas. Furthermore, The Authors Show How To Combine The Various Algorithms Introduced With Approximate Dynamic Programming Methods That Reduce The Size Of The State Space And Ameliorate The Effects Of Dimensionality. The Self-contained Approach Of This Book Will Appeal Not Only To Researchers In Mdps, Stochastic Modeling And Control, And Simulation But Will Be A Valuable Source Of Instruction And Reference For Students Of Control And Operations Research.--book Jacket. 1. Markov Decision Processes -- 2. Multi-stage Adaptive Sampling Algorithms -- 3. Population-based Evolutionary Approaches -- 4. Model Reference Adaptive Search -- 5. On-line Control Methods Via Simulation. Hyeong Soo Chang ... [et Al.]. Includes Bibliographical References (. [177]-185) And Index. Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. This book brings the state-of-the-art research together for the first time. It provides practical modeling methods for many real-world problems with high dimensionality or complexity which have not hitherto been treatable with Markov decision processes. This volume brings together state-of-the-art research and presents it in amanner that makes it accessible to researchers with varying interests and backgrounds. In addition to providing numerous specific algorithms, the exposition includes both illustrated numerical examples and theoretical convergence results.
