Numerical Analysis for Statisticians (Statistics and Computing)

Every advance in computer architecture and software tempts statisticians to tackle numerically harder problems. To do so intelligently requires a good working knowledge of numerical analysis. This book equips students to craft their own software and to understand the advantages and disadvantages of different numerical methods. Issues of numerical stability, accurate approximation, computational complexity, and mathematical modeling share the limelight in a broad yet rigorous overview of those parts of numerical analysis most relevant to statisticians. In this second edition, the material on optimization has been completely rewritten. There is now an entire chapter on the MM algorithm in addition to more comprehensive treatments of constrained optimization, penalty and barrier methods, and model selection via the lasso. There is also new material on the Cholesky decomposition, Gram-Schmidt orthogonalization, the QR decomposition, the singular value decomposition, and reproducing kernel Hilbert spaces. The discussions of the bootstrap, permutation testing, independent Monte Carlo, and hidden Markov chains are updated, and a new chapter on advanced MCMC topics introduces students to Markov random fields, reversible jump MCMC, and convergence analysis in Gibbs sampling. Numerical Analysis for Statisticians can serve as a graduate text for a course surveying computational statistics. With a careful selection of topics and appropriate supplementation, it can be used at the undergraduate level. It contains enough material for a graduate course on optimization theory. Because many chapters are nearly self-contained, professional statisticians will also find the book useful as a reference. Kenneth Lange is the Rosenfeld Professor of Computational Genetics in the Departments of Biomathematics and Human Genetics and the Chair of the Department of Human Genetics, all in the UCLA School of Medicine. His research interests include human genetics, population modeling, biomedical imaging, computational statistics, high-dimensional optimization, and applied stochastic processes. Springer previously published his books Mathematical and Statistical Methods for Genetic Analysis, 2nd ed., Applied Probability, and Optimization. He has written over 200 research papers and produced with his UCLA colleague Eric Sobel the computer program Mendel, widely used in statistical genetics. Front Matter....Pages I-XX Recurrence Relations....Pages 1-11 Power Series Expansions....Pages 13-25 Continued Fraction Expansions....Pages 27-38 Asymptotic Expansions....Pages 39-54 Solution of Nonlinear Equations....Pages 55-75 Vector and Matrix Norms....Pages 77-91 Linear Regression and Matrix Inversion....Pages 93-111 Eigenvalues and Eigenvectors....Pages 113-128 Singular Value Decomposition....Pages 129-142 Splines....Pages 143-155 Optimization Theory....Pages 157-188 The MM Algorithm....Pages 189-221 The EM Algorithm....Pages 223-247 Newton’s Method and Scoring....Pages 249-276 Local and Global Convergence....Pages 277-296 Advanced Optimization Topics....Pages 297-332 Concrete Hilbert Spaces....Pages 333-361 Quadrature Methods....Pages 363-377 The Fourier Transform....Pages 379-393 The Finite Fourier Transform....Pages 395-411 Wavelets....Pages 413-429 Generating Random Deviates....Pages 431-458 Independent Monte Carlo....Pages 459-476 Permutation Tests and the Bootstrap....Pages 477-501 Finite-State Markov Chains....Pages 503-526 Markov Chain Monte Carlo....Pages 527-550 Advanced Topics in MCMC....Pages 551-579 Back Matter....Pages 581-600 Numerical analysis is the study of computation and its accuracy, stability and often its implementation on a computer. This book focuses on the principles of numerical analysis and is intended to equip those readers who use statistics to craft their own software and to understand the advantages and disadvantages of different numerical methods.