Probability, Statistics, and Stochastic Processes

Thoroughly updated to showcase the interrelationships between probability, statistics, and stochastic processes, Probability, Statistics, and Stochastic Processes, Second Edition prepares readers to collect, analyze, and characterize data in their chosen fields. Beginning with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions, the book goes on to present limit theorems and simulation. The authors combine a rigorous, calculus-based development of theory with an intuitive approach that appeals to readers' sense of reason and logic. Including more than 400 examples that help illustrate concepts and theory, the Second Edition features new material on statistical inference and a wealth of newly added topics, including: Consistency of point estimators Large sample theory Bootstrap simulation Multiple hypothesis testing Fisher's exact test and Kolmogorov-Smirnov test Martingales, renewal processes, and Brownian motion One-way analysis of variance and the general linear model Extensively class-tested to ensure an accessible presentation, Probability, Statistics, and Stochastic Processes, Second Edition is an excellent book for courses on probability and statistics at the upper-undergraduate level. The book is also an ideal resource for scientists and engineers in the fields of statistics, mathematics, industrial management, and engineering. Table of Contents Preface xi Preface to the First Edition xiii 1 Basic Probability Theory 1 1.1 Introduction 1 1.2 Sample Spaces and Events 3 1.3 The Axioms of Probability 7 1.4 Finite Sample Spaces and Combinatorics 15 1.4.1 Combinatorics 17 1.5 Conditional Probability and Independence 27 1.6 The Law of Total Probability and Bayes’ Formula 41 Problems 63 2 Random Variables 76 2.1 Introduction 76 2.2 Discrete Random Variables 77 2.3 Continuous Random Variables 82 2.4 Expected Value and Variance 95 2.5 Special Discrete Distributions 111 2.6 The Exponential Distribution 123 2.7 The Normal Distribution 127 PROBABILITY, STATISTICS, AND STOCHASTIC PROCESSES......Page 1 CONTENTS......Page 7 Preface......Page 13 Preface to the First Edition......Page 15 1.1 Introduction......Page 17 1.2 Sample Spaces and Events......Page 19 1.3 The Axioms of Probability......Page 23 1.4 Finite Sample Spaces and Combinatorics......Page 31 1.4.1 Combinatorics......Page 33 1.5 Conditional Probability and Independence......Page 43 1.5.1 Independent Events......Page 49 1.6 The Law of Total Probability and Bayes’ Formula......Page 57 1.6.1 Bayes’ Formula......Page 63 1.6.2 Genetics and Probability......Page 70 1.6.3 Recursive Methods......Page 71 Problems......Page 79 2.1 Introduction......Page 92 2.2 Discrete Random Variables......Page 93 2.3 Continuous Random Variables......Page 98 2.3.1 The Uniform Distribution......Page 106 2.3.2 Functions of Random Variables......Page 108 2.4 Expected Value and Variance......Page 111 2.4.1 The Expected Value of a Function of a Random Variable......Page 116 2.4.2 Variance of a Random Variable......Page 120 2.5.1 Indicators......Page 127 2.5.2 The Binomial Distribution......Page 128 2.5.3 The Geometric Distribution......Page 132 2.5.4 The Poisson Distribution......Page 133 2.5.6 Describing Data Sets......Page 137 2.6 The Exponential Distribution......Page 139 2.7 The Normal Distribution......Page 143 2.8.1 The Lognormal Distribution......Page 147 2.8.2 The Gamma Distribution......Page 149 2.8.3 The Cauchy Distribution......Page 150 2.8.4 Mixed Distributions......Page 151 2.9 Location Parameters......Page 153 2.10 The Failure Rate Function......Page 155 2.10.1 Uniqueness of the Failure Rate Function......Page 157 Problems......Page 160 3.2 The Joint Distribution Function......Page 172 3.3 Discrete Random Vectors......Page 174 3.4 Jointly Continuous Random Vectors......Page 176 3.5 Conditional Distributions and Independence......Page 180 3.5.1 Independent Random Variables......Page 184 3.6.1 Real-Valued Functions of Random Vectors......Page 188 3.6.2 The Expected Value and Variance of a Sum......Page 192 3.6.3 Vector-Valued Functions of Random Vectors......Page 198 3.7 Conditional Expectation......Page 201 3.7.1 Conditional Expectation as a Random Variable......Page 205 3.7.2 Conditional Expectation and Prediction......Page 207 3.7.3 Conditional Variance......Page 208 3.7.4 Recursive Methods......Page 209 3.8 Covariance and Correlation......Page 212 3.8.1 The Correlation Coefficient......Page 217 3.9 The Bivariate Normal Distribution......Page 225 3.10 Multidimensional Random Vectors......Page 232 3.10.1 Order Statistics......Page 234 3.10.2 Reliability Theory......Page 239 3.10.3 The Multinomial Distribution......Page 241 3.10.4 The Multivariate Normal Distribution......Page 242 3.10.5 Convolution......Page 243 3.11.1 The Probability Generating Function......Page 247 3.11.2 The Moment Generating Function......Page 253 3.12 The Poisson Process......Page 256 3.12.1 Thinning and Superposition......Page 260 Problems......Page 263 4.1 Introduction......Page 279 4.2 The Law of Large Numbers......Page 280 4.3 The Central Limit Theorem......Page 284 4.3.1 The Delta Method......Page 289 4.4.1 Discrete Limits......Page 291 4.4.2 Continuous Limits......Page 293 Problems......Page 294 5.1 Introduction......Page 297 5.2 Random Number Generation......Page 298 5.3 Simulation of Discrete Distributions......Page 299 5.4 Simulation of Continuous Distributions......Page 301 5.5 Miscellaneous......Page 306 Problems......Page 308 6.2 Point Estimators......Page 310 6.2.1 Estimating the Variance......Page 318 6.3 Confidence Intervals......Page 320 6.3.1 Confidence Interval for the Mean in the Normal Distribution with Known Variance......Page 323 6.3.2 Confidence Interval for an Unknown Probability......Page 324 6.4.1 The Method of Moments......Page 328 6.4.2 Maximum Likelihood......Page 331 6.4.3 Evaluation of Estimators with Simulation......Page 338 6.4.4 Bootstrap Simulation......Page 340 6.5 Hypothesis Testing......Page 343 6.5.1 Large Sample Tests......Page 348 6.5.2 Test for an Unknown Probability......Page 349 6.6.1 P-Values......Page 350 6.6.2 Data Snooping......Page 351 6.6.3 The Power of a Test......Page 352 6.6.4 Multiple Hypothesis Testing......Page 354 6.7 Goodness of Fit......Page 355 6.7.1 Goodness-of-Fit Test for Independence......Page 362 6.7.2 Fisher’s Exact Test......Page 365 6.8 Bayesian Statistics......Page 367 6.8.1 Noninformative priors......Page 375 6.8.2 Credibility Intervals......Page 378 6.9.1 Nonparametric Hypothesis Testing......Page 379 6.9.2 Comparing Two Samples......Page 386 6.9.3 Nonparametric Confidence Intervals......Page 391 Problems......Page 394 7.1 Introduction......Page 407 7.2 Sampling Distributions......Page 408 7.3 Single Sample Inference......Page 411 7.3.1 Inference for the Variance......Page 412 7.3.2 Inference for the Mean......Page 415 7.4.1 Inference about Means......Page 418 7.4.2 Inference about Variances......Page 423 7.5.1 One-Way Analysis of Variance......Page 425 7.5.2 Multiple Comparisons: Tukey’s Method......Page 428 7.5.3 Kruskal–Wallis Test......Page 429 7.6 Linear Regression......Page 431 7.6.1 Prediction......Page 438 7.6.2 Goodness of Fit......Page 440 7.6.3 The Sample Correlation Coefficient......Page 441 7.6.4 Spearman’s Correlation Coefficient......Page 445 7.7 The General Linear Model......Page 447 Problems......Page 452 8.1 Introduction......Page 460 8.2 Discrete -Time Markov Chains......Page 461 8.2.1 Time Dynamics of a Markov Chain......Page 463 8.2.2 Classification of States......Page 466 8.2.3 Stationary Distributions......Page 470 8.3.1 The Simple Random Walk......Page 480 8.3.2 Multidimensional Random Walks......Page 484 8.3.3 Branching Processes......Page 485 8.4 Continuous -Time Markov Chains......Page 491 8.4.1 Stationary Distributions and Limit Distributions......Page 496 8.4.2 Birth–Death Processes......Page 500 8.4.3 Queueing Theory......Page 504 8.4.4 Further Properties of Queueing Systems......Page 507 8.5 Martingales......Page 510 8.5.1 Martingale Convergence......Page 511 8.5.2 Stopping Times......Page 513 8.6 Renewal Processes......Page 518 8.6.1 Asymptotic Properties......Page 520 8.7 Brownian Motion......Page 525 8.7.1 Hitting Times......Page 528 8.7.2 Variations of the Brownian Motion......Page 531 Problems......Page 533 Appendix A: Tables......Page 543 Appendix B: Answers to Selected Problems......Page 551 Further Reading......Page 567 Index......Page 569 "This book provides a unique and balanced approach to probability, statistics, and stochastic processes. Readers gain a solid foundation in all three fields that serves as a stepping stone to more advanced investigations into each area. The Second Edition features new coverage of analysis of variance (ANOVA), consistency and efficiency of estimators, asymptotic theory for maximum likelihood estimators, empirical distribution function and the Kolmogorov-Smirnov test, general linear models, multiple comparisons, Markov chain Monte Carlo (MCMC), Brownian motion, martingales, and renewal theory. Many new introductory problems and exercises have also been added. This book combines a rigorous, calculus-based development of theory with a more intuitive approach that appeals to readers' sense of reason and logic, an approach developed through the author's many years of classroom experience. The book begins with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions. The next two chapters introduce limit theorems and simulation. Also included is a chapter on statistical inference with a focus on Bayesian statistics, which is an important, though often neglected, topic for undergraduate-level texts. Markov chains in discrete and continuous time are also discussed within the book. More than 400 examples are interspersed throughout to help illustrate concepts and theory and to assist readers in developing an intuitive sense of the subject. Readers will find many of the examples to be both entertaining and thought provoking. This is also true for the carefully selected problems that appear at the end of each chapter"-- "This book provides a unique and balanced approach to probability, statistics, and stochastic processes. Readers gain a solid foundation in all three fields that serves as a stepping stone to more advanced investigations into each area. The Second Edition features new coverage of analysis of variance (ANOVA), consistency and efficiency of estimators, asymptotic theory for maximum likelihood estimators, empirical distribution function and the Kolmogorov-Smirnov test, general linear models, multiple comparisons, Markov chain Monte Carlo (MCMC), Brownian motion, martingales, and renewal theory"-- Machine generated contents note: Preface v1. Basic Probability Theory 11.1 Introduction 11.2 Sample Spaces and Events 31.3 The Axioms of Probability 71.4 Finite Sample Spaces and Combinatorics 161.5 Conditional Probability and Independence 291.6 The Law of Total Probability and Bayes' Formula 432. Random Variables 792.1 Introduction 792.2 Discrete Random Variables 812.3 Continuous Random Variables 862.4 Expected Value and Variance 992.5 Special Discrete Distributions 1152.6 The Exponential Distribution 1282.7 The Normal Distribution 1322.8 Other Distributions 1372.9 Location Parameters 1422.10 The Failure Rate Function 1453. Joint Distributions 1613.1 Introduction 1613.2 The Joint Distribution Function 1613.3 Discrete Random Vectors 1633.4 Jointly Continuous Random Vectors 1663.5 Conditional Distributions and Independence 1693.6 Functions of Random Vectors 1783.7 Conditional Expectation 1913.8 Covariance and Correlation 2023.9 The Bivariate Normal Distribution 2163.10 Multidimensional Random Vectors 2233.11 Generating Functions 2383.12 The Poisson Process 2484. Limit Theorems 2714.1 Introduction 2714.2 The Law of Large Numbers 2724.3 The Central Limit Theorem 2764.4 Convergence in Distribution 2835. Simulation 2895.1 Introduction 2895.2 Random-Number Generation 2905.3 Simulation of Discrete Distributions 2915.4 Simulation of Continuous Distributions 2935.5 Miscellaneous 2986. Statistical Inference 3036.1 Introduction 3036.2 Point Estimators 3036.3 Confidence Intervals 3116.4 Estimation Methods 3216.5 Hypothesis Testing 3366.6 Further Topics in Hypothesis Testing 3446.7 Goodness of Fit 3496.8 Bayesian Statistics 3616.9 Nonparametric Methods 3737. Linear Models 4017.1 Introduction 4017.2 Sampling Distributions 4027.3 Single Sample Inference 4067.4 Comparing Two Samples 4127.5 Analysis of Variance 4197.6 Linear Regression 4267.7 The General Linear Model 4438. Stochastic Processes 4558.1 Introduction 4558.2 Discrete-Time Markov Chains 4568.3 Random Walks and Branching Processes 4768.4 Continuous-Time Markov Chains 4888.5 Martingales 5068.6 Renewal Processes 5178.7 Brownian Motion 522Appendix A. Tables 541Appendix B. Answers to Selected Problems 551References 563Index 565 . A mathematical and intuitive approach to probability, statistics, and stochastic processes. This textbook provides a unique, balanced approach to probability, statistics, and stochastic processes. Readers gain a solid foundation in all three fields that serves as a stepping stone to more advanced investigations into each area. This text combines a rigorous, calculus-based development of theory with a more intuitive approach that appeals to readers' sense of reason and logic, an approach developed through the author's many years of classroom experience. The text begins with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions. The next two chapters introduce limit theorems and simulation. Also included is a chapter on statistical inference with a section on Bayesian statistics, which is an important, though often neglected, topic for undergraduate-level texts. Markov chains in discrete and continuous time are also discussed within the book. More than 400 examples are interspersed throughout the text to help illustrate concepts and theory and to assist the reader in developing an intuitive sense of the subject. Readers will find many of the examples to be both entertaining and thought provoking. This is also true for the carefully selected problems that appear at the end of each chapter. This book is an excellent text for upper-level undergraduate courses. While many texts treat probability theory and statistical inference or probability theory and stochastic processes, this text enables students to become proficient in all three of these essential topics. For students in science and engineering who may take only one course in probability theory, mastering all three areas will better prepare them to collect, analyze, and characterize data in their chosen fields This Book Provides A Unique And Balanced Approach To Probability, Statistics, And Stochastic Processes. Readers Gain A Solid Foundation In All Three Fields That Serves As A Stepping Stone To More Advanced Investigations Into Each Area. The Second Edition Features New Coverage Of Analysis Of Variance (anova), Consistency And Efficiency Of Estimators, Asymptotic Theory For Maximum Likelihood Estimators, Empirical Distribution Function And The Kolmogorov-smirnov Test, General Linear Models, Multiple Comparisons, Markov Chain Monte Carlo (mcmc), Brownian Motion, Martingales, And Renewal Theory. Many New Introductory Problems And Exercises Have Also Been Added. This Book Combines A Rigorous, Calculus-based Development Of Theory With A More Intuitive Approach That Appeals To Readers' Sense Of Reason And Logic, An Approach Developed Through The Author's Many Years Of Classroom Experience. The Book Begins With Three Chapters That Develop Probability Theory And Introduce The Axioms Of Probability, Random Variables, And Joint Distributions. The Next Two Chapters Introduce Limit Theorems And Simulation. Also Included Is A Chapter On Statistical Inference With A Focus On Bayesian Statistics, Which Is An Important, Though Often Neglected, Topic For Undergraduate-level Texts. Markov Chains In Discrete And Continuous Time Are Also Discussed Within The Book. More Than 400 Examples Are Interspersed Throughout To Help Illustrate Concepts And Theory And To Assist Readers In Developing An Intuitive Sense Of The Subject. Readers Will Find Many Of The Examples To Be Both Entertaining And Thought Provoking. This Is Also True For The Carefully Selected Problems That Appear At The End Of Each Chapter-- Peter Olofsson, Mikael Andersson. Includes Bibliographical References (p. 551-552) And Index. Praise for the First Edition'... an excellent textbook... well organized and neatly written.'—Mathematical Reviews'... amazingly interesting...'—Technometrics Thoroughly updated to showcase the interrelationships between probability, statistics, and stochastic processes, Probability, Statistics, and Stochastic Processes, Second Edition prepares readers to collect, analyze, and characterize data in their chosen fields. Beginning with three chapters that develop probability theory and introduce the axioms of probability, random variables, and joint distributions, the book goes on to present limit theorems and simulation. The authors combine a rigorous, calculus-based development of theory with an intuitive approach that appeals to readers'sense of reason and logic. Including more than 400 examples that help illustrate concepts and theory, the Second Edition features new material on statistical inference and a wealth of newly added topics, including: Consistency of point estimators Large sample theory Bootstrap simulation Multiple hypothesis testing Fisher's exact test and Kolmogorov-Smirnov test Martingales, renewal processes, and Brownian motion One-way analysis of variance and the general linear model Extensively class-tested to ensure an accessible presentation, Probability, Statistics, and Stochastic Processes, Second Edition is an excellent book for courses on probability and statistics at the upper-undergraduate level. The book is also an ideal resource for scientists and engineers in the fields of statistics, mathematics, industrial management, and engineering.