Categorical Data Analysis

**Praise for the Second Edition** "A must-have book for anyone expecting to do research and/or applications in categorical data analysis." —__Statistics in Medicine__ "It is a total delight reading this book." —__Pharmaceutical Research__ "If you do any analysis of categorical data, this is an essential desktop reference." —__Technometrics__ The use of statistical methods for analyzing categorical data has increased dramatically, particularly in the biomedical, social sciences, and financial industries. Responding to new developments, this book offers a comprehensive treatment of the most important methods for categorical data analysis. __Categorical Data Analysis, Third Edition__ summarizes the latest methods for univariate and correlated multivariate categorical responses. Readers will find a unified generalized linear models approach that connects logistic regression and Poisson and negative binomial loglinear models for discrete data with normal regression for continuous data. This edition also features: * An emphasis on logistic and probit regression methods for binary, ordinal, and nominal responses for independent observations and for clustered data with marginal models and random effects models * Two new chapters on alternative methods for binary response data, including smoothing and regularization methods, classification methods such as linear discriminant analysis and classification trees, and cluster analysis * New sections introducing the Bayesian approach for methods in that chapter * More than 100 analyses of data sets and over 600 exercises * Notes at the end of each chapter that provide references to recent research and topics not covered in the text, linked to a bibliography of more than 1,200 sources * A supplementary website showing how to use R and SAS; for all examples in the text, with information also about SPSS and Stata and with exercise solutions __Categorical Data Analysis, Third Edition__ is an invaluable tool for statisticians and methodologists, such as biostatisticians and researchers in the social and behavioral sciences, medicine and public health, marketing, education, finance, biological and agricultural sciences, and industrial quality control. Contents ... 9 Preface ... 15 CHAPTER 1 Introduction: Distributions and Inference for Categorical Data ... 19 1.1 CATEGORICAL RESPONSE DATA ... 19 1.1.1 Response-Explanatory Variable Distinction ... 20 1.1.2 Binary-Nominal-Ordinal Scale Distinction ... 20 1.1.3 Discrete-Continuous Variable Distinction ... 21 1.1.4 Quantitative-Qualitative Variable Distinction ... 21 1.1.S Organization of Book and Online Computing Appendix ... 22 1.2 DISTRIBUTIONS FOR CATEGORICAL DATA ... 23 1.2.1 Binomial Distribution ... 23 1.2.2 Multinomial Distribution ... 24 1.2.3 Poisson Distribution ... 24 1.2.4 Overdispersion ... 25 1.2.5 Connection Between Poisson and Multinomial Distributions ... 25 1.2.6 The Chi-Squared Distribution ... 26 1.3 STATISTICAL INFERENCE FOR CATEGORICAL DATA ... 26 1.3.1 Likelihood Functions and Maximum Likelihood Estimation ... 27 1.3.2 Likelihood Function and ML Estimate for Binomial Parameter ... 27 1.3.3 Wald-Likelihood Ratio Score Test Triad ... 28 1.3.4 Constructing Confi dence Intervals by Inverting Tests ... 30 1.4 STATISTICAL INFERENCE FOR BINOMIAL PARAME TERS ... 31 1.4.1 Tests About a Binomial Parameter ... 31 1.4.2 Confidence Intervals for a Binomial Parameter ... 32 1.4.3 Example: Estimating the Proportion of Vegetarians ... 33 1.4.4 Exact Small-Sample Inference and the Mid P-Value ... 34 1.5 STATISTICAL INFERENCE FOR MULTINOMIAL PARAMETERS ... 35 1.5.1 Estimation of Multinomial Parameters ... 35 1.5.2 Pearson Chi-Squared Test of a Specifi ed Multinomial ... 36 1.5.3 Likelihood-Ratio Chi-Squared Test of a Specifi ed Multinomial ... 36 1.5.4 Example: Testing Mendel's Theories ... 37 1.5.5 Testing with Estimated Expected Frequencies ... 38 1.5.6 Example: Pneumonia Infections in Calves ... 38 1.5.7 Chi-Squared Theoretical Justifi cation ... 40 1.6 BAYESIAN INFERENCE FOR BINOMIAL AND MULTINOMIAL PARAMETERS ... 40 1.6.1 The Bayesian Approach to Statistical Inference ... 40 1.6.2 Binomial Estimation: Beta and Logit-Normal Prior Distributions ... 42 1.6.3 Multinomial Estimation: Dirichlet Prior Distributions ... 43 1.6.4 Example: Estimating Vegetarianism Revisited ... 44 1.6.5 Binomial and Multinomial Estimation: Improper Priors ... 44 NOTES ... 45 EXERCISES ... 46 CHAPTER 2 Describing Contingency Tables ... 55 2.1 PROBABILITY STRUCTURE FOR CONTINGENCY TABLES ... 55 2.1.1 Contingency Tables ... 55 2.1.2 Joint/Marginal/Conditional Distributions for Contingency Tables ... 56 2.1.3 Example: Sensitivity and Specifi city for Medical Diagnoses ... 57 2.1.4 Independence of Categorical Variables ... 58 2.1.5 Poisson, Binomial, and Multinomial Sampling ... 58 2.1.6 Example: Seat Belts and Auto Accident Injuries ... 59 2.1.7 Example: Case-Control Study of Cancer and Smoking ... 60 2.1.8 Ty pes of Studies: Observational Versus Experimental ... 61 2.2 COMPARING TWO PROPORTIONS ... 61 2.2.1 Difference of Proportions ... 62 2.2.2 Relative Risk ... 62 2.2.3 Odds Ratio ... 62 2.2.4 Properties of the Odds Ratio ... 63 2.2.S Example: Association Between Heart Attacks and Aspirin Use ... 64 2.2.6 Case-Control Studies and the Odds Ratio ... 64 2.2. 7 Relationship B etween Odds Ratio and Relative Risk ... 65 2.3 CONDITIONAL ASSOCIATION IN STRATIFIED 2x2 TABLES ... 65 2.3.1 Partial Tables ... 66 2.3.2 Example: Racial Characteristics and the Death Penalty ... 66 2.3.3 Conditional and Marginal Odds Ratios ... 68 2.3.4 Marginal Independence Versus Conditional Independence ... 69 2.3.5 Homogeneous Association ... 71 2.3.6 Collapsibility: Identical Conditional and Marginal Associations ... 71 2.4 MEASURING ASSOCIATION IN IxJ TABLES ... 72 2.4.1 Odds Ratios in IxJ Tables ... 72 2.4.2 Association Factors ... 73 2.4.3 Summary Measures of Association ... 74 2.4.4 Ordinal Trends: Concordant and Discordant Pairs ... 74 2.4.5 Ordinal Measure of Association: Gamma ... 75 2.4.6 Probabilistic Comparisons of Tw o Ordinal Distributions ... 76 2.4.7 Example: Comparing Pain Ratings After Surgery ... 77 2.4.8 Correlation for Underlying Normality ... 77 NOTES ... 78 EXERCISES ... 78 CHAPTER 3 Inference for Two-Way Contingency Tables ... 87 3.1 CONFIDENCE INTERVALS FOR ASSOCIATION PARAMETERS ... 87 3.1.1 Interval Estimation of the Odds Ratio ... 87 3.1.2 Example: Seat-Belt Use and Traffic Deaths ... 88 3.1.3 Interval Estimation of Diff erence of Proportions and Relative Risk ... 89 3.1.4 Example: Aspirin and Heart Attacks Revisited ... 89 3.1.5 Deriving Standard Errors with the Delta Method ... 90 3.1.6 Delta Method Applied to the Sample Logit ... 91 3.1.7 Delta Method for the Log Odds Ratio ... 91 3.1.8 Simultaneous Confi dence Intervals for Multiple Comparisons ... 93 3.2 TESTING INDEPENDENCE IN TWO-WAY CONTINGENCY TABLES ... 93 3.2.1 Pearson and Likelihood-Ratio Chi-Squared Tests ... 93 3.2.2 Example: Education and Belief in God ... 95 3.2.3 Adequacy of Chi-Squared Approximations ... 95 3.2.4 Chi-Squared and Comparing Proportions in 2x2 Tables ... 96 3.2.5 Score Confi dence Intervals Comparing Proportions ... 96 3.2.6 Profi le Likelihood Confi dence Intervals ... 97 3.3 FOLLOWING-UP CHI-SQUARED TESTS ... 98 3.3.1 Pearson Residuals and Standardized Residuals ... 98 3.3.2 Example: Education and Belief in God Revisited ... 99 3.3.3 Partitioning Chi-Squared ... 99 3.3.4 Example: Origin of Schizophrenia ... 101 3.3.S Rules for Partitioning ... 102 3.3.6 Summarizing the Association ... 102 3.3.7 Limitations of Chi-Squared Tests ... 102 3.3.8 Why Consider Independence If It's Unlikely to Be True? ... 103 3.4 TWO-WAY TABLES WITH ORDERED CLASSIFICATIONS ... 104 3.4.1 Linear Trend Alternative to Independence ... 104 3.4.2 Example: Is Happiness Associated with Political Ideology? ... 105 3.4.3 Monotone Trend Alternatives to Independence ... 105 3.4.4 Extra Power with Ordinal Tests ... 106 3.4.5 Sensitivity to Choice of Scores ... 106 3.4.6 Example: Infant Birth Defects by Maternal Alcohol Consumption ... 107 3.4.7 Trend Tests for Ix2 and 2xJ Tables ... 108 3.4.8 Nominal-Ordinal Tables ... 108 3.5 SMALL-SAMPLE INFERENCE FOR CONTINGENCY TABLES ... 108 3.5.1 Fisher's Exact Test for 2x2 Tables ... 108 3.5.2 Example: Fisher's Tea Drinker ... 109 3.5.3 Two-Sided P-Values for Fisher's Exact Test ... 110 3.5.4 Confidence Intervals Based on Conditional Likelihood ... 110 3.5.5 Discreteness and Conservatism Issues ... 111 3.5.6 Small-Sample Unconditional Tests of Independence ... 111 3.5.7 Conditional Versus Unconditional Tests ... 112 3.6 BAYESIAN INFERENCE FOR TWO-WAY CONTINGENCY TABLES ... 114 3.6.1 Prior Distributions for Comparing Proportions in 2x2 Tables ... 114 3.6.2 Posterior Probabilities Comparing Proportions ... 115 3.6.3 Posterior Intervals for Association Parameters ... 115 3.6.4 Example: Urn Sampling Gives Highly Unbalanced Treatment Allocation ... 116 3.6.5 Highest Posterior Density Intervals ... 116 3.6.6 Testing Independence ... 117 3.6.7 Empirical Bayes and Hierarchical Bayesian Approaches ... 118 3.7 EXTENSIONS FOR MULTIWAY TABLES AND NONTABULATED RESPONSES ... 118 3.7.1 Categorical Data Need Not Be Contingency Tables ... 118 NOTES ... 119 EXERCISES ... 121 CHAPTER 4 Introduction to Generalized Linear Models ... 131 4.1 THE GENERALIZED LINEAR MODEL ... 131 4.1.1 Components of Generalized Linear Models ... 132 4.1.2 Binomial Logit Models for Binary Data ... 132 4.1.3 Poisson Loglinear Models for Count Data ... 133 4.1.4 Generalized Linear Models for Continuous Responses ... 133 4.1.5 Deviance of a GLM ... 133 4.1.6 Advantages of GLMs Versus Transforming the Data ... 134 4.2 GENERALIZED LINEAR MODELS FOR BINARY DATA ... 135 4.2.1 Linear Probability Model ... 135 4.2.2 Example: Snoring and Heart Disease ... 136 4.2.3 Logistic Regression Model ... 137 4.2.4 Binomial GLM for 2x2 Contingency Tables ... 138 4.2.5 Probit and Inverse cdf Link Functions ... 139Black,notBold,notItalic,open,TopLeftZoom,358,2,0.0 4.2.6 Latent Tolerance Motivation for Binary Response Models ... 140 4.3 GENERALIZED LINEAR MODELS FOR COUNTS AND RATES ... 140 4.3.1 Poisson Loglinear Models ... 141 4.3.2 Example: Horseshoe Crab Mating ... 141 4.3.3 Overdispersion for Poisson GLMs ... 144 4.3.4 Negative Binomial GLMs ... 145 4.3.5 Poisson Regression for Rates Using Offsets ... 146 4.3.6 Example: Modeling Death Rates for Heart Valve Operations ... 146 4.3.7 Poisson GLM of Independence in Two-Way Contingency Tables ... 148 4.4 MOMENTS AND LIKELIHOOD FOR GENERALIZED LINEAR MODELS ... 148 4.4.1 The Exponential Dispersion Family ... 148 4.4.2 Mean and Variance Functions for the Random Component ... 149 4.4.3 Mean and Variance Functions for Poisson and Binomial GLMs ... 150 4.4.4 Systematic Component and Link Function of a GLM ... 150 4.4.S Likelihood Equations for a GLM ... 151 4.4.6 The Key Role of the Mean-Variance Relationship ... 152 4.4.7 Likelihood Equations for Binomial GLMs ... 152 4.4.8 Asymptotic Covariance Matrix of Model Parameter Estimators ... 153 4.4.9 Likelihood Equations and cov (p) for Poisson Loglinear Model ... 154 4.5 INFERENCE AND MODEL CHECKING FOR GENERALIZED LINEAR MODELS ... 154 4.5.1 Deviance and Goodness of Fit ... 154 4.5.2 Deviance for Poisson GLMs ... 155 4.5.3 Deviance for Binomial GLMs: Grouped Versus Ungrouped Data ... 155 4.5.4 Likelihood-Ratio Model Comparison Using the Deviances ... 156 4.5.S Score Tests for Goodness of Fit and for Model Comparison ... 157 4.5.6 Residuals for GLMs ... 158 4.5.7 Covariance Matrices for Fitted Values and Residuals ... 160 4.5.8 The Bayesian Approach for GLMs ... 160 4.6 FITTING GENERALIZED LINEAR MODELS ... 161 4.6.1 Newton-Raphson Method ... 161 4.6.2 Fisher Scoring Method ... 162 4.6.3 Newton-Raphson and Fisher Scoring for Binary Data ... 163 4.6.4 ML as Iterative Reweighted Least Squares ... 164Black,notBold,notItalic,open,TopLeftZoom,284,2,0.0 4.6.5 Simplifi cations for Canonical Link Functions ... 165 4.7 QUASI-LIKELIHOOD AND GENERALIZED LINEAR MODELS ... 167 4.7.1 Mean-Variance Relationship Determines Quasi-likelihood Estimates ... 167 4.7.2 Overdispersion for Poisson GLMs and Quasi-likelihood ... 167 4.7.3 Overdispersion for Binomial GLMs and Quasi-likelihood ... 168 4.7.4 Example: Teratology Overdispersion ... 169 NOTES ... 170 EXERCISES ... 171 CHAPTER 5 Logistic Regression ... 181 5.1 INTERPRETING PARAMETERS IN LOGISTIC REGRESSION ... 181 5.1.1 Interpreting p: Odds, Probabilities, and Linear Approximations ... 182 5.1.2 Looking at the Data ... 183 5.1.3 Example: Horseshoe Crab Mating Revisited ... 184 5.1.4 Logistic Regression with Retrospective Studies ... 186 5.1.5 Logistic Regression Is Implied by Normal Explanatory Variables ... 187 5.2 INFERENCE FOR LOGISTIC REGRESSION ... 187 5.2.1 Inference About Model Parameters and Probabilities ... 187 5.2.2 Example: Inference for Horseshoe Crab Mating Data ... 188 5.2.3 Checking Goodness of Fit: Grouped and Ungrouped Data ... 189 5.2.4 Example: Model Goodness of Fit for Horseshoe Crab Data ... 190 5.2.5 Checking Goodness of Fit with Ungrouped Data by Grouping ... 190 5.2.6 Wald Inference Can Be Suboptimal ... 192 5.3 LOGISTIC MODELS WITH CATEGORICAL PREDICTORS ... 193 5.3.1 ANOVA-Type Representation of Factors ... 193 5.3.2 Indicator Variables Represent a Factor ... 193 5.3.3 Example: Alcohol and Infant Malformation Revisited ... 194 5.3.4 Linear Logit Model for Ix2 Contingency Tables ... 195 5.3.5 Cochran-Armitage Trend Test ... 195 5.3.6 Example: Alcohol and Infant Malformation Revisited ... 197 5.3.7 Using Directed Models Can Improve Inferential Power ... 197 5.3.8 Noncentral Chi-Squared Distribution and Power for Narrower Alternatives ... 198 5.3.9 Example: Skin Damage and Leprosy ... 199 5.3.10 Model Smoothing Improves Precision of Estimation ... 200 5.4 MULTIPLE LOGISTIC REGRESSION ... 200 5.4.1 Logistic Models for Multiway Contingency Tables ... 201 5.4.2 Example: AIDS and AZT Use ... 202 5.4.3 Goodness of Fit as a Likelihood-Ratio Test ... 204 5.4.4 Model Comparison by Comparing Deviances ... 205 5.4.5 Example: Horseshoe Crab Satellites Revisited ... 205 5.4.6 Quantitative Treatment of Ordinal Predictor ... 207Black,notBold,notItalic,open,TopLeftZoom,292,2,0.0 5.4.7 Probability-Based and Standardized Interpretations ... 208 5.4.8 Estimating an Average Causal Eff ect ... 209 5.5 FITTING LOGISTIC REGRESSION MODELS ... 210 5.5.1 Likelihood Equations for Logistic Regression ... 210 5.5.2 Asymptotic Covariance Matrix of Parameter Estimators ... 211 5.5.3 Distribution of Probability Estimators ... 212 5.5.4 Newton-Raphson Method Applied to Logistic Regression ... 212 NOTES ... 213 EXERCISES ... 214 CHAPTER 6 Building, Checking, and Applying Logistic Regression Models ... 225 6.1 STRATEGIES IN MODEL SELECTION ... 225 6.1.1 How Many Explanatory Variables Can Be in the Model? ... 226 6.1.2 Example: Horseshoe Crab Mating Data Revisited ... 226 6.1.3 Stepwise Procedures: Forward Selection and Backward Elimination ... 227 6.1.4 Example: Backward Elimination for Horseshoe Crab Data ... 228 6.1.5 Model Selection and the "Correct" Model ... 229 6.1.6 AIC: Minimizing Distance of the Fit from the Tr uth ... 230 6.1.7 Example: Using Causal Hypotheses to Guide Model Building ... 231 6.1.8 Alternative Strategies, Including Model Averaging ... 233 6.2 LOGISTIC REGRESSION DIAGNOSTICS ... 233 6.2.1 Residuals: Pearson, Deviance, and Standardized ... 233 6.2.2 Example: Heart Disease and Blood Pressure ... 234 6.2.3 Example: Admissions to Graduate School at Florida ... 236 6.2.4 Infl uence Diagnostics for Logistic Regression ... 238 6.3 SUMMARIZING THE PREDICTIVE POWER OF A MODEL ... 239 6.3.1 Summarizing Predictive Power: Rand R-Squared Measures ... 239 6.3.2 Summarizing Predictive Power: Likelihood and Deviance Measures ... 240 6.3.3 Summarizing Predictive Power: Classifi cation Tables ... 241 6.3.4 Summarizing Predictive Power: ROC Curves ... 242 6.3.S Example: Evaluating Predictive Power for Horseshoe Crab Data ... 242 6.4 MANTEL-HAENSZEL AND RELATED METHODS FOR MULTIPLE 2x2 TABLES ... 243 6.4.1 Using Logistic Models to Test Conditional Independence ... 244 6.4.2 Cochran-Mantel-Haenszel Test of Conditional Independence ... 245 6.4.3 Example: Multicenter Clinical Trial Revisited ... 246 6.4.4 CMH Test Is Advantageous for Sparse Data ... 246 6.4.S Estimation of Common Odds Ratio ... 247 6.4.6 Meta-analyses for Summarizing Multiple 2x2 Tables ... 248 6.4. 7 Meta-analyses for Multiple 2x2 Tables: Diff erence of Proportions ... 249 6.4.8 Collapsibility and Logistic Models for Contingency Tables ... 250 6.4.9 Testing Homogeneity of Odds Ratios ... 250 6.4.10 Summarizing Heterogeneity in Odds Ratios ... 251 6.4.11 Propensity Scores in Observational Studies ... 251 6.5 DETECTING A ND DEALING WITH INFINITE ESTIMATES ... 251 6.5.1 Complete or Quasi-complete Separation ... 252 6.5.2 Example: Multicenter Clinical Trial with Few Successes ... 253 6.5.3 Remedies When at Least One ML Estimate Is Infi nite ... 254 6.6 SAMPLE SIZE AND POWER CONSIDERATIONS ... 255 6.6.1 Sample Size and Power for Comparing Two Proportions ... 255 6.6.2 Sample Size Determination in Logistic Regression ... 256 6.6.3 Sample Size in Multiple Logistic Regression ... 257 6.6.4 Power for Chi-Squared Tests in Contingency Tables ... 257 6.6.5 Power for Testing Conditional Independence ... 258 6.6.6 Effects of Sample Size on Model Selection and Inference ... 259 NOTES ... 259 EXERCISES ... 261 CHAPTER 7 Alternative Modeling of Binary Response Data ... 269 7.1 PROBIT AND COMPLEMENTARY LOG-LOG MODELS ... 269 7.1.l Probit Models: Three Latent Variable Motivations ... 270 7.1.2 Probit Models: Interpreting Eff ects ... 270 7.1.3 Probit Model Fitting ... 271 7.1.4 Example: Modeling Flour Beetle Mortality ... 272 7.1.5 Complementary Log-Log Link Models ... 273 7.1.6 Example: Beetle Mortality Revisited ... 275 7.2 BAYESIAN INFERENCE FOR BINARY REGRESSION ... 275 7.2.1 Prior Specifi cations for Binary Regression Models ... 275 7.2.2 Example: Risk Factors for Endometrial Cancer Grade ... 276 7 .2.3 Bayesian Logistic Regression for Retrospective Studies ... 278 7.2.4 Probability-Based Prior Specifi cations for Binary Regression Models ... 278 7.2.5 Example: Modeling the Probability a Trauma Patient Survives ... 279 7.2.6 Bayesian Fitting for Probit Models ... 281 7.2.7 Bayesian Model Checking for Binary Regression ... 283 7.3 CONDITIONAL LOGISTIC REGRESSION ... 283 7.3.1 Conditional Likelihood ... 283 7.3.2 Small-Sample Inference for a Logistic Regression Parameter ... 285 7.3.3 Small-Sample Conditional Inference for 2x2 Contingency Tables ... 285 7.3.4 Small-Sample Conditional Inference for Linear Logit Model ... 286 7.3.5 Small-Sample Tests of Conditional Independence in 2x2 x K Tables ... 287 7.3.6 Example: Promotion Discrimination ... 287 7.3.7 Discreteness Complications of Using Exact Conditional Inference ... 288 7.4 SMOOTHING: KERNELS, PENALIZED LIKELIHOOD, GENERALIZED ADDITIVE MODELS ... 288 7.4.l How Much Smoothing The Variance -Bias Trade-off ... 288 7.4.2 Kernel Smoothing ... 289 7.4.3 Example: Smoothing to Portray Probability of Kyphosis ... 290 7.4.4 Nearest Neighbors Smoothing ... 290 7.4.5 Smoothing Using Penalized Likelihood Estimation ... 291 7.4.6 Why Shrink Estimates Toward 0? ... 293 7.4.7 Firth's Penalized Likelihood for Logistic Regression ... 293 7.4.8 Example: Complete Separation but Finite Logistic Estimates ... 293 7.4.9 Generalized Additive Models ... 294 7.4.10 Example: GAMs for Horseshoe Crab Mating Data ... 295 7.4.11 Advantages -Disadvantages of Various Smoothing Methods ... 295 7.5 ISSUES IN ANALYZING HIGH-DIMENSIONAL CATEGORICAL DATA ... 296 7.5.l Issues in Selecting Explanatory Variables ... 296 7.5.2 Adjusting for Multiplicity: The Bonferroni Method ... 297 7.5.3 Adjusting for Multiplicity: The False Discovery Rate ... 298 7.5.4 Other Variable Selection Methods with High-Dimensional Data ... 299 7.5.S Examples: High-Dimensional Applications in Genomics ... 300 7.5.6 Example: Motif Discovery for Protein Sequences ... 301 7.5.7 Example: The Netfl ix Prize ... 302 7.5.8 Example: Credit Scoring ... 303 NOTES ... 303 EXERCISES ... 305 CHAPTER 8 Models for Multinomial Responses ... 311 8.1 NOMINAL RESPONSES: BASELINE-CATEGORY LOGIT MODELS ... 311 8.1.1 Baseline-Category Logits ... 311 8.1.2 Example: Alligator Food Choice ... 312 8.1.3 Estimating Response Probabilities ... 314 8.1.4 Fitting Baseline-Category Logistic Models ... 315 8.1.5 Multicategory Logit Model as a Multivariate GLM ... 317 8.1.6 Multinomial Probit Models ... 317 8.1.7 Example: Eff ect of Menu Pricing ... 318 8.2 ORDINAL RESPONSES: CUMULATIVE LOGIT MODELS ... 319 8.2.1 Cumulative Logits ... 319 8.2.2 Proportional Odds Form of Cumulative Logit Model ... 319 8.2.3 Latent Variable Motivation for Proportional Odds Structure ... 321 8.2.4 Example: Happiness and Traumatic Events ... 322 8.2.S Checking the Proportional Odds Assumption ... 324 8.3 ORDINAL RESPONSES: ALTERNATIVE MODELS ... 326 8.3.1 Cumulative Link Models ... 326 8.3.2 Cumulative Probit and Log-Log Models ... 326 8.3.3 Example: Happiness Revisited with Cumulative Probits ... 327 8.3.4 Adjacent-Categories Logit Models ... 327 8.3.5 Example: Happiness Revisited ... 328 8.3.6 Continuation-Ratio Logit Models ... 329 8.3.7 Example: Developmental Toxicity Study with Pregnant Mice ... 330 8.3.8 Stochastic Ordering Location Eff ects Versus Dispersion Eff ects ... 331 8.3.9 Summarizing Predictive Power of Explanatory Variables ... 332 8.4 TESTING CONDITIONAL INDEPENDENCE IN IxJ x K TABLES ... 332 8.4.1 Testing Conditional Independence Using Multinomial Models ... 332 8.4.2 Example: Homosexual Marriage and Religious Fundamentalism ... 334 8.4.3 Generalized Cochran-Mantel-Haenszel Tests for IxJ x K Tables ... 335 8.4.4 Example: Homosexual Marriage Revisited ... 337 8.4.5 Related Score Tests for Multinomial Logit Models ... 337 8.5 DISCRETE-CHOICE MODELS ... 338 8.5.1 Conditional Logits for Characteristics of the Choices ... 338 8.5.2 Multinomial Logit Model Expressed as Discrete-Choice Model ... 339 8.5.3 Example: Shopping Destination Choice ... 339 8.5.4 Multinomial Probit Discrete-Choice Models ... 339 8.5.5 Extensions: Nested Logit and Mixed Logit Models ... 340 8.5.6 Extensions: Discrete Choice with Ordered Categories ... 340 8.6 BAYESIAN MODELING OF MULTINOMIAL RESPONSES ... 341 8.6.1 Bayesian Fitting of Cumulative Link Models ... 341 8.6.2 Example: Cannabis Use and Mother's Age ... 342 8.6.3 Bayesian Fitting of Multinomial Logit and Probit Models ... 343 8.6.4 Example: Alligator Food Choice Revisited ... 344 NOTES ... 344 EXERCISES ... 347 CHAPTER 9 Loglinear Models for Contingency Tables ... 357 9.1 LOGLINEAR MODELS FOR TWO-WAY TABLES ... 357 9.1.1 Independence Model for a Two-Way Table ... 357 9.1.2 Interpretation of Loglinear Model Parameters ... 358 9.1.3 Saturated Model for a Two-Way Table ... 358 9.1.4 Alternative Parameter Constraints ... 359 9.1.5 Hierarchical Versus Nonhierarchical Models ... 359 9.1.6 Multinomial Models for Cell Probabilities ... 360 9.2 LOGLINEAR MODELS FOR INDEPENDENCE AND INTERACTION IN THREE-WAY TABLES ... 360 9.2.1 Types of Independence ... 360 9.2.2 Homogeneous Association and Three-Factor Interaction ... 362 9.2.3 Interpretation of Loglinear Model Parameters ... 363 9.2.4 Example: Alcohol, Cigarette, and Marijuana Use ... 364 9.3 INFERENCE FOR LOGLINEAR MODELS ... 366 9.3.1 Chi-Squared Goodness-of-Fit Tests ... 366 9.3.2 Inference about Conditional Associations ... 366 9.4 LOGLINEAR MODELS FOR HIGHER DIMENSIONS ... 368 9.4.1 Models for Four-Way Contingency Tables ... 368 9.4.2 Example: Automobile Accidents and Seat-Belt Use ... 368 9.4.3 Large Samples and Statistical Versus Practical Signifi cance ... 370 9.4.4 Dissimilarity Index ... 370 9.5 LOGLINEAR-LOGISTIC MODEL CONNECTION ... 371 9.5.1 Using Logistic Models to Interpret Loglinear Models ... 371 9.5.2 Example: Auto Accidents and Seat-Belts Revisited ... 372 9.5.3 Equivalent Loglinear and Logistic Models ... 372 9.5.4 Example: Detecting Gene-Environment Interactions in Case-Control Studies ... 373 9.6 LOG LINEAR MODEL FITTING: LIKELIHOOD EQUATIONS AND ASYMPTOTIC DISTRIBUTIONS ... 374 9.6.1 Minimal Suffi cient Statistics ... 374 9.6.2 Likelihood Equations for Loglinear Models ... 375 9.6.3 Unique ML Estimates Match Data in Suffi cient Marginal Tables ... 376 9.6.4 Direct Versus Iterative Calculation of Fitted Values ... 376 9.6.S Decomposable Models ... 377 9.6.6 Chi-Squared Goodness-of-Fit Tests ... 377 9.6.7 Covariance Matrix of ML Parameter Estimators ... 378 9.6.8 Connection Between Multinomial and Poisson Loglinear Models ... 379 9.6.9 Distribution of Probability Estimators ... 380 9.6.10 Proof of Uniqueness of ML Estimates ... 381 9.6.11 Pseudo ML for Complex Sampling Designs ... 381 9.7 LOGLINEA R MODEL FITTING: ITERATIVE METHODS AND THEIR APPLICATION ... 382 9.7.1 Newton-Raphson Method ... 382 9.7.2 Iterative Proportional Fitting ... 383 9.7.3 Comparison ofIPF and Newton-Raphson Iterative Methods ... 384 9.7.4 Raking a Table: Contingency Table Standardization ... 385 NOTES ... 386 EXERCISES ... 387Black,notBold,notItalic,open,TopLeftZoom,165,2,0.0 CHAPTER 10 Building and Extending Loglinear Models ... 395 IO.I CONDITIONAL INDEPENDENCE GRAPHS AND COLLAPSIBILITY ... 395 IO.I.I Conditional Independence Graphs ... 395 10.1.2 Graphical Loglinear Models ... 396 10.1.3 Collapsibility in Three-Way Contingency Tables ... 397 10.1.4 Collapsibility for Multiway Tables ... 398 10.2 MODEL SELECTION AND COMPARISON ... 398 10.2.1 Considerations in Model Selection ... 398 10.2.2 Example: Model Building for Student Survey ... 399 10.2.3 Loglinear Model Comparison Statistics ... 401 10.2.4 Partitioning Chi-Squared with Model Comparisons ... 402 10.2.5 Identical Marginal and Conditional Tests of Independence ... 402 10.3 RESIDUALS FOR DETECTING CELL-SPECIFIC LACK OF FIT ... 403 10.3.1 Residuals for Loglinear Models ... 403 10.3.2 Example: Student Survey Revisited ... 403 10.3.3 Identical Loglinear and Logistic Standardized Residuals ... 404 10.4 MODELING ORDINAL ASSOCIATIONS ... 404 10.4.1 Linear-by-Linear Association Model for Two-Way Tables ... 405 10.4.2 Corresponding Logistic Model for Adjacent Responses ... 406 10.4.3 Likelihood Equations and Model Fitting ... 407 10.4.4 Example: Sex and Birth Control Opinions Revisited ... 407 10.4.5 Directed Ordinal Test of Independence ... 409 10.4.6 Row Effects and Column Effects Association Models ... 409 10.4.7 Example: Estimating Category Scores for Premarital Sex ... 410 10.4.8 Ordinal Variables in Models for Multiway Tables ... 410 10.S GENERALIZED LOGLINEAR AND ASSOCIATION MODELS, CORRELATION MODELS, AND CORRESPONDENCE ANALYSIS ... 411 10.S.1 Generalized Loglinear Model ... 411 10.5.2 Multiplicative Row and Column Eff ects Model ... 412 10.5.3 Example: Mental Health and Parents' SES ... 413 10.5.4 Correlation Models ... 413 10.5.5 Correspondence Analysis ... 414 10.5.6 Model Selection and Score Choice for Ordinal Variables ... 416 10.6 EMPTY CELLS AND SPARSENESS IN MODELING CONTINGENCY TABLES ... 416 10.6.1 Empty Cells: Sampling Versus Structural Zeros ... 416 10.6.2 Existence of Estimates in Loglinear Models ... 416 10.6.3 Eff ects of Sparseness on X2, G2, and Model-Based Tests ... 418 10.6.4 Alternative Sparse Data Asymptotics ... 419 10.6.5 Adding Constants to Cells of a Contingency Table ... 419 10.7 BAYESIAN LOGLINEAR MODELING ... 419 10.7.1 Estimating Loglinear Model Parameters in Two-Way Tables ... 420 10.7.2 Example: Polarized Opinions by Political Party ... 420 10.7.3 Bayesian Loglinear Modeling of Multidimensional Tables ... 421 10.7.4 Graphical Conditional Independence Models ... 422 NOTES ... 422 EXERCISES ... 425 CHAPTER 11 Models for Matched Pairs ... 431 11.1 COMPARING DEPENDENT PROPORTIONS ... 432 11.1.2 McNemar Test Comparing Dependent Proportions ... 433 11.1.3 Example: Changes in Presidential Election Voting ... 433 11.1.4 Increased Precision with Dependent Samples ... 434 11.1.5 Small-Sample Test Comparing Dependent Proportions ... 434 11.1.6 Connection Between McNemar and Cochran-Mantel-Haenszel Tests ... 435 11.1.7 Subject-Specifi c and Population- Averaged (Marginal) Tables ... 436 11.2 CONDITIONAL LOGISTIC REGRESSION FOR BINARY MATCHED PA IRS ... 436 11.2.1 Subject-Specific Versus Marginal Models for Matched Pairs ... 436 11.2.2 Logistic Models with Subject-Specific Probabilities ... 437 11.2.3 Conditional ML Inference for Binary Matched Pairs ... 438 11.2.4 Random Effects in Binary Matched-Pairs Model ... 439 11.2.S Conditional Logistic Regression for Matched Case-Control Studies ... 439 11.2.6 Conditional Logistic Regression for Matched Pairs with Multiple Predictors ... 440 11.2.7 Marginal Models and Subject-Specifi c Models: Extensions ... 441 11.3 MARGINAL MODELS FOR SQUARE CONTINGENCY TABLES ... 442 11.3.1 Marginal Models for Nominal Classifi cations ... 442 11.3.2 Example: Regional Migration ... 443 11.3.3 Marginal Models for Ordinal Classifi cations ... 443 11.3.4 Example: Opinions on Premarital and Extramarital Sex ... 444 11.4 SYMMETRY, QUASI-SYMMETRY, AND QUASI-INDEPENDENCE ... 444 11.4.1 Symmetry as Logistic and Loglinear Models ... 445 11.4.2 Quasi-symmetry ... 445 11.4.3 Marginal Homogeneity and Quasi-symmetry ... 447 11.4.4 Quasi-independence ... 447 11.4.5 Example: Migration Revisited ... 448 11.4.6 Ordinal Quasi-symmetry ... 449 11.4.7 Example: Premarital and Extramarital Sex Revisited ... 450 11.5 MEASURING AGREEMENT BETWEEN OBSERVERS ... 450 11.5.1 Agreement: Departures from Independence ... 451 11.5.2 Using Quasi-independence to Analyze Agreement ... 451 11.5.3 Quasi-symmetry and Agreement Modeling ... 452 11.5.4 Kappa: A Summary Measure of Agreement ... 452 11.5.5 Weighted Kappa: Quantifying Disagreement ... 453 11.S.6 Extensions to Multiple Observers ... 453 11.6 BRADLEY-TERRY MODEL FOR PAIRED PREFERENCES ... 454 11.6.1 Bradley-Terry Model ... 454 11.6.2 Example: Major League Baseball Rankings ... 454 11.6.3 Example: Home Team Advantage in Baseball ... 455 11.6.4 Bradley-Terry Model and Quasi-symmetry ... 456 11.6.S Extensions to Ties and Ordinal Pairwise Evaluations ... 457 11.7 MARGINAL MODELS AND QUASI-SYMMETRY MODELS FOR MATCHED SETS ... 457 11.7.1 Marginal Homogeneity, Complete Symmetry, and Quasi-symmetry ... 457 11.7.2 Types of Marginal Symmetry ... 458 11.7.3 Comparing Binary Marginal Distributions in Multiway Tables ... 458 11.7.4 Example: Attitudes Toward Legalized Abortion ... 459 11.7.S Marginal Homogeneity for a Multicategory Response ... 460 11.7.6 Wald and Generalized CMH Score Tests of Marginal Homogeneity ... 460 NOTES ... 461 EXERCISES ... 463 CHAPTER 12 Clustered Categorical Data: Marginal and Transitional Models ... 473 12.1 MARGINAL MODELING: MAXIMUM LIKELIHOOD APPROACH ... 474 12.1.1 Example: Longitudinal Study of Mental Depression ... 474 12.1.2 Modeling a Repeated Multinomial Response ... 476 12.1.3 Example: Insomnia Clinical Trial ... 476 12.1.4 ML Fitting of Marginal Logistic Models: Constraints on Cell Probabilities ... 477 12.1.5 ML Fitting of Marginal Logistic Models: Other Methods ... 479 12.2 MARGINAL MODELING: GENERALIZED ESTIMATING EQUATIONS (GEEs) APPROACH ... 480 12.2.1 Generalized Estimating Equations Methodology: Basic Idea Machine generated contents note: Preface 1. Introduction: Distributions and Inference for Categorical Data 1 1.1 Categorical Response Data, 1 1.2 Distributions for Categorical Data 1.3 Statistical Inference for Categorical Data 1.4 Statistical Inference for Binomial Parameters 1.5 Statistical Inference for Multinomial Parameters 1.6 Bayesian Inference for Binomial and Multinomial Parameters Notes Exercises 2. Describing Contingency Tables 2.1 Probability Structure for Contingency Tables 2.2 Comparing Two Proportions 2.3 Conditional Association in Stratified 2x2 Tables 2.4 Measuring Association in I x J Tables Notes Exercises 3. Inference for Two-Way Contingency Tables 3.1 Confidence Intervals for Association Parameters 3.2 Testing Independence in Two-Way Contingency Tables 3.3 Following-Up Chi-Squared Tests 3.4 Two-Way Tables with Ordered Classifications 3.5 Small-Sample Inference for Contingency Tables 3.6 Bayesian Inference for Two-Way Contingency Tables 3.7 Extensions for Multiway Tables and Nontabulated Responses Notes Exercises 4. Introduction to Generalized Linear Models 4.1 The Generalized Linear Model 4.2 Generalized Linear Models for Binary Data 4.3 Generalized Linear Models for Counts and Rates 4.4 Moments and Likelihood for Generalized Linear Models 4.5 Inference and Model Checking for Generalized Linear Models 4.6 Fitting Generalized Linear Models 4.7 Quasi-Likelihood and Generalized Linear Models Notes Exercises 5. Logistic Regression 5.1 Interpreting Parameters in Logistic Regression 5.2 Inference for Logistic Regression 5.3 Logistic Models with Categorical Predictors 5.4 Multiple Logistic Regression 5.5 Fitting Logistic Regression Models Notes Exercises 6. Building, Checking, and Applying Logistic Regression Models 6.1 Strategies in Model Selection 6.2 Logistic Regression Diagnostics 6.3 Summarizing the Predictive Power of a Model 6.3 Mantel-Haenszel and Related Methods for Multiple 2x2 Tables 6.4 Detecting and Dealing with Infinite Estimates 6.5 Sample Size and Power Considerations Notes Exercises 7. Alternative Modeling of Binary Response Data 7.1 Probit and Complementary Log-Log Models 7.2 Bayesian Inference for Binary Regression 7.3 Conditional Logistic Regression 7.4 Smoothing: Kernels, Penalized Likelihood, Generalized Additive Models 7.5 Issues in Analyzing High-Dimensional Categorical Data Notes Exercises 8. Models for Multinomial Responses 8.1 Nominal Responses: Baseline-Category Logit Models 8.2 Ordinal Responses: Cumulative Logit Models 8.3 Ordinal Responses: Alternative Models 8.4 Testing Conditional Independence in I ? J ? K Tables 8.5 Discrete-Choice Models 8.6 Bayesian Modeling of Multinomial Responses Notes Exercises 9. Loglinear Models for Contingency Tables 9.1 Loglinear Models for Two-Way Tables 9.2 Loglinear Models for Independence and Interaction in Three-Way Tables 9.3 Inference for Loglinear Models 9.4 Loglinear Models for Higher Dimensions 9.5 The Loglinear?Logistic Model Connection 9.6 Loglinear Model Fitting: Likelihood Equations and Asymptotic Distributions 9.7 Loglinear Model Fitting: Iterative Methods and their Application Notes Exercises 10. Building and Extending Loglinear Models 10.1 Conditional Independence Graphs and Collapsibility 10.2 Model Selection and Comparison 10.3 Residuals for Detecting Cell-Specific Lack of Fit 10.4 Modeling Ordinal Associations 10.5 Generalized Loglinear and Association Models, Correlation Models, and Correspondence Analysis 10.6 Empty Cells and Sparseness in Modeling Contingency Tables 10.7 Bayesian Loglinear Modeling Notes Exercises 11. Models for Matched Pairs 11.1 Comparing Dependent Proportions 11.2 Conditional Logistic Regression for Binary Matched Pairs 11.3 Marginal Models for Square Contingency Tables 11.4 Symmetry, Quasi-symmetry, and Quasi-independence 11.5 Measuring Agreement Between Observers 11.6 Bradley-Terry Model for Paired Preferences 11.7 Marginal Models and Quasi-symmetry Models for Matched Sets Notes Exercises 12. Clustered Categorical Data: Marginal and Transitional Models 12.1 Marginal Modeling: Maximum Likelihood Approach 12.2 Marginal Modeling: Generalized Estimating Equations Approach 12.3 Quasi-likelihood and Its GEE Multivariate Extension: Details 12.4 Transitional Models: Markov Chain and Time Series Models Notes Exercises 13. Clustered Categorical Data: Random Effects Models 13.1 Random Effects Modeling of Clustered Categorical Data 13.2 Binary Responses: The Logistic-Normal Model 13.3 Examples of Random Effects Models for Binary Data 13.4 Random Effects Models for Multinomial Data 13.5 Multilevel Models 13.6 GLMM Fitting, Inference, and Prediction 13.7 Bayesian Multivariate Categorical Modeling Notes Exercises 14. Other Mixture Models for Discrete Data 14.1 Latent Class Models 14.2 Nonparametric Random Effects Models 14.3 Beta-Binomial Models 14.4 Negative Binomial Regression 14.5 Poisson Regression with Random Effects Notes Exercises 15. Non-Model-Based Classification and Clustering 15.2 Classification: Linear Discriminant Analysis 15.3 Classification: Tree-Structured Prediction 15.4 Cluster Analysis for Categorical Data Notes Exercises 16. Large- and Small-Sample Theory for Parametric Models 16.1 Delta Method 16.2 Asymptotic Distributions of Estimators of Model Parameters and Cell Probabilities 16.3 Asymptotic Distributions of Residuals and Goodness-of-Fit Statistics 16.4 Asymptotic Distributions for Logit/Loglinear Models 16.5 Small-Sample Significance Tests for Contingency Tables 16.6 Small-Sample Confidence Intervals for Categorical Data 16.7 Alternative Estimation Theory for Parametric Models Notes Exercises 17. Historical Tour of Categorical Data Analysis 17.1 Pearson-Yule Association Controversy 17.2 R. A. Fisher's Contributions 17.3 Logistic Regression 17.4 Multiway Contingency Tables and Loglinear Models 17.5 Bayesian Methods for Categorical Data 17.6 A Look Forward, and Backward Appendix A. Statistical Software for Categorical Data Analysis Appendix B. Chi-Squared Distribution Values References Author Index Example Index Subject Index. A Classic In Its Own Right, This Book Continues To Provide An Introduction To Modern Generalized Linear Models For Categorical Variables. The Text Emphasizes Methods That Are Most Commonly Used In Practical Application, Such As Classical Inferences For Two- And Three-way Contingency Tables, Logistic Regression, Loglinear Models, Models For Multinomial (nominal And Ordinal) Responses, And Methods For Repeated Measurement And Other Forms Of Clustered, Correlated Response Data. Chapter Headings Remain Essentially With The Exception Of A New One On Bayesian Inference For Parametric Models. Other Major Changes Include An Expansion Of Clustered Data, New Research On Analysis Of Data Sets With Robust Variables, Extensive Discussions Of Ordinal Data, More On Interpretation, And Additional Exercises Throughout The Book. R And Sas Are Now Showcased As The Software Of Choice. An Author Web Site With Solutions, Commentaries, Software Programs, And Data Sets Is Available-- Machine Generated Contents Note: Preface 1. Introduction: Distributions And Inference For Categorical Data 1 1.1 Categorical Response Data, 1 1.2 Distributions For Categorical Data 1.3 Statistical Inference For Categorical Data 1.4 Statistical Inference For Binomial Parameters 1.5 Statistical Inference For Multinomial Parameters 1.6 Bayesian Inference For Binomial And Multinomial Parameters Notes Exercises 2. Describing Contingency Tables 2.1 Probability Structure For Contingency Tables 2.2 Comparing Two Proportions 2.3 Conditional Association In Stratified 2x2 Tables 2.4 Measuring Association In I X J Tables Notes Exercises 3.^ Alan Agresti. Includes Bibliographical References And Index.