Discovering Knowledge in data An Introduction to Data Mining

The second edition of a highly praised, successful reference on data mining, with thorough coverage of big data applications, predictive analytics, and statistical analysis. Includes new chapters on Multivariate Statistics, Preparing to Model the Data, and Imputation of Missing Data, and an Appendix on Data Summarization and Visualization Offers extensive coverage of the R statistical programming language Contains 280 end-of-chapter exercises Includes a companion website with further resources for all readers, and Powerpoint slides, a solutions manual, and suggested projects for instructors who adopt the book DISCOVERING KNOWLEDGE IN DATA 3 Contents 7 Preface 13 1 An Introduction to Data Mining 21 1.1 What is Data Mining? 21 1.2 Wanted: Data Miners 22 1.3 The Need for Human Direction of Data Mining 23 1.4 The Cross-Industry Standard Practice for Data Mining 24 1.4.1 Crisp-DM: The Six Phases 25 1.5 Fallacies of Data Mining 26 1.6 What Tasks Can Data Mining Accomplish? 28 1.6.1 Description 28 1.6.2 Estimation 28 1.6.3 Prediction 30 1.6.4 Classification 30 1.6.5 Clustering 32 1.6.6 Association 34 References 34 Exercises 35 2 Data Preprocessing 36 2.1 Why do We Need to Preprocess the Data? 37 2.2 Data Cleaning 37 2.3 Handling Missing Data 39 2.4 Identifying Misclassifications 42 2.5 Graphical Methods for Identifying Outliers 42 2.6 Measures of Center and Spread 43 2.7 Data Transformation 46 2.8 Min-Max Normalization 46 2.9 Z-Score Standardization 47 2.10 Decimal Scaling 48 2.11 Transformations to Achieve Normality 48 2.12 Numerical Methods for Identifying Outliers 55 2.13 Flag Variables 56 2.14 Transforming Categorical Variables into Numerical Variables 57 2.15 Binning Numerical Variables 58 2.16 Reclassifying Categorical Variables 59 2.17 Adding an Index Field 59 2.18 Removing Variables that are Not Useful 59 2.19 Variables that Should Probably Not Be Removed 60 2.20 Removal of Duplicate Records 61 2.21 A Word About Id Fields 61 THE R ZONE 62 References 68 Exercises 68 Hands-On Analysis 70 3 Exploratory Data Analysis 71 3.1 Hypothesis Testing Versus Exploratory Data Analysis 71 3.2 Getting to Know the Data Set 72 3.3 Exploring Categorical Variables 75 3.4 Exploring Numeric Variables 82 3.5 Exploring Multivariate Relationships 89 3.6 Selecting Interesting Subsets of the Data for Further Investigation 91 3.7 Using EDA to Uncover Anomalous Fields 91 3.8 Binning Based on Predictive Value 92 3.9 Deriving New Variables: Flag Variables 94 3.10 Deriving New Variables: Numerical Variables 97 3.11 Using EDA to Investigate Correlated Predictor Variables 97 3.12 Summary 100 THE R ZONE 102 Reference 108 Exercises 108 Hands-On Analysis 109 4 Univariate Statistical Analysis 111 4.1 Data Mining Tasks in Discovering Knowledge in Data 111 4.2 Statistical Approaches to Estimation and Prediction 112 4.3 Statistical Inference 113 4.4 How Confident are We in Our Estimates? 114 4.5 Confidence Interval Estimation of the Mean 115 4.6 How to Reduce the Margin of Error 117 4.7 Confidence Interval Estimation of the Proportion 118 4.8 Hypothesis Testing for the Mean 119 4.9 Assessing the Strength of Evidence Against the Null Hypothesis 121 4.10 Using Confidence Intervals to Perform Hypothesis Tests 122 4.11 Hypothesis Testing for the Proportion 124 THE R ZONE 125 Reference 126 Exercises 126 5 Multivariate Statistics 129 5.1 Two-Sample t-Test for Difference in Means 130 5.2 Two-Sample Z-Test for Difference in Proportions 131 5.3 Test for Homogeneity of Proportions 132 5.4 Chi-Square Test for Goodness of Fit of Multinomial Data 134 5.5 Analysis of Variance 135 5.6 Regression Analysis 138 5.7 Hypothesis Testing in Regression 142 5.8 Measuring the Quality of a Regression Model 143 5.9 Dangers of Extrapolation 143 5.10 Confidence Intervals for the Mean Value of Given 145 5.11 Prediction Intervals for a Randomly Chosen Value of Given 145 5.12 Multiple Regression 146 5.13 Verifying Model Assumptions 147 THE R ZONE 151 Reference 155 Exercises 155 Hands-On Analysis 156 6 Preparing to Model the Data 158 6.1 Supervised Versus Unsupervised Methods 158 6.2 Statistical Methodology and Data Mining Methodology 159 6.3 Cross-Validation 159 6.4 Overfitting 161 6.5 BIAS–Variance Trade-Off 162 6.6 Balancing the Training Data Set 164 6.7 Establishing Baseline Performance 165 THE R ZONE 166 Reference 167 Exercises 167 7 k-Nearest Neighbor Algorithm 169 7.1 Classification Task 169 7.2 κ-Nearest Neighbor Algorithm 170 7.3 Distance Function 173 7.4 Combination Function 176 7.4.1 Simple Unweighted Voting 176 7.4.2 Weighted Voting 176 7.5 Quantifying Attribute Relevance: Stretching the Axes 178 7.6 Database Considerations 178 7.7 κ-Nearest Neighbor Algorithm for Estimation and Prediction 179 7.8 Choosing κ 180 7.9 Application of κ-Nearest Neighbor Algorithm Using IBM/SPSS Modeler 180 THE R ZONE 182 Exercises 183 Hands-On Analysis 184 8 Decision Trees 185 8.1 What is a Decision Tree? 185 8.2 Requirements for Using Decision Trees 187 8.3 Classification and Regression Trees 188 8.4 C4.5 Algorithm 194 8.5 Decision Rules 199 8.6 Comparison of the C5.0 and Cart Algorithms Applied to Real Data 200 THE R ZONE 203 References 204 Exercises 205 Hands-On Analysis 205 9 Neural Networks 207 9.1 Input and Output Encoding 208 9.2 Neural Networks for Estimation and Prediction 210 9.3 Simple Example of a Neural Network 211 9.4 Sigmoid Activation Function 213 9.5 Back-Propagation 214 9.5.1 Gradient Descent Method 214 9.5.2 Back-Propagation Rules 215 9.5.3 Example of Back-Propagation 216 9.6 Termination Criteria 218 9.7 Learning Rate 218 9.8 Momentum Term 219 9.9 Sensitivity Analysis 221 9.10 Application of Neural Network Modeling 222 THE R ZONE 224 References 227 Exercises 227 Hands-On Analysis 227 10 Hierarchical and k-Means Clustering 229 10.1 The Clustering Task 229 10.2 Hierarchical Clustering Methods 232 10.3 Single-Linkage Clustering 233 10.4 Complete-Linkage Clustering 234 10.5 κ-Means Clustering 235 10.6 Example of κ-Means Clustering at Work 236 10.7 Behavior of MSB, MSE, and PSEUDO-F as the κ-Means Algorithm Proceeds 239 10.8 Application of κ-Means Clustering Using SAS Enterprise Miner 240 10.9 Using Cluster Membership to Predict Churn 243 THE R ZONE 244 References 246 Exercises 246 Hands-On Analysis 246 11 Kohonen Networks 248 11.1 Self-Organizing Maps 248 11.2 Kohonen Networks 250 11.2.1 Kohonen Networks Algorithm 251 11.3 Example of a Kohonen Network Study 251 11.4 Cluster Validity 255 11.5 Application of Clustering Using Kohonen Networks 255 11.6 Interpreting the Clusters 257 11.6.1 Cluster Profiles 260 11.7 Using Cluster Membership as Input to Downstream Data Mining Models 262 THE R ZONE 263 References 265 Exercises 265 Hands-On Analysis 265 12 Association Rules 267 12.1 Affinity Analysis and Market Basket Analysis 267 12.1.1 Data Representation for Market Basket Analysis 268 12.2 Support, Confidence, Frequent Itemsets, and the a Priori Property 269 12.3 How Does the a Priori Algorithm Work? 271 12.3.1 Generating Frequent Itemsets 271 12.3.2 Generating Association Rules 273 12.4 Extension from Flag Data to General Categorical Data 275 12.5 Information-Theoretic Approach: Generalized Rule Induction Method 276 12.5.1 J-Measure 277 12.6 Association Rules are Easy to do Badly 278 12.7 How can we Measure the Usefulness of Association Rules? 279 12.8 Do Association Rules Represent Supervised or Unsupervised Learning? 280 12.9 Local Patterns Versus Global Models 281 THE R ZONE 282 References 283 Exercises 283 Hands-On Analysis 284 13 Imputation of Missing Data 286 13.1 Need for Imputation of Missing Data 286 13.2 Imputation of Missing Data: Continuous Variables 287 13.3 Standard Error of the Imputation 290 13.4 Imputation of Missing Data: Categorical Variables 291 13.5 Handling Patterns in Missingness 292 THE R ZONE 293 Reference 296 Exercises 296 Hands-On Analysis 296 14 Model Evaluation Techniques 297 14.1 Model Evaluation Techniques for the Description Task 298 14.2 Model Evaluation Techniques for the Estimation and Prediction Tasks 298 14.3 Model Evaluation Techniques for the Classification Task 300 14.4 Error Rate, False Positives, and False Negatives 300 14.5 Sensitivity and Specificity 303 14.6 Misclassification Cost Adjustment to Reflect Real-World Concerns 304 14.7 Decision Cost/Benefit Analysis 305 14.8 Lift Charts and Gains Charts 306 14.9 Interweaving Model Evaluation with Model Building 309 14.10 Confluence of Results: Applying a Suite of Models 310 THE R ZONE 311 Reference 311 Exercises 311 Hands-On Analysis 311 Appendix Data Summarization and Visualization 314 Part 1 Summarization 1: Building Blocks of Data Analysis 314 Part 2 Visualization: Graphs and Tables for Summarizing and Organizing Data 316 2.1 Categorical Variables 316 2.2 Quantitative Variables 317 Part 3 Summarization 2: Measures of Center, Variability, and Position 321 Part 4 Summarization and Visualization of Bivariate Relationships 324 Index 329