Data Mining Algorithms in C++ : Data Patterns and Algorithms for Modern Applications

Find the various relationships among variables that can be present in big data as well as other data sets. This book also covers information entropy, permutation tests, combinatorics, predictor selections, and eigenvalues to give you a well-rounded view of data mining and algorithms in C++. Furthermore, Data Mining Algorithms in C++ includes classic techniques that are widely available in standard statistical packages, such as maximum likelihood factor analysis and varimax rotation. After reading and using this book, you'll come away with many code samples and routines that can be repurposed into your own data mining tools and algorithms toolbox. This will allow you to integrate these techniques in your various data and analysis projects. You will: Discover useful data mining techniques and algorithms using the C++ programming language Carry out permutation tests Work with the various relationships and screening types for these relationships Master predictor selections Use the DATAMINE program. Read more... Abstract: Find the various relationships among variables that can be present in big data as well as other data sets. This book also covers information entropy, permutation tests, combinatorics, predictor selections, and eigenvalues to give you a well-rounded view of data mining and algorithms in C++. Furthermore, Data Mining Algorithms in C++ includes classic techniques that are widely available in standard statistical packages, such as maximum likelihood factor analysis and varimax rotation. After reading and using this book, you'll come away with many code samples and routines that can be repurposed into your own data mining tools and algorithms toolbox. This will allow you to integrate these techniques in your various data and analysis projects. You will: Discover useful data mining techniques and algorithms using the C++ programming language Carry out permutation tests Work with the various relationships and screening types for these relationships Master predictor selections Use the DATAMINE program Table of Contents About the Author About the Technical Reviewers Introduction Chapter 1: Information and Entropy Entropy Entropy of a Continuous Random Variable Partitioning a Continuous Variable for Entropy An Example of Improving Entropy Joint and Conditional Entropy Code for Conditional Entropy Mutual Information Fano’s Bound and Selection of Predictor Variables Confusion Matrices and Mutual Information Extending Fano’s Bound for Upper Limits Simple Algorithms for Mutual Information The TEST_DIS Program Continuous Mutual Information The Parzen Window Method Adaptive Partitioning The TEST_CON Program Asymmetric Information Measures Uncertainty Reduction Transfer Entropy: Schreiber’s Information Transfer Chapter 2: Screening for Relationships Simple Screening Methods Univariate Screening Bivariate Screening Forward Stepwise Selection Forward Selection Preserving Subsets Backward Stepwise Selection Criteria for a Relationship Ordinary Correlation Nonparametric Correlation Accommodating Simple Nonlinearity Chi-Square and Cramer’s V Mutual Information and Uncertainty Reduction Multivariate Extensions Permutation Tests A Modestly Rigorous Statement of the Procedure A More Intuitive Approach Serial Correlation Can Be Deadly Permutation Algorithms Outline of the Permutation Test Algorithm Permutation Testing for Selection Bias Combinatorially Symmetric Cross Validation The CSCV Algorithm An Example of CSCV OOS Testing Univariate Screening for Relationships Three Simple Examples Bivariate Screening for Relationships Stepwise Predictor Selection Using Mutual Information Maximizing Relevance While Minimizing Redundancy Code for the Relevance Minus Redundancy Algorithm An Example of Relevance Minus Redundancy A Superior Selection Algorithm for Binary Variables FREL for High-Dimensionality, Small Size Datasets Regularization Interpreting Weights Bootstrapping FREL Monte Carlo Permutation Tests of FREL General Statement of the FREL Algorithm Multithreaded Code for FREL Some FREL Examples Chapter 3: Displaying Relationship Anomalies Marginal Density Product Actual Density Marginal Inconsistency Mutual Information Contribution Code for Computing These Plots Comments on Showing the Display Chapter 4: Fun with Eigenvectors Eigenvalues and Eigenvectors Principal Components (If You Really Must) The Factor Structure Is More Interesting A Simple Example Rotation Can Make Naming Easier Code for Eigenvectors and Rotation Eigenvectors of a Real Symmetric Matrix Factor Structure of a Dataset Varimax Rotation Horn’s Algorithm for Determining Dimensionality Code for the Modified Horn Algorithm Clustering Variables in a Subspace Code for Clustering Variables Separating Individual from Common Variance Log Likelihood the Slow, Definitional Way Log Likelihood the Fast, Intelligent Way The Basic Expectation Maximization Algorithm Code for Basic Expectation Maximization Accelerating the EM Algorithm Code for Quadratic Acceleration with DECME-2s Putting It All Together Thoughts on My Version of the Algorithm Measuring Coherence Code for Tracking Coherence Coherence in the Stock Market Chapter 5: Using the DATAMINE Program File/Read Data File File/Exit Screen/Univariate Screen Screen/Bivariate Screen Screen/Relevance Minus Redundancy Screen/FREL Analyze/Eigen Analysis Analyze/Factor Analysis Analyze/Rotate Analyze/Cluster Variables Analyze/Coherence Plot/Series Plot/Histogram Plot/Density Index Discover hidden relationships among the variables in your data, and learn how to exploit these relationships. This book presents a collection of data-mining algorithms that are effective in a wide variety of prediction and classification applications. All algorithms include an intuitive explanation of operation, essential equations, references to more rigorous theory, and commented C++ source code. Many of these techniques are recent developments, still not in widespread use. Others are standard algorithms given a fresh look. In every case, the focus is on practical applicability, with all code written in such a way that it can easily be included into any program. The Windows-based DATAMINE program lets you experiment with the techniques before incorporating them into your own work. What you'll learn Monte-Carlo permutation tests provide statistically sound assessment of relationships present in your data. Combinatorially symmetric cross validation reveals whether your model has true power or has just learned noise by overfitting the data. Feature weighting as regularized energy-based learning ranks variables according to their predictive power when there is too little data for traditional methods. The eigenstructure of a dataset enables clustering of variables into groups that exist only within meaningful subspaces of the data. Plotting regions of the variable space where there is disagreement between marginal and actual densities, or where contribution to mutual information is high, provides visual insight into anomalous relationships. Who this book is for The techniques presented in this book and in the DATAMINE program will be useful to anyone interested in discovering and exploiting relationships among variables. Although all code examples are written in C++, the algorithms are described in sufficient detail that they can easily be programmed in any language. Discover hidden relationships among the variables in your data, and learn how to exploit these relationships. This book presents a collection of data-mining algorithms that are effective in a wide variety of prediction and classification applications. All algorithms include an intuitive explanation of operation, essential equations, references to more rigorous theory, and commented C++ source code. Many of these techniques are recent developments, still not in widespread use. Others are standard algorithms given a fresh look. In every case, the focus is on practical applicability, with all code written in such a way that it can easily be included into any program. The Windows-based DATAMINE program lets you experiment with the techniques before incorporating them into your own work. What You'll Learn Use Monte-Carlo permutation tests to provide statistically sound assessments of relationships present in your data Discover how combinatorially symmetric cross validation reveals whether your model has true power or has just learned noise by overfitting the data Work with feature weighting as regularized energy-based learning to rank variables according to their predictive power when there is too little data for traditional methods See how the eigenstructure of a dataset enables clustering of variables into groups that exist only within meaningful subspaces of the data Plot regions of the variable space where there is disagreement between marginal and actual densities, or where contribution to mutual information is high Who This Book Is For Anyone interested in discovering and exploiting relationships among variables. Although all code examples are written in C++, the algorithms are described in sufficient detail that they can easily be programmed in any language. Find the various relationships among variables that can be present in big data as well as other data sets. This book also covers information entropy, permutation tests, combinatorics, predictor selections, and eigenvalues to give you a well-rounded view of data mining and algorithms in C++. Furthermore, 䡴ta Mining Algorithms in C++鮣ludes classic techniques that are widely available in standard statistical packages, such as maximum likelihood factor analysis and varimax rotation. After reading and using this book, you'll come away with many code samples and routines that can be repurposed into your own data mining tools and algorithms toolbox. This will allow you to integrate these techniques in your various data and analysis projects. ٯu will: Discover useful data mining techniques and algorithms using the C++ programming language Carry out permutation tests Work with the various relationships and screening types for these relationships Master predictor selections Use the DATAMINE program#xE000