Representation in Machine Learning

This book provides a concise but comprehensive guide to representation, which forms the core of Machine Learning (ML). State-of-the-art practical applications involve a number of challenges for the analysis of high-dimensional data. Unfortunately, many popular ML algorithms fail to perform, in both theory and practice, when they are confronted with the huge size of the underlying data. Solutions to this problem are aptly covered in the book. In addition, the book covers a wide range of representation techniques that are important for academics and ML practitioners alike, such as Locality Sensitive Hashing (LSH), Distance Metrics and Fractional Norms, Principal Components (PCs), Random Projections and Autoencoders. Several experimental results are provided in the book to demonstrate the discussed techniques��� effectiveness Preface 6 Overview 6 Audience 7 Organization 7 Contents 8 Acronyms 10 1 Introduction 11 1.1 Machine Learning (ML) System 11 1.2 Main Steps in an ML System 13 1.2.1 Data Collection/Acquisition 13 1.2.2 Feature Engineering and Representation 17 1.2.3 Model Selection 24 1.2.4 Model Estimation 24 1.2.5 Model Validation 24 1.2.6 Model Explanation 25 1.3 Data Sets Used 25 1.4 Summary 26 References 26 2 Representation 27 2.1 Introduction 27 2.2 Representation in Problem Solving 28 2.3 Representation of Data Items 29 2.4 Representation of Classes 35 2.5 Representation of Clusters 36 2.6 Summary 38 References 38 3 Nearest Neighbor Algorithms 39 3.1 Introduction 39 3.2 Nearest Neighbors in High-Dimensional Spaces 40 3.3 Fractional Norms 48 3.4 Locality Sensitive Hashing (LSH) and Applications 51 3.5 Summary 54 References 55 4 Representation Using Linear Combinations 56 4.1 Introduction 56 4.2 Feature Selection 56 4.3 Principal Component Analysis 61 4.4 Random Projections 65 4.5 Non-negative Matrix Factorization 67 4.6 Summary 70 References 71 5 Non-linear Schemes for Representation 72 5.1 Introduction 72 5.2 Optimization Schemes for Representation 72 5.3 Visualization 73 5.4 Autoencoders for Representation 83 5.5 Experimental Results: ORL Data Set 88 5.6 Experimental Results: MNIST Data Set 89 5.7 Summary 94 References 94 6 Conclusions 96 References 98 Index 99