Machine Learning A First Course for Engineers and Scientists [draft]

Notation Introduction Machine learning exemplified About this book Further reading Supervised learning: a first approach Supervised machine learning A distance-based method: k-NN A rule-based method: Decision trees Further reading Basic parametric models and a statistical perspective on learning Linear regression Classification and logistic regression Polynomial regression and regularization Generalized linear models Further reading Derivation of the normal equations Understanding, evaluating and improving the performance Expected new data error Enew: performance in production Estimating Enew The training error–generalization gap decomposition of Enew The bias-variance decomposition of Enew Additional tools for evaluating binary classifiers Further reading Learning parametric models Principles of parametric modeling Loss functions and likelihood-based models Regularization Parameter optimization Optimization with large datasets Hyperparameter optimization Further reading Neural networks and deep learning The neural network model Training a neural network Convolutional neural networks Dropout Further reading Derivation of the backpropagation equations Ensemble methods: Bagging and boosting Bagging Random forests Boosting and AdaBoost Gradient boosting Further reading Nonlinear input transformations and kernels Creating features by nonlinear input transformations Kernel ridge regression Support vector regression Kernel theory Support vector classification Further reading The representer theorem Derivation of support vector classification The Bayesian approach and Gaussian processes The Bayesian idea Bayesian linear regression The Gaussian process Practical aspects of the Gaussian process Other Bayesian methods in machine learning Further reading The multivariate Gaussian distribution Generative models and learning from unlabeled data The Gaussian mixture model and discriminant analysis Cluster analysis Deep generative models Representation learning and dimensionality reduction Further reading User aspects of machine learning Defining the machine learning problem Improving a machine learning model What if we cannot collect more data? Practical data issues Can I trust my machine learning model? Further reading Ethics in machine learning Fairness and error functions Misleading claims about performance Limitations of training data Further reading Notation Bibliography