Introduction to Machine Learning with Applications in Information Security (Chapman & Hall/CRC Machine Learning & Pattern Recognition)

Introduction to Machine Learning with Applications in Information Security, Second Edition provides a classroom-tested introduction to a wide variety of machine learning and deep learning algorithms and techniques, reinforced via realistic applications. The book is accessible and doesn't prove theorems, or dwell on mathematical theory. The goal is to present topics at an intuitive level, with just enough detail to clarify the underlying concepts. The book covers core classic machine learning topics in depth, including Hidden Markov Models (HMM), Support Vector Machines (SVM), and clustering. Additional machine learning topics include k-Nearest Neighbor (k-NN), boosting, Random Forests, and Linear Discriminant Analysis (LDA). The fundamental deep learning topics of backpropagation, Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Recurrent Neural Networks (RNN) are covered in depth. A broad range of advanced deep learning architectures are also presented, including Long Short-Term Memory (LSTM), Generative Adversarial Networks (GAN), Extreme Learning Machines (ELM), Residual Networks (ResNet), Deep Belief Networks (DBN), Bidirectional Encoder Representations from Transformers (BERT), and Word2Vec. Finally, several cutting-edge deep learning topics are discussed, including dropout regularization, attention, explainability, and adversarial attacks. Most of the examples in the book are drawn from the field of information security, with many of the machine learning and deep learning applications focused on malware. The applications presented serve to demystify the topics by illustrating the use of various learning techniques in straightforward scenarios. Some of the exercises in this book require programming, and elementary computing concepts are assumed in a few of the application sections. However, anyone with a modest amount of computing experience should have no trouble with this aspect of the book. Instructor resources, including PowerPoint slides, lecture videos, and other relevant materialare provided on an accompanying website: http://www.cs.sjsu.edu/~stamp/ML/ Introduction to Machine Learning with Applications in Information Security, Second Edition provides a classroom-tested introduction to a wide variety of machine learning and deep learning algorithms and techniques, reinforced via realistic applications. Cover 1 Half Title 2 Series Page 3 Title Page 4 Copyright Page 5 Dedication 6 Contents 8 Preface 16 About the Author 20 Acknowledgments 22 1. What is Machine Learning? 24 1.1. Introduction 24 1.2. About This Book 26 1.3. Necessary Background 26 1.4. A Note on Terminology 27 1.5. A Few Too Many Notes 28 I. Classic Machine Learning 30 2. A Revealing Introduction to Hidden Markov Models 32 2.1. Introduction and Background 32 2.2. Tree Rings and Temperature 34 2.3. Notation 36 2.4. The Three Problems 40 2.5. The Three Solutions 41 2.5.1. Scoring 41 2.5.2. Uncovering Hidden States 43 2.5.3. Training 44 2.6. Dynamic Programming 46 2.7. HMM Scaling 49 2.8. All Together Now 51 2.9. English Text Example 55 2.10. The Bottom Line 59 2.11. Problems 59 3. Principles of Principal Component Analysis 68 3.1. Introduction 68 3.2. Background 69 3.2.1. A Brief Review of Linear Algebra 70 3.2.2. Geometric View of Eigenvectors 74 3.2.3. Covariance Matrix 76 3.3. Principal Component Analysis 79 3.4. SVD Basics 85 3.5. All Together Now 86 3.5.1. Training Phase 86 3.5.2. Scoring Phase 88 3.6. A Numerical Example 90 3.7. The Bottom Line 93 3.8. Problems 94 4. A Reassuring Introduction to Support Vector Machines 104 4.1. Introduction 104 4.2. Constrained Optimization 112 4.2.1. Lagrange Multipliers 114 4.2.2. Lagrangian Duality 119 4.3. A Closer Look at SVM 121 4.3.1. Training and Scoring 123 4.3.2. Scoring Revisited 126 4.3.3. Support Vectors 126 4.3.4. Training and Scoring Re-revisited 127 4.3.5. The Kernel Trick 129 4.4. All Together Now 132 4.5. A Note on Quadratic Programming 133 4.6. The Bottom Line 137 4.7. Problems 137 5. A Comprehensible Collection of Clustering Concepts 146 5.1. Introduction 146 5.2. Overview and Background 147 5.3. K-Means 149 5.4. Measuring Cluster Quality 154 5.4.1. Internal Validation 156 5.4.2. External Validation 163 5.4.3. Visualizing Clusters 164 5.5. EM Clustering 167 5.5.1. Maximum Likelihood Estimator 169 5.5.2. An Elementary EM Example 170 5.5.3. EM Algorithm 174 5.5.4. Gaussian Mixture Example 179 5.6. The Bottom Line 186 5.7. Problems 187 6. Many Mini Topics 194 6.1. Introduction 194 6.2. k-Nearest Neighbors 194 6.3. Boost Your Knowledge of Boosting 197 6.3.1. Football Analogy 197 6.3.2. AdaBoost 198 6.3.3. Examples 202 6.4. Random Forest 208 6.5. Linear Discriminant Analysis 214 6.5.1. LDA Training 215 6.5.2. Numerical Example 222 6.6. The Bottom Line 225 6.7. Problems 225 II. Deep Learning 230 7. Deep Thoughts on Deep Learning 232 7.1. Introduction 232 7.2. A Brief History of Neural Networks 233 7.2.1. McCulloch-Pitts Neuron 233 7.2.2. Perceptron 234 7.2.3. Multilayer Perceptron 235 7.2.4. AI Winters and AI Summers 237 7.3. Why Deep Learning? 238 7.4. Decisions, Decisions 239 7.5. Basic Deep Learning Architectures 242 7.5.1. Feedforward Neural Networks 242 7.5.2. Convolutional Neural Networks 243 7.5.3. Recurrent Neural Networks 251 7.6. The Bottom Line 255 7.7. Problems 255 8. Onward to Backpropagation 260 8.1. Introduction 260 8.2. Automatic Differentiation 260 8.3. Backpropagation Example 268 8.3.1. Gradient Descent 268 8.3.2. MLP Example 271 8.4. Backpropagation Through Time 276 8.4.1. Vanishing and Exploding Gradients 277 8.4.2. Mitigating Gradient Issues 280 8.5. The Bottom Line 281 8.6. Problems 282 9. A Deeper Dive into Deep Learning 290 9.1. Introduction 290 9.2. Long Short-Term Memory 291 9.3. Gated Recurrent Unit 294 9.4. Generative Adversarial Networks 296 9.4.1. Generative and Discriminative Models 296 9.4.2. GAN Basics 299 9.4.3. GAN Training 300 9.5. Extreme Learning Machines 302 9.6. Residual Networks 304 9.7. Boltzmann Machines 306 9.7.1. Restricted Boltzmann Machine 307 9.7.2. Deep Belief Networks 308 9.7.3. Contrastive Divergence 309 9.8. Graph Neural Networks 315 9.9. Transfer Learning 318 9.10. The Bottom Line 319 9.11. Problems 320 10. Alphabet Soup of Deep Learning Topics 328 10.1. Introduction 328 10.2. Word Embedding Techniques 329 10.2.1. TF-IDF 330 10.2.2. HMM2Vec and PCA2Vec 331 10.2.3. Word2Vec 335 10.2.4. BERT 338 10.3. Multipart Methods 340 10.3.1. Ensembles 340 10.3.2. Combination Architectures 342 10.4. Overfitting 342 10.4.1. Regularization 343 10.4.2. Dropout 344 10.5. Attention 347 10.6. Explainability 349 10.7. Adversarial Attacks 353 10.8. The Bottom Line 356 10.9. Problems 357 III. Applications 364 11. HMMs for Classic Cryptanalysis 366 11.1. Introduction 366 11.2. Simple Substitutions 367 11.2.1. Jakobsen’s Algorithm 367 11.2.2. HMMs and Simple Substitutions 374 11.3. Homophonic Substitutions 380 11.4. Vigenere Cipher 383 11.4.1. Vigenere Cipher Example 383 11.4.2. Friedman Test 383 11.4.3. Experimental Results 386 11.5. Conclusion and Future Work 389 12. Image Spam Detection 392 12.1. Introduction 392 12.2. Eigenfaces 392 12.3. Eigenspam 394 12.3.1. PCA Experiments 395 12.3.2. Detection Results 397 12.4. SVM for Image Spam Detection 398 12.4.1. SVM Experiments 401 12.4.2. Improved Dataset 404 12.5. Conclusion and Future Work 406 13. Image-Based Malware Analysis 408 13.1. Introduction 408 13.2. Background 409 13.2.1. Transfer Learning Architectures 409 13.2.2. Dataset 409 13.3. Deep Learning Experiments and Results 411 13.3.1. MLP 412 13.3.2. CNN 414 13.3.3. RNN 417 13.3.4. Transfer Learning 419 13.3.5. Discussion 421 13.4. Conclusions and Future Work 422 14. Malware Evolution Detection 424 14.1. Introduction 424 14.2. Related Work 425 14.3. Design and Implementation 427 14.3.1. Dataset 427 14.3.2. Feature Extraction 430 14.3.3. Experimental Design 430 14.4. SVM Experimental Results 433 14.4.1. Juxtaposed Malware Families 433 14.4.2. Zbot Experiments 434 14.5. Additional Experiments 436 14.6. Conclusions and Future Work 437 IV. Extras 440 15. Experimental Design and Analysis 442 15.1. Introduction 442 15.2. Experimental Design 443 15.3. Accuracy 446 15.4. ROC Curves 450 15.5. Imbalance Problem 452 15.6. PR Curves 455 15.7. Accuracy, Loss, Overfitting, and Underfitting 457 15.8. The Bottom Line 459 15.9. Problems 459 16. Epilogue 464 16.1. Introduction 464 16.2. Summarizing Proust 464 16.3. The Goldilocks Principle 465 16.4. Machine Learning and Science Fiction 468 References 472 Index 488 Computer,and,network,security;,cruptography;,intrusion,detection;,malware,detection;,pattern,recognition;,statistical,learning. Computer and network security,cruptography,intrusion detection,malware detection,pattern recognition,statistical learning. Introduction to Machine Learning with Applications in Information Security, Second Edition provides a classroom-tested introduction to a wide variety of machine learning and deep learning algorithms and techniques, reinforced via realistic applications. The book is accessible and doesn't prove theorems, or dwell on mathematical theory. The goal is to present topics at an intuitive level, with just enough detail to clarify the underlying concepts.The book covers core classic machine learning topics in depth, including Hidden Markov Models (HMM), Support Vector Machines (SVM), and clustering. Additional machine learning topics include k-Nearest Neighbor (k-NN), boosting, Random Forests, and Linear Discriminant Analysis (LDA). The fundamental deep learning topics of backpropagation, Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Recurrent Neural Networks (RNN) are covered in depth. A broad range of advanced deep learning architectures are also presented, including Long Short-Term Memory (LSTM), Generative Adversarial Networks (GAN), Extreme Learning Machines (ELM), Residual Networks (ResNet), Deep Belief Networks (DBN), Bidirectional Encoder Representations from Transformers (BERT), and Word2Vec. Finally, several cutting-edge deep learning topics are discussed, including dropout regularization, attention, explainability, and adversarial attacks.Most of the examples in the book are drawn from the field of information security, with many of the machine learning and deep learning applications focused on malware. The applications presented serve to demystify the topics by illustrating the use of various learning techniques in straightforward scenarios. Some of the exercises in this book require programming, and elementary computing concepts are assumed in a few of the application sections. However, anyone with a modest amount of computing experience should have no trouble with this aspect of the book.Instructor resources, including PowerPoint slides, lecture videos, and other relevant material are provided on an accompanying website: http://www.cs.sjsu.edu/~stamp/ML/. "Introduction to Machine Learning with Applications in Information Security, Second Edition provides a classroom-tested introduction to a wide variety of machine learning and deep learning algorithms and techniques, reinforced via realistic applications. The book is accessible and doesn't prove theorems, or dwell on mathematical theory. The goal is to present topics at an intuitive level, with just enough detail to clarify the underlying concepts. The book covers core classic machine learning topics in depth, including Hidden Markov Models (HMM), Support Vector Machines (SVM), and clustering. Additional machine learning topics include k-Nearest Neighbor (k-NN), boosting, Random Forests, and Linear Discriminant Analysis (LDA). The fundamental deep learning topics of backpropagation, Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Recurrent Neural Networks (RNN) are covered in depth. A broad range of advanced deep learning architectures are also presented, including Long Short-Term Memory (LSTM), Generative Adversarial Networks (GAN), Extreme Learning Machines (ELM), Residual Networks (ResNet), Deep Belief Networks (DBN), Bidirectional Encoder Representations from Transformers (BERT), and Word2Vec. Finally, several cutting-edge deep learning topics are discussed, including dropout regularization, attention, explainability, and adversarial attacks. Most of the examples in the book are drawn from the field of information security, with many of the machine learning and deep learning applications focused on malware. The applications presented serve to demystify the topics by illustrating the use of various learning techniques in straightforward scenarios. Some of the exercises in this book require programming, and elementary computing concepts are assumed in a few of the application sections. However, anyone with a modest amount of computing experience should have no trouble with this aspect of the book"-- Provided by publisher