**Solve machine learning problems using probabilistic graphical models implemented in Python with real-world applications** * Stretch the limits of machine learning by learning how graphical models provide an insight on particular problems, especially in high dimension areas such as image processing and NLP * Solve real-world problems using Python libraries to run inferences using graphical models * A practical, step-by-step guide that introduces readers to representation, inference, and learning using Python libraries best suited to each task If you are a data scientist who knows about machine learning and want to enhance your knowledge of graphical models, such as Bayes network, in order to use them to solve real-world problems using Python libraries, this book is for you.This book is intended for those who have some Python and machine learning experience, or are exploring the machine learning field. * Create Bayesian networks and make inferences * Learn the structure of causal Bayesian networks from data * Gain an insight on algorithms that run inference * Explore parameter estimation in Bayes nets with PyMC sampling * Understand the complexity of running inference algorithms in Bayes networks * Discover why graphical models can trump powerful classifiers in certain problems With the increasing prominence in machine learning and data science applications, probabilistic graphical models are a new tool that machine learning users can use to discover and analyze structures in complex problems. The variety of tools and algorithms under the PGM framework extend to many domains such as natural language processing, speech processing, image processing, and disease diagnosis. You've probably heard of graphical models before, and you're keen to try out new landscapes in the machine learning area. This book gives you enough background information to get started on graphical models, while keeping the math to a minimum. Cover 1 Copyright 3 Credits 4 About the Author 5 About the Reviewers 6 www.PacktPub.com 8 Table of Contents 10 Preface 14 Chapter 1: Probability 18 The theory of probability 18 Goals of probabilistic inference 21 Conditional probability 22 The chain rule 22 The Bayes rule 22 Interpretations of probability 24 Random variables 26 Marginal distribution 26 Joint distribution 27 Independence 27 Conditional independence 28 Types of queries 29 Probability queries 29 MAP queries 29 Summary 31 Chapter 2: Directed Graphical Models 32 Graph terminology 32 Python digression 33 Independence and independent parameters 33 The Bayes network 36 The chain rule 37 Reasoning patterns 37 Causal reasoning 38 Evidential reasoning 40 Inter-causal reasoning 40 D-separation 42 The D-separation example 44 Blocking and unblocking a V-structure 46 Factorization and I-maps 47 The Naive Bayes model 47 The Naïve Bayes example 49 Summary 50 Chapter 3: Undirected Graphical Models 52 Pairwise Markov networks 52 The Gibbs distribution 54 An induced Markov network 56 Factorization 56 Flow of influence 57 Active trail and separation 58 Structured prediction 58 Problem of correlated features 59 The CRF representation 59 The CRF example 60 The factorization-independence tango 61 Summary 62 Chapter 4: Structure Learning 64 The structure learning landscape 65 Constraint-based structure learning 65 Part I 65 Part II 66 Part III 67 Summary of constraint-based approaches 73 Score-based learning 73 The likelihood score 74 The Bayesian information criterion score 75 The Bayesian score 76 Summary of score-based learning 81 Summary 81 Chapter 5: Parameter Learning 82 The likelihood function 84 Parameter learning example using MLE 85 MLE for Bayesian networks 87 Bayesian parameter learning example using MLE 88 Data fragmentation 90 Effect of data fragmentation on parameter estimation 90 Bayesian parameter estimation 92 An example of Bayesian methods for parameter learning 93 Bayesian estimation for the Bayesian network 98 Example of Bayesian estimation 98 Summary 104 Chapter 6: Exact Inference Using Graphical Models 106 Complexity of inference 106 Real-world issues 107 Using the Variable Elimination algorithm 107 Marginalizing factors that are not relevant 110 Factor reduction to filter on evidence 111 Shortcomings of the brute-force approach 113 Using the Variable Elimination approach 113 Complexity of Variable Elimination 119 Graph perspective 120 Learning the induced width from the graph structure 122 The tree algorithm 123 The four stages of the junction tree algorithm 124 Using the junction tree algorithm for inference 125 Stage 1.1 – moralization 126 Stage 1.2 – triangulation 127 Stage 1.3 – building the join tree 127 Stage 2 – initializing potentials 128 Stage 3 – message passing 128 Summary 132 Chapter 7: Approximate Inference Methods 134 The optimization perspective 134 Belief propagation on general graphs 135 Creating a cluster graph to run LBP 136 Message passing in LBP 137 Steps in the LBP algorithm 138 Improving the convergence of LBP 139 Applying LBP to segment an image 139 Understanding energy-based models 141 Visualizing unary and pairwise factors on a 3 x 3 grid 142 Creating the model for image segmentation 143 Applications of LBP 148 Sampling-based methods 149 Forward sampling 149 The accept-reject sampling method 150 The Markov Chain Monte Carlo sampling process 151 The Markov property 151 The Markov chain 152 Reaching a steady state 153 Sampling using a Markov chain 153 Gibbs sampling 154 Steps in the Gibbs sampling procedure 154 An example of Gibbs sampling 155 Summary 158 Appendix: References 160 Index 164 www.it-ebooks.info 'This is a short, practical guide that allows data scientists to understand the concepts of Graphical models and enables them to try them out using small Python code snippets, without being too mathematically complicated. If you are a data scientist who knows about machine learning and want to enhance your knowledge of graphical models, such as Bayes network, in order to use them to solve real-world problems using Python libraries, this book is for you.This book is intended for those who have some Python and machine learning experience, or are exploring the machine learning field.' This is a short, practical guide that allows data scientists to understand the concepts of Graphical models and enables them to try them out using small Python code snippets, without being too mathematically complicated. If you are a data scientist who knows about machine learning and want to enhance your knowledge of graphical models, such as Bayes network, in order to use them to solve real-world problems using Python libraries, this book is for you. This book is intended for those who have some Python and machine learning experience, or are exploring the machine learning field
