Building Machine Learning Systems with Python - Second Edition

Get more from your data through creating practical machine learning systems with Python About This Book Build your own Python-based machine learning systems tailored to solve any problem Discover how Python offers a multiple context solution for create machine learning systems Practical scenarios using the key Python machine learning libraries to successfully implement in your projects Who This Book Is For This book primarily targets Python developers who want to learn and use Python's machine learning capabilities and gain valuable insights from data to develop effective solutions for business problems. In Detail Using machine learning to gain deeper insights from data is a key skill required by modern application developers and analysts alike. Python is a wonderful language to develop machine learning applications. As a dynamic language, it allows for fast exploration and experimentation. With its excellent collection of open source machine learning libraries you can focus on the task at hand while being able to quickly try out many ideas. This book shows you exactly how to find patterns in your raw data. You will start by brushing up on your Python machine learning knowledge and introducing libraries. You'll quickly get to grips with serious, real-world projects on datasets, using modeling, creating recommendation systems. Later on, the book covers advanced topics such as topic modeling, basket analysis, and cloud computing. These will extend your abilities and enable you to create large complex systems. With this book, you gain the tools and understanding required to build your own systems, tailored to solve your real-world data analysis problems. Cover 1 Copyright 3 Credits 4 About the Authors 5 About the Reviewers 7 www.PacktPub.com 9 Table of Contents 10 Preface 16 Chapter 1: Getting Started with Python Machine Learning 22 Machine learning and Python – a dream team 23 What the book will teach you (and what it will not) 24 What to do when you are stuck 25 Getting started 26 Introduction to NumPy, SciPy, and matplotlib 27 Installing Python 27 Chewing data efficiently with NumPy and intelligently with SciPy 27 Learning NumPy 28 Indexing 30 Handling nonexisting values 31 Comparing the runtime 32 Learning SciPy 33 Our first (tiny) application of machine learning 34 Reading in the data 35 Preprocessing and cleaning the data 36 Choosing the right model and learning algorithm 38 Before building our first model... 39 Starting with a simple straight line 39 Towards some advanced stuff 41 Stepping back to go forward – another look at our data 43 Training and testing 47 Answering our initial question 48 Summary 49 Chapter 2: Classifying with Real-world Examples 50 The Iris dataset 51 Visualization is a good first step 51 Building our first classification model 53 Evaluation – holding out data and cross-validation 57 Building more complex classifiers 60 A more complex dataset and a more complex classifier 62 Learning about the Seeds dataset 62 Features and feature engineering 63 Nearest neighbor classification 64 Classifying with scikit-learn 64 Looking at the decision boundaries 66 Binary and multiclass classification 68 Summary 70 Chapter 3: Clustering – Finding Related Posts 72 Measuring the relatedness of posts 73 How not to do it 73 How to do it 74 Preprocessing – similarity measured as a similar number of common words 75 Converting raw text into a bag of words 75 Counting words 76 Normalizing word count vectors 79 Removing less important words 80 Stemming 81 Stop words on steroids 84 Our achievements and goals 86 Clustering 87 K-means 87 Getting test data to evaluate our ideas on 91 Clustering posts 93 Solving our initial challenge 94 Another look at noise 96 Tweaking the parameters 97 Summary 98 Chapter 4: Topic Modeling 100 Latent Dirichlet allocation 101 Building a topic model 102 Comparing documents by topics 107 Modeling the whole of Wikipedia 110 Choosing the number of topics 113 Summary 115 Chapter 5: Classification – Detecting Poor Answers 116 Sketching our roadmap 117 Learning to classify classy answers 117 Tuning the instance 117 Tuning the classifier 117 Fetching the data 118 Slimming the data down to chewable chunks 119 Preselection and processing of attributes 119 Defining what is a good answer 121 Creating our first classifier 121 Starting with kNN 121 Engineering the features 122 Training the classifier 124 Measuring the classifier's performance 124 Designing more features 125 Deciding how to improve 128 Bias-variance and their tradeoff 129 Fixing high bias 129 Fixing high variance 130 High bias or low bias 130 Using logistic regression 133 A bit of math with a small example 133 Applying logistic regression to our post classification problem 135 Looking behind accuracy – precision and recall 137 Slimming the classifier 141 Ship it! 142 Summary 142 Chapter 6: Classification II – Sentiment Analysis 144 Sketching our roadmap 144 Fetching the Twitter data 145 Introducing the Naïve Bayes classifier 145 Getting to know the Bayes' theorem 146 Being naïve 147 Using Naïve Bayes to classify 148 Accounting for unseen words and other oddities 152 Accounting for arithmetic underflows 153 Creating our first classifier and tuning it 155 Solving an easy problem first 156 Using all classes 159 Tuning the classifier's parameters 162 Cleaning tweets 167 Taking the word types into account 169 Determining the word types 169 Successfully cheating using SentiWordNet 171 Our first estimator 173 Putting everything together 176 Summary 177 Chapter 7: Regression 178 Predicting house prices with regression 178 Multidimensional regression 182 Cross-validation for regression 183 Penalized or regularized regression 184 L1 and L2 penalties 185 Using Lasso or ElasticNet in scikit-learn 186 Visualizing the Lasso path 187 P-greater-than-N scenarios 188 An example based on text documents 189 Setting hyperparameters in a principled way 191 Summary 195 Chapter 8: Recommendations 196 Rating predictions and recommendations 196 Splitting into training and testing 198 Normalizing the training data 199 A neighborhood approach to recommendations 201 A regression approach to recommendations 205 Combining multiple methods 207 Basket analysis 209 Obtaining useful predictions 211 Analyzing supermarket shopping baskets 211 Association rule mining 215 More advanced basket analysis 217 Summary 218 Chapter 9: Classification – Music Genre Classification 220 Sketching our roadmap 220 Fetching the music data 221 Converting into a WAV format 221 Looking at music 222 Decomposing music into sine wave components 224 Using FFT to build our first classifier 226 Increasing experimentation agility 226 Training the classifier 228 Using a confusion matrix to measure accuracy in multiclass problems 228 An alternative way to measure classifier performance using receiver-operator characteristics 231 Improving classification performance with Mel Frequency Cepstral Coefficients 235 Summary 239 Chapter 10: Computer Vision 240 Introducing image processing 240 Loading and displaying images 241 Thresholding 243 Gaussian blurring 244 Putting the center in focus 246 Basic image classification 249 Computing features from images 250 Writing your own features 251 Using features to find similar images 253 Classifying a harder dataset 255 Local feature representations 256 Summary 260 Chapter 11: Dimensionality Reduction 262 Sketching our roadmap 263 Selecting features 263 Detecting redundant features using filters 263 Correlation 264 Mutual information 267 Asking the model about the features using wrappers 272 Other feature selection methods 274 Feature extraction 275 About principal component analysis 275 Sketching PCA 276 Applying PCA 276 Limitations of PCA and how LDA can help 278 Multidimensional scaling 279 Summary 283 Chapter 12: Bigger Data 284 Learning about big data 285 Using jug to break up your pipeline into tasks 285 An introduction to tasks in jug 286 Looking under the hood 289 Using jug for data analysis 290 Reusing partial results 293 Using Amazon Web Services 295 Creating your first virtual machines 297 Installing Python packages on Amazon Linux 303 Running jug on our cloud machine 304 Automating the generation of clusters with StarCluster 305 Summary 309 Appendix: Where to Learn More Machine Learning 312 Online courses 312 Books 312 Question and answer sites 313 Blogs 313 Data sources 314 Getting competitive 314 All that was left out 314 Summary 315 Index 316 Get more from your data through creating practical machine learning systems with Python About This BookBuild your own Python-based machine learning systems tailored to solve any problemDiscover how Python offers a multiple context solution for create machine learning systemsPractical scenarios using the key Python machine learning libraries to successfully implement in your projectsWho This Book Is ForThis book primarily targets Python developers who want to learn and use Python's machine learning capabilities and gain valuable insights from data to develop effective solutions for business problems. What You Will LearnBuild a classification system that can be applied to text, images, or soundsUse NumPy, SciPy, scikit-learn scientific Python open source libraries for scientific computing and machine learningExplore the mahotas library for image processing and computer visionBuild a topic model for the whole of WikipediaEmploy Amazon Web Services to run analysis on the cloudDebug machine learning problemsGet to grips with recommendations using basket analysisRecommend products to users based on past purchasesIn DetailUsing machine learning to gain deeper insights from data is a key skill required by modern application developers and analysts alike. Python is a wonderful language to develop machine learning applications. As a dynamic language, it allows for fast exploration and experimentation. With its excellent collection of open source machine learning libraries you can focus on the task at hand while being able to quickly try out many ideas. This book shows you exactly how to find patterns in your raw data. You will start by brushing up on your Python machine learning knowledge and introducing libraries. Youll quickly get to grips with serious, real-world projects on datasets, using modeling, creating recommendation systems. Later on, the book covers advanced topics such as topic modeling, basket analysis, and cloud computing. These will extend your abilities and enable you to create large complex systems. With this book, you gain the tools and understanding required to build your own systems, tailored to solve your real-world data analysis problems.

About This Book

• Build full-featured web applications, such as Spring MVC applications, efficiently that will get you up and running with Spring web development
• Reuse working code snippets handy for integration scenarios such as Twitter, e-mail, FTP, databases, and many others
• An advanced guide which includes Java programs to integrate Spring with Thymeleaf

Who This Book Is For

If you are a Java developer with experience in developing applications with Spring, then this book is perfect for you. A good working knowledge of Spring programming conventions and applying dependency injections is recommended to make the most of this book.