**Explore intuitive data analysis techniques and powerful machine learning methods using over 130 practical recipes** About This Book* A practical and concise guide to using Haskell when getting to grips with data analysis * Recipes for every stage of data analysis, from collection to visualization * In-depth examples demonstrating various tools, solutions and techniques Who This Book Is ForThis book shows functional developers and analysts how to leverage their existing knowledge of Haskell specifically for high-quality data analysis. A good understanding of data sets and functional programming is assumed. What You Will Learn* Obtain and analyze raw data from various sources including text files, CSV files, databases, and websites * Implement practical tree and graph algorithms on various datasets * Apply statistical methods such as moving average and linear regression to understand patterns * Fiddle with parallel and concurrent code to speed up and simplify time-consuming algorithms * Find clusters in data using some of the most popular machine learning algorithms * Manage results by visualizing or exporting data In DetailThis book will take you on a voyage through all the steps involved in data analysis. It provides synergy between Haskell and data modeling, consisting of carefully chosen examples featuring some of the most popular machine learning techniques. You will begin with how to obtain and clean data from various sources. You will then learn how to use various data structures such as trees and graphs. The meat of data analysis occurs in the topics involving statistical techniques, parallelism, concurrency, and machine learning algorithms, along with various examples of visualizing and exporting results. By the end of the book, you will be empowered with techniques to maximize your potential when using Haskell for data analysis. Cover 1 Copyright 3 Credits 4 About the Author 5 About the Reviewers 6 www.PacktPub.com 8 Table of Contents 10 Preface 16 Chapter 1: The Hunt for Data 22 Introduction 23 Harnessing data from various sources 23 Accumulating text data from a file path 26 Catching I/O code faults 28 Keeping and representing data from a CSV file 30 Examining a JSON file with the aeson package 33 Reading an XML file using the HXT package 36 Capturing table rows from an HTML page 39 Understanding how to perform HTTP GET requests 41 Learning how to perform HTTP POST requests 43 Traversing online directories for data 44 Using MongoDB queries in Haskell 47 Reading from a remote MongoDB server 49 Exploring data from a SQLite database 51 Chapter 2: Integrity and Inspection 54 Introduction 55 Trimming excess whitespace 55 Ignoring punctuation and specific characters 57 Coping with unexpected or missing input 58 Validating records by matching regular expressions 61 Lexing and parsing an e-mail address 63 Deduplication of nonconflicting data items 64 Deduplication of conflicting data items 67 Implementing a frequency table using Data.List 70 Implementing a frequency table using Data.MultiSet 71 Computing the Manhattan distance 73 Computing the Euclidean distance 75 Comparing scaled data using the Pearson correlation coefficient 77 Comparing sparse data using cosine similarity 78 Chapter 3: The Science of Words 80 Introduction 81 Displaying a number in another base 81 Reading a number from another base 83 Searching for a substring using Data.ByteString 84 Searching a string using the Boyer-Moore-Horspool algorithm 86 Searching a string using the Rabin-Karp algorithm 88 Splitting a string on lines, words, or arbitrary tokens 90 Finding the longest common subsequence 92 Computing a phonetic code 93 Computing the edit distance 95 Computing the Jaro-Winkler distance between two strings 96 Finding strings within one-edit distance 99 Fixing spelling mistakes 101 Chapter 4: Data Hashing 106 Introduction 107 Hashing a primitive data type 107 Hashing a custom data type 110 Running popular cryptographic hash functions 112 Running a cryptographic checksum on a file 115 Performing fast comparisons between data types 117 Using a high-performance hash table 118 Using Google's CityHash hash functions for strings 121 Computing a Geohash for location coordinates 122 Using a bloom filter to remove unique items 123 Running MurmurHash, a simple but speedy hashing algorithm 125 Measuring image similarity with perceptual hashes 127 Chapter 5: The Dance with Trees 132 Introduction 133 Defining a binary tree data type 133 Defining a rose tree (multiway tree) data type 135 Traversing a tree depth-first 136 Traversing a tree breadth-first 138 Implementing a Foldable instance for a tree 140 Calculating the height of a tree 142 Implementing a binary search tree data structure 144 Verifying the order property of a binary search tree 146 Using a self-balancing tree 148 Implementing a min-heap data structure 150 Encoding a string using a Huffman tree 153 Decoding a Huffman code 156 Chapter 6: Graph Fundamentals 158 Introduction 159 Representing a graph from a list of edges 159 Representing a graph from an adjacency list 160 Conducting a topological sort on a graph 162 Traversing a graph depth-first 164 Traversing a graph breadth-first 165 Visualizing a graph using Graphviz 166 Using Directed Acyclic Word Graphs 167 Working with hexagonal and square grid networks 169 Finding maximal cliques in a graph 171 Determining whether any two graphs are isomorphic 172 Chapter 7: Statistics and Analysis 174 Introduction 175 Calculating a moving average 175 Calculating a moving median 177 Approximating a linear regression 180 Approximating a quadratic regression 182 Obtaining the covariance matrix from samples 183 Finding all unique pairings in a list 185 Using the Pearson correlation coefficient 186 Evaluating a Bayesian network 188 Creating a data structure for playing cards 190 Using a Markov chain to generate text 193 Creating n-grams from a list 194 Creating a neural network perceptron 195 Chapter 8: Clustering and Classification 200 Introduction 201 Implementing the k-means clustering algorithm 201 Implementing hierarchical clustering 205 Using a hierarchical clustering library 208 Finding the number of clusters 211 Clustering words by their lexemes 213 Classifying the parts of speech of words 215 Identifying key words in a corpus of text 216 Training a parts of speech tagger 219 Implementing a decision tree classifier 220 Implementing a k-Nearest Neighbors classifier 225 Visualizing points using Graphics.EasyPlot 228 Chapter 9: Parallel and Concurrent Design 230 Introduction 231 Using the Haskell Runtime System options 231 Evaluating a procedure in parallel 232 Controlling parallel algorithms in sequence 234 Forking I/O actions for concurrency 235 Communicating with a forked I/O action 236 Killing forked threads 238 Parallelizing pure functions using the Par monad 240 Mapping over a list in parallel 242 Accessing tuple elements in parallel 243 Implementing MapReduce to count word frequencies 244 Manipulating images in parallel using Repa 247 Benchmarking runtime performance in Haskell 250 Using the criterion package to measure performance 252 Benchmarking runtime performance in the terminal 254 Chapter 10: Real-time Data 256 Introduction 257 Streaming Twitter for real-time sentiment analysis 257 Reading IRC chat room messages 263 Responding to IRC messages 264 Polling a web server for latest updates 266 Detecting real-time file directory changes 267 Communicating in real time through sockets 269 Detecting faces and eyes through a camera stream 271 Streaming camera frames for template matching 274 Chapter 11: Visualizing Data 278 Introduction 279 Plotting a line chart using Google's Chart API 279 Plotting a pie chart using Google's Chart API 282 Plotting bar graphs using Google's Chart API 284 Displaying a line graph using gnuplot 287 Displaying a scatter plot of two-dimensional points 289 Interacting with points in a three-dimensional space 291 Visualizing a graph network 294 Customizing the looks of a graph network diagram 296 Rendering a bar graph in JavaScript using D3.js 299 Rendering a scatter plot in JavaScript using D3.js 301 Diagramming a path from a list of vectors 303 Chapter 12: Exporting and Presenting 308 Introduction 309 Exporting data to a CSV file 309 Exporting data as JSON 310 Using SQLite to store data 312 Saving data to a MongoDB database 313 Presenting results in an HTML web page 315 Creating a LaTeX table to display results 317 Personalizing messages using a text template 319 Exporting matrix values to a file 320 Index 322
In Detail
This book will take you on a voyage through all the steps involved in data analysis. It provides synergy between Haskell and data modeling, consisting of carefully chosen examples featuring some of the most popular machine learning techniques.
You will begin with how to obtain and clean data from various sources. You will then learn how to use various data structures such as trees and graphs. The meat of data analysis occurs in the topics involving statistical techniques, parallelism, concurrency, and machine learning algorithms, along with various examples of visualizing and exporting results. By the end of the book, you will be empowered with techniques to maximize your potential when using Haskell for data analysis.
Approach
Step-by-step recipes filled with practical code samples and engaging examples demonstrate Haskell in practice, and then the concepts behind the code.
Who this book is for
This book shows functional developers and analysts how to leverage their existing knowledge of Haskell specifically for high-quality data analysis. A good understanding of data sets and functional programming is assumed.
In Detail This book will take you on a voyage through all the steps involved in data analysis. It provides synergy between Haskell and data modeling, consisting of carefully chosen examples featuring some of the most popular machine learning techniques. You will begin with how to obtain and clean data from various sources. You will then learn how to use various data structures such as trees and graphs. The meat of data analysis occurs in the topics involving statistical techniques, parallelism, concurrency, and machine learning algorithms, along with various examples of visualizing and exporting results. By the end of the book, you will be empowered with techniques to maximize your potential when using Haskell for data analysis.Approach Step-by-step recipes filled with practical code samples and engaging examples demonstrate Haskell in practice, and then the concepts behind the code.Who this book is for This book shows functional developers and analysts how to leverage their existing knowledge of Haskell specifically for high-quality data analysis. A good understanding of data sets and functional programming is assumed
