Deep Learning with TensorFlow : Explore neural networks and build intelligent systems with Python, 2nd Edition

Delve into neural networks, implement deep learning algorithms, and explore layers of data abstraction with the help of this comprehensive TensorFlow guide ## Key Features * Learn how to implement advanced techniques in deep learning with Google's brainchild, TensorFlow * Explore deep neural networks and layers of data abstraction with the help of this comprehensive guide * Real-world contextualization through some deep learning problems concerning research and application ## Book Description Deep learning is the step that comes after machine learning, and has more advanced implementations. Machine learning is not just for academics anymore, but is becoming a mainstream practice through wide adoption, and deep learning has taken the front seat. As a data scientist, if you want to explore data abstraction layers, this book will be your guide. This book shows how this can be exploited in the real world with complex raw data and has been fully updated to the latest version of TensorFlow 1.x. Throughout the book, you'll learn how to implement deep learning algorithms for machine learning systems and integrate them into your product offerings, including search, image recognition, and language processing. Additionally, you'll learn how to analyze and improve the performance of deep learning models. This can be done by comparing algorithms against benchmarks, along with machine intelligence, to learn from the information and determine ideal behaviors within a specific context. After finishing the book, you will be familiar with machine learning techniques, in particular the use of TensorFlow for deep learning, and will be ready to apply your knowledge to research or commercial projects. ## What you will learn * Learn about machine learning landscapes along with the historical development and progress of deep learning * Learn about deep machine intelligence and GPU computing with the latest TensorFlow 1.x * Access public datasets and utilize them using TensorFlow to load, process, and transform data * Use TensorFlow on real-world datasets, including images, text, and more * Learn how to evaluate the performance of your deep learning models * Using deep learning for scalable object detection and mobile computing * Train machines quickly to learn from data by exploring reinforcement learning techniques * Explore active areas of deep learning research and applications ## Who This Book Is For The book is intended for a general audience of people interested in machine learning and machine intelligence. A rudimentary level of programming in one language is assumed, as is a basic familiarity with computer science techniques and technologies, including a basic awareness of computer hardware and algorithms. Some competence in mathematics is needed to the level of elementary linear algebra and calculus. Cover 1 Copyright 3 Packt Upsell 5 Contributors 6 Table of Contents 11 Preface 19 Chapter 1: Getting Started with Deep Learning 27 A soft introduction to machine learning 28 Supervised learning 30 Unbalanced data 32 Unsupervised learning 32 Reinforcement learning 34 What is deep learning? 34 Artificial neural networks 37 The biological neurons 38 The artificial neuron 39 How does an ANN learn? 41 ANNs and the backpropagation algorithm 41 Weight optimization 42 Stochastic gradient descent 43 Neural network architectures 44 Deep Neural Networks (DNNs) 44 Multilayer perceptron 45 Deep Belief Networks (DBNs) 46 Convolutional Neural Networks (CNNs) 48 AutoEncoders 50 Recurrent Neural Networks (RNNs) 52 Emergent architectures 53 Deep learning frameworks 53 Summary 56 Chapter 2: A First Look at TensorFlow 57 A general overview of TensorFlow 58 What's new in TensorFlow v1.6? 59 Nvidia GPU support optimized 60 Introducing TensorFlow Lite 60 Eager execution 61 Optimized Accelerated Linear Algebra (XLA) 61 Installing and configuring TensorFlow 62 TensorFlow computational graph 63 TensorFlow code structure 67 Eager execution with TensorFlow 70 Data model in TensorFlow 72 Tensor 72 Rank and shape 74 Data type 76 Variables 79 Fetches 80 Feeds and placeholders 81 Visualizing computations through TensorBoard 83 How does TensorBoard work? 83 Linear regression and beyond 85 Linear regression revisited for a real dataset 93 Summary 99 Chapter 3: Feed-Forward Neural Networks with TensorFlow 101 Feed-forward neural networks (FFNNs) 102 Feed-forward and backpropagation 103 Weights and biases 105 Activation functions 107 Using sigmoid 110 Using tanh 110 Using ReLU 110 Using softmax 111 Implementing a feed-forward neural network 111 Exploring the MNIST dataset 112 Softmax classifier 114 Implementing a multilayer perceptron (MLP) 121 Training an MLP 122 Using MLPs 124 Dataset description 125 Preprocessing 127 A TensorFlow implementation of MLP for client-subscription assessment 129 Deep Belief Networks (DBNs) 137 Restricted Boltzmann Machines (RBMs) 138 Construction of a simple DBN 141 Unsupervised pre-training 142 Supervised fine-tuning 143 Implementing a DBN with TensorFlow for client-subscription assessment 144 Tuning hyperparameters and advanced FFNNs 152 Tuning FFNN hyperparameters 152 Number of hidden layers 152 Number of neurons per hidden layer 153 Weight and biases initialization 154 Selecting the most suitable optimizer 155 GridSearch and randomized search for hyperparameters tuning 156 Regularization 156 Dropout optimization 159 Summary 162 Chapter 4: Convolutional Neural Networks 165 Main concepts of CNNs 166 CNNs in action 168 LeNet5 169 Implementing a LeNet-5 step by step 170 AlexNet 178 Transfer learning 180 Pretrained AlexNet 180 Dataset preparation 182 Fine-tuning implementation 183 VGG 186 Artistic style learning with VGG-19 188 Input images 189 Content extractor and loss 190 Style extractor and loss 193 Merger and total loss 194 Training 194 Inception-v3 197 Exploring Inception with TensorFlow 198 Emotion recognition with CNNs 199 Testing the model on your own image 211 Source code 213 Summary 216 Chapter 5: Optimizing TensorFlow Autoencoders 217 How does an autoencoder work? 218 Implementing autoencoders with TensorFlow 221 Improving autoencoder robustness 226 Implementing a denoising autoencoder 227 Implementing a convolutional autoencoder 233 Encoder 233 Decoder 234 Fraud analytics with autoencoders 243 Description of the dataset 243 Problem description 244 Exploratory data analysis 245 Training, validation, and testing set preparation 249 Normalization 250 Autoencoder as an unsupervised feature learning algorithm 250 Evaluating the model 255 Summary 259 Chapter 6: Recurrent Neural Networks 261 Working principles of RNNs 262 Implementing basic RNNs in TensorFlow 265 RNN and the long-term dependency problem 269 Bi-directional RNNs 270 RNN and the gradient vanishing-exploding problem 272 LSTM networks 275 GRU cell 278 Implementing an RNN for spam prediction 279 Data description and preprocessing 279 Developing a predictive model for time series data 286 Description of the dataset 286 Pre-processing and exploratory analysis 288 LSTM predictive model 290 Model evaluation 293 An LSTM predictive model for sentiment analysis 296 Network design 296 LSTM model training 297 Visualizing through TensorBoard 315 LSTM model evaluation 317 Human activity recognition using LSTM model 320 Dataset description 320 Workflow of the LSTM model for HAR 322 Implementing an LSTM model for HAR 323 Summary 333 Chapter 7: Heterogeneous and Distributed Computing 335 GPGPU computing 336 The GPGPU history 336 The CUDA architecture 337 The GPU programming model 338 The TensorFlow GPU setup 339 Update TensorFlow 339 GPU representation 340 Using a GPU 340 GPU memory management 342 Assigning a single GPU on a multi-GPU system 342 The source code for GPU with soft placement 343 Using multiple GPUs 344 Distributed computing 346 Model parallelism 346 Data parallelism 347 The distributed TensorFlow setup 349 Summary 351 Chapter 8: Advanced TensorFlow Programming 353 tf.estimator 353 Estimators 354 Graph actions 354 Parsing resources 354 Flower predictions 355 TFLearn 359 Installation 360 Titanic survival predictor 360 PrettyTensor 363 Chaining layers 363 Normal mode 363 Sequential mode 364 Branch and join 364 Digit classifier 364 Keras 368 Keras programming models 369 Sequential model 369 Functional API 374 Summary 379 Chapter 9: Recommendation Systems Using Factorization Machines 381 Recommendation systems 382 Collaborative filtering approaches 382 Content-based filtering approaches 384 Hybrid recommender systems 384 Model-based collaborative filtering 384 Movie recommendation using collaborative filtering 385 The utility matrix 385 Description of the dataset 388 Ratings data 388 Movies data 388 Users data 389 Exploratory analysis of the MovieLens dataset 390 Implementing a movie RE 396 Training the model with the available ratings 397 Inferencing the saved model 406 Generating the user-item table 406 Clustering similar movies 408 Movie rating prediction by users 412 Finding top k movies 413 Predicting top k similar movies 414 Computing user-user similarity 415 Evaluating the recommender system 416 Factorization machines for recommendation systems 419 Factorization machines 420 Cold-start problem and collaborative-filtering approaches 421 Problem definition and formulation 423 Dataset description 424 Workflow of the implementation 425 Preprocessing 427 Training the FM model 434 Improved factorization machines 439 Neural factorization machines 440 Dataset description 440 Using NFM for the movie recommendation 441 Summary 446 Chapter 10: Reinforcement Learning 447 The RL problem 448 OpenAI Gym 449 OpenAI environments 450 The env class 450 Installing and running OpenAI Gym 451 The Q-Learning algorithm 452 The FrozenLake environment 453 Deep Q-learning 457 Deep Q neural networks 457 The Cart-Pole problem 459 Deep Q-Network for the Cart-Pole problem 461 The Experience Replay method 462 Exploitation and exploration 463 The Deep Q-Learning training algorithm 464 Summary 471 Other Books You May Enjoy 473 Leave a review – let other readers know what you think 475 Index 477 Delve into neural networks, implement deep learning algorithms, and explore layers of data abstraction with the help of TensorFlow. About This Book Learn how to implement advanced techniques in deep learning with Google's brainchild, TensorFlow Explore deep neural networks and layers of data abstraction with the help of this comprehensive guide Gain real-world contextualization through some deep learning problems concerning research and application Who This Book Is For The book is for people interested in machine learning and machine intelligence. A rudimentary level of programming in one language is assumed, as is a basic familiarity with computer science techniques and technologies, including a basic awareness of computer hardware and algorithms. Some competence in mathematics is needed to the level of elementary linear algebra and calculus. What You Will Learn Apply deep machine intelligence and GPU computing with TensorFlow Access public datasets and use TensorFlow to load, process, and transform the data Discover how to use the high-level TensorFlow API to build more powerful applications Use deep learning for scalable object detection and mobile computing Train machines quickly to learn from data by exploring reinforcement learning techniques Explore active areas of deep learning research and applications In Detail Deep learning is a branch of machine learning algorithms based on learning multiple levels of abstraction. Neural networks, which are at the core of deep learning, are being used in predictive analytics, computer vision, natural language processing, time series forecasting, and to perform a myriad of other complex tasks. This book is conceived for developers, data analysts, machine learning practitioners and deep learning enthusiasts who want to build powerful, robust, and accurate predictive models with the power of TensorFlow, combined with other open source Python libraries. Throughout the book, you'll learn how to develop deep learning applications for machine learning systems using Feedforward Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Autoencoders, and Factorization Machines. Discover how to attain deep learning programming on GPU in a distributed way. You'll come away with an in-depth knowledge of machine learning techniques and the skills to apply them to real-world projects. Style and approach This step-by-step guide explores common, and not so common, deep neural networks, and shows ho.. Delve into neural networks, implement deep learning algorithms, and explore layers of data abstraction with the help of TensorFlow. Key FeaturesLearn how to implement advanced techniques in deep learning with Google's brainchild, TensorFlowExplore deep neural networks and layers of data abstraction with the help of this comprehensive guideGain real-world contextualization through some deep learning problems concerning research and applicationBook DescriptionDeep learning is a branch of machine learning algorithms based on learning multiple levels of abstraction. Neural networks, which are at the core of deep learning, are being used in predictive analytics, computer vision, natural language processing, time series forecasting, and to perform a myriad of other complex tasks. This book is conceived for developers, data analysts, machine learning practitioners and deep learning enthusiasts who want to build powerful, robust, and accurate predictive models with the power of TensorFlow, combined with other open source Python libraries. Throughout the book, you'll learn how to develop deep learning applications for machine learning systems using Feedforward Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Autoencoders, and Factorization Machines. Discover how to attain deep learning programming on GPU in a distributed way. You'll come away with an in-depth knowledge of machine learning techniques and the skills to apply them to real-world projects. What you will learnApply deep machine intelligence and GPU computing with TensorFlowAccess public datasets and use TensorFlow to load, process, and transform the dataDiscover how to use the high-level TensorFlow API to build more powerful applicationsUse deep learning for scalable object detection and mobile computingTrain machines quickly to learn from data by exploring reinforcement learning techniquesExplore active areas of deep learning research and applicationsWho this book is forThe book is for people interested in machine learning and machine intelligence. A rudimentary level of programming in one language is assumed, as is a basic familiarity with computer science techniques and technologies, including a basic awareness of computer hardware and algorithms. Some competence in mathematics is needed to the level of elementary linear algebra and calculus. Compliant with TensorFlow 1.7, this book introduces the core concepts of deep learning. Get implementation and research details on cutting-edge architectures and apply advanced concepts to your own projects. Develop your knowledge of deep neural networks through hands-on model building and examples of real-world data collection