Building Chatbots with Python : Using Natural Language Processing and Machine Learning

Build your own chatbot using Python and open source tools. This book begins with an introduction to chatbots where you will gain vital information on their architecture. You will then dive straight into natural language processing with the natural language toolkit (NLTK) for building a custom language processing platform for your chatbot. With this foundation, you will take a look at different natural language processing techniques so that you can choose the right one for you. The next stage is to learn to build a chatbot using the API.ai platform and define its intents and entities. During this example, you will learn to enable communication with your bot and also take a look at key points of its integration and deployment. The final chapter of __Building Chatbots with Python__ teaches you how to build, train, and deploy your very own chatbot. Using open source libraries and machine learning techniques you will learn to predict conditions for your bot and develop a conversational agent as a web application. Finally you will deploy your chatbot on your own server with AWS. **What You Will Learn** * Gain the basics of natural language processing using Python * Collect data and train your data for the chatbot * Build your chatbot from scratch as a web app * Integrate your chatbots with Facebook, Slack, and Telegram * Deploy chatbots on your own server **Who This Book Is For** Intermediate Python developers who have no idea about chatbots. Developers with basic Python programming knowledge can also take advantage of the book. Table of Contents 5 About the Author 10 About the Technical Reviewer 11 Acknowledgments 12 Introduction 13 Chapter 1: The Beloved Chatbots 16 Popularity of Chatbots Usage 17 The Zen of Python and Why It Applies to Chatbots? 18 The Need for Chatbots 20 The Business Perspective 20 Chatbots Bring Revenue 21 A Glimpse of Chatbot Usage 23 Customers Prefer Chatbots 24 The Developer’s Perspective 25 Feature Releases and Bug Fixes 26 Market Demand 26 Learning Curve 27 Industries Impacted by Chatbots 27 Brief Timeline of Chatbots 28 1950 28 1966 29 1972 29 1981 29 1985 29 1992 29 1995 29 1996 30 2001 30 2006 30 2010 30 2012 30 2014 30 2015 31 2016 31 2017 31 What Kind of Problems Can I Solve Using Chatbots? 31 Can the Problem be Solved by Simple Question and Answer or Back-and-Forth Communication? 32 Does It Have Highly Repetitive Issues That Require Either Analyzing or Fetching of Data? 32 Can Your Bot’s Task be Automated and Fixed? 33 A QnA Bot 33 Starting With Chatbots 35 Decision Trees in Chatbots 35 Using Decision Trees in Chatbots 36 How Does a Decision Tree Help? 36 The Best Chatbots/Bot Frameworks 40 Components of a Chatbot and Terminologies Used 41 Intent 42 Entities 42 Utterances 42 Training the Bot 43 Confidence Score 43 Chapter 2: Natural Language Processing for Chatbots 44 Why Do I Need to Know Natural Language Processing to Build a Chatbot? 44 What Is spaCy? 46 Benchmarks Results of spaCy 46 What Does spaCy Provide? 47 World’s Fastest Library 47 Get Things Done 47 Deep Learning 47 Features of spaCy 47 Installation and Prerequisites 48 What Are SpaCy Models? 50 Fundamental Methods of NLP for Building Chatbots 52 POS Tagging 52 Stemming and Lemmatization 57 Named-Entity Recognition 59 Stop Words 62 Dependency Parsing 64 What Are Ancestors in Dependency Parsing? 65 What Are Children in Dependency Parsing? 66 Interactive Visualization for Dependency Parsing 66 What Is the Use of Dependency Parsing in Chatbots? 68 Noun Chunks 69 Finding Similarity 70 Good to Know Things in NLP for Chatbots 73 Tokenization 74 Regular Expressions 75 Summary 76 Chapter 3: Building Chatbots the Easy Way 77 Introduction to Dialogflow 77 Getting Started 79 Building a Food-Ordering Chatbot 79 Deciding the Scope 79 Listing Intents 80 Listing Entities 80 Building a Food Ordering Chatbot 80 Getting Started With Dialogflow 81 Points to Remember When Creating Intents 85 Creating Intents and Adding Utterances 86 Adding Default Response to the Intent 86 Item Description Intent and Belonging Entities 87 Understanding and Replying Back to the User 91 Order Status Intent 92 User_Order_ID Intent 93 User_Thanks Intent 95 Deploying Dialogflow Chatbot on the Web 96 Integrate Dialogflow Chatbot on Facebook Messenger 100 Setting Up Facebook 100 Creating a Facebook App 101 Setting Up the Dialogflow Console 102 Callback URLs 103 Verify Token 103 Access Tokens 103 Configuring Webhooks 104 Testing the Messenger Bot 105 Fulfillment 110 Enabling Webhook 112 Checking the Response 115 Summary 117 Chapter 4: Building Chatbots the Hard Way 118 What Is Rasa NLU? 119 Why Should I Use Rasa NLU? 119 Diving Straight Into Rasa NLU 120 Installing Rasa 120 Deciding a Pipeline in Rasa 121 Training and Building a Chatbot From Scratch 122 Building a Horoscope Bot 122 Conversation Script Between the Horoscope Bot and the User 123 Preparing Data for Chatbot 124 Creating Data for Model in JSON Format 125 Visualizing and Modifying Rasa’s JSON Data Format 127 Training the Chatbot Model 129 Creating a Configuration File 130 Writing Python Code to Train the Model and Predict 131 Training the Model 132 Predicting From the Model 132 Dialog Management Using Rasa Core 134 Understanding More on Rasa Core and Dialog System 135 Understanding Rasa Concepts 138 Action 138 Slots 139 Templates 140 Creating Domain File for the Chatbot 140 Writing Custom Actions of the Chatbot 143 Data Preparation for Training the Bot 146 Creating Story Data 147 Interactive Learning 149 Training the Chatbot Agent Model 149 Real-Time Training by Reinforcement 153 rasa-nlu-sdk 155 Exporting Conversations As Stories 163 Testing the Bot 165 Test Case 1 165 Test Case 2 166 Summary 166 Chapter 5: Deploying Your Chatbot 168 First Steps 168 Rasa’s Credential Management 168 Deploying the Chatbot on Facebook 170 Creating an App on Heroku 170 Setting Up Heroku on Your Local System 171 Creating and Setting Up an App at Facebook 171 Creating and Deploying Rasa Actions Server App on Heroku 174 Creating Rasa Chatbot API App 176 Creating a Standalone Script for Facebook Messenger Chatbot 177 Verifying the Deployment of Our Dialog Management App on Heroku 180 Integrating Webhook With Facebook 180 Post-Deployment Verification: Facebook Chatbot 182 Deploying the Chatbot on Slack 184 Creating a Standalone Script for Slack Chatbot 184 Editing your Procfile 188 Final Deployment of Slack Bot to Heroku 188 Subscribe to Slack Events 188 Subscribe to Bot Events 189 Post-Deployment Verification: Slack Bot 190 Deploying the Chatbot on Your Own 191 Writing a Script for Your Own Chatbot Channel 192 Writing the Procfile and Deploying to the Web 194 Verifying Your Chatbot APIs 194 Creating the Chatbot UI 196 Summary 200 Index 202 Front Matter ....Pages i-xix The Beloved Chatbots (Sumit Raj)....Pages 1-28 Natural Language Processing for Chatbots (Sumit Raj)....Pages 29-61 Building Chatbots the Easy Way (Sumit Raj)....Pages 63-103 Building Chatbots the Hard Way (Sumit Raj)....Pages 105-154 Deploying Your Chatbot (Sumit Raj)....Pages 155-188 Back Matter ....Pages 189-192