Python microservices development : build, test, deploy, and scale microservices in Python

Key Features* A very useful guide for Python developers who are shifting to the new microservices-based development * A concise, up-to-date guide to building efficient and lightweight microservices in Python using Flask, Tox, and other tools * Learn to use Docker containers, CoreOS, and Amazon Web Services to deploy your services Book DescriptionWe often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other. This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you'll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them. You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker's role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services. This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microservices. What you will learn* Explore what microservices are and how to design them * Use Python 3, Flask, Tox, and other tools to build your services using best practices * Learn how to use a TDD approach * Discover how to document your microservices * Configure and package your code in the best way * Interact with other services * Secure, monitor, and scale your services * Deploy your services in Docker containers, CoreOS, and Amazon Web Services About the Author**Tarek Ziadé** is a Python developer, located in the countryside near Dijon, France. He works at Mozilla in the services team. He founded a French Python user group called Afpy, and he has written several books about Python in French and English. When he is not hacking on his computer or hanging out with his family, he's spending time between his two passions, running and playing the trumpet. You can visit his personal blog (Fetchez le Python) and follow him on Twitter (@tarek\_ziade). You can also take a look at one of his books on Amazon, Expert Python Programming, published by Packt. Table of Contents1. Understanding Microservices 2. Working with flask 3. Test-driven development and documentation, the virtuous cycle 4. Designing Forrest 5. Interacting with other services 6. Monitoring your services 7. Securing your services 8. Consuming the Microservices 9. Packaging Forrest 10. Dockerizing your services 11. Deploying on AWS 12. Thinking ahead Cover 1 Copyright 3 Credits 5 About the Author 6 About the Reviewer 7 www.PacktPub.com 8 Customer Feedback 9 Table of Contents 10 Preface 15 Introduction 21 Chapter 1: Understanding Microservices 22 Origins of Service-Oriented Architecture 23 The monolithic approach 24 The microservice approach 28 Microservice benefits 30 Separation of concerns 30 Smaller projects 30 Scaling and deployment 31 Microservices pitfalls 32 Illogical splitting 32 More network interactions 33 Data storing and sharing 33 Compatibility issues 34 Testing 34 Implementing microservices with Python 35 The WSGI standard 36 Greenlet and Gevent 37 Twisted and Tornado 39 asyncio 40 Language performances 43 Summary 45 Chapter 2: Discovering Flask 47 Which Python? 49 How Flask handles requests 49 Routing 53 Variables and converters 54 The url_for function 57 Request 58 Response 59 Flask built-in features 61 The session object 61 Globals 62 Signals 63 Extensions and middlewares 65 Templates 66 Configuration 68 Blueprints 70 Error handling and debugging 71 Custom error handler 72 The debug mode 74 A microservice skeleton 75 Summary 78 Chapter 3: Coding, Testing, and Documenting - the Virtuous Cycle 79 Different kinds of tests 81 Unit tests 82 Functional tests 85 Integration tests 87 Load tests 88 End-to-end tests 91 Using WebTest 93 Using pytest and Tox 94 Developer documentation 97 Continuous Integration 102 Travis-CI 103 ReadTheDocs 104 Coveralls 105 Summary 108 Chapter 4: Designing Runnerly 109 The Runnerly application 110 User stories 110 Monolithic design 112 Model 112 View and Template 113 Background tasks 117 Strava token 120 Authentication and authorization 121 Putting together the monolithic design 125 Splitting the monolith 126 Data Service 128 Using Open API 2.0 129 More splitting 132 Summary 133 Chapter 5: Interacting with Other Services 135 Synchronous calls 136 Using Session in a Flask app 137 Connection pooling 141 HTTP cache headers 143 Improving data transfer 146 GZIP compression 146 Binary payloads 148 Putting it together 151 Asynchronous calls 151 Task queues 152 Topic queues 153 Publish/subscribe 158 RPC over AMQP 158 Putting it together 159 Testing 159 Mocking synchronous calls 159 Mocking asynchronous calls 161 Mocking Celery 161 Mocking other asynchronous calls 163 Summary 164 Chapter 6: Monitoring Your Services 165 Centralizing logs 166 Setting up Graylog 168 Sending logs to Graylog 171 Adding extra fields 173 Performance metrics 175 System metrics 175 Code metrics 178 Web server metrics 180 Summary 182 Chapter 7: Securing Your Services 183 The OAuth2 protocol 184 Token-based authentication 186 The JWT standard 187 PyJWT 189 X.509 certificate-based authentication 190 The TokenDealer microservice 193 The POST/oauth/token implementation 194 Using TokenDealer 198 Web application firewall 200 OpenResty - Lua and nginx 202 Rate and concurrency limiting 205 Other OpenResty features 207 Securing your code 208 Asserting incoming data 208 Limiting your application scope 212 Using Bandit linter 213 Summary 216 Chapter 8: Bringing It All Together 218 Building a ReactJS dashboard 219 The JSX syntax 220 React components 221 ReactJS and Flask 224 Using Bower, npm, and Babel 225 Cross-origin resource sharing 229 Authentication and authorization 232 Interacting with Data Service 232 Getting the Strava token 233 JavaScript authentication 235 Summary 237 Chapter 9: Packaging and Running Runnerly 239 The packaging toolchain 240 A few definitions 241 Packaging 242 The setup.py file 242 The requirements.txt file 247 The MANIFEST.in file 249 Versioning 250 Releasing 253 Distributing 255 Running all microservices 258 Process management 260 Summary 264 Chapter 10: Containerized Services 265 What is Docker? 266 Docker 101 268 Running Flask in Docker 270 The full stack - OpenResty, Circus and Flask 272 OpenResty 273 Circus 275 Docker-based deployments 278 Docker Compose 279 Introduction to Clustering and Provisioning 281 Summary 284 Chapter 11: Deploying on AWS 285 AWS overview 286 Routing - Route53, ELB, and AutoScaling 288 Execution - EC2 and Lambda 288 Storage - EBS, S3, RDS, ElasticCache, and CloudFront 290 Messaging - SES, SQS, and SNS 291 Simple Email Service (SES) 291 Simple Queue Service (SQS) 292 Simple Notification Service (SNS) 292 Provisioning and deployment - CloudFormation and ECS 293 Deploying on AWS - the basics 294 Setting up your AWS account 294 Deploying on EC2 with CoreOS 298 Deploying with ECS 302 Route53 308 Summary 310 Chapter 12: What Next? 311 Iterators and generators 312 Coroutines 315 The asyncio library 317 The aiohttp framework 318 Sanic 319 Asynchronous versus synchronous 320 Summary 323 Index 324 Humble bundle_CDP.pdf 1 Table of Contents -1 Test -1 Index -1 Key Features A very useful guide for Python developers who are shifting to the new microservices-based development A concise, up-to-date guide to building efficient and lightweight microservices in Python using Flask, Tox, and other tools Learn to use Docker containers, CoreOS, and Amazon Web Services to deploy your services Book Description We often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other. This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you'll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them. You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker's role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services. This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microservices. What you will learn Explore what microservices are and how to design them Use Python 3, Flask, Tox, and other tools to build your services using best practices Learn how to use a TDD approach Discover how to document your microservices Configure and package your code in the best way Interact with other services Secure, monitor, and scale your services Deploy your services in Docker containers, CoreOS, and Amazon Web Services About the Author Tarek Ziadé is a Python developer, located in the countryside near Dijon, France. He works at Mozilla in the services team. He founded a French Python user group called Afpy, and he has written several books about Python in French and English. When he is not hacking on his computer or hanging out with his family, he's spending time between his two passions, running and playing the trumpet. You can visit his personal blog (Fetchez le Python) and follow him on Twitter (@tarek_ziade). You can also take a look at one of his books on Amazon, Expert Python Programming, published by Packt. Table of Contents Understanding Microservices Working with flask Test-driven development and documentation, the virtuous cycle Designing Forrest Interacting with other services Monitoring your services Securing your services Consuming the Microservices Packaging Forrest Dockerizing your services Deploying on AWS Thinking ahead A practical approach to conquering the complexities of Microservices using the Python tooling ecosystemAbout This Book* A very useful guide for Python developers who are shifting to the new microservices-based development* A concise, up-to-date guide to building efficient and lightweight microservices in Python using Flask, Tox, and other tools* Learn to use Docker containers, CoreOS, and Amazon Web Services to deploy your servicesWho This Book Is ForThis book is for developers who have basic knowledge of Python, the command line, and HTTP-based application principles, and those who want to learn how to build, test, scale, and manage Python 3 microservices. No prior experience of writing microservices in Python is assumed. What You Will Learn* Explore what microservices are and how to design them* Use Python 3, Flask, Tox, and other tools to build your services using best practices* Learn how to use a TDD approach* Discover how to document your microservices* Configure and package your code in the best way* Interact with other services* Secure, monitor, and scale your services* Deploy your services in Docker containers, CoreOS, and Amazon Web ServicesIn DetailWe often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other. This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you'll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them. You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker's role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services. This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microservices. Style and approachThis book is an linear, easy-to-follow guide on how to best design, write, test, and deploy your microservices. It includes real-world examples that will help Python developers create their own Python microservice using the most efficient methods A practical approach to conquering the complexities of Microservices using the Python tooling ecosystem About This Book A very useful guide for Python developers who are shifting to the new microservices-based development A concise, up-to-date guide to building efficient and lightweight microservices in Python using Flask, Tox, and other tools Learn to use Docker containers, CoreOS, and Amazon Web Services to deploy your services Who This Book Is For This book is for developers who have basic knowledge of Python, the command line, and HTTP-based application principles, and those who want to learn how to build, test, scale, and manage Python 3 microservices. No prior experience of writing microservices in Python is assumed. What You Will Learn Explore what microservices are and how to design them Use Python 3, Flask, Tox, and other tools to build your services using best practices Learn how to use a TDD approach Discover how to document your microservices Configure and package your code in the best way Interact with other services Secure, monitor, and scale your services Deploy your services in Docker containers, CoreOS, and Amazon Web Services In Detail We often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other. This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you'll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them. You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker's role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services. This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microse..