Daniel Arbuckle's Mastering Python : building powerful Python applications

Key Features* Covers the latest and advanced concepts of Python such as parallel processing with Python 3.6 * Explore the Python language from its basic installation and setup to concepts such as reactive programming and microservices * Get introduced to the mechanism for rewriting code in a compiled language along with ctypes and Cython tools Book DescriptionDaniel Arbuckle's Mastering Python covers the basics of operating in a Python development environment, before moving on to more advanced topics. Daniel presents you with real-world solutions to Python 3.6 and advanced-level concepts, such as reactive programming, microservices, ctypes, and Cython tools. You don't need to be familiar with the Python language to use this book, as Daniel starts with a Python primer. Throughout, Daniel highlights the major aspects of managing your Python development environment, shows you how to handle parallel computation, and helps you to master asynchronous I/O with Python 3.6 to improve performance. Finally, Daniel will teach you the secrets of metaprogramming and unit testing in Python, helping you acquire the perfect skillset to be a Python expert. Daniel will get you up to speed on everything from basic programming practices to high-end tools and techniques, things that will help set you apart as a successful Python programmer. What you will learn* Get to grips with the basics of operating in a Python development environment * Build Python packages to efficiently create reusable code * Become proficient at creating tools and utility programs in Python * Use the Git version control system to protect your development environment from unwanted changes * Harness the power of Python to automate other software * Distribute computational tasks across multiple processors * Handle high I/O loads with asynchronous I/O to get a smoother performance * Take advantage of Python's metaprogramming and programmable syntax features * Get acquainted with the concepts behind reactive programming and RxPy About the Author**Daniel Arbuckle** gained his PhD in Computer Science from the University of Southern California. He has published numerous papers along with several books and video courses, and he is both a teacher of computer science and a professional programmer. Table of Contents1. Python Primer 2. Setting Up 3. Making a Package 4. Basic Best Practices 5. Making a Command-Line Utility 6. Parallel Processing 7. Coroutines and Asynchronous I/O 8. Metaprogramming 9. Unit Testing 10. Reactive Programming 11. Microservices 12. Extension Modules and Compiled Code Cover......Page 1 Copyright......Page 3 Credits......Page 4 About the Author......Page 5 www.PacktPub.com......Page 6 Customer Feedback......Page 7 Table of Contents......Page 8 Preface......Page 15 Python basic syntax and block structure......Page 20 Functions......Page 22 Variables......Page 24 Expressions......Page 25 Classes......Page 26 Flow control statements......Page 27 Indentation......Page 28 Dictionaries......Page 29 List......Page 30 Tuple......Page 31 Set......Page 32 Comprehension......Page 33 First-class functions and classes......Page 34 The defaultdict class......Page 36 Attributes......Page 37 Different types of packages......Page 38 What's new in modern Python......Page 40 The changes in the syntactic......Page 41 Changes in packages......Page 42 Other changes in Python packages......Page 43 Summary......Page 44 Downloading and installing Python......Page 45 Installing Python......Page 46 Opening a command-line window......Page 48 Python interactive shell......Page 49 The pip tool for packages......Page 52 Managing installed packages......Page 53 Using Package Index......Page 56 Legalities and licenses of the Python Package Index......Page 57 Summary......Page 58 Creating an empty package......Page 59 Turning a regular folder into a package......Page 60 Importing all package modules......Page 61 Adding modules to the package......Page 63 Module loading with namespace packages......Page 64 The Package structure and interface......Page 65 Accessing code from other modules......Page 67 Importing a cyclic dependency......Page 68 Resolving attribute errors raised due to cyclic dependencies......Page 69 Adding static data files to the package......Page 70 Summary......Page 71 Chapter 4: Basic Best Practices......Page 72 PEP 8 — guidelines for Python code......Page 73 Code indentation......Page 74 Formatting recommendations......Page 75 Using version control......Page 77 Committing the changes in Git......Page 78 Undoing the changes......Page 79 Branches......Page 80 Merging codes......Page 82 The mergetool command......Page 83 Using venv to create a stable and isolated work area......Page 84 Activating a virtual environment......Page 85 pip in virtual environments......Page 86 PEP 257 and docutils......Page 87 Sphinx......Page 88 Turning docstrings into HTML......Page 90 Using doctest to test documentation examples......Page 94 Testing examples using doctest......Page 95 Summary......Page 96 Chapter 5: Making a Command-Line Utility......Page 97 Making a package executable via Python -m......Page 98 Pipeline program......Page 99 Creating an ArgumentParser object......Page 101 Setting the name of argument......Page 102 nargs......Page 104 Python's built-in functions - print and input......Page 106 The pprint package......Page 107 The cmd class......Page 108 The Pipeline user interface......Page 110 Subprocess and its variants......Page 112 The PIPE constant......Page 115 The wait method......Page 116 Finishing up our code example......Page 117 Creating launches for our program......Page 118 Summary......Page 119 Using the concurrent.futures package......Page 120 The concurrent.futures module......Page 122 Calling ProcessPoolExecutor......Page 123 Using the submit method......Page 125 The done and result methods......Page 127 The wait and as_completed functions......Page 129 The cancel method......Page 130 Using the multiprocessing packages......Page 131 Process class in the multiprocessing module......Page 132 Queues......Page 133 Pipes......Page 134 Manager......Page 135 The event object......Page 137 The condition object......Page 138 Summary......Page 139 Chapter 7: Coroutines and Asynchronous I/O......Page 140 The difference between asynchronous processing and parallel processing......Page 141 Multithreading is not good for servers......Page 142 Cooperative coroutine scheduler versus coroutine......Page 143 The coroutine scheduler......Page 144 Creating a coroutine......Page 145 ensure_future......Page 146 The run_forever/run_until_complete methods......Page 147 Closing event_loop......Page 149 asyncio's future objects......Page 150 Synchronization primitives......Page 152 The wait coroutine......Page 153 The gather coroutine......Page 155 The asyncio Queue class......Page 156 Creating a simple client in asyncio......Page 157 Handling client disconnections......Page 159 Summary......Page 160 Chapter 8: Metaprogramming......Page 161 Using the @ syntax in a function decorator......Page 162 Global decorator - @staticmethod......Page 163 Attributes......Page 164 Enclosing the function in a wrapper......Page 165 Function annotations......Page 166 Accessing annotation data......Page 167 Annotations as input to function decorators......Page 168 Keyword arguments......Page 169 Inspecting the package signature function......Page 170 Class decorators......Page 171 Modifying class attributes......Page 172 The factory function......Page 173 Class definitions......Page 174 Metaclasses......Page 176 The __prepare__method......Page 177 The __new__ method......Page 178 Context managers......Page 180 Defining a context manager as a generator......Page 181 Synchronous-coroutine-based context managers......Page 182 Descriptors......Page 184 Using @property to create a descriptor......Page 185 Writing descriptors as classes......Page 186 Summary......Page 188 Chapter 9: Unit Testing......Page 189 Understanding the principle of unit testing......Page 190 What is a unit test?......Page 191 Structuring a test file......Page 193 assert methods......Page 194 Comparing what happens to what should happen in unit tests......Page 196 Using unittest.mock......Page 197 Preconfiguring mock objects......Page 198 assert methods of mock objects......Page 200 The unittest.mock patch function......Page 201 Using unittest's test discovery......Page 202 Unittest's discovery tool......Page 203 Command-line options in unit test discovery......Page 204 Running our tests with nose......Page 206 The cover-package option......Page 207 Summary......Page 208 Chapter 10: Reactive Programming......Page 209 The concept of reactive programming......Page 210 Building a simple reactive programming framework......Page 211 Observables......Page 212 Emitting events......Page 213 Building the observable sequence......Page 214 Illustrating a stream of animal events......Page 215 Composing an observable sequence......Page 216 Using the reactive extensions for Python (RxPY)......Page 217 Translating our zoo demo into Rx......Page 218 Observable factory methods......Page 219 Combining and processing observable sequences......Page 220 The Observable.create method......Page 222 The Observable.select_many method......Page 223 Empty, return_value, and from_iterable factory methods......Page 224 The where factory method......Page 225 Summary......Page 226 Microservices and the advantages of process isolation......Page 227 Applying the microservice architecture to web servers......Page 228 Building high-level microservices with Flask......Page 229 Building a microservice to maintain a database......Page 230 Making Flask handle a request......Page 231 Running and connecting to our microservice using Flask......Page 234 Installing nameko......Page 236 Running and connecting a microservice using nameko......Page 238 Interacting with a microservice manually using the nameko shell......Page 239 Interacting with a microservice by creating another microservice......Page 240 Summary......Page 241 Chapter 12: Extension Modules and Compiled Code......Page 242 The downsides of compiled code......Page 243 Locating and linking a dynamic library......Page 245 Assigning attributes to a function......Page 247 Using a pointer as a parameter of a function......Page 248 Providing data structure layouts......Page 249 Working with Cython......Page 251 Additional import methods in Cython......Page 252 Writing extension modules in Cython......Page 253 Methods to increase the execution speed of Python code......Page 254 Using cpdef in a Cython class......Page 256 Compiling an extension module in Python......Page 257 Summary......Page 259 Index......Page 260 Gain a thorough understanding of operating in a Python development environment, and some of the most important advanced topics with Daniel Arbuckle. This dynamic, concise book is full of real-world solutions for Python 3.6 problems, and advanced-level concepts such as reactive programming, microservices, ctypes and Cython. About This Book • Covers the latest and advanced concepts of Python such as parallel processing with Python 3.6 • Explore the Python language from its basic installation and setup to concepts such as reactive programming and microservices • Get introduced to the mechanism for rewriting code in a compiled language along with ctypes and Cython tools Who This Book Is For If you are a programmer and are familiar with the basics of Python, and you want to broaden your knowledge base to develop projects better and faster, this book is for you. Even if you are not familiar with Python, Daniel Arbuckle's Mastering Python starts with the basics and takes you on a journey to become an expert in the technology. What You Will Learn • Get to grips with the basics of operating in a Python development environment • Build Python packages to efficiently create reusable code • Become proficient at creating tools and utility programs in Python • Use the Git version control system to protect your development environment from unwanted changes • Harness the power of Python to automate other software • Distribute computational tasks across multiple processors • Handle high I/O loads with asynchronous I/O to get a smoother performance • Take advantage of Python's metaprogramming and programmable syntax features • Get acquainted with the concepts behind reactive programming and RxPy In Detail Daniel Arbuckle's Mastering Python covers the basics of operating in a Python development environment, before moving on to more advanced topics. Daniel presents you with real-world solutions to Python 3.6 and advanced-level concepts, such as reactive programming, microservices, ctypes, and Cython tools. You don't need to be familiar with the Python language to use this book, as Daniel starts with a Python primer. Throughout, Daniel highlights the major aspects of managing your Python development environment, shows you how to handle parallel computation, and helps you to master asynchronous I/O with Python 3.6 to improve performance. Finally, Daniel will teach you the secrets of metaprogramming and unit testing in Python, helping you acquire the perfect skillset to be a Python expert. Daniel will get you up to speed on everything from basic programming practices to high-end tools and techniques, things that will help set you apart as a successful Python programmer. Style and Approach Daniel Arbuckle's Mastering Python covers basic to advanced-level concepts in computer science. If you are a beginner, then Daniel will help you get started. If you are experienced, he will expand your knowledge base.