“Hellmann’s writing has become an indispensable resource for me and many others as it fills a critical gap in Python Documentation with examples.” — Jesse Noller, Python Core Developer and PSF Board Member Master the Powerful Python Standard Library through Real Code Examples The Python Standard Library contains hundreds of modules for interacting with the operating system, interpreter, and Internet—all extensively tested and ready to jump-start your application development. The Python Standard Library by Example introduces virtually every important area of the Python 2.7 library through concise, stand-alone source code/output examples, designed for easy learning and reuse. Building on his popular Python Module of the Week blog series, author and Python expert Doug Hellmann focuses on “showing” not “telling.” He explains code behavior through downloadable examples that fully demonstrate each feature. You’ll find practical code for working with text, data types, algorithms, math, file systems, networking, the Internet, XML, email, cryptography, concurrency, runtime and language services, and much more. Each section fully covers one module, and links to valuable additional resources, making this book an ideal tutorial and reference. Coverage includes Manipulating text with string, textwrap, re, and difflib Implementing data structures: collections, array, queue, struct, copy, and more Reading, writing, and manipulating files and directories Regular expression pattern matching Exchanging data and providing for persistence Archiving and data compression Managing processes and threads Using application “building blocks”: parsing command-line options, prompting for passwords, scheduling events, and logging Testing, debugging, and compilation Controlling runtime configuration Using module and package utilities If you’re new to Python, this book will quickly give you access to a whole new world of functionality. If you’ve worked with Python before, you’ll discover new, powerful solutions and better ways to use the modules you’ve already tried. Contents......Page 10 Tables......Page 32 Foreword......Page 34 Acknowledgments......Page 38 About the Author......Page 40 INTRODUCTION......Page 42 1 TEXT......Page 44 1.1.1 Functions......Page 45 1.1.2 Templates......Page 46 1.1.3 Advanced Templates......Page 48 1.2.1 Example Data......Page 50 1.2.3 Removing Existing Indentation......Page 51 1.2.4 Combining Dedent and Fill......Page 52 1.2.5 Hanging Indents......Page 53 1.3 Re—Regular Expressions......Page 54 1.3.2 Compiling Expressions......Page 55 1.3.3 Multiple Matches......Page 56 1.3.4 Pattern Syntax......Page 57 1.3.5 Constraining the Search......Page 69 1.3.6 Dissecting Matches with Groups......Page 71 1.3.7 Search Options......Page 78 1.3.8 Looking Ahead or Behind......Page 86 1.3.9 Self-Referencing Expressions......Page 91 1.3.10 Modifying Strings with Patterns......Page 97 1.3.11 Splitting with Patterns......Page 99 1.4 Difflib—Compare Sequences......Page 102 1.4.1 Comparing Bodies of Text......Page 103 1.4.2 Junk Data......Page 106 1.4.3 Comparing Arbitrary Types......Page 107 2 DATA STRUCTURES......Page 110 2.1.1 Counter......Page 111 2.1.2 Defaultdict......Page 115 2.1.3 Deque......Page 116 2.1.4 Namedtuple......Page 120 2.1.5 OrderedDict......Page 123 2.2.1 Initialization......Page 125 2.2.3 Arrays and Files......Page 126 2.2.4 Alternate Byte Ordering......Page 127 2.3 Heapq—Heap Sort Algorithm......Page 128 2.3.1 Example Data......Page 129 2.3.2 Creating a Heap......Page 130 2.3.3 Accessing Contents of a Heap......Page 131 2.3.4 Data Extremes from a Heap......Page 133 2.4.1 Inserting in Sorted Order......Page 134 2.4.2 Handling Duplicates......Page 136 2.5.1 Basic FIFO Queue......Page 137 2.5.2 LIFO Queue......Page 138 2.5.3 Priority Queue......Page 139 2.5.4 Building a Threaded Podcast Client......Page 140 2.6.2 Packing and Unpacking......Page 143 2.6.3 Endianness......Page 144 2.6.4 Buffers......Page 146 2.7 Weakref—Impermanent References to Objects......Page 147 2.7.1 References......Page 148 2.7.3 Proxies......Page 149 2.7.4 Cyclic References......Page 150 2.7.5 Caching Objects......Page 155 2.8 Copy—Duplicate Objects......Page 158 2.8.2 Deep Copies......Page 159 2.8.3 Customizing Copy Behavior......Page 160 2.8.4 Recursion in Deep Copy......Page 161 2.9.1 Printing......Page 164 2.9.2 Formatting......Page 165 2.9.4 Recursion......Page 166 2.9.6 Controlling Output Width......Page 167 3.1 Functools—Tools for Manipulating Functions......Page 170 3.1.1 Decorators......Page 171 3.1.2 Comparison......Page 179 3.2 Itertools—Iterator Functions......Page 182 3.2.1 Merging and Splitting Iterators......Page 183 3.2.2 Converting Inputs......Page 186 3.2.3 Producing New Values......Page 187 3.2.4 Filtering......Page 189 3.2.5 Grouping Data......Page 192 3.3 Operator—Functional Interface to Built-in Operators......Page 194 3.3.2 Comparison Operators......Page 195 3.3.3 Arithmetic Operators......Page 196 3.3.4 Sequence Operators......Page 198 3.3.5 In-Place Operators......Page 199 3.3.6 Attribute and Item “Getters”......Page 200 3.3.7 Combining Operators and Custom Classes......Page 202 3.3.8 Type Checking......Page 203 3.4 Contextlib—Context Manager Utilities......Page 204 3.4.1 Context Manager API......Page 205 3.4.2 From Generator to Context Manager......Page 208 3.4.3 Nesting Contexts......Page 209 3.4.4 Closing Open Handles......Page 210 4.1 Time—Clock Time......Page 214 4.1.2 Processor Clock Time......Page 215 4.1.3 Time Components......Page 217 4.1.4 Working with Time Zones......Page 218 4.1.5 Parsing and Formatting Times......Page 220 4.2 Datetime—Date and Time Value Manipulation......Page 221 4.2.1 Times......Page 222 4.2.2 Dates......Page 223 4.2.3 Timedeltas......Page 226 4.2.4 Date Arithmetic......Page 227 4.2.5 Comparing Values......Page 228 4.2.6 Combining Dates and Times......Page 229 4.2.7 Formatting and Parsing......Page 230 4.2.8 Time Zones......Page 231 4.3.1 Formatting Examples......Page 232 4.3.2 Calculating Dates......Page 235 5.1 Decimal—Fixed and Floating-Point Math......Page 238 5.1.1 Decimal......Page 239 5.1.2 Arithmetic......Page 240 5.1.3 Special Values......Page 241 5.1.4 Context......Page 242 5.2.1 Creating Fraction Instances......Page 248 5.2.3 Approximating Values......Page 251 5.3.1 Generating Random Numbers......Page 252 5.3.2 Seeding......Page 253 5.3.3 Saving State......Page 254 5.3.4 Random Integers......Page 255 5.3.5 Picking Random Items......Page 256 5.3.6 Permutations......Page 257 5.3.7 Sampling......Page 259 5.3.8 Multiple Simultaneous Generators......Page 260 5.3.9 SystemRandom......Page 262 5.3.10 Nonuniform Distributions......Page 263 5.4.1 Special Constants......Page 264 5.4.2 Testing for Exceptional Values......Page 265 5.4.3 Converting to Integers......Page 267 5.4.4 Alternate Representations......Page 268 5.4.5 Positive and Negative Signs......Page 270 5.4.6 Commonly Used Calculations......Page 271 5.4.7 Exponents and Logarithms......Page 275 5.4.8 Angles......Page 279 5.4.9 Trigonometry......Page 281 5.4.10 Hyperbolic Functions......Page 284 5.4.11 Special Functions......Page 285 6 THE FILE SYSTEM......Page 288 6.1.1 Parsing Paths......Page 289 6.1.2 Building Paths......Page 293 6.1.3 Normalizing Paths......Page 294 6.1.4 File Times......Page 295 6.1.5 Testing Files......Page 296 6.1.6 Traversing a Directory Tree......Page 297 6.2 Glob—Filename Pattern Matching......Page 298 6.2.2 Wildcards......Page 299 6.2.3 Single Character Wildcard......Page 300 6.2.4 Character Ranges......Page 301 6.3.1 Test Data......Page 302 6.3.2 Reading Specific Lines......Page 303 6.3.4 Error Handling......Page 304 6.3.5 Reading Python Source Files......Page 305 6.4.1 Temporary Files......Page 306 6.4.3 Temporary Directories......Page 309 6.4.4 Predicting Names......Page 310 6.4.5 Temporary File Location......Page 311 6.5.1 Copying Files......Page 312 6.5.2 Copying File Metadata......Page 315 6.5.3 Working with Directory Trees......Page 317 6.6.1 Reading......Page 320 6.6.2 Writing......Page 321 6.6.3 Regular Expressions......Page 324 6.7.1 Unicode Primer......Page 325 6.7.2 Working with Files......Page 328 6.7.3 Byte Order......Page 330 6.7.4 Error Handling......Page 332 6.7.5 Standard Input and Output Streams......Page 336 6.7.6 Encoding Translation......Page 339 6.7.7 Non-Unicode Encodings......Page 341 6.7.8 Incremental Encoding......Page 342 6.7.9 Unicode Data and Network Communication......Page 344 6.7.10 Defining a Custom Encoding......Page 348 6.8.1 Examples......Page 355 6.9.1 Simple Matching......Page 356 6.9.2 Filtering......Page 358 6.9.3 Translating Patterns......Page 359 6.10.1 Listing Directory Contents......Page 360 6.10.2 Annotated Listings......Page 362 6.11 FIlecmp—Compare Files......Page 363 6.11.1 Example Data......Page 364 6.11.2 Comparing Files......Page 366 6.11.3 Comparing Directories......Page 368 6.11.4 Using Differences in a Program......Page 369 7 DATA PERSISTENCE AND EXCHANGE......Page 374 7.1 Pickle—Object Serialization......Page 375 7.1.2 Encoding and Decoding Data in Strings......Page 376 7.1.3 Working with Streams......Page 377 7.1.4 Problems Reconstructing Objects......Page 379 7.1.6 Circular References......Page 381 7.2.1 Creating a New Shelf......Page 384 7.2.2 Writeback......Page 385 7.2.3 Specific Shelf Types......Page 387 7.3.1 Database Types......Page 388 7.3.2 Creating a New Database......Page 389 7.3.4 Error Cases......Page 390 7.4 Whichdb—Identify DBM-Style Database Formats......Page 391 7.5 Sqlite3—Embedded Relational Database......Page 392 7.5.1 Creating a Database......Page 393 7.5.2 Retrieving Data......Page 396 7.5.3 Query Metadata......Page 398 7.5.4 Row Objects......Page 399 7.5.5 Using Variables with Queries......Page 400 7.5.6 Bulk Loading......Page 403 7.5.7 Defining New Column Types......Page 404 7.5.8 Determining Types for Columns......Page 407 7.5.9 Transactions......Page 409 7.5.10 Isolation Levels......Page 413 7.5.12 Exporting the Contents of a Database......Page 417 7.5.13 Using Python Functions in SQL......Page 419 7.5.14 Custom Aggregation......Page 421 7.5.15 Custom Sorting......Page 422 7.5.16 Threading and Connection Sharing......Page 424 7.5.17 Restricting Access to Data......Page 425 7.6.1 Parsing an XML Document......Page 428 7.6.2 Traversing the Parsed Tree......Page 429 7.6.3 Finding Nodes in a Document......Page 431 7.6.4 Parsed Node Attributes......Page 432 7.6.5 Watching Events While Parsing......Page 434 7.6.6 Creating a Custom Tree Builder......Page 437 7.6.7 Parsing Strings......Page 439 7.6.8 Building Documents with Element Nodes......Page 441 7.6.9 Pretty-Printing XML......Page 442 7.6.10 Setting Element Properties......Page 444 7.6.11 Building Trees from Lists of Nodes......Page 446 7.6.12 Serializing XML to a Stream......Page 449 7.7.1 Reading......Page 452 7.7.2 Writing......Page 453 7.7.3 Dialects......Page 454 7.7.4 Using Field Names......Page 459 8.1 Zlib—GNU zlib Compression......Page 462 8.1.1 Working with Data in Memory......Page 463 8.1.2 Incremental Compression and Decompression......Page 464 8.1.3 Mixed Content Streams......Page 465 8.1.4 Checksums......Page 466 8.1.5 Compressing Network Data......Page 467 8.2 Ggzip—Read and Write GNU Zip Files......Page 471 8.2.1 Writing Compressed Files......Page 472 8.2.2 Reading Compressed Data......Page 474 8.2.3 Working with Streams......Page 475 8.3.1 One-Shot Operations in Memory......Page 477 8.3.2 Incremental Compression and Decompression......Page 479 8.3.3 Mixed Content Streams......Page 480 8.3.4 Writing Compressed Files......Page 481 8.3.5 Reading Compressed Files......Page 483 8.3.6 Compressing Network Data......Page 484 8.4.1 Testing Tar Files......Page 489 8.4.2 Reading Metadata from an Archive......Page 490 8.4.3 Extracting Files from an Archive......Page 491 8.4.5 Using Alternate Archive Member Names......Page 494 8.4.6 Writing Data from Sources Other than Files......Page 495 8.4.7 Appending to Archives......Page 496 8.4.8 Working with Compressed Archives......Page 497 8.5.2 Reading Metadata from an Archive......Page 498 8.5.3 Extracting Archived Files from an Archive......Page 500 8.5.4 Creating New Archives......Page 501 8.5.6 Writing Data from Sources Other than Files......Page 503 8.5.7 Writing with a ZipInfo Instance......Page 504 8.5.8 Appending to Files......Page 505 8.5.9 Python ZIP Archives......Page 507 8.5.10 Limitations......Page 508 9.1 Hashlib—Cryptographic Hashing......Page 510 9.1.3 SHA-1 Example......Page 511 9.1.4 Creating a Hash by Name......Page 512 9.1.5 Incremental Updates......Page 513 9.2 Hmac—Cryptographic Message Signing and Verification......Page 514 9.2.2 SHA vs. MD5......Page 515 9.2.3 Binary Digests......Page 516 9.2.4 Applications of Message Signatures......Page 517 10.1 Subprocess—Spawning Additional Processes......Page 522 10.1.1 Running External Commands......Page 523 10.1.2 Working with Pipes Directly......Page 527 10.1.3 Connecting Segments of a Pipe......Page 530 10.1.4 Interacting with Another Command......Page 531 10.1.5 Signaling between Processes......Page 533 10.2 Signal—Asynchronous System Events......Page 538 10.2.1 Receiving Signals......Page 539 10.2.2 Retrieving Registered Handlers......Page 540 10.2.4 Alarms......Page 542 10.2.6 Signals and Threads......Page 543 10.3.1 Thread Objects......Page 546 10.3.2 Determining the Current Thread......Page 548 10.3.3 Daemon vs. Non-Daemon Threads......Page 550 10.3.4 Enumerating All Threads......Page 553 10.3.5 Subclassing Thread......Page 554 10.3.6 Timer Threads......Page 556 10.3.7 Signaling between Threads......Page 557 10.3.8 Controlling Access to Resources......Page 558 10.3.9 Synchronizing Threads......Page 564 10.3.10 Limiting Concurrent Access to Resources......Page 565 10.3.11 Thread-Specific Data......Page 567 10.4.1 Multiprocessing Basics......Page 570 10.4.2 Importable Target Functions......Page 571 10.4.3 Determining the Current Process......Page 572 10.4.4 Daemon Processes......Page 573 10.4.5 Waiting for Processes......Page 575 10.4.6 Terminating Processes......Page 577 10.4.7 Process Exit Status......Page 578 10.4.8 Logging......Page 580 10.4.9 Subclassing Process......Page 581 10.4.10 Passing Messages to Processes......Page 582 10.4.11 Signaling between Processes......Page 586 10.4.12 Controlling Access to Resources......Page 587 10.4.13 Synchronizing Operations......Page 588 10.4.14 Controlling Concurrent Access to Resources......Page 589 10.4.15 Managing Shared State......Page 591 10.4.16 Shared Namespaces......Page 592 10.4.17 Process Pools......Page 594 10.4.18 Implementing MapReduce......Page 596 11.1 Socket—Network Communication......Page 602 11.1.1 Addressing, Protocol Families, and Socket Types......Page 603 11.1.2 TCP/IP Client and Server......Page 613 11.1.3 User Datagram Client and Server......Page 621 11.1.4 UNIX Domain Sockets......Page 624 11.1.5 Multicast......Page 628 11.1.6 Sending Binary Data......Page 632 11.1.7 Nonblocking Communication and Timeouts......Page 634 11.2 Select—Wait for I/O Efficiently......Page 635 11.2.1 Using select()......Page 636 11.2.2 Nonblocking I/O with Timeouts......Page 642 11.2.3 Using poll()......Page 644 11.2.4 Platform-Specific Options......Page 649 11.3.2 Server Objects......Page 650 11.3.5 Echo Example......Page 651 11.3.6 Threading and Forking......Page 657 11.4.1 Servers......Page 660 11.4.2 Clients......Page 662 11.4.3 The Event Loop......Page 664 11.4.4 Working with Other Event Loops......Page 666 11.4.5 Working with Files......Page 669 11.5.1 Message Terminators......Page 670 11.5.2 Server and Handler......Page 671 11.5.3 Client......Page 673 11.5.4 Putting It All Together......Page 675 12 THE INTERNET......Page 678 12.1.1 Parsing......Page 679 12.1.2 Unparsing......Page 682 12.1.3 Joining......Page 683 12.2.1 HTTP GET......Page 685 12.2.2 HTTP POST......Page 687 12.2.3 Threading and Forking......Page 689 12.2.4 Handling Errors......Page 690 12.2.5 Setting Headers......Page 691 12.3.1 Simple Retrieval with Cache......Page 692 12.3.2 Encoding Arguments......Page 694 12.3.3 Paths vs. URLs......Page 696 12.4.1 HTTP GET......Page 698 12.4.2 Encoding Arguments......Page 701 12.4.4 Adding Outgoing Headers......Page 702 12.4.5 Posting Form Data from a Request......Page 704 12.4.6 Uploading Files......Page 705 12.4.7 Creating Custom Protocol Handlers......Page 708 12.5.1 Base64 Encoding......Page 711 12.5.2 Base64 Decoding......Page 712 12.5.3 URL-Safe Variations......Page 713 12.5.4 Other Encodings......Page 714 12.6.1 Robots.txt......Page 715 12.6.2 Testing Access Permissions......Page 716 12.6.3 Long-Lived Spiders......Page 717 12.7 Cookie—HTTP Cookies......Page 718 12.7.2 Morsels......Page 719 12.7.3 Encoded Values......Page 721 12.7.4 Receiving and Parsing Cookie Headers......Page 722 12.7.5 Alternative Output Formats......Page 723 12.7.6 Deprecated Classes......Page 724 12.8.1 UUID 1—IEEE 802 MAC Address......Page 725 12.8.2 UUID 3 and 5—Name-Based Values......Page 727 12.8.3 UUID 4—Random Values......Page 729 12.8.4 Working with UUID Objects......Page 730 12.9.1 Encoding and Decoding Simple Data Types......Page 731 12.9.2 Human-Consumable vs. Compact Output......Page 733 12.9.3 Encoding Dictionaries......Page 735 12.9.4 Working with Custom Types......Page 736 12.9.5 Encoder and Decoder Classes......Page 738 12.9.6 Working with Streams and Files......Page 741 12.9.7 Mixed Data Streams......Page 742 12.10 Xmlrpclib—Client Library for XML-RPC......Page 743 12.10.1 Connecting to a Server......Page 745 12.10.2 Data Types......Page 747 12.10.3 Passing Objects......Page 750 12.10.4 Binary Data......Page 751 12.10.6 Combining Calls into One Message......Page 753 12.11.1 A Simple Server......Page 755 12.11.2 Alternate API Names......Page 757 12.11.3 Dotted API Names......Page 759 12.11.4 Arbitrary API Names......Page 760 12.11.5 Exposing Methods of Objects......Page 761 12.11.6 Dispatching Calls......Page 763 12.11.7 Introspection API......Page 765 13.1 Smtplib—Simple Mail Transfer Protocol Client......Page 768 13.1.1 Sending an Email Message......Page 769 13.1.2 Authentication and Encryption......Page 771 13.1.3 Verifying an Email Address......Page 773 13.2.1 Mail Server Base Class......Page 775 13.2.3 Proxy Server......Page 778 13.3 Imaplib—IMAP4 Client Library......Page 779 13.3.2 Connecting to a Server......Page 780 13.3.4 Listing Mailboxes......Page 782 13.3.5 Mailbox Status......Page 785 13.3.6 Selecting a Mailbox......Page 786 13.3.7 Searching for Messages......Page 787 13.3.8 Search Criteria......Page 788 13.3.9 Fetching Messages......Page 790 13.3.10 Whole Messages......Page 793 13.3.11 Uploading Messages......Page 794 13.3.12 Moving and Copying Messages......Page 796 13.3.13 Deleting Messages......Page 797 13.4 Mailbox—Manipulate Email Archives......Page 799 13.4.1 Mbox......Page 800 13.4.2 Maildir......Page 803 13.4.3 Other Formats......Page 809 14 APPLICATION BUILDING BLOCKS......Page 810 14.1 Getopt—Command-Line Option Parsing......Page 811 14.1.2 Short-Form Options......Page 812 14.1.4 A Complete Example......Page 813 14.1.6 GNU-Style Option Parsing......Page 816 14.2.1 Creating an OptionParser......Page 818 14.2.2 Short- and Long-Form Options......Page 819 14.2.3 Comparing with getopt......Page 820 14.2.4 Option Values......Page 822 14.2.5 Option Actions......Page 825 14.2.6 Help Messages......Page 831 14.3 Argparse—Command-Line Option and Argument Parsing......Page 836 14.3.4 Parsing a Command Line......Page 837 14.3.5 Simple Examples......Page 838 14.3.6 Automatically Generated Options......Page 846 14.3.7 Parser Organization......Page 848 14.3.8 Advanced Argument Processing......Page 856 14.4.1 Configuring......Page 864 14.4.2 Completing Text......Page 865 14.4.3 Accessing the Completion Buffer......Page 869 14.4.4 Input History......Page 873 14.4.5 Hooks......Page 875 14.5.1 Example......Page 877 14.5.2 Using getpass without a Terminal......Page 878 14.6.1 Processing Commands......Page 880 14.6.2 Command Arguments......Page 881 14.6.3 Live Help......Page 883 14.6.4 Auto-Completion......Page 884 14.6.5 Overriding Base Class Methods......Page 886 14.6.6 Configuring Cmd through Attributes......Page 888 14.6.7 Running Shell Commands......Page 889 14.6.8 Alternative Inputs......Page 890 14.6.9 Commands from sys.argv......Page 892 14.7.1 Quoted Strings......Page 893 14.7.2 Embedded Comments......Page 895 14.7.4 Including Other Sources of Tokens......Page 896 14.7.5 Controlling the Parser......Page 897 14.7.6 Error Handling......Page 899 14.7.7 POSIX vs. Non-POSIX Parsing......Page 900 14.8 ConfigParser—Work with Configuration Files......Page 902 14.8.2 Reading Configuration Files......Page 903 14.8.3 Accessing Configuration Settings......Page 905 14.8.4 Modifying Settings......Page 910 14.8.5 Saving Configuration Files......Page 912 14.8.6 Option Search Path......Page 913 14.8.7 Combining Values with Interpolation......Page 916 14.9.1 Logging in Applications vs. Libraries......Page 919 14.9.3 Rotating Log Files......Page 920 14.9.4 Verbosity Levels......Page 921 14.9.5 Naming Logger Instances......Page 923 14.10.1 Converting M3U Files to RSS......Page 924 14.10.2 Progress Metadata......Page 927 14.10.3 In-Place Filtering......Page 928 14.11.1 Examples......Page 931 14.11.2 When Are atexit Functions Not Called?......Page 932 14.11.3 Handling Exceptions......Page 934 14.12 Sched—Timed Event Scheduler......Page 935 14.12.1 Running Events with a Delay......Page 936 14.12.2 Overlapping Events......Page 937 14.12.4 Canceling Events......Page 938 15.1 Gettext—Message Catalogs......Page 940 15.1.2 Creating Message Catalogs from Source Code......Page 941 15.1.3 Finding Message Catalogs at Runtime......Page 944 15.1.4 Plural Values......Page 946 15.1.5 Application vs. Module Localization......Page 948 15.1.6 Switching Translations......Page 949 15.2.1 Probing the Current Locale......Page 950 15.2.2 Currency......Page 956 15.2.3 Formatting Numbers......Page 957 15.2.5 Dates and Times......Page 958 16 DEVELOPER TOOLS......Page 960 16.1.2 HTML Help......Page 961 16.2 Doctest—Testing through Documentation......Page 962 16.2.1 Getting Started......Page 963 16.2.2 Handling Unpredictable Output......Page 965 16.2.3 Tracebacks......Page 969 16.2.4 Working around Whitespace......Page 971 16.2.5 Test Locations......Page 977 16.2.6 External Documentation......Page 980 16.2.7 Running Tests......Page 983 16.2.8 Test Context......Page 986 16.3.2 Running Tests......Page 990 16.3.3 Test Outcomes......Page 991 16.3.4 Asserting Truth......Page 993 16.3.5 Testing Equality......Page 994 16.3.6 Almost Equal?......Page 995 16.3.7 Testing for Exceptions......Page 996 16.3.8 Test Fixtures......Page 997 16.3.9 Test Suites......Page 998 16.4.1 Supporting Functions......Page 999 16.4.2 Working with Exceptions......Page 1000 16.4.3 Working with the Stack......Page 1004 16.5 Cgitb—Detailed Traceback Reports......Page 1006 16.5.2 Enabling Detailed Tracebacks......Page 1007 16.5.3 Local Variables in Tracebacks......Page 1009 16.5.4 Exception Properties......Page 1012 16.5.6 Logging Tracebacks......Page 1013 16.6 Pdb—Interactive Debugger......Page 1016 16.6.1 Starting the Debugger......Page 1017 16.6.2 Controlling the Debugger......Page 1020 16.6.3 Breakpoints......Page 1031 16.6.4 Changing Execution Flow......Page 1043 16.6.5 Customizing the Debugger with Aliases......Page 1050 16.6.6 Saving Configuration Settings......Page 1052 16.7 Trace—Follow Program Flow......Page 1053 16.7.2 Tracing Execution......Page 1054 16.7.3 Code Coverage......Page 1055 16.7.4 Calling Relationships......Page 1058 16.7.5 Programming Interface......Page 1059 16.7.6 Saving Result Data......Page 1061 16.8 Profile and pstats—Performance Analysis......Page 1063 16.8.1 Running the Profiler......Page 1064 16.8.2 Running in a Context......Page 1067 16.8.3 Pstats: Saving and Working with Statistics......Page 1068 16.8.4 Limiting Report Contents......Page 1069 16.8.5 Caller / Callee Graphs......Page 1070 16.9.1 Module Contents......Page 1072 16.9.2 Basic Example......Page 1073 16.9.3 Storing Values in a Dictionary......Page 1074 16.9.4 From the Command Line......Page 1076 16.10.1 Compiling One Directory......Page 1078 16.10.2 Compiling sys.path......Page 1079 16.11 Pyclbr—Class Browser......Page 1080 16.11.1 Scanning for Classes......Page 1082 16.11.2 Scanning for Functions......Page 1083 17 RUNTIME FEATURES......Page 1086 17.1.1 Import Path......Page 1087 17.1.2 User Directories......Page 1088 17.1.3 Path Configuration Files......Page 1090 17.1.4 Customizing Site Configuration......Page 1092 17.1.5 Customizing User Configuration......Page 1094 17.1.6 Disabling the site Module......Page 1095 17.2.1 Interpreter Settings......Page 1096 17.2.2 Runtime Environment......Page 1103 17.2.3 Memory Management and Limits......Page 1106 17.2.4 Exception Handling......Page 1112 17.2.5 Low-Level Thread Support......Page 1115 17.2.6 Modules and Imports......Page 1121 17.2.7 Tracing a Program as It Runs......Page 1142 17.3.1 Process Owner......Page 1149 17.3.2 Process Environment......Page 1152 17.3.4 Pipes......Page 1153 17.3.6 File System Permissions......Page 1157 17.3.7 Directories......Page 1159 17.3.8 Symbolic Links......Page 1160 17.3.9 Walking a Directory Tree......Page 1161 17.3.10 Running External Commands......Page 1162 17.3.11 Creating Processes with os.fork()......Page 1163 17.3.12 Waiting for a Child......Page 1166 17.3.14 File System Permissions......Page 1168 17.4.1 Interpreter......Page 1170 17.4.2 Platform......Page 1171 17.4.3 Operating System and Hardware Info......Page 1172 17.4.4 Executable Architecture......Page 1174 17.5.1 Current Usage......Page 1175 17.5.2 Resource Limits......Page 1176 17.6.1 Tracing References......Page 1179 17.6.2 Forcing Garbage Collection......Page 1182 17.6.3 Finding References to Objects that Cannot Be Collected......Page 1187 17.6.4 Collection Thresholds and Generations......Page 1189 17.6.5 Debugging......Page 1192 17.7.1 Configuration Variables......Page 1201 17.7.2 Installation Paths......Page 1204 17.7.3 Python Version and Platform......Page 1208 18 LANGUAGE TOOLS......Page 1210 18.1.1 Categories and Filtering......Page 1211 18.1.2 Generating Warnings......Page 1212 18.1.3 Filtering with Patterns......Page 1213 18.1.4 Repeated Warnings......Page 1215 18.1.5 Alternate Message Delivery Functions......Page 1216 18.1.6 Formatting......Page 1217 18.1.7 Stack Level in Warnings......Page 1218 18.2.2 How Abstract Base Classes Work......Page 1219 18.2.4 Implementation through Subclassing......Page 1220 18.2.5 Concrete Methods in ABCs......Page 1222 18.2.6 Abstract Properties......Page 1223 18.3 Dis—Python Bytecode Disassembler......Page 1227 18.3.2 Disassembling Functions......Page 1228 18.3.3 Classes......Page 1230 18.3.4 Using Disassembly to Debug......Page 1231 18.3.5 Performance Analysis of Loops......Page 1233 18.3.6 Compiler Optimizations......Page 1239 18.4.1 Example Module......Page 1241 18.4.2 Module Information......Page 1242 18.4.3 Inspecting Modules......Page 1244 18.4.4 Inspecting Classes......Page 1245 18.4.5 Documentation Strings......Page 1247 18.4.6 Retrieving Source......Page 1248 18.4.7 Method and Function Arguments......Page 1250 18.4.8 Class Hierarchies......Page 1251 18.4.9 Method Resolution Order......Page 1253 18.4.10 The Stack and Frames......Page 1254 18.5.1 Base Classes......Page 1257 18.5.2 Raised Exceptions......Page 1258 18.5.3 Warning Categories......Page 1274 19.1 Imp—Python’s Import Mechanism......Page 1276 19.1.2 Module Types......Page 1277 19.1.3 Finding Modules......Page 1278 19.1.4 Loading Modules......Page 1279 19.2.1 Example......Page 1281 19.2.2 Finding a Module......Page 1282 19.2.3 Accessing Code......Page 1283 19.2.4 Source......Page 1284 19.2.6 Data......Page 1285 19.3.1 Package Import Paths......Page 1288 19.3.2 Development Versions of Packages......Page 1290 19.3.3 Managing Paths with PKG Files......Page 1292 19.3.4 Nested Packages......Page 1294 19.3.5 Package Data......Page 1296 M......Page 1300 Z......Page 1301 A......Page 1302 B......Page 1305 C......Page 1306 D......Page 1309 E......Page 1312 F......Page 1314 G......Page 1316 I......Page 1318 L......Page 1321 M......Page 1322 N......Page 1324 O......Page 1325 P......Page 1326 Q......Page 1329 R......Page 1330 S......Page 1332 T......Page 1337 U......Page 1340 W......Page 1341 X......Page 1342 Z......Page 1343 &;Hellmann&;s writing has become an indispensable resource for me and many others as it fills a critical gap in Python Documentation with examples.&; &; Jesse Noller, Python Core Developer and PSF Board Member Master the Powerful Python Standard Library through Real Code Examples The Python Standard Library contains hundreds of modules for interacting with the operating system, interpreter, and Internet&;all extensively tested and ready to jump-start your application development. The Python Standard Library by Example (2 Volume Set) introduces virtually every important area of the Python 2.7 library through concise, stand-alone source code/output examples, designed for easy learning and reuse. Building on his popular Python Module of the Week blog series, author and Python expert Doug Hellmann focuses on &;showing&; not &;telling.&; He explains code behavior through downloadable examples that fully demonstrate each feature. You&;ll find practical code for working with text, data types, algorithms, math, file systems, networking, the Internet, XML, email, cryptography, concurrency, runtime and language services, and much more. Each section fully covers one module, and links to valuable additional resources, making this book an ideal tutorial and reference. Coverage includes If you&;re new to Python, this book will quickly give you access to a whole new world of functionality. If you&;ve worked with Python before, you&;ll discover new, powerful solutions and better ways to use the modules you&;ve already tried.