Beginning python by james payne

.xxviiPart I: Dipping Your Toe into Python Chapter 1: Programming Basics and Strings.. Introduction xxvii How Programming is Different from Using a Computer 3 Programming Copes with Chan

Using Python 2.6 and Python 3.1

Introduction xxvii

Part I: Dipping Your Toe into Python Chapter 1: Programming Basics and Strings 3

Chapter 2: Numbers and Operators 15

Chapter 3: Variables — Names for Values 31

Part II: Python Language and the Standard Library Chapter 4: Making Decisions 51

Chapter 5: Functions 71

Chapter 6: Classes and Objects 93

Chapter 7: Organizing Programs 111

Chapter 8: Files and Directories 127

Chapter 9: Other Features of the Language 143

Chapter 10: Building a Module 157

Chapter 11: Text Processing 189

Part III: Putting Python to Work Chapter 12: Testing 207

Chapter 13: Writing a GUI with Python 227

Chapter 14: Accessing Databases 239

Chapter 15: Using Python for XML 265

Chapter 16: Network Programming 287

Continues

Chapter 17: Extension Programming with C 337

Chapter 18: Numerical Programming 367

Chapter 19: An Introduction to Django 387

Chapter 20: Web Applications and Web Services 407

Chapter 21: Integrating Java with Python 481

Part IV: Appendices Appendix A: Answers to the Exercises 515

Appendix B: Online Resources 549

Appendix C: What’s New in Python 3.1 553

Appendix D: Glossary 559

Index 569

Python®

Using Python 2.6 and Python 3.1

James Payne

Beginning Python®: Using Python 2.6 and Python 3.1

Copyright © 2010 by Wiley Publishing, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-0-470-41463-7

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any

means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections

107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or

authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood

Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600 Requests to the Publisher for permission should be

addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201)

748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties

with respect to the accuracy or completeness of the contents of this work and specifically disclaim all

warranties, including without limitation warranties of fitness for a particular purpose No warranty may be

created or extended by sales or promotional materials The advice and strategies contained herein may not

be suitable for every situation This work is sold with the understanding that the publisher is not engaged in

rendering legal, accounting, or other professional services If professional assistance is required, the services of a

competent professional person should be sought Neither the publisher nor the author shall be liable for

damages arising here from The fact that an organization or Web site is referred to in this work as a citation

and/or a potential source of further information does not mean that the author or the publisher endorses the

information the organization or Web site may provide or recommendations it may make Further, readers

should be aware that Internet Web sites listed in this work may have changed or disappeared between when

this work was written and when it is read

For general information on our other products and services please contact our Customer Care Department

within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be

available in electronic books

Library of Congress Control Number: 2009936814

Trademarks: Wiley, the Wiley logo, Wrox, the Wrox logo, Wrox Programmer to Programmer, and related trade

dress are trademarks or registered trademarks of John Wiley & Sons, Inc and/or its affiliates, in the United

States and other countries, and may not be used without written permission Python is a registered trademark

of Python Software Foundation All other trademarks are the property of their respective owners Wiley

Publishing, Inc is not associated with any product or vendor mentioned in this book

Ronnie and Sharon Payne, who raised me to believe I could do anything I put my mind to, even when I told them I wanted to be Santa Claus For my brother, Ron, who read my work even when

it was bad, Dorjan, Eric, Clem, and Nick because they know things about me and

will tell them if I don’t include them.

James Payne (Margate, FL) is Editor-in-Chief of Developer Shed, Inc and has been writing and

programming since the age of seven years old Proficient in many languages, he has written over

400 articles covering practically every major programming language As a contractor, he develops

proprietary software for the financial industry using Python and likes to dabble in Django in his

spare time

Introduction xxvii

How Programming is Different from Using a Computer 3

Programming Copes with Change 5

Installing Python 3.1 on Non-Windows Systems 6

Understanding Different Quotes 9

Joining Strings with the Print() Function 12

Names You Can’t Use and Some Rules 34

Tuples — Unchanging Sequences of Data 34

Lists — Changeable Sequences of Data 37

Dictionaries — Groupings of Data Indexed by Name 39

Treating a String Like a List 41

Referencing the Last Elements 43

Growing Lists by Appending Sequences 45

Using Lists to Temporarily Store Data 45

More Than or Equal, Less Than or Equal 55

Looking for the Results of More Than One Comparison 56

How to Do Something — Again and Again 60

Summary 67

Describing a Function in the Function 75 The Same Name in Two Different Places 76

Asking a Function to Use a Value You Provide 79

Setting a Default Value for a Parameter—Just in Case 83 Calling Functions from within Other Functions 84 Functions Inside of Functions 86 Flagging an Error on Your Own Terms 87

Looking Ahead: How You Want to Use Objects 95

How Code Can Be Made into an Object 96

Summary 107 Exercises 108

Importing a Module So That You Can Use It 112 Making a Module from Pre-existing Code 113 Using Modules — Starting with the Command Line 115 Changing How Import Works — Bringing in More 118

Bringing Everything into the Current Scope 120

Re-importing Modules and Packages 121

Summary 124

Exercises 125

Obtaining Information about Files 136

Renaming, Moving, Copying, and Removing Files 137

Creating and Removing Directories 140

Summary 142

Exercises 142

Getopt — Getting Options from the Command Line 149

Using More Than One Process 152

Threads — Doing Many Things in the Same Process 154

Summary 156

Exercises 156

Chapter 10: Building a Module 157

Defining Object-Oriented Programming 163

Defining Module-Specific Errors 166

Running a Module as a Program 178

Summary 187 Exercises 188

Working with Regular Expressions and the re Module 199 Summary 203 Exercises 204

Implementing a Search Utility in Python 216

A More Powerful Python Search 222

Summary 225

Putting the Widgets to Work 231

Controlling Widget Appearances 233

Radio Buttons and Checkboxes 235

Summary 238

Exercises 238

Creating Persistent Dictionaries 241

Accessing Persistent Dictionaries 243

Deciding When to Use DBM and When to Use a Relational Database 245

Working with Transactions and Committing the Results 260

Examining Module Capabilities and Metadata 261

Summary 262

Exercises 263

Chapter 15: Using Python for XML 265

A Hierarchical Markup Language 265

What Are Document Models For? 268

Other Advantages of Schemas 271

Comparing Protocols and Programming Languages 289

The Internet Protocol Stack 290

A Little Bit About the Internet Protocol 292

Sending Mail with SMTP and smtplib 303

Parsing a Local Mail Spool with mailbox 305

Fetching Mail from a POP3 Server with poplib 307

Fetching Mail from an IMAP Server with imaplib 309

Webmail Applications Are Not E-mail Applications 313

Binding to an External Hostname 316

Design of the Python Chat Server 323

The Python Chat Server Protocol 323

Single-Threaded Multitasking with select 331

Miscellaneous Considerations for Protocol Design 333

The Peer-to-Peer Architecture 333

Summary 334

Exercises 335

Passing Parameters from Python to C 342

Summary 366 Exercises 366

Creating a Model: First Steps — Configure the Database Settings 401

Working with Models: Installation 404

Summary 405

The Web Server Makes a Deal with the CGI Script 419

CGI’s Special Environment Variables 420

Accepting User Input through HTML Forms 422

The cgi Module: Parsing HTML Forms 423

The BittyWiki Web Interface 432

REST Quick Start: Finding Bargains on Amazon.com 443

Introducing WishListBargainFinder 445

Giving BittyWiki a REST API 448

Wiki Search-and-Replace Using the REST Web Service 451

XML-RPC 456

Exposing the BittyWiki API through XML-RPC 460

Wiki Search-and-Replace Using the XML-RPC Web Service 463

Exposing a SOAP Interface to BittyWiki 468

Wiki Search-and-Replace Using the SOAP Web Service 470

Human-Readable API Documentation 473

Choosing a Web Service Standard 478

For Consumers of Web Services 479 For Producers of Web Services 479 Using Web Applications as Web Services 480

Summary 480 Exercises 480

Running Jython Interactively 484

Controlling the jython Script 486

Using Java Classes in Jython 489 Accessing Databases from Jython 494 Writing Java EE Servlets in Jython 500

Calling Jython Scripts from Java 508

Handling Differences between C-Python and Jython 510 Summary 511 Exercises 512

time on already-known information.

I wanted to write this book because I love Python I love it! And I want to share my love with you And, maybe you’ll grow to love it as I do

Who This Book Is For

If you’re computer-literate, and want to learn a fun programming language to better control your

computer, this book is for you

If you are a system administrator who wants to learn a great language to help you better manage and

configure systems and networks, this book is for you

If you already know Python, but are wondering what cool new features are available in version 3.1, this book is for you

In summary, this book is for anyone interested in exploring Python programming with the newest and

most full-featured, easy-to-use version, 3.1

What This Book Covers

This book is designed to cover Python 3.1 Python 3.1, released in 2009, is the latest major revision of the Python programming language Since Python is a cross-platform language, the content and examples in the book are applicable in any platform (unless specified otherwise) When there is a choice to be made

as to platform independence, the examples will be as cross-platform as possible

In addition, since Python 3.1 is relatively new, not all supporting libraries have been updated to work in Python 3.x In those instances where this is the case and it is felt that the theory still needs to be

expounded upon, Python 2.6 will be used in lieu of version 3.1

How This Book Is Str uctured

As might be expected from a “Beginning” book, the book begins with an introduction to the language

From there, you’ll move through the core of the language, then move on to more advanced and

specialized topics The book is divided up into four parts

Part I — Dipping Your Toe into Python

The first part will allow you to, as the title suggests, dip your toe in

Programming Basics and Strings

First you’ll be introduced to Python This chapter will explore what Python is, and why it is so useful

and powerful Also explored will be Python’s history from its early development to the newest version,

which is the focus of this book You’ll also learn about the scope of Python’s reach, and all the different

areas of application development in which Python plays a part Finally, you’ll learn to work with your

first data type — strings

Numbers and Operators

This chapter will guide you through the basics of working with numbers and operators You will learn

the different types of numbers, how to perform simple — and complex — equations, and work with the

various operators You will also learn about order of precedence and formatting numbers

Variables — Names for Values

Ultimately, programming languages help you to manage different types of information — in other

words, data An understanding of data types and how they are represented in Python is essential to

programming in Python This chapter will help you to understand the best ways to represent different

data types in Python

Part II — Python Language and the Standard Library

Of course, the core piece of knowledge you need to use a language is to know the language itself, and

familiarize yourself with its syntax and modules This part will start small, with data types and

variables, and gradually introduce additional concepts until you have all the information you need to

develop fully functional Python programs

You’ll want to read through these chapters sequentially –– each chapter builds on the information

presented in the previous chapter.

Making Decisions

Ultimately, there will come a point when your program must make a decision — do I take this path or

that path? And what happens when I take that path? In this chapter, you will learn how to compare data,

such as deciding if one value is greater than another, and use repetition to repeat repetitive tasks

This chapter will help you to expand on your Python knowledge by introducing you to functional

programming Functions allow you to take advantage of powerful concepts like parameter passing and code reuse You’ll learn how to use functions to make your code more efficient and flexible

Classes and Objects

Here you will be shown what objects are and learn to create classes You will learn how to define them, create objects in your classes, write methods, and discuss the scope of your objects

Organizing Programs

When your programs get larger, you’ll want to divide them up into separate components This chapter

will discuss Python modules You’ll also explore packages, which are nothing but collections of modules

Files and Directories

An important part of everyday programming is learning to work with files and directories This chapter focuses on creating, modifying, and working with files in general In addition, you will learn how to

obtain data from files and how to interact with the various directories

Other Features of the Language

Here you will learn about some of the other features the language has to offer, including how to make

decisions with lists, string substitutions with dictionaries, and some of the featured modules

Building a Module

Modules help you save time by allowing you to reuse snippets of code It also ensures fewer errors, as

the module you use will have been tested and used many times before Here, we will learn to create our own modules, as well as import and work with pre-existing modules –– something that makes Python

particularly powerful

Text Processing

There are so many things you can do with text in programming and in essence, text is the key to

effectively communicating with your user After all, without it, the only thing you are left with is images

In this chapter you learn to process text in a variety of ways, including: working with regular

expressions, searching for files, and searching for files of a particular type

Part III — Putting Python to Work

So, now that you know what Python is, and how to work with the language, what’s next, you ask? This final part explores many of the programming topics you’ll likely encounter or want to explore These can

be looked at sequentially, or in any order you like these chapters are independent of each other

Testing

There is only one way to ensure your program works before it is in the hands of the user, and that is by

Writing a GUI with Python

Thus far in the book, all the programs work through the command line In this chapter, you’ll be

introduced to the concept of GUI programming You’ll also walk through creating a few GUI programs

with Tkinter, the most popular GUI toolkit used by Python programmers

Accessing Databases

Databases store information that your program can use for an infinite amount of reasons It also acts as a

place for you to store information, and later retrieve that information for a given task In this chapter you

learn about the different types of databases and how to work with them

Using Python for XML

XML is a powerful tool for processing data on the Internet Here, you will learn the basics of XML

including the difference between schema and DTD, basic syntax, how to create and validate your own

XML, and more advanced topics such as using lxml

Network Programming

Now that the Internet has wormed its way into our everyday lives, and has become more of a necessity

than a privilege, learning to programmatically send e-mails and allow users to communicate across the

web is essential In this chapter, you will learn how to do just that

Extension Programming with C

This chapter delves into programming with the C language, including working with C frameworks

and modules, the basics of C, and passing parameters from Python to C, and then returning value back

to Python

Numerical Programming

Numbers were touched on briefly in the beginning of this book; now it is time to delve more deeply

below the surface Here you will learn all there is to know about integers and floating point numbers, as

well as complex numbers, arrays, and working with built-in math functions and modules

An Introduction to Django

Django is a web application framework written in Python, which utilizes the model-view-architecture

pattern Originally created for managing news websites, Django has become popular for its ease of use,

allowing programmers to create complex websites in a simple fashion, including database-focused sites

Here we will learn the basics of Django

Web Applications and Web Services

Here you will learn the foundations of working with web applications and web services You will learn

about the REST architecture, as well as how to work with HTTP Requests and Responses

Integrating Java with Python

In this chapter you learn the basics of Java, building a strong foundation before you delve blending the two languages together You will learn the various modules that allow you to work with Java in Python and how to create simple, yet effective applications

Part IV: Appendices

In the back of the book, there are some useful appendices to further your knowledge and fun with

Python:

❑ Answers to the Exercises

❑ Online Resources –– where do you go from here?

❑ What’s New in Python 3.1

❑ Glossary of terms

What You Need to Use This Book

There are some minimal requirements to use the material in this book The following are

recommendations, as Python itself runs on many different platforms However, the first chapters assume that you have access to a GUI such as is available in Windows, Mac OS X, or the X Window system on

UNIX and Linux Naturally, some chapters, such as the GUI chapter, require the GUI as well, and

chapters involving networking will make much more sense if a network connection is in place

Following are the suggested minimum requirements:

❑ A PC running Linux, a BSD UNIX, or Windows running at 500MHz or faster, or a G3 or later

Macintosh running Mac OS X version 10.2 or later

❑ 256MB of memory (at a minimum)

❑ A graphical user interface native to the platform you are on

❑ Necessary access to the computer you are on so that you may install required software

❑ Network access to a TCP/IP network such as the Internet or a campus network

❑ Internet access to download required software

Conventions

To help you get the most from the text and keep track of what’s happening, we’ve used a number of

conventions throughout the book

The Try It Out is an exercise you should work through, following the text in the book.

1 They usually consist of a set of steps.

2 Each step has a number

3 Follow the steps through with your copy of the database.

How It Works

After each Try It Out, the code you’ve typed will be explained in detail.

Boxes like this one hold important, not-to-be forgotten information that is directly

relevant to the surrounding text.

Notes, tips, hints, tricks, and asides to the current discussion are offset and placed in italics like this.

As for styles in the text:

❑ We highlight new terms and important words when we introduce them.

❑ We show keyboard strokes like this: Ctrl+A

Try It Out

❑ We show file names, URLs, and code within the text like so: persistence.properties.

❑ We present code in two different ways:

We use a monofont type with no highlighting for most code examples

We use bold highlighting to emphasize code that’s particularly important

in the present context

Source Code

As you work through the examples in this book, you may choose either to type in all the code manually

or to use the source code files that accompany the book All of the source code used in this book is

available for download at http://www.wrox.com Once at the site, simply locate the book’s title (either

by using the Search box or by using one of the title lists) and click the Download Code link on the book’s detail page to obtain all the source code for the book

Because many books have similar titles, you may find it easiest to search by ISBN; this book’s ISBN is 978-0-470-41463-7.

Once you download the code, just decompress it with your favorite compression tool Alternately, you

can go to the main Wrox code download page at http://www.wrox.com/dynamic/books/download.aspx to see the code available for this book and all other Wrox books

Errata

We make every effort to ensure that there are no errors in the text or in the code However, no one is

perfect, and mistakes do occur If you find an error in one of our books, like a spelling mistake or faulty piece of code, we would be very grateful for your feedback By sending in errata, you may save another reader hours of frustration and at the same time you will be helping us provide even higher quality

information

To find the errata page for this book, go to http://www.wrox.com and locate the title using the Search box or one of the title lists Then, on the book details page, click the Errata link On this page you can

view all errata that has been submitted for this book and posted by Wrox editors A complete book list

including links to each book’s errata is also available at www.wrox.com/misc-pages/booklist.shtml

If you don’t spot “your” error on the Errata page, go to www.wrox.com/contact/techsupport.shtml

and complete the form there to send us the error you have found We’ll check the information and, if

appropriate, post a message to the book’s errata page and fix the problem in subsequent editions of

the book

p2p.wrox.com

For author and peer discussion, join the P2P forums at p2p.wrox.com The forums are a Web-based

system for you to post messages relating to Wrox books and related technologies and interact with other

you read this book, but also as you develop your own applications To join the forums, just follow

these steps:

1 Go to p2p.wrox.com and click the Register link

2 Read the terms of use and click Agree.

3 Complete the required information to join as well as any optional information you wish to

provide and click Submit

4 You will receive an e-mail with information describing how to verify your account and complete

the joining process

You can read messages in the forums without joining P2P, but in order to post your own messages, you

must join.

Once you join, you can post new messages and respond to messages other users post You can read

messages at any time on the Web If you would like to have new messages from a particular forum

e-mailed to you, click the Subscribe to this Forum icon by the forum name in the forum listing

For more information about how to use the Wrox P2P, be sure to read the P2P FAQs for answers to

questions about how the forum software works as well as many common questions specific to P2P and

Wrox books To read the FAQs, click the FAQ link on any P2P page

Part I

Dipping Your Toe

into Python

Chapter 1: Programming Basics and Strings

Chapter 2: Numbers and Operators

Chapter 3: Variables — Names for Values

1

Programming Basics

and Strings

This chapter is a gentle introduction to the practice of programming in Python Python is a very

rich language with many features, so it is important to learn to walk before you learn to run

Chapters 1 through 3 provide a basic introduction to common programming ideas, explained in

easily digestible paragraphs with simple examples

If you are already an experienced programmer interested in Python, you may want to read this

chapter quickly and take note of the examples, but until Chapter 3 you will be reading material

with which you ’ ve probably already gained some familiarity in another language

If you are a novice programmer, by the end of this chapter you will learn the following:

Some guiding principles for programming Directions for your first interactions with a programming language — Python

The exercises at the end of the chapter provide hands - on experience with the basic information

that you have learned

How Programming is Different

from Using a Computer

The first thing you need to understand about computers when you ’ re programming is that you

control the computer Sometimes the computer doesn ’ t do what you expect, but even when it

doesn ’ t do what you want the first time, it should do the same thing the second and third

time — until you take charge and change the program

❑

❑

Part I: Dipping Your Toe into Python

extra work and various harassments while you ’ re already trying to accomplish something However,

after you ’ ve learned how to program, you gain an understanding of how this situation has come to

pass, and perhaps you ’ ll find that you can do better than some of the programmers whose software

you ’ ve used

Note that programming in a language like Python, an interpreted language, means that you are not going

to need to know a whole lot about computer hardware, memory, or long sequences of 0s and 1s You are

going to write in text form like you are used to reading and writing but in a different and simpler

language Python is the language, and like English or any other language(s) you speak, it makes sense to

other people who already speak the language Learning a programming language can be even easier,

however, because programming languages aren ’ t intended for discussions, debates, phone calls, plays,

movies, or any kind of casual interaction They ’ re intended for giving instructions and ensuring that

those instructions are followed Computers have been fashioned into incredibly flexible tools that have

found a use in almost every business and task that people have found themselves doing, but they are

still built from fundamentally understandable and controllable pieces

Programming is Consistency

In spite of the complexity involved in covering all of the disciplines into which computers have crept, the

basic computer is still relatively simple in principle The internal mechanisms that define how a

computer works haven ’ t changed a lot since the 1950s when transistors were first used in computers

In all that time, this core simplicity has meant that computers can, and should, be held to a high standard

of consistency What this means to you, as the programmer, is that anytime you tell a computer to

metaphorically jump, you must tell it how high and where to land, and it will perform that jump — over

and over again for as long as you specify The program should not arbitrarily stop working or change

how it works without you facilitating the change

Programming is Control

Programming a computer is very different from creating a program, as the word applies to people in real

life In real life, you ask people to do things, and sometimes you have to struggle mightily to ensure that

your wishes are carried out — for example, if you plan a party for 30 people and assign two of them to

bring the chips and dip and they bring the drinks instead, it is out of your control

With computers that problem doesn ’ t exist The computer does exactly what you tell it to do As you can

imagine, this means that you must pay some attention to detail to ensure that the computer does just

what you want it to do

One of the goals of Python is to program in blocks that enable you to think about larger and larger

projects by building each project as pieces that behave in well - understood ways This is a key goal of a

programming style known as object - oriented programming The guiding principle of this style is that you

can create reliable pieces that still work when you piece them together, that are understandable, and that

are useful This gives you, the programmer, control over how the parts of your programs run, while

enabling you to extend your program as the problems you ’ re solving evolve

Programming Copes with Change

Programs are run on computers that handle real - world problems; and in the real world, plans and

circumstances frequently change Because of these shifting circumstances, programmers rarely get the

opportunity to create perfectly crafted, useful, and flexible programs Usually, you can achieve only two

of these goals The changes that you will have to deal with should give you some perspective and lead

you to program cautiously With sufficient caution, you can create programs that know when they ’ re

being asked to exceed their capabilities, and they can fail gracefully by notifying their users that they ’ ve stopped In the best cases, you can create programs that explain what failed and why Python offers

especially useful features that enable you to describe what conditions may have occurred that prevented your program from working

What All That Means Together

Taken together, these beginning principles mean that you ’ re going to be introduced to programming as a way of telling a computer what tasks you want it to do, in an environment where you are in control You will be aware that sometimes accidents can happen and that these mistakes can be accommodated

through mechanisms that offer you some discretion regarding how these conditions will be handled,

including recovering from problems and continuing to work

The F irst Steps

The absolute first step you need to take before you can begin programming in Python is to download

and install Python version 3.1 Navigate to www.python.org/download and choose the newest version

of Python You will be taken to a page with instructions on how to download the appropriate version for your computer For instance, if you are running Windows, it may say Windows x86 MSI Installer (3.0)

Programs are written in a form called source code Source code contains the

instruc-tions that the language follows, and when the source code is read and processed, the instructions that you ’ ve put in there become the actions that the computer takes.

Just as authors and editors have specialized tools for writing for magazines, books, or online

publications, programmers also need specialized tools As a starting Python programmer, the right tool for the job is the Python IDLE GUI (graphical user interface)

Once the download is finished, double - click it to run the program Your best bet is to accept the default prompts Python offers you This process may take a few minutes, depending on your system

After setup is complete, you will want to test to make sure it is installed properly Click the Windows

Start menu and go to All Programs You will see Python 3.0 in the menu Choose IDLE (Python GUI) and wait for the program to load