Advance Praise for Head First Python Part 4 docx

Extend try with finallyWhen you have a situation where code must always run no matter what errors occur, add that code to your try statement’s finally suite: try: man_file = open'man_

Extend try with finally

When you have a situation where code must always run no matter what errors

occur, add that code to your try statement’s finally suite:

try:

man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w')

print(man, file=man_file) print(other, file=other_file)

except IOError:

print('File error.')

finally:

man_file.close() other_file.close()

If no runtime errors occur, any code in the finally suite executes Equally,

if an IOError occurs, the except suite executes and then the finally

suite runs

No matter what, the code in the finally suite always runs.

By moving your file closing code into your finally suite, you are reducing

the possibility of data corruption errors

This is a big improvement, because you’re now ensuring that files are closed

properly (even when write errors occur)

But what about those errors?

How do you find out the specifics of the error?

no dumb questiohns

of unwanted whitespace, you assigned the result back to the

line_spoken variable Surely invoking the strip() method on

line_spoken changed the string it refers to?

A: No, that’s not what happens Strings in Python are immutable,

which means that once a string is created, it cannot be changed.

any unwanted whitespace, right?

A: Yes and no What actually happens is that invoking the

strip() method on the line_spoken string creates a

new string with leading and trailing whitespace removed The new

string is then assigned to line_spoken, replacing the data that

was referred to before In effect, it is as if you changed line_

spoken, when you’ve actually completely replaced the data it

refers to.

A: Python’s built-in memory management technology reclaims the

RAM it was using and makes it available to your program That is,

unless some other Python data object is also referring to the string.

A: It is conceivable that another data object is referring to the

string referred to by line_spoken For example, let’s assume

you have some code that contains two variables that refer to the

same string, namely “Flying Circus.” You then decide that one of

the variables needs to be in all UPPERCASE, so you invoke the

upper() method on it The Python interperter takes a copy of the

string, converts it to uppercase, and returns it to you You can then

assign the uppercase data back to the variable that used to refer to

the original data.

another variable referring to it?

A: Precisely That’s why strings are immutable, because you never

know what other variables are referring to any particular string.

referring to any one particular string?

A: It does, but only for the purposes of garbage collection If you have a line of code like print('Flying Circus'), the string is not referred to by a variable (so any variable reference counting that’s going on isn’t going to count it) but is still a valid string object (which might be referred to by a variable) and it cannot have its data changed under any circumstances.

assigned to them?

A: That’s correct Python variables contain a reference to a data object.The data object contains the data and, because you can conceivably have a string object used in many different places throughout your code, it is safest to make all strings immutable so that no nasty side effects occur.

place”?

A: No, not really Once you get used to how strings work, it becomes less of an issue In practice, you’ll find that this issue rarely trips you up.

A: Yes, a few There’s the tuple, which is an immutable list Also, all of the number types are immutable

something is immutable?

A: Don’t worry: you’ll know If you try to change an immutable value, Python raises a TypeError exception.

Python, aren’t they?

A: Yes Exceptions make the world go ’round.

Knowing the type of error is not enough

When a file I/O error occurs, your code displays a generic “File Error”

message This is too generic How do you know what actually happened?

Maybe the problem

is that you can’t open

the file?

It could be that the file can be opened but not written to?

Yeah, or it could be

a permission error, or maybe your disk is full?

Who knows?

It turns out that the Python interpreter knows…and it will give up the details

if only you’d ask

When an error occurs at runtime, Python raises an exception of the specific

type (such as IOError, ValueError, and so on) Additionally, Python

creates an exception object that is passed as an argument to your except

suite

Let’s use IDLE to see how this works.

Traceback (most recent call last):

File "<pyshell#8>", line 7, in <module>

data.close()

NameError: name 'data' is not defined

There’s your error message, but…

…what’s this?!? Another exception was raised and it killed your code.

As the file doesn’t exist, the data file object wasn’t created, which subsequently makes it impossible to call the

close() method on it, so you end up with a NameError A quick fix is to add a small test to the finally

suite to see if the data name exists before you try to call close() The locals() BIF returns a collection of names defined in the current scope Let’s exploit this BIF to only invoke close() when it is safe to do so:

finally:

if 'data' in locals():

data.close()

File error

No extra exceptions this time

Just your error message.

Here you’re searching the collection returned by the locals() BIF for the string data If you find it, you can assume the file was opened successfully and safely call the close() method

If some other error occurs (perhaps something awful happens when your code calls the print() BIF), your

exception-handling code catches the error, displays your “File error” message and, finally, closes any opened file

The “in” operator tests for membership This is just the bit of code that needs to change Press Alt-P to

edit your code at IDLE’s shell.

When an exception is raised and handled by your except suite, the Python interpreter passes an exception object

into the suite A small change makes this exception object available to your code as an identifier:

except IOError as err:

print('File error: ' + err)

Traceback (most recent call last):

File "<pyshell#18>", line 5, in <module>

print('File error:' + err)

TypeError: Can't convert 'IOError' object to str implicitly

Give your exception object

using the str() BIF:

except IOError as err:

print('File error: ' + str(err))

File error: [Errno 2] No such file or directory: 'missing.txt'

Use the “str()” BIF to force the exception object to behave like a string.

And you now get a specific error message that tells you exactly what went wrong.

Of course, all this extra logic is starting to obscure the

real meaning of your code.

Wouldn’t it be dreamy if there were a way to take advantage of these mechanisms without the code bloat? I guess it’s just a fantasy But when you try to run your code with this change made, another exception is raised:

Now, with this final change, your code is behaving exactly as expected:

try with

Use with to work with files

Because the use of the try/except/finally pattern is so common when

it comes to working with files, Python includes a statement that abstracts

away some of the details The with statement, when used with files, can

dramatically reduce the amount of code you have to write, because it negates

the need to include a finally suite to handle the closing of a potentially

opened data file Take a look:

try:

data = open('its.txt', "w")

print("It's ", file=data)

except IOError as err:

print('File error: ' + str(err))

print('File error: ' + str(err))

This is the usual “try/

except/finally” pattern.

The use of “with”

negates the need for the “finally” suite.

When you use with, you no longer have to worry about closing any opened

files, as the Python interpreter automatically takes care of this for you The

with code on the the right is identical in function to that on the left At Head

First Labs, we know which approach we prefer

Geek Bits

The with statement takes advantage of a Python technology

called the context management protocol

Grab your pencil and rewrite this try/except/finally code to use

with instead Here’s your code with the appropriate finally

suite added:

try:

man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w')

print(man, file=man_file) print(other, file=other_file) except IOError as err:

print('File error: ' + str(err)) finally:

no finally

You were to grab your pencil and rewrite this try/except/finally

code to use with instead Here’s your code with the appropriate

finally suite added:

try:

man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w')

print(man, file=man_file) print(other, file=other_file) except IOError as err:

print('File error: ' + str(err)) finally:

print(other, file=other_file) except IOError as err:

print(‘File error: ' + str(err))

with open('man_data.txt', 'w') as man_file, open('other_data.txt’, 'w’) as other_file:

print(man, file=man_file) print(other, file=other_file)

Using two “with”

statements to rewrite

the code without the

“finally” suite.

Or combine the two “open()” calls into one

Test DriveAdd your with code to your program, and let’s confirm that it continues to function as expected Delete the two data files you created with the previous version of your program and then load your newest code into IDLE and give

it a spin

No errors in the IDL E

shell appears to indica te

that the program ran

successfully.

If you check your folder, your two data files should’ve reappeared Let’s take a closer look at the data file’s contents

by opening them in your favorite text editor (or use IDLE)

Here’s what the man said.

Here’s what the other man said.

You’ve saved the lists in two files containing what the Man said and what the Other

man said Your code is smart enough to handle any exceptions that Python or

your operating system might throw at it

Well done This is really coming along.

unsuitable format

Default formats are unsuitable for files

Although your data is now stored in a file, it’s not really in a useful format

Let’s experiment in the IDLE shell to see what impact this can have

Use a with statement to open your data file and display a single line from it:

>>> with open('man_data.txt') as mdf:

print(mdf.readline())

['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't

an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!']

Yikes! It would appear your list is converted to a large string by print()

when it is saved Your experimental code reads a single line of data from the

file and gets all of the data as one large chunk of text…so much for your code

saving your list data.

What are your options for dealing with this problem?

Note: no need to close your file, because “with” does that for you.

Geek Bits

By default, print() displays your data in a format that mimics how your list data is actually stored by the Python interpreter

The resulting output is not really meant to be processed further…

its primary purpose is to show you, the Python programmer, what your list data “looks like” in memory.

I guess I could write some custom

parsing code to process the “internal

format” used by “print()” It shouldn’t

take me all that long

It might be worth looking at using something other than a plain

“print()” to format the data prior

to saving it to the data file? I’d certainly look into it.

Parsing the data in the file is a possibility…although it’s complicated by all those

square brackets, quotes, and commas Writing the required code is doable,

but it is a lot of code just to read back in your saved data

Of course, if the data is in a more easily parseable format, the task would likely be

easier, so maybe the second option is worth considering, too?

Can you think of a function you created from earlier

in this book that might help here?

nonstandard output

Why not modify print lol()?

Recall your print_lol() function from Chapter 2, which takes any list (or

list of lists) and displays it on screen, one line at a time And nested lists can

be indented, if necessary

This functionality sounds perfect! Here’s your code from the nester.py

module (last seen at the end of Chapter 2):

This code currently displays your data on the screen.

Amending this code to print to a disk file instead of the screen (known as

standard output) should be relatively straightforward You can then save your

data in a more usable format

Standard Output The default place where your code writes its data when the “print()” BIF is used This is typically the screen

In Python, standard output is referred to as “sys.stdout” and

is importable from the Standard Library’s “sys” module.

Let’s add a fourth argument to your print_lol() function to identify a place to write your data to Be sure to give your argument a default value of sys.stdout, so that it continues to write to the screen if no file object is specified when the function is invoked

Fill in the blanks with the details of your new argument (Note: to save on space, the comments

have been removed from this cod, but be sure to update your comments in your nester.py module after you’ve amended your code.)

1

def print_lol(the_list, indent=False, level=0, ):

for each_item in the_list:

What needs to happen to the code in your with statement now that your amended print_lol()

function is available to you?

2

List the name of the module(s) that you now need to import into your program in order to support your

amendments to print_lol()

3

extend your function

You were to add a fourth argument to your print_lol() function to identify a place to write your data to, being sure to give your argument a default value of sys.stdout so that it continues to write to the screen if no file object is specified when the function is invoked

You were to fill in the blanks with the details of your new argument (Note: to save on space, the

comments have been removed from this code, but be sure to update those in your nester.py module after you’ve amended your code)

1

def print_lol(the_list, indent=False, level=0, ):

for each_item in the_list:

What needs to happen to the code in your with statement now that your amended print_lol() function is available to you?

The code needs to be adjusted so that instead of using the

“print()” BIF, the code needs to invoke “print_lol()” instead.

The program needs to import the amended “nester” module.

Note: the signature has changed.

Adjust the two calls to “print()”

to use the new argument.

Add the fourth argument and give it a default value.

Test DriveBefore taking your code for a test drive, you need to do the following:

1 Make the necessary changes to nester and install the amended module into your Python

environment (see Chapter 2 for a refresher on this) You might want to upload to PyPI, too

2 Amend your program so that it imports nester and uses print_lol() instead of print()

within your with statement Note: your print_lol() invocation should look something like this:

When you are ready, take your latest program for a test drive and let’s see what happens:

Let’s check the contents of the files to see what they look like now

As before, there’s no

output on screen.

What the man said is now legible.

And here’s what the other man said.

This is looking good By amending your nester module, you’ve provided a

facility to save your list data in a legible format It’s now way easier on the eye

But does this make it any easier to read the data back in?

brittle code

Hang on a second haven’t you been here before? You’ve already written code to read in lines from a data file and put ‘em into lists do you like going around

in circles?!?

That’s a good point.

This problem is not unlike the problem from the beginning of the chapter, in that you’ve got lines of text in a disk file that you need to process, only now

you have two files instead of one

You know how to write the code to process your new files, but writing custom code like this is specific to the format that you’ve created for this

problem This is brittle: if the data format changes,

your custom code will have to change, too

Ask yourself: is it worth it?

Head First: Hello, CC, how are you today?

Custom Code: Hi, I’m great! And when I’m not

great, there’s always something I can do to fix things

Nothing’s too much trouble for me Here: have a

seat

Head First: Why, thanks.

Custom Code: Let me get that for you It’s my

new custom SlideBack&Groove™, the 2011 model,

with added cushions and lumbar support…and it

automatically adjusts to your body shape, too How

does that feel?

Head First: Actually [relaxes], that feels kinda

groovy

Custom Code: See? Nothing’s too much trouble

for me I’m your “go-to guy.” Just ask; absolutely

anything’s possible when it’s a custom job

Head First: Which brings me to why I’m here I

have a “delicate” question to ask you

Custom Code: Go ahead, shoot I can take it.

Head First: When is custom code appropriate?

Custom Code: Isn’t it obvious? It’s always

appropriate

Head First: Even when it leads to problems down

the road?

Custom Code: Problems?!? But I’ve already told

you: nothing’s too much trouble for me I live to

customize If it’s broken, I fix it

Head First: Even when a readymade solution

might be a better fit?

Custom Code: Readymade? You mean (I hate to

say it): off the shelf?

Head First: Yes Especially when it comes to

writing complex programs, right?

Custom Code: What?!? That’s where I excel:

creating beautifully crafted custom solutions for folks with complex computing problems

Head First: But if something’s been done before,

why reinvent the wheel?

Custom Code: But everything I do is

custom-made; that’s why people come to me…

Head First: Yes, but if you take advantage of other

coders’ work, you can build your own stuff in half

the time with less code You can’t beat that, can you?

Custom Code: “Take advantage”…isn’t that like

exploitation?

Head First: More like collaboration, sharing,

participation, and working together.

Custom Code: [shocked] You want me to give my

code…away?

Head First: Well…more like share and share alike

I’ll scratch your back if you scratch mine How does that sound?

Custom Code: That sounds disgusting.

Head First: Very droll [laughs] All I’m saying is

that it is not always a good idea to create everything from scratch with custom code when a good enough solution to the problem might already exist

Custom Code: I guess so…although it won’t be as

perfect a fit as that chair

Head First: But I will be able to sit on it!

Custom Code: [laughs] You should talk to my

buddy Pickle…he’s forever going on about stuff like

this And to make matters worse, he lives in a library

Head First: I think I’ll give him a shout Thanks! Custom Code: Just remember: you know where to

find me if you need any custom work done

Custom Code Exposed

This week’s interview:

When is custom code appropriate?

in a pickle

Pickle your data

Python ships with a standard library called pickle, which can save and load

almost any Python data object, including lists

Once you pickle your data to a file, it is persistent and ready to be read into

another program at some later date/time:

['Is this the right room for an

argument?', "No you haven't!",

'When?', "No you didn't!", "You

didn't!", 'You did not!', 'Ah!

(taking out his wallet and paying)

Just the five minutes.', 'You

most certainly did not!', "Oh

no you didn't!", "Oh no you

didn't!", "Oh look, this isn't

an argument!", "No it isn't!",

"It's just contradiction!", 'It

IS!', 'You just contradicted

me!', 'You DID!', 'You did just

then!', '(exasperated) Oh, this

is futile!!', 'Yes it is!']

[‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’,

“No you didn’t!”, “You didn’t!”, ‘You did not!’,

‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh

no you didn’t!”, “Oh no this isn’t an argument!”,

“No it isn’t!”, “It’s

‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated)

‘Yes it is!’]

Your data as it appears

Your pickled data

Feed your Python data to pickle.

Out comes the pickled version of your data.

You can, for example, store your pickled data on disk, put it in a database,

or transfer it over a network to another computer

When you are ready, reversing this process unpickles your persistent pickled

data and recreates your data in its original form within Python’s memory:

['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh

no you didn't!", "Oh no you didn't!", "Oh look, this isn't

an argument!", "No it isn't!",

"It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this

[‘Is this the right room

for an argument?’, “No

you haven’t!”, ‘When?’,

“No you didn’t!”, “You

didn’t!”, ‘You did not!’,

‘Ah! (taking out his wallet and paying) Just the five

minutes.’, ‘You most

certainly did not!’, “Oh

no you didn’t!”, “Oh no

this isn’t an argument!”,

“No it isn’t!”, “It’s

‘It IS!’, ‘You just

contradicted me!’, ‘You

DID!’, ‘You did just

then!’, ‘(exasperated)

‘Yes it is!’]

Your data is recreated

in Python’s memory, exactly as before.

The same pickle engine Your

pickled

data

Feed your pickled

data to pickle Out comes the Python version of

Save with dump and restore with load

Using pickle is straightforward: import the required module, then use

dump() to save your data and, some time later, load() to restore it The

only requirement when working with pickled files is that they have to be

opened in binary access mode:

import pickle

with open('mydata.pickle', 'wb') as mysavedata:

pickle.dump([1, 2, 'three'], mysavedata)

with open('mydata.pickle', 'rb') as myrestoredata:

a_list = pickle.load(myrestoredata) print(a_list)

What if something goes wrong?

If something goes wrong when pickling or unpickling your data, the pickle

module raises an exception of type PickleError

The “b” tells Python to open your data files

in BINARY mode.

Once your data is back in your program, you can

treat it like any other data object.

w Here’s a snippet of your code as it currently stands Grab your

pencil and strike out the code you no longer need, and then replace it with code that uses the facilities of pickle instead

Add any additional code that you think you might need, too.

try:

with open('man_data.txt', 'w') as man_file, open('other_data.txt', 'w') as other_file: nester.print_lol(man, fh=man_file)

nester.print_lol(other, fh=other_file)

except IOError as err:

print('File error: ' + str(err))

significance of the pickle

print(‘Pickling error: ‘ + str(perr))

Here’s a snippet of your code as it currently stands You were to grab your pencil and strike out the code you no longer need, and then replace it with code that uses the facilities pickle instead You were also to add any additional code that you think you might need.

try:

with open('man_data.txt', 'w') as man_file, open('other_data.txt', 'w') as other_file: nester.print_lol(man, fh=man_file)

nester.print_lol(other, fh=other_file)

except IOError as err:

print('File error: ' + str(err))

import pickle

except pickle.PickleError as perr:

pickle.dump(man, man_file) pickle.dump(other, other_file) print('Pickling error: ' + str(perr))

Import “pickle” near the top of your program.

Replace the two calls to “nester.print_lol()” with calls to “pickle.dump()”.

Don’t forget to handle any exceptions that can occur.

Change the access mode to

be “writeable, binary”.

provide four How is this possible?

A: When you invoke a Python function in your code, you have options, especially when the function provides default values for some arguments If you use positional arguments, the position of the argument in your function invocation dictates what data is assigned to which argument When the function has arguments that also provide default values, you do not need to always worry about positional arguments being assigned values

Q: OK, you’ve completely lost me Can you explain?

A: Consider print(), which has this signature: print(value, sep=' ', end='\n', file=sys.stdout) By default, this BIF displays to standard output (the screen), because it has an argument called file with a default value of sys.stdout The file argument is the fourth positional argument However, when you want to send data to something other than the screen, you do not need

to (nor want to have to) include values for the second and third positional arguments They have default values anyway, so you need to provide values for them only if the defaults are not what you want If all you want to do is to send data to a file, you invoke the print() BIF like this:

print("Dead Parrot Sketch", file='myfavmonty.txt') and the fourth positional argument uses the value you specify, while the other positional arguments use their defaults In Python, not only do the BIFs work this way, but your custom functions

Test DriveLet’s see what happens now that your code has been amended to use the standard pickle module instead of your custom nester module Load your amended code into IDLE and press F5 to run it.

So, once again, let’s check the contents of the files to see what they look like now:

Once again, you

get no visual clue

that something has

It appears to have worked…but these files look like gobbledygook! What gives?

Recall that Python, not you, is pickling your data To do so efficiently, Python’s

pickle module uses a custom binary format (known as its protocol) As you

can see, viewing this format in your editor looks decidedly weird

Don’t worry: it is supposed to look like this.

idle session

pickle really shines when you load some previously pickled data into another program And, of course, there’s nothing to stop you from using pickle with nester After all, each module is designed to serve different purposes Let’s demonstrate with a handful of lines of code within IDLE’s shell Start by importing any required modules:

>>> import pickle

>>> import nester

No surprises there, eh?

Next up: create a new identifier to hold the data that you plan to unpickle.Create an empty list called new_man:

>>> new_man = []

Yes, almost too exciting for words, isn’t it? With your list created let’s load your pickled data into it As you are

working with external data files, it’s best if you enclose your code with try/except:

>>> try:

with open('man_data.txt', 'rb') as man_file:

new_man = pickle.load(man_file) except IOError as err:

print('File error: ' + str(err))

except pickle.PickleError as perr:

print('Pickling error: ' + str(perr))

This code is not news to you either However, at this point, your data has been unpickled and assigned to the

new_man list It’s time for nester to do its stuff:

You did just then!

(exasperated) Oh, this is futile!!

Generic file I/O with pickle is the way to go!

Now, no matter what data you create and process in your Python programs, you have a simple, tested, tried- and-true mechanism for saving and restoring your data How cool is that?

Python takes care of your file I/O details, so you can concentrate on what

your code actually does or needs to do

As you’ve seen, being able to work with, save, and restore data in lists is a

breeze, thanks to Python But what other data structures does Python

support out of the box?

Let’s dive into Chapter 5 to find out.

python toolbox

Your Python Toolbox

You’ve got Chapter 4 under your belt and you’ve added some key Python techiques to your toolbox

ƒ The finally suite is always executed

no matter what exceptions occur within a

try/except statement.

ƒ An exception object is passed into the except suite and can be assigned to

an identifier using the as keyword

ƒ The str() BIF can be used to access the stringed representation of any data object that supports the conversion

ƒ The locals() BIF returns a collection

of variables within the current scope

ƒ The in operator tests for membership

ƒ The “+” operator concatenates two strings when used with strings but adds two numbers together when used with numbers

ƒ The with statement automatically arranges to close all opened files, even when exceptions occur The with statement uses the as keyword, too

ƒ sys.stdout is what Python calls

“standard output” and is available from the standard library’s sys module

ƒ The standard library’s pickle module lets you easily and efficiently save and restore Python data objects to disk

ƒ The pickle.dump() function saves data to disk

ƒ The pickle.load() function restores data from disk

Python Lingo

• “Immutable typ es” - data typ

es

in Python that, onc e assigned

a value, cannot hav e that value changed.

• “Pickling” - the proc ess of saving a data objec t to persistenc

e storage.

• “Unpickling” - the proc ess of restoring a sav ed data object from persistenc e storage.

Life could be so much

easier if only she’d let me

help her extract, sort, and

comprehend her data

Work that data!

Data comes in all shapes and sizes, formats and encodings.

To work effectively with your data, you often have to manipulate and transform it into a

common format to allow for efficient processing, sorting, and storage In this chapter, you’ll

explore Python goodies that help you work your data up into a sweat, allowing you to

achieve data-munging greatness So, flip the page, and let’s not keep the coach waiting…