Learning the bash shell - unix shell programming

Đây là bộ sách tiếng anh cho dân công nghệ thông tin chuyên về bảo mật,lập trình.Thích hợp cho những ai đam mê về công nghệ thông tin,tìm hiểu về bảo mật và lập trình.

Learning the bash Shell, 3rd Edition

Table of Contents

1 bash Basics

1.1 What Is a Shell?

1.2 Scope of This Book

1.3 History of UNIX Shells

1.3.1 The Bourne Again Shell

1.3.2 Features of bash

1.4 Getting bash

1.5 Interactive Shell Use

1.5.1 Commands, Arguments, and Options 1.6 Files

1.8.2 Background Jobs and Priorities

1.9 Special Characters and Quoting

1.10 Help

2 Command-Line Editing

2.1 Enabling Command-Line Editing

2.2 The History List

2.3 emacs Editing Mode

2.4.4 Moving Around in the History List 2.4.5 Character-Finding Commands 2.4.6 Textual Completion

2.4.7 Miscellaneous Commands

2.5 The fc Command

2.6 History Expansion

2.7 readline

2.7.1 The readline Startup File

2.7.2 Key Bindings Using bind

2.8 Keyboard Habits

3 Customizing Your Environment

3.1 The bash_profile, bash_logout, and bashrc Files

4.1 Shell Scripts and Functions

4.3.4 Extended Pattern Matching 4.4 Command Substitution

4.5 Advanced Examples: pushd and popd

5.5 while and until

6 Command-Line Options and Typed Variables

8.1 Process IDs and Job Numbers

8.4.1 Traps and Functions

8.4.2 Process ID Variables and Temporary Files

9.1 Basic Debugging Aids

11.1 What's That Do?

11.1.1 Comments

11.1.2 Variables and Constants 11.2 Starting Up

11.3 Potential Problems

11.4 Don't Use bash

12 bash for Your System

12.1 Obtaining bash

12.2 Unpacking the Archive

12.3 What's in the Archive

A.1 The Bourne Shell

A.2 The IEEE 1003.2 POSIX Shell Standard A.3 The Korn Shell

B.2 Prompt String Customizations

B.3 Built-In Commands and Reserved Words B.4 Built-In Shell Variables

B.5 Test Operators

B.6 set Options

B.7 shopt Options

B.8 I/O Redirection

B.9 emacs Mode Commands

B.10 vi Control Mode Commands

C Loadable Built-Ins

D Programmable Completion

Learning the bash Shell, 3rd Edition

Cameron Newham

Copyright © 2009 O'Reilly Media, Inc

O'Reilly Media, Inc

The first thing users of the UNIX or Linux operating

systems come face to face with is the shell "Shell" is the

UNIX term for a user interface to the system—somethingthat lets you communicate with the computer via thekeyboard and the display Shells are just separateprograms that encapsulate the system, and, as such, thereare many to choose from

Systems are usually set up with a "standard" shell thatnew users adopt without question However, some ofthese standard shells are rather old and lack manyfeatures of the newer shells This is a shame, becauseshells have a large bearing on your working environment.Since changing shells is as easy as changing hats, there is

no reason not to change to the latest and greatest in shelltechnology

Of the many shells to choose from, this book introduces

the Bourne Again shell (bash for short), a modern

general-purpose shell Other useful modern shells are the

Korn shell (ksh) and the "Tenex C shell" (tcsh); both are

also the subjects of O'Reilly handbooks

bash Versions

This book is relevant to all versions of bash, although

older versions lack some of the features of the mostrecent version.[1] You can easily find out which version

you are using by typing echo $BASH_VERSION The

earliest public version of bash was 1.0, and the most

recent is 3.0 (released in July 2004) If you have an olderversion, you might like to upgrade to the latest one

Chapter 12shows you how to go about it

[ 1 ] Throughout this book we have clearly marked withfootnotes the features that are not present in the earlierversions

Summary of bash Features

bash is a backward-compatible evolutionary successor to

the Bourne shell that includes most of the C shell's majoradvantages as well as features from the Korn shell and afew new features of its own Features appropriated fromthe C shell include:

• Directory manipulation, with the pushd, popd, and dirs commands.

• Job control, including the fg and bg commands

and the ability to stop jobs with CTRL-Z

• Brace expansion, for generating arbitrary strings

• Tilde expansion, a shorthand way to refer todirectories

• Aliases, which allow you to define shorthandnames for commands or command lines

• Command history, which lets you recallpreviously entered commands

bash's major new features include:

• Command-line editing, allowing you to use vi- or emacs-style editing commands on your command

commands, including test, expr, getopt, and echo,

has been integrated into the shell itself, enablingcommon programming tasks to be done morecleanly and efficiently

• Control structures, especially the select construct,

which enables easy menu generation

• New options and variables that give you moreways to customize your environment

• One dimensional arrays that allow easyreferencing and manipulation of lists of data

• Dynamic loading of built-ins, plus the ability towrite your own and load them into the runningshell

Intended Audience

This book is designed to address casual UNIX and Linuxusers who are just above the "raw beginner" level Youshould be familiar with the process of logging in, enteringcommands, and doing simple things with files Although

Chapter 1 reviews concepts such as the tree-like file anddirectory scheme, you may find that it moves too quickly

if you're a complete neophyte In that case, we

recommend the O'Reilly handbook, Learning the UNIX Operating System, by Jerry Peek, Grace Todino, and John

Strang

If you're an experienced user, you may wish to skip

Chapter 1altogether But if your experience is with the Cshell, you may find that Chapter 1 reveals a few subtle

differences between the bash and C shells.

No matter what your level of experience is, you willundoubtedly learn many things in this book that will

make you a more productive bash user—from major

features down to details at the "nook-and-cranny" levelthat you may not have been aware of

If you are interested in shell programming (writing shell

scripts and functions that automate everyday tasks or

serve as system utilities), you should also find this bookuseful However, we have deliberately avoided drawing astrong distinction between interactive shell use (entering

commands during a login session) and shellprogramming We see shell programming as a natural,inevitable outgrowth of increasing experience as a user.Accordingly, each chapter depends on those previous to

it, and although the first three chapters are orientedtoward interactive use only, subsequent chapters describeinteractive, user-oriented features in addition toprogramming concepts

This book aims to show you that writing useful shellprograms doesn't require a computing degree Even if youare completely new to computing, there is no reason why

you shouldn't be able to harness the power of bash within

a short time

Toward that end, we decided not to spend too much time

on features of exclusive interest to low-level systemsprogrammers Concepts like file descriptors and specialfile types might only confuse the casual user, andanyway, we figure those of you who understand suchthings are smart enough to extrapolate the necessaryinformation from our cursory discussions

Code Examples

This book is full of examples of shell commands andprograms designed to be useful in your everyday life as auser, not just to illustrate the feature being explained In

Chapter 4 and onwards, we include various programming

problems, which we call tasks, that illustrate particular

shell programming concepts Some tasks have solutionsthat are refined in subsequent chapters The later chaptersalso include programming exercises, many of which build

on the tasks in the chapter

Feel free to use any code you see in this book and to pass

it along to friends and colleagues We especiallyencourage you to modify and enhance it yourself

If you want to try examples but you don't use bash as

your login shell, you must put the following line at thetop of each shell script:

#!/bin/bash

If bash isn't installed as the file /bin/bash, substitute its

pathname in the above

Chapter Summary

If you want to investigate specific topics rather than readthe entire book through, here is a chapter-by-chaptersummary:

Chapter 1 introduces bash and tells you how to install it

as your login shell Then it surveys the basics ofinteractive shell use, including overviews of the UNIXfile and directory scheme, standard I/O, and backgroundjobs

Chapter 2 discusses the shell's command history

mechanism (including the emacs- and vi-editing modes),

history substitution and the fc history command, and key

bindings with readline and bind.

Chapter 3 covers ways to customize your shellenvironment without programming by using the startupand environment files Aliases, options, and shellvariables are the customization techniques discussed

Chapter 4 is an introduction to shell programming Itexplains the basics of shell scripts and functions, anddiscusses several important "nuts-and-bolts"programming features: string manipulation operators,brace expansion, command-line arguments (positionalparameters), and command substitution

Chapter 5 continues the discussion of shell programming

by describing command exit status, conditional

expressions, and the shell's flow-control structures: if, for, case, select, while, and until.

Chapter 6goes into depth about positional parameters andcommand-line option processing, then discusses specialtypes and properties of variables, integer arithmetic, andarrays

Chapter 7 gives a detailed description of bash I/O This

chapter covers all of the shell's I/O redirectors, as well as

the line-at-a-time I/O commands read and echo It also

discusses the shell's command-line processing mechanism

and the eval command.

Chapter 8 covers process-related issues in detail It startswith a discussion of job control, then gets into variouslow-level information about processes, including processIDs, signals, and traps The chapter then moves to ahigher level of abstraction to discuss coroutines andsubshells

Chapter 9 discusses various debugging techniques, liketrace and verbose modes, and the "fake" signal traps Itthen presents in detail a useful shell tool, written using

the shell itself: a bash debugger.

Chapter 10 gives information for system administrators,including techniques for implementing system-wide shellcustomization and features related to system security

Chapter 11 discusses ways to make bash scripts more

maintainable

Chapter 12 shows you how to go about getting bash and

how to install it on your system It also outlines what to

do in the event of problems along the way

Appendix A compares bash to several similar shells,

including the standard Bourne shell, the POSIX shell

standard, the Korn shell (ksh), the public-domain Korn shell (pdksh), and the Z Shell (zsh).

Appendix B contains lists of shell invocation options,built-in commands, built-in variables, conditional test

operators, options, I/O redirection, and emacs- and vi-editing mode commands.

Appendix C gives information on writing and compilingyour own loadable built-ins

Appendix D looks at the basics of programmablecompletion

Conventions Used in This Handbook

We leave it as understood that when you enter a shellcommand, you press RETURN at the end RETURN islabeled ENTER on some keyboards

Characters called CTRL-X, where X is any letter, are

entered by holding down the CTRL (or CTL, orCONTROL) key and pressing that letter Although wegive the letter in uppercase, you can press the letterwithout the SHIFT key

Other special characters are LINEFEED (which is thesame as CTRL-J), BACKSPACE (same as CTRL-H),ESC, TAB, and DEL (sometimes labeled DELETE orRUBOUT)

This book uses the following font conventions:

Italic

Used for UNIX filenames, commands not built intothe shell (which are files anyway), and shell

functions Italic is also used for dummy parameters

that should be replaced with an actual value, to

distinguish the vi and emacs programs from their bash modes, and to highlight special terms the first

Used for bash built-in commands, aliases, variables,

and options, as well as command lines when they are

within regular text Bold is used for all elements

typed in by the user within regular text

Square Brackets

Used in Chapter 2 to show the position of the cursor

on the command line being edited Forexample:grep -l Alice < ~cam/book/[a]iw

We use UNIX as a shorthand for "UNIX and Linux."Purists will correctly insist that Linux is notUNIX—but as far as this book is concerned, theybehave identically.

We'd Like to Hear from You

Please address comments and questions concerning thisbook to the publisher:

O'Reilly Media, Inc

1005 Gravenstein Highway North

http://www.oreilly.com

Using Code Examples

This book is here to help you get your job done Ingeneral, you may use the code in this book in yourprograms and documentation You do not need to contact

us for permission unless you're reproducing a significantportion of the code For example, writing a program thatuses several chunks of code from this book does notrequire permission Selling or distributing a CD-ROM of

examples from O'Reilly books does require permission.

Answering a question by citing this book and quotingexample code does not require permission Incorporating

a significant amount of example code from this book into

your product's documentation does require permission.

We appreciate, but do not require, attribution Anattribution usually includes the title, author, publisher,

and ISBN For example "Learning the bash Shell, Third

Edition, by Cameron Newham and Bill Rosenblatt.Copyright 2005 O'Reilly Media, Inc., 0-596-00965-8."

Safari Enabled

When you see a Safari® Enabled icon on the cover ofyour favorite technology book, that means the book isavailable online through the O'Reilly Network SafariBookshelf

Safari offers a solution that's better than e-books It's avirtual library that lets you easily search thousands of toptech books, cut and paste code samples, downloadchapters, and find quick answers when you need the mostaccurate, current information Try it for free at

http://safari.oreilly.com

Acknowledgments for the First Edition

This project has been an interesting experience andwouldn't have been possible without the help of a number

of people Firstly, I'd like to thank Brian Fox and Chet

Ramey for creating bash and making it the polished

product it is today Thanks also to Chet Ramey for

promptly answering all of my questions on bash and

pointing out my errors

Many thanks to Bill Rosenblatt for Learning the korn Shell, on which this book is based; Michael O'Reilly and

Michael Malone at iiNet Technologies for their usefulcomments and suggestions (and my net.connection!);Chris Thorne, Justin Twiss, David Quin-Conroy, and mymum for their comments, suggestions, and corrections;Linus Torvalds for the Linux operating system which

introduced me to bash and was the platform for all of my

work on the book; Brian Fox for providing a short history

of bash; David Korn for information on the latest Korn

shell Thanks also to Depeche Mode for "101" as abackdrop while I worked, Laurence Durbridge for being alikable pest and never failing to ask "Finished the bookyet?" and Adam (for being in my book)

The sharp eyes of our technical reviewers picked upmany mistakes Thanks to Matt Healy, Chet Ramey, Bill

Reynolds, Bill Rosenblatt, and Norm Walsh for takingtime out to go through the manuscript.

The crew at O'Reilly were indispensable in getting thisbook out the door I'd like to thank Lenny Muellner forproviding me with the formatting tools for the job, ChrisReilley for the figures, and Edie Freedman for the coverdesign On the production end, I'd like to thank DavidSewell for his copyediting, Clairemarie Fisher O'Learyfor managing the production process, Michael Deutschand Jane Ellin for their production assistance, EllenSiever for tools support, Kismet McDonough forproviding quality assurance, and Seth Maislin for theindex

I'm grateful to Frank Willison for taking me up on myfirst piece of email to ORA: "What about a book on

bash?"

Last but by no means least, a big thank you to my editor,Mike Loukides, who helped steer me through this project

Acknowledgments for the Second Edition

Thanks to all the people at O'Reilly Gigi Estabrook wasthe editor for the second edition Nicole Gipson Arigowas the production editor and project manager NancyWolfe Kotary and Ellie Fountain Maden performedquality control checks Seth Maislin wrote the index EdieFreedman designed the cover, and Nancy Priest designedthe interior format of the book Lenny Muellnerimplemented the format in troff Robert Romano updatedthe illustrations for the second edition

Acknowledgments for the Third Edition

Thanks to the production people at O'Reilly and to theindexer

Thanks to Chet Ramey for once again swiftly answering

my queries on bash and for providing helpful comments

on the book I'd also like to thank Ian Macdonald for hisfeedback on Programmable Completion

Chapter 1 bash Basics

Since the early 1970s, when it was first created, theUNIX operating system has become more and morepopular During this time it has branched out intodifferent versions, and taken on such names as Ultrix,AIX, Xenix, SunOS, and Linux Starting onminicomputers and mainframes, it has moved ontodesktop workstations and even personal computers used

at work and home No longer a system used only byacademics and computing wizards at universities andresearch centers, UNIX is used in many businesses,schools, and homes As time goes on, more people willcome into contact with UNIX

You may have used UNIX at your school, office, or home

to run your applications, print documents, and read yourelectronic mail But have you ever thought about theprocess that happens when you type a command and hitRETURN?

Several layers of events take place whenever you enter acommand, but we're going to consider only the top layer,

known as the shell Generically speaking, a shell is any

user interface to the UNIX operating system, i.e., anyprogram that takes input from the user, translates it intoinstructions that the operating system can understand, andconveys the operating system's output back to the user

Figure 1-1shows the relationship between user, shell, andoperating system.

Figure 1-1 The shell is a layer around the UNIX operating system

There are various types of user interfaces bash belongs to

the most common category, known as character-baseduser interfaces These interfaces accept lines of textualcommands that the user types in; they usually producetext-based output Other types of interfaces include the

increasingly common graphical user interfaces (GUI),

which add the ability to display arbitrary graphics (notjust typewriter characters) and to accept input from amouse or other pointing device, touch-screen interfaces(such as those on some bank teller machines), and so on

What Is a Shell?

The shell's job, then, is to translate the user's commandlines into operating system instructions For example,consider this command line:

sort -n phonelist > phonelist.sorted

This means, "Sort lines in the file phonelist in numerical order, and put the result in the file phonelist.sorted."

Here's what the shell does with this command:

1 Breaks up the line into the pieces sort, -n, phonelist, >, and phonelist.sorted These pieces

are called words

2 Determines the purpose of the words: sort is a command, -n and phonelist are arguments, and > and phonelist.sorted, taken together, are I/O

instructions

3 Sets up the I/O according to > phonelist.sorted

(output to the file phone list.sorted) and somestandard, implicit instructions

4 Finds the command sort in a file and runs it with

the option -n (numerical order) and the argument

phonelist (input filename)

Of course, each of these steps really involves severalsubsteps, each of which includes a particular instruction

to the underlying operating system

Remember that the shell itself is not UNIX—just the userinterface to it UNIX is one of the first operating systems

to make the user interface independent of the operatingsystem

Scope of This Book

In this book you will learn about bash, which is one of

the most recent and powerful of the major UNIX shells

There are two ways to use bash: as a user interface and as

a programming environment

This chapter and the next cover interactive use These twochapters should give you enough background to use theshell confidently and productively for most of youreveryday tasks

After you have been using the shell for a while, you willundoubtedly find certain characteristics of yourenvironment (the shell's "look and feel") that you wouldlike to change, and tasks that you would like to automate

Chapter 3shows several ways of doing this

Chapter 3 also prepares you for shell programming, thebulk of which is covered in Chapter 4through Chapter 6.You need not have any programming experience tounderstand these chapters and learn shell programming

Chapter 7andChapter 8give more complete descriptions

of the shell's I/O and process-handling capabilities, while

Chapter 9 discusses various techniques for debuggingshell programs

You'll learn a lot about bash in this book; you'll also learn

about UNIX utilities and the way the UNIX operating

system works in general It's possible to become avirtuoso shell programmer without any previousprogramming experience At the same time, we'vecarefully avoided going into excessive detail about UNIXinternals We maintain that you shouldn't have to be aninternals expert to use and program the shell effectively,and we won't dwell on the few shell features that areintended specifically for low-level systems programmers.

History of UNIX Shells

The independence of the shell from the UNIX operating

system per se has led to the development of dozens of

shells throughout UNIX history—although only a fewhave achieved widespread use

The first major shell was the Bourne shell (named after itsinventor, Steven Bourne); it was included in the firstpopular version of UNIX, Version 7, starting in 1979

The Bourne shell is known on the system as sh Although

UNIX has gone through many, many changes, the Bourneshell is still popular and essentially unchanged SeveralUNIX utilities and administration features depend on it

The first widely used alternative shell was the C shell, or

csh This was written by Bill Joy at the University of

California at Berkeley as part of the Berkeley SoftwareDistribution (BSD) version of UNIX that came out acouple of years after Version 7

The C shell gets its name from the resemblance of itscommands to statements in the C ProgrammingLanguage, which makes the shell easier for programmers

on UNIX systems to learn It supports a number ofoperating system features (e.g., job control; see Chapter

8) that were unique to BSD UNIX but by now havemigrated to most other modern versions It also has a few