Unix Shell Programming Third Edition phần 3 docx

To sort the file instead by the third colon-delimited field which contains what is known as your userid, you would want an arithmetic sort, skipping the first two fields +2n, specifying

If you had to sort your data file by the y value—that is, the second number in each line—you could

tell sort to skip past the first number on the line by using the option

+1n

instead of -n The +1 says to skip the first field Similarly, +5n would mean to skip the first five fields

on each line and then sort the data numerically Fields are delimited by space or tab characters bydefault If a different delimiter is to be used, the -t option must be used

$ sort +1n data Skip the first field in the sort

is taken as the delimiter character.

Look at our sample password file again:

$ cat /etc/passwd

root:*:0:0:The super User:/:/usr/bin/ksh

steve:*:203:100::/users/steve:/usr/bin/ksh

bin:*:3:3:The owner of system files:/:

cron:*:1:1:Cron Daemon for periodic tasks:/:

george:*:75:75::/users/george:/usr/lib/rsh

pat:*:300:300::/users/pat:/usr/bin/ksh

uucp:*:5:5::/usr/spool/uucppublic:/usr/lib/uucp/uucico

asg:*:6:6:The Owner of Assignable Devices:/:

sysinfo:*:10:10:Access to System Information:/:/usr/bin/sh

To sort the file instead by the third colon-delimited field (which contains what is known as your user

id), you would want an arithmetic sort, skipping the first two fields (+2n), specifying the colon

character as the field delimiter (-t:):

$ sort +2n -t: /etc/passwd Sort by user id

root:*:0:0:The Super User:/:/usr/bin/ksh

cron:*:1:1:Cron Daemon for periodic tasks:/:

bin:*:3:3:The owner of system files:/:

uucp:*:5:5::/usr/spool/uucppublic:/usr/lib/uucp/uucico

asg:*:6:6:The Owner of Assignable Devices:/:

sysinfo:*:10:10:Access to System Information:/:/usr/bin/sh

Other options to sort enable you to skip characters within a field, specify the field to end the sort on,

merge sorted input files, and sort in "dictionary order" (only letters, numbers, and spaces are usedfor the comparison) For more details on these options, look under sort in your Unix User's Manual.

uniq

The uniq command is useful when you need to find duplicate lines in a file The basic format of thecommand is

uniq in_file out_file

In this format, uniq copies in_file to out_file, removing any duplicate lines in the process uniq's

definition of duplicated lines are consecutive-occurring lines that match exactly.

If out_file is not specified, the results will be written to standard output If in_file is also not specified,

uniq acts as a filter and reads its input from standard input

Here are some examples to see how uniq works Suppose that you have a file called names withcontents as shown:

Tony still appears twice in the preceding output because the multiple occurrences are not consecutive

in the file, and thus uniq's definition of duplicate is not satisfied To remedy this situation, sort isoften used to get the duplicate lines adjacent to each other The result of the sort is then run throughuniq:

$ sort names | uniq

Frequently, you'll be interested in finding the duplicate entries in a file The -d option to uniq should

be used for such purposes: It tells uniq to write only the duplicated lines to out_file (or standard

output) Such lines are written just once, no matter how many consecutive occurrences there are

$ sort names | uniq -d List duplicate lines

Tony

$

As a more practical example, let's return to our /etc/passwd file This file contains information abouteach user on the system It's conceivable that over the course of adding and removing users fromthis file that perhaps the same username has been inadvertently entered more than once You caneasily find such duplicate entries by first sorting /etc/passwd and piping the results into uniq -d asdone previously:

$ sort /etc/passwd | uniq -d Find duplicate entries in /etc/passwd

by using cut to extract the username from each line of the password file before sending it to uniq

$ sort /etc/passwd | cut -f1 -d: | uniq -d Find duplicates

$ grep -n 'harry' /etc/passwd

If you now want to remove the second cem entry, you could use sed:

$ sed '166d' /etc/passwd > /tmp/passwd Remove duplicate

$ mv /tmp/passwd /etc/passwd

mv: /etc/passwd: 444 modey

mv: cannot unlink /etc/passwd

$

Naturally, /etc/passwd is one of the most important files on a Unix system As such, only the

superuser is allowed to write to the file That's why the mv command failed

2 Tony

$

Two other options that won't be described enable you to tell uniq to ignore leading characters/fields

on a line For more information, consult your Unix User's Manual.

We would be remiss if we neglected to mention the programs awk and perl that can be useful whenwriting shell programs However, to do justice to these programs requires more space than we can

provide in this text We'll refer you to the document Awk—A Pattern Scanning and Processing

Language, by Aho, et al., in the Unix Programmer's Manual, Volume II for a description of awk

Kernighan and Pike's The Unix Programming Environment (Prentice Hall, 1984) contains a detailed

discussion of awk Learning Perl and Programming Perl, both from O'Reilly and Associates, present a

good tutorial and reference on the language, respectively

2: What will be the effect of the following commands?

who | grep 'mary'

who | grep '^mary'

grep '[Uu]nix' ch?/*

ls -l | sort +4n

sed '/^$/d' text > text.out

sed 's/\([Uu]nix\)/\1(TM)/g' text > text.out

date | cut -c12-16

date | cut -c5-11,25- | sed 's/\([0-9]\{1,2\}\)/\1,/'

3: Write the commands to

a.

Find all logged-in users with usernames of at least four characters.

In this chapter, you'll learn how to write your own commands and how to use shell variables.

of example, let's type this pipeline into a file We'll call the file nu (for number of users), and its

contents will be just the pipeline shown previously:

$ cat nu

who | wc –l

$

To execute the commands contained inside the file nu, all you now have to do is type nu as the

command name to the shell: [1]

[1] Note that the error produced here varies between different shells.

$ nu

sh: nu: cannot execute

$

Oops! We forgot to mention one thing Before you can execute a program this way, you must change

the file's permissions and make it executable This is done with the change mode command chmod

To add execute permission to the file nu, you simply type

chmod +x file(s)

The +x says make the file(s) that follow executable The shell requires that a file be both readable

and executable by you before you can execute it

$ ls -l nu

-rw-rw-r 1 steve steve 12 Jul 10 11:42 nu

$ chmod +x nu Make it executable

This time it worked

You can put any commands at all inside a file, make the file executable, and then execute its

contents simply by typing its name to the shell It's that simple and that powerful

The standard shell mechanisms such as I/O redirection and pipes can be used on your own programs

Suppose that you're working on a proposal called sys.caps and that the following command

sequence is needed every time you want to print a new copy of the proposal:

tbl sys.caps | nroff -mm –Tlp | lp

Once again, you can save yourself some typing by simply placing this command sequence into afile—let's call it run—making it executable, and then just typing the name run whenever you want toget a new copy of the proposal:

in, and your current working directory You know that the three command sequences you need to use

to get this information are date, who | wc -l, and pwd:

$ stats Try it out

Wed Jul 10 11:55:50 EDT 2002

13

The current date and time is:

Wed Jul 10 12:00:27 EDT 2002

The number of users on the system is:

The current date and time is: Wed Jul 10 12:00:27 EDT 2002

Comments

The shell programming language would not be complete without a comment statement A comment is

a way for you to insert remarks or comments inside the program that otherwise have no effect on itsexecution

Whenever the shell encounters the special character # at the start of a word, it takes whatever

characters follow the # to the end of the line as comments and simply ignores them [2] If the #starts the line, the entire line is treated as a comment by the shell Here are examples of valid

comments:

[2] Note that the # may be your default erase character If so, to enter the character into an editor such as ed

or vi, you'll have to "escape" it by preceding it with a \ Alternatively, you can change your erase character to something else with the stty command

# Here is an entire commentary line

who | wc –l # count the number of users

Let's go back to the stats program and insert some comments:

$ cat stats

#

# stats prints: date, number of users logged on,

# and current working directory

Variables

Like virtually all programming languages, the shell allows you to store values into variables A shell

variable begins with an alphabetic or underscore (_) character and is followed by zero or more

alphanumeric or underscore characters

To store a value inside a shell variable, you simply write the name of the variable, followed

immediately by the equals sign =, followed immediately by the value you want to store in the

In the shell language, you can't put those spaces in

Second, unlike most other programming languages, the shell has no concept whatsoever of data

types Whenever you assign a value to a shell variable, no matter what it is, the shell simply

interprets that value as a string of characters So when you assigned 1 to the variable count

previously, the shell simply stored the character 1 inside the variable count, making no observation

whatsoever that an integer value was being stored in the variable

If you're used to programming in a language such as C or Pascal, where all variables must be

declared, you're in for another readjustment Because the shell has no concept of data types,

variables are not declared before they're used; they're simply assigned values when you want to usethem

As you'll see later in this chapter, the shell does support integer operations on shell variables thatcontain strings that are also valid numbers through special built-in operations

Because the shell is an interpretive language, you can assign values to variables directly at yourterminal:

$ count=1 Assign character 1 to count

$ my_bin=/users/steve/bin Assign /users/steve/bin to my_bin

$

So now that you know how to assign values to variables, what good is it? Glad you asked

Displaying the Values of Variables

The echo command is used to display the value stored inside a shell variable To do this, you simplywrite

Figure 5.1 echo $count.

You can have the value of more than one variable substituted at a time:

Figure 5.2 —echo $my_bin $count.

The values of variables can be used anywhere on the command line, as the next examples illustrate:

$ cd $my_bin Change to my bin directory

$ pwd

/users/steve/bin

$ number=99

$ echo There are $number bottles of beer on the wall

There are 99 bottles of beer on the wall

So you see, even the name of a command can be stored inside a variable Because the shell performsits substitution before determining the name of the program to execute and its arguments, it scansthe line

$command $option $file

and turns it into

wc -l names

Then it executes wc, passing the two arguments -l and names

Variables can even be assigned to other variables, as shown in the next example:

The Null Value

What do you think happens when you try to display the value of a variable that has no value assigned

to it? Try it and see:

$ echo $nosuch Never assigned it a value

So you see no characters were substituted by the shell for the value of nosuch

A variable that contains no value is said to contain the null value It is the default case for variables

that you never store values in When the shell performs its variable substitution, any values that are

null are completely removed from the command line, without a trace:

$ wc $nosuch -l $nosuch $nosuch names

which explains why it works

Sometimes you may want to explicitly set a variable null in a program This can be done by simplyassigning no value to the variable, as in

is not equivalent to the three previous ones because it assigns a single space character to dataflag;

that's different from assigning no characters to it.

Filename Substitution and Variables

Here's a puzzle for you: If you type

echo $x

was executed is as follows:

The shell scanned the line, substituting * as the value of x

The ${variable} Construct

Suppose that you have the name of a file stored in the variable filename If you wanted to renamethat file so that the new name was the same as the old, except with an X added to the end, your firstimpulse would be to type

mv $filename $filenameX

When the shell scans this command line, it substitutes the value of the variable filename and also

the value of the variable filenameX The shell thinks filenameX is the full name of the variablebecause it's composed entirely of valid variable name characters To avoid this problem, you candelimit the end of the variable name by enclosing the entire name (but not the leading dollar sign) in

a pair of curly braces, as in

${filename}X

This removes the ambiguity, and the mv command then works as desired:

mv $filename ${filename}X

Remember that the braces are necessary only if the last character of the variable name is followed by

an alphanumeric character or an underscore

Built-in Integer Arithmetic

The POSIX standard shell provides a mechanism for performing integer arithmetic on shell variables

called arithmetic expansion Note that some older shells do not support this feature.

The format for arithmetic expansion is

$((expression))

where expression is an arithmetic expression using shell variables and operators Valid shell variables

are those that contain numeric values (leading and trailing whitespace is allowed) Valid operatorsare taken from the C programming language and are listed in Appendix A , "Shell Summary."

The result of computing expression is substituted on the command line For example,

echo $((i+1))

adds one to the value in the shell variable i and prints the result Notice that the variable i doesn'thave to be preceded by a dollar sign That's because the shell knows that the only valid items thatcan appear in arithmetic expansions are operators, numbers, and variables If the variable is notdefined or contains a NULL string, its value is assumed to be zero So if we have not assigned anyvalue yet to the variable a , we can still use it in an integer expression:

$ echo $a Variable a not set

Note that assignment is a valid operator, and the value of the assignment is substituted in the second

echo command in the preceding example

Parentheses may be used freely inside expressions to force grouping, as in

echo $((i = (i + 10) * j))

If you want to perform an assignment without echo or some other command, you can move the

assignment before the arithmetic expansion.

So to multiply the variable i by 5 and assign the result back to i you can write

i=$(( i * 5 ))

Note that spaces are optional inside the double parentheses, but are not allowed when the

assignment is outside them

Finally, to test to see whether i is greater than or equal to 0 and less than or equal to 100, you can

That concludes our introduction to writing commands and using variables The next chapter goes into

detail on the quoting mechanisms in the shell

write the commands in terms of these variables to

List the contents of the documents directory

Chapter 6 Can I Quote You on That?

IN THIS CHAPTER

The Single Quote

The Double Quote

The single quote character '

The double quote character "

The backslash character \

The back quote character `

The first two and the last characters in the preceding list must occur in pairs, whereas the backslashcharacter is unary in nature Each of these quotes has a distinct meaning to the shell We'll coverthem in separate sections of this chapter

The Single Quote

There are several reasons that you might need to use quotes in the shell One of these is to keepcharacters otherwise separated by whitespace characters together Let's look at an example Here's afile called phonebook that contains names and phone numbers:

To look up someone in our phonebook file—which has been kept small here for the sake of

example—you use grep:

$ grep Alice phonebook

Alice Chebba 973-555-2015

$

Look what happens when you look up Susan:

$ grep Susan phonebook

Susan Goldberg 201-555-7776

Susan Topple 212-555-4932