Linux programming unleash phần 4 docx

If you send data through a message queue between two separate grams obviously, the normal way to use message queues, only one program should create the message queue using the IPC_CREATf

private use, such as a custom malloc()implementation MAP_FIXEDcauses the kernel toplace the map at a specific address If the address is already in use or otherwise unavail-able,mmap()fails If MAP_FIXEDis not specified and addressis unavailable, the kernelwill attempt to place the region elsewhere in memory MAP_LOCKEDallows processes withroot privilege to lock the region into memory so it will never be swapped to disk Userspace programs cannot use MAP_LOCKED, a security feature that prevents unauthorizedprocesses from locking all available memory, which would essentially bring the system

to a standstill

When you have finished using a memory mapped file, call munmap()to unmap the regionand return the memory to the operating system This function uses the following proto-type:

int munmap(void *start, size_t length);

The startargument points to the beginning of the region to unmap, and lengthcates how much of the region to unmap After a memory block has been unmapped, fur-ther attempts to access startwill cause a segmentation fault (generate a SIGSEGV) When

indi-a process terminindi-ates, indi-all memory mindi-aps indi-are unmindi-apped The munmap()function returns 0

on success or, on failure, -1 and sets errno

Themsync()function writes a mapped file to disk It uses the following prototype:

int msync(const void *start, size_t length, int flags);

Call msync()to update the disk file with changes made to the in-core map The region toflush to disk begins at the startaddress; length bytes will be flushed The flagsargu-ment is a bitwise OR of one or more of the following:

MS_ASYNC Schedules a write and returns

MS_SYNC Data are written before msync()returns

MS_INVALIDATE Invalidate other maps of the same file so they will be

updated with new data

The mprotect()function modifies the protection on a memory map This function usesthe following prototype:

int protect(const void *addr, size_t len, int prot);

System Programming

P ART II

254

This function call modifies the protections on the memory region that begins at addrtothe protections specified in prot, a bitwise OR of one or more of the flags listed in Table14.1 It returns zero on success or, on failure, -1 and sets errno.

Locking Memory

Without going into the nitty-gritty details of how it works, memory locking means venting a memory area from being swapped to disk In a multitasking, multiuser systemsuch as Linux, areas of system memory (RAM) not in active use may be temporarilywritten to disk (swapped out) in order for that memory to be put to other uses Lockingthe memory sets a flag that prevents it from being swapped out

pre-There are four functions for locking and unlocking memory:mlock(),mlockall(),

munlock(), and munlockall() Their prototypes are listed below

int mlock(const void *addr, size_t len);

int munlock(void *addr, size_t len);

int mlockall(int flags);

int munlockall(void);

The memory region to be locked or unlocked is specified in addrand lenindicates howmuch of the region to lock or unlock Values for flagsmay be one or both of MCL_CUR- RENT, which requests that all pages are locked before the call returns, or MCL_FUTURE,indicating that all pages added to the process’ address space should be locked As noted

in the discussion of mmap(), only processes with root privilege may lock or unlock ory regions

Use the mremap()function to change the size of a mapped file This function uses thefollowing prototype:

void *mremap(void *old_addr, size_t old_len,

➥size_t new_len, unsigned long flags);

You will occasionally need to resize a memory region, which is the reason for this tion An analogue of the realloc()call discussed earlier,mremap()resizes the memoryregion beginning at old_addr, originally with size old_len, to new_len flagsindicateswhether the region can be moved in memory if necessary MREMAP_MAYMOVEpermits theaddress to change; if not specified, the resize operation fails mremap()returns theaddress of the resized region or NULL on failure

Implementing cat(1) Using Memory Maps

Listing 14.2 illustrates using memory mapped files Although it is a naive cat(1)mentation, it clearly demonstrates using memory mapped files

imple-L ISTING 14.2 A cat(1)IMPLEMENTATIONUSINGMEMORYMAPS

34 /* map the input file */

35 if((src = mmap(0, len, PROT_READ, MAP_SHARED,

43 munmap(src, len);

44

45 exit(0);

46 } 47

fread()or fgets()call, as we do in the fprintf()statement on line 39 We return tothe memory region to the kernel on line 43 by calling munmap() perror(), used in theutility function err_quit(), was introduced in Chapter 13, “Handling Errors.”

From a practical point of view, using a memory mapped file in this example was overkillbecause it buys us little in terms of performance or code length In situations where per-formance is crucial or when you are dealing with time-sensitive operations, however,memory mapped files can be a definite plus Memory mapping can also be valuable inhigh security situations Because processes running with root privilege can lock memorymapped files into memory, preventing them from being swapped to disk by Linux’smemory manager, sensitive data such as password files will be less susceptible to scannerprograms Of course, in such a situation, the memory region would have to be set to

PROT_NONEso that other process cannot read the region

Now that you know more about memory maps, the next section examines a few tools tohelp you debug memory problems

Finding and Fixing Memory Problems

This section covers a few tools that help you locate memory management problems inyour code Because C assumes you know what you are doing, most C compilers ignoreuses of uninitialized memory, buffer overruns, and buffer underruns Nor do most com-pilers catch memory leaks or dangling pointers The tools discussed in this section make

up for these compiler shortcomings

A Problem Child

Listing 14.3 is beset with bugs, including the following:

• A memory leak (line 18)

• Overruns the end of dynamically allocated heap memory (lines 22 and 28)

• Underruns a memory buffer (line 32)

• Frees the same buffer twice (lines 36 and37)

• Accesses freed memory (lines 40 and41)

• Clobbers statically allocated stack and global memory (lines 48 and 44,respectively)

L ISTING 14.3 A PROGRAM WITHMEMORYBUGS

Actually, most compilers accept various switches and options that enable them

to catch some subset of the errors just mentioned gcc , for example, has the Wall option (discussed in Chapter 3, “GNU cc”) In general, however, compilers

-do not detect all memory problems, making the tools covered in this section quite valuable.

20 /* Overrun buf a little bit */

39 /* access free()ed memory */

40 strcpy(buf, “This will blow up”);

41 fprintf(stdout, “FREED : %s\n”, buf);

42

43 /* Trash the global variable */

44 strcpy(g_buf, “global boom”);

45 fprintf(stdout, “GLOBAL : %s\n”, g_buf);

46

47 /* Trash the local variable */

48 strcpy(l_buf, “local boom”);

49 fprintf(stdout, “LOCAL : %s\n”, l_buf);

On other systems, especially those configured to allow core dumps, the sample programmay dump core on the second call to free()(line 37)

Using mpr and check to Locate Memory Problems

The first tool covered here is Taj Khattra’s mprpackage, available from your favoriteMetalab mirror (ftp://metalab.unc.edu/pub/Linux/devel/lang/c/mpr-1.9.tar.gz)

It can be used to find memory leaks, but it does not find memory corruption errors Inaddition,mpralso generates allocation statistics and patterns, but those features will not

be covered in this section mpr’s method uses simple brute force: it logs all allocation andfree requests to an external log file that is later processed using mpr’s utilities

To use mpr, download and compile it The package includes several utility programs and

a static library,libmpr.a, that you link your program against To compile Listing 14.3,the command line was:

$ gcc -g badmem.c -o badmem -lmpr -L $HOME/lib

Be sure to use the -gswitch to generate debugging symbols because some of mpr’s grams require them Recall from Chapter 3, “GNU cc,” that -lmprlinks badmemagainst

pro-libmpr.a, and -L $HOME/libprepends $HOME/libto the library search path Once theprogram is compiled and linked, set the environment variables $MPRPCand $MPRFI mpr

uses $MPRPCto traverse and display the call chain for each allocation and free request,while $MPRFIdefines a pipeline command for logging (and, optionally, filtering) mpr’soutput

$ export MPRPC=`mprpc badmem`

$ export MPRFI=”cat > badmem.log”

With these preliminary steps out of the way, execute the program If all goes as planned,you should wind up with a file named badmem.login the current directory It will looksomething like the following:

m:134522506:134516229:134514813:10:134561792 m:134522506:134516229:134514826:5:134565888 f:134522614:134520869:134514880:134565888 m:134522506:134516229:134514890:5:134565888 f:134522614:134520869:134514975:134565888 f:134522614:134520869:134514987:134565888

This isn’t very informative as is, but the mprdocumentation explains the format The logfile provides the raw material for mpr’s utility programs, which parse, slice, dice, andjulienne the log to create meaningful information

System Programming

P ART II

260

To view memory leaks, use mprlk:

The output indicates that on line 18 of badmem.c, in the main()function, we malloc()

10 bytes of memory that we never free (the long decimal number is the call chaincounter, which mprand its utilities use precisely to track each allocation and freerequest) Looking back at Listing 14.3, this is exactly correct

I mentioned a moment ago that mprcannot detect memory corruption errors While this

is true,mprincludes the mcheck()function from GNU’s malloc()library, which enablesyou to detect buffer overruns, buffer underruns, and multiple free()s of the same block

In fact,mprcompilesmcheck()intolibmpr.aby default So, the good news is that thebuffer overruns and underruns in Listing 14.3 will cause it to abort unless you specifical-

ly instruct mprnot to use mcheck() The bad news is that mcheck()is not terribly mative—it merely complains about a problem and leaves the programmer to determinewhere the problem occurs Compiled with mcheck(), the sample program aborts eachtime we clobber memory:

infor-$ /badmem LITTLE : abcde mcheck: memory clobbered past end of allocated block IOT trap/Abort

After fixing the first overrun, the program gets a little farther:

$ /badmem LITTLE : abcde BIG : abcdefgh UNDERRUN: abcdefgh mcheck: memory clobbered before allocated block IOT trap/Abort

Interestingly,mcheck()ignores the larger overrun on line 28, but dies, as you wouldexpect, when the program underruns the buffer on line 32 After fixing these two errors,

mcheck()complains about freeing memory twice, as shown in the following code:

$ /badmem LITTLE : abcde BIG : abcdefgh UNDERRUN:

mcheck: block freed twice IOT trap/Abort

Fixing the other errors is left as an exercise for you

Electric Fence

The next tool covered is Electric Fence, written by Bruce Perens Electric Fence does notcatch memory leaks, but it does an excellent job of detecting buffer overruns You canobtain it from ftp://metalab.unc.edu/pub/Linux/devel/lang/c,although manyLinux distributions also ship with it

Electric Fence uses a system’s virtual memory hardware to detect illegal memory

access-es, stopping a program on the first instruction that causes a boundary violation It plishes this by replacing the normal malloc() with its own malloc(),and allocating asmall section of memory after the requested allocation that the process is not permitted

accom-to access As a result, buffer overruns cause a memory access violation, which aborts theprogram with a SIGSEGV(segmentation violation) If your system is configured to allowcore files (execute ulimit -cto get and set the size of core files allowed), you can thenuse a debugger to isolate the location of the overrun Like mpr, to use Electric Fence youhave to link your program against a special library,libefence.a:

$ gcc -ggdb badmem.c -o badmem -lefence

The compile command used the -ggdboption to generate extra gdb-compatible ging symbols When executed, the program aborts and dumps core:

debug-$ /badmem

Electric Fence 2.0.5 Copyright © 1987-1995 Bruce Perens.

LITTLE : abcde Segmentation fault (core dumped)

Next, using the core file, run badmemfrom the gdbdebugger (just follow the example forthe time being, because gdb is covered indetail in Chapter 36, “Debugging: GNU

gdb”)

$ gdb badmem GNU gdb 4.17

System Programming

P ART II

262

Copyright 1998 Free Software Foundation, Inc.

GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions

Type “show copying” to see the conditions.

There is absolutely no warranty for GDB Type “show warranty” for details.

This GDB was configured as “i386-COL-linux”

(gdb) run Starting program: /home/kwall/projects/unleashed/src/14/badmem

Electric Fence 2.0.5 Copyright (C) 1987-1995 Bruce Perens.

LITTLE : abcde

Program received signal SIGSEGV, Segmentation fault.

strcpy (dest=0x40003ff8 “abcdefgh”, src=0x8055e0c “abcdefgh”) at strcpy.c:35

strcpy.c:35: No such file or directory.

The second line from the bottom of the listing makes it crystal clear that there is a lem at line 28 in badmem.cin the main()function Once you fix this problem, you wouldthen recompile and rerun the program, and, if it aborts again, repeat the debug/fix/recom-pile sequence Once you’ve thoroughly debugged all of your code, recompile withoutlinking against Electric Fence, and you should be set

prob-But wait, Electric Fence caught the big overrun on line 28, but it missed the little overrun

on line 22 How could this be? This peculiar behavior results from the way the CPUaligns allocated memory Most modern CPUs require that memory blocks be aligned ontheir natural word size Intel x86 CPUs, for example, require that memory regions begin

at addresses evenly divisible by four, so malloc()calls ordinarily return pieces of memory aligned accordingly Electric Fence does the same So, a request for five bytesactually results in eight bytes being allocated in order to meet the memory alignmentrequirements! As a result, the small buffer overrun on line 22 slips through the fence

Fortunately, Electric Fence allows you to control its alignment behavior using the ronment variable $EF_ALIGNMENT Its default value is sizeof(int), but if you set it tozero (0), Electric Fence will detect smaller overruns After setting $EF_ALIGNMENTto 0,recompiling, and rerunning the program, Electric Fence catches the small overrun at line 22:

Program received signal SIGSEGV, Segmentation fault.

strcpy (dest=0x40003ffb “abcde”, src=0x8055df8 “abcde”) at strcpy.c:35 strcpy.c:35: No such file or directory.

Electric Fence recognizes three other environment variables that control its behavior:

EF_PROTECT_BELOW=1for detecting buffer underruns; EF_PROTECT_FREE=1for detectingaccess to free()ed memory; and EF_ALLOW_MALLOC_0=1, which allows programs to

malloc()zero bytes of memory

Use a Lint Brush

The traditional tool for detecting code problems is lint Although lintis a proprietaryprogram, an open source (possibly better) version of it, LCLint, by David Evans,released under an MIT-style license (see Chapter 39, “Licensing”) can be obtained from

http://www.sds.lcs.mit.edu/lclint/and most free software repositories on theInternet LCLint is a hugely capable program that accomplishes far more than merelyfinding memory problems Unfortunately, there is only time and space to highlight itsabilities with respect to memory irregularities You are strongly encouraged to visit theWeb site, download the package, and invest some time and effort in learning to useLCLint—you will be glad you did

Using LCLint is simple: just execute it (lclintis the command name, while LCLint isthe formal name), providing the name of one or more source files as arguments:

$ lclint badmem.c LCLint 2.4b —- 18 Apr 98

badmem.c: (in function main) badmem.c:22:9: Possibly null storage buf passed as non-null param:

strcpy (buf, )

A possibly null pointer is passed as a parameter corresponding to a

➥formal parameter with no /*@null@*/ annotation If NULL may

➥be used for this parameter, add a /*@null@*/ annotation to the

➥function parameter declaration (-nullpass will suppress message) badmem.c:21:8: Storage buf may become null

The first line names the function in which the error is found The second line identifiesthe filename, the line and column number, and the error message In this case,lclint

detected that the program may be passing a NULL argument,buf, to strcpy(), which

System Programming

P ART II

264

requires a non-NULL argument The third line provides a hint of more information aboutthe error, and the last line provides additional location information, where appropriate.

badmem.c:37:7: Dead storage buf passed as out parameter: buf Memory is used after it has been released (either by passing as an only param or assigning to and only global (-usereleased will suppress message)

badmem.c:36:7: Storage buf is released

This message indicates that badmem.cuses the bufvariable on line 37 after bufwasreleased on line 36

badmem.c:41:36: Variable buf used after being released badmem.c:37:7: Storage buf released

Summary

This chapter covered a potpourri of memory management tools and techniques Itreviewed the standard C functions for obtaining and manipulating memory regions andalso looked at using memory mapped files using the mmap()family of system calls

Finally, you were introduced to three tools for locating and correcting memory bugs:mpr,Electric Fence, and LCLint

266

Interprocess Communication and Network

• TCP/IP and Socket Programming 295

• UDP: The User Data Protocol 311

• Using Multicast Sockets 317

• Non-blocking Socket I/O 325

• A C++ Class Library for TCP Sockets 331

• Using Libraries 341

• Device Drivers 359

by Mark Watson

This chapter starts the discussion of building distributed applications using InterprocessCommunication (IPC) Linux provides a powerful platform for building distributed applications The World Wide Web (WWW) could be considered the ultimate distributedapplication, and more Web servers run on Linux than any other operating system Even

if you are building applications to run on a single computer, it still often makes sense

to use IPC to break up a program into modules, with well-defined interfaces definedacross IPC boundaries Chapters 16–23 cover “classic” methods of IPC, including the following:

• Non-blocking socket I/O

• A C++ class library for IPCThis chapter, as well as Chapters 16 through 22, uses the C language for all programexamples Chapter 23 uses the C++ language All programs for these chapters are located

on the CD-ROM in the IPC directory, which contains the following subdirectories:C++

MULTICASTPIPESSHAREDUDPMESSAGEQNOBLOCKSEMAPHORESSOCKETSYou should copy the entire directory tree under the IPCdirectory to a convenient work-ing directory on your local disk The text of the chapters in this Part refers to the exam-ple source code in the subdirectories of the IPC directory The chapter text also containssmall “snippets” of the example programs for use in explaining how the example pro-grams work You might find it useful to print out the complete listings of the sample pro-grams to augment reading of the text

Interprocess Communication and Network Programming

P ART III

270

This Part will not cover other useful (and higher level) techniques for building distributedapplications like CORBA, Remote Procedure Calls (RPC), Distributed ComputingEnvironment (DCE), and Java’s Remote Method Invocation (RMI).

Updates to the examples in this Part and corrected errors will be posted to

http://www.markwatson.com/books/linux_prog.html

Introduction to Using Pipes

Pipes are a one-way communication channel and are accessed through a socket tor Operations on pipes look like operations on local files Two short examples will beused to illustrate both unnamed and named pipes Typically, unnamed pipes are used forcommunication between a parent process and a child (or forked) process Named pipesare most useful for communication between different programs that share the same filesystem Pipes are easy to use, but lack the generality of sockets (which are covered inChapter 19, “TCP/IP and Socket Programming”)

descrip-Unnamed Pipes

Unnamed pipes are used when a Linux program forks off a separate process after usingthepipelibrary call to create two file descriptors (one for each end of the pipe) The file

unnamed_pipe.cimplements both ends of a pipe reader/writer by creating two input/

output file descriptors:

int file_descriptors[2];

pipe(file_descriptors);

and then forking off a new process using the forklibrary call:

pid_t spawned_process_pid = fork();

After the call to fork, there are two copies of the program executing, so we need tocheck which process is the original (spawning or parent) process, and which process isthe copied (spawned or child) process:

if(spawned_process_pid == 0) { printf(“in the spawned (child) process \n”);

} else { printf(“in the spawning (parent) process \n”);

}

In this snippet of code, we simply printed out whether the current process was the parent

or child process In a real application, the functionality of parent and child processes isusually quite different In this example, the spawned (child) process closes the input filedescriptor and writes some data to the output file descriptor:

Introduction to IPC: Using Pipes

#define INPUT 0

#define OUTPUT 1 close(file_descriptors[INPUT]);

write(file_descriptors[OUTPUT],

“test data”, strlen(“test data”));

The original (spawning or parent) process closes the output file descriptor and reads datafrom the input file descriptor:

close(file_descriptors[OUTPUT]);

// wait for data sent by the spawned (child) process:

returned_count = read(file_descriptors[INPUT],

buf, sizeof(buf));

Unnamed pipes are the standard method for communication between parent and childprocesses In Chapter 19, you will learn how to write socket- based client and server pro-grams As you will see, sockets are often the right technology for communicationbetween programs in the general case that one program does not spawn another, andwhen programs are running on different computer systems

A very common use of unnamed pipes is in server applications that spawn off one ormore processes to handle long-running computational tasks Figure 15.1 shows an exam-ple of a server program that uses a child (or spawned) process to handle service requests

Interprocess Communication and Network Programming

P ART III

272

Input queue

Remove next service request

Return processed request to requestor Parent process

Process service request Spawned work process

Internet Service Requests

F IGURE 15.1

A server process spawns off a child process to handle long-running tasks.

Named Pipes

Named pipes (also referred to as FIFOs) can only be used for communication betweentwo processes that share a common file system One advantage of named pipes overunnamed pipes is the ability to communicate between two processes that are started inde-pendently—where one process does not fork off a new process) The following manpages will be useful when writing applications that use named pipes:

• man 2 open—Opens a named pipe created with mknod

• man 2 read—Reads from a pipe

• man 2 write—Writes to a pipeThe directory src/IPC/PIPESon the CD-ROM contains two example programs,

read_pipe.cand write_pipe.c,and a “read me” file explaining how to set up a namedpipe using the UNIX mknod utility Although it is also possible to create unnamed pipesusing the UNIX pipesystem call, I find named pipes to be more convenient Namedpipes are used in the following example I assume that you have copied the entire

srcdirectory from the CD-ROM; change directory to src/IPC/PIPESand execute the following:

mknod a_pipe p make

The mknodutility is used to create special files mknodtakes both an argument thatappears before the filename, and a file type that appears after the filename on the com-mand line The argument pindicates that the special file a_pipeis a FIFO file When

mknodis used to create a FIFO file, no arguments are allowed before the filename

You can then run the read_pipeand write_pipeexample programs in two separate minal windows As you can see in the read_pipe.csource file, opening and readingfrom a named pipe is exactly like reading from a “normal” file:

ter-FILE * in_file;

char buf[80];

in_file = fopen(“a_pipe”, “r”);

if (in_file == NULL) { perror(“Error in fopen”);

exit(1);

} fread(buf, 1, 80, in_file);

printf(“received from pipe: %s\n”, buf);

The system utility function perroris useful for both printing a string message argument(in this example, this is “Error in fopen”) and the last system error Because named pipescan be used with the standard file I/O functions fopen,fread, and fwrite, named pipesare simple to use Also, in the write_pipe.csource file, we see that writing to a namedpipe is the same as writing to a “normal” file:

fwrite(buf, 1, 80, out_file);

fclose(out_file);

Using named pipes is a simple way to pass data between two programs that have access

to the same file system When reading the source to the example programs, you can look

at the man pages for fopen,fread,fwrite, and fclosefor documentation on these system calls These example programs simply pass a C-style string as data You may alsowant to pass binary data like a structof a C++ object (if the struct or object contains nopointers or references to other objects) through a pipe If you are passing binary databetween computers with a different CPU type, you may have to worry about incompati-ble data types for primitive types like int, long, float, and packing of structs and C++objects If you are programming distributed applications for heterogeneous computerenvironments, then you may want to use RPC or CORBA

Summary

Pipes provide a simple method for IPC between a parent process and a child usingunnamed pipes and between any two processes running on the same machine usingnamed pipes The next seven chapters will cover alternative methods of IPC Eventhough the use of sockets (covered in Chapter 19) is the most general method for imple-menting IPC, you should know how to program using pipes because many older UNIXutilities were written using them

Interprocess Communication and Network Programming

P ART III

274

Message queues are similar to using pipes but have the advantage of allowing messages

to be tagged with specific message types A message recipient can ask for the next able message ignoring message type (by specifying a desired message type of zero), orthe next message of a specific type by specifying a positive non-zero message type.Message queues are used for communication between processes that are running on thesame computer In Chapter 19, “TCP/IP and Socket Programming,” socket programming

avail-is used to provide a solution for communication between programs running on differentcomputers Any corrections and updates for this chapter can be found at

http://www.markwatson.com/books/linux_prog.html.Like pipes, message queues are largely interesting for historical reasons Many olderUNIX programs use message queues

Creating a Sample Message Queue Program

This section uses a sample program,message_q.c,to show how to set up a messagequeue, and then uses it to send and receive a message This single sample program writes

to and reads from a message queue In real applications, one program typically writes to

a message queue and another program reads from the message queue When reading thesource to the sample program message_q.c, you will want to check the man pages for

msgget,msgsnd,msgrcv, and msgctl In the example in the following chapter,

“Semaphores,” the ftoksystem function is used to get a unique key based on the currentdirectory name; ftokis also used here:

key_t unique_key = ftok(“.”, ‘a’); // ‘a’ can be any character

The ftoksystem call provides a convenient way to calculate a key based on the currentdirectory where a program is running If we start multiple programs from the same direc-tory and use the preceding call to ftokto calculate a unique key, then they can access acommon message queue The msggetsystem function gets a queue identifier based onthe unique key:

int id = msgget(unique_key, IPC_CREAT | 0666);

The first argument to msggetis a unique key that we calculated by calling ftok The ond argument specifies the file permissions for the message queue In this example, thesame program reads and writes from the message queue, so the call to msggetuses the

sec-IPC_CREATflag If you send data through a message queue between two separate grams (obviously, the normal way to use message queues), only one program should create the message queue using the IPC_CREATflag The sample program message_q.c

pro-deletes the message queue before terminating, but it is also possible to create a message

Interprocess Communication and Network Programming

P ART III

276

queue and to reuse it with many programs The value of the variable id(the return valuefrom the call to msgget) will be passed to all calls to msgsnd,msgrcv, and msgctltoidentify the message queue The structure data type msgbufis defined in the file

/usr/include/linux/msg.h:

/* message buffer for msgsnd and msgrcv calls */

struct msgbuf { long mtype; /* type of message */

char mtext[1]; /* message text */

};

In applications using message queues, we need to define our own data structure like bufbecause the msgbufstruct contains only one character In the sample program, weallocate 80 bytes for message data In message_q.c, we define:

msg-struct amsgbuf { long mtype;

We can then use the msgsndsystem call to add the message to the message queue:

msgsnd(id, (struct msgbuf *)&mq_test_buf, sizeof(“test message”) + 1, 0);

When calling msgsnd, the second argument is the address of a struct like msgbuf; thethird argument is the length of the data in the msgbufstruct (in this case the field mtext)

Passing a zero value for the fourth (last) argument to msgsndeffectively turns off errorchecking The data copied into mq_test_buf.mtextis a null terminated string, so Iadded one to the size of the string to allow for the null termination character Whenreceiving messages, I usually use the flag IPC_NOWAITso that the msgrcvsystem callreturns immediately, even if no message is available:

int status = msgrcv(id, (struct msgbuf *)&mq_test_buf,

80, 123, IPC_NOWAIT);

Here, the return value, saved in the variable status, equals -1 if no message is available

The third argument (80 in this example) specifies the size of the mtextfield in the sage buffer The fourth argument (123 in this example) specifies the message type to bereturned Only messages with the field mtypeequaling 123 will be returned in this exam-ple call to msgrcv If the fourth argument is zero, then the next available message will bereturned, regardless of the value of the mtextfield in the message buffer The fifth argu-ment (IPC_NOWAITin this example) specifies whether the call to msgrcvshould wait for amessage if none is currently available Using IPC_NOWAITcauses the call to msgrcvto

immediately return if no messages are available Using IPC_NOWAITeffectively turns offerror checking You can use the value zero instead of IPC_NOWAITif you want the call to

msgrvc to block, waiting for a message

When you are done using a message queue, you can close it using the msgctlsystemcall:

msgctl(id, IPC_RMID, 0);

The second argument to msgctl(IPC_RMIDin this example) specifies a command The

IPC_RMIDcommand indicates that the message queue identified by the value of the able idshould be removed; any current blocking (or waiting) calls to msgrcvfor thisqueue will immediately return without a message; the system errnovariable will be set

vari-to EIDRM.Message queues are a great technique for reliably passing data between two programsrunning on the same computer Another technique, socket programming, will be used inChapter 19 Sockets are useful for solving the general communications problem—IPCbetween programs running on different computers

Running the Sample Message Queue Program

Assuming that you have copied the IPCdirectory contents from the CD-ROM to yourlocal file system, open a new command window, change the directory to IPC/MESSAGEQ,make the test program, and run it by typing the following commands:

make message_q

You should see the following output:

markw@colossus:/home/markw/MyDocs/LinuxBook/src/IPC/MESSAGEQ > make

cc -o message_q message_q.c markw@colossus:/home/markw/MyDocs/LinuxBook/src/IPC/MESSAGEQ > message_q unique key=1627570461

message queue id=129 Sending message

message of type 123 received with data: test message markw@colossus:/home/markw/MyDocs/LinuxBook/src/IPC/MESSAGEQ >

The sample program prints out a unique key that is calculated from the program’s tory pathname The test message is sent and received on message queue with id equal to

direc-129 You will probably get different message queue id numbers when you rerun the ple program

sam-Interprocess Communication and Network Programming

P ART III

278

There are two very useful UNIX utility programs that you should use when you aredeveloping and running programs using shared memory, semaphores, and/or messagequeues:ipcsand ipcrm The ipcsutility prints out information on currently allocatedshared memory, semaphores, and message queues The ipcrmutility is useful for freeing system resources that were not freed because a program crashed or was not written correctly.

280

Shared memory is the fastest method of IPC for processes that are running on the samecomputer One process can create a shared memory area and other processes can access

it You can use shared memory when you need very high performance IPC betweenprocesses running on the same machine There are two ways that shared memory is oftenused: mapping the /dev/memdevice and memory mapping a file It is much safer tomemory map files, rather than /dev/mem(which can crash a system), but memorymapped files have the overhead of using the file system This chapter looks at a singleexample that memory maps /dev/mem

Interprocess Communication and Network Programming

You will see in this chapter that you need to configure your Linux system to dedicatesome real memory for shared memory allocations This dedicated memory cannot beused by Linux or application programs

Configuring Linux to Use Shared Memory

You will have to allocate a small block of memory when you boot up Linux by ing your /etc/lilo.configfile and rebooting your system Here are the changes that Imade to my /etc/lilo.configfile to allow using shared memory:

modify-# Linux bootable (use 1 megabyte for shared memory) image = /vmlinuz

append=”mem=63m”

root = /dev/hda2 label = linux

# Linux bootable partition config ends

I added the appendstatement to my lilo.configfile to indicate that my system onlyhas 63 megabytes of physical memory My system actually has 64 megabytes of physicalmemory, so this effectively reserves 1 megabyte of physical memory as shared memory

Sample Program Using Shared Memory

The source file shared_mem.cshows a very simple example of reading and writingshared memory We will use the library functions mmapand munmapin this example; I rec-ommend that you read the man pages for both mmapand munmap The following C codesets up shared memory to use (assuming that it starts at the 63 megabyte address):

#define ADDRESS (63*0x100000) void main() {

char *mem_pointer;

int f;

if ((f=open(“/dev/mem”, O_RDWR)) < 0) { printf(“Error opening /dev/mem\n”);

exit(1);

} mem_pointer = (char *)mmap(0, 8192,

PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED,

f, ADDRESS);

In this code example, we use the openfunction to open the shared memory device, just

as we would open a disk file for reading and writing The first argument to mmap(zero inthis example) specifies the starting location for a block of shared memory A value ofzero specifies that mmapshould allocate the requested block of shared memory at thebeginning of the space that is available for shared memory The second argument specifies the allocation size block of shared memory In this call to mmap, we are onlydeclaring the use of 8192 bytes of shared memory (we allocated a whole megabyte whenbooting Linux; see the file /etc/lilo.conf) The third argument is used to specify pro-tections for the block of shared memory; in this example we have specified that sharedmemory pages can be read and written to The fourth argument specifies flags for sharedmemory; in this example the shared memory pages are mapped as a file and sharable

between processes The fifth argument to mmapis a file handle; in this case, the value ofthe variable fwas set as the returned value for calling openon the shared memory device(/dev/mem) The sixth argument to mmapis the physical address of the shared memoryblock; in this example, the expression (63*0x100000)evaluates to 63 megabytes, thestart of the shared memory area reserved at Linux boot up time (remember our changes

to /etc/lilo.conf)

Figure 17.1 shows two programs accessing the same block of shared (physical) memory

Interprocess Communication and Network Programming

P ART III

284

F IGURE 17.1

Two programs reading and writ- ing into a shared memory segment.

Block of shared memory (used by programs A and B)

Physical shared memory allocated at boot up time

Program A

Program B

The following code reads and writes the first two bytes of shared memory every two seconds:

for (i=0; i<10; i++) { printf(“Test iteration %d\n”, i);

printf(“first two bytes: %d %d\n”, mem_pointer[0], mem_pointer[1]);

mem_pointer[0] = 2*i; // write into shared memory mem_pointer[1] = 3*i; // write into shared memory printf(“first two bytes: %d %d\n”,

mem_pointer[0], mem_pointer[1]); // read from shared memory sleep(2); // wait 2 seconds

by different executing programs

The best way to insure atomic read/writes to shared memory is to use a semaphore (seeChapter 18, “Semaphores”) to prevent more than one program from accessing sharedmemory for either reading or writing at one time The problem is that a process writing

to shared memory might be interrupted by the kernel, and another program reading theshared memory might be executed and read partially written data For some applications,

it might be safe to simply use a single byte of shared memory for a flag that indicatesthat some process is using shared memory If a process writing to shared memory setsthis flag byte to a non-zero value and then is interrupted before updating (writing) data

in shared memory, any other process that is run can check the single flag byte and seethat shared memory is in use Then, it can sleep for a while before attempting to re-access shared memory

Running the Shared Memory Program Example

You must have followed the shared memory configuration instructions in the first section

of this chapter before running the shared_mem.cexample program Change directories to

IPC/SHAREDand type the following command to build the example program:

make

Then, do a “su root” to get root privileges and run the shared_memexample program;

you should see the following output:

markw@colossus:/home/markw/MyDocs/LinuxBook/src/IPC/SHARED > make

cc -o shared_mem shared_mem.c markw@colossus:/home/markw/MyDocs/LinuxBook/src/IPC/SHARED > su Password:

colossus:/home/markw/MyDocs/LinuxBook/src/IPC/SHARED # /shared_mem Test iteration 0

first two bytes: 0 0 first two bytes: 0 0 Test iteration 1 first two bytes: 0 0 first two bytes: 2 3 Test iteration 2 first two bytes: 2 3 first two bytes: 4 6 Test iteration 3 first two bytes: 4 6 first two bytes: 6 9 Test iteration 4 first two bytes: 6 9 first two bytes: 8 12 Test iteration 5 first two bytes: 8 12

first two bytes: 10 15 Test iteration 6 first two bytes: 10 15 first two bytes: 12 18 Test iteration 7 first two bytes: 12 18 first two bytes: 14 21 Test iteration 8 first two bytes: 14 21 first two bytes: 16 24 Test iteration 9 first two bytes: 16 24 first two bytes: 18 27 colossus:/home/markw/MyDocs/LinuxBook/src/IPC/SHARED #

There are two very useful UNIX utility programs that you should use when you aredeveloping and running programs using shared memory:ipcsand ipcrm The ipcsutili-

ty prints out information on currently allocated shared memory, semaphores, and sage queues Theipcrmutility is useful for freeing system resources that were not freedbecause a program crashed or was not written correctly

mes-Summary

Using shared memory is a great technique when you need the fastest possible mance for sharing large amounts of data The example in this chapter ignored the prob-lems caused by two or more programs writing to shared memory at the same time onmulti-processor computers In the next chapter, you will see how to use semaphores tocoordinate access to shared resources

perfor-Interprocess Communication and Network Programming

P ART III

286

Semaphores are data objects that are used to coordinate actions between separateprocesses Semaphores are frequently used to share resources that can only be used byone process at a time In Chapter 17, “Shared Memory,” you saw a simple example ofusing shared memory between two processes; in this shared memory example, the possi-ble problems of both processes simultaneously accessing the same shared memory areignored You can avoid these potential problems by using semaphores to coordinate writeaccess to shared memory and access to other system resources.

The kernel Linux operating system needs to maintain the state of semaphores, rather thanuser processes If you have the Linux kernel source code installed on your system, youcan examine the include file sem.hto see the definition of the semid_dsdata structurethat is used by the kernel for maintaining semaphore state information A semaphore isreally a set of data; you can individually use each element of the set In this chapter, youwill use the following three system calls to create, use, and release semaphores:

• semget—Returns an integer semaphore index that is assigned by the kernel

• semop—Performs operations on the semaphore set

• semctl—Performs control operations on the semaphore setUsing semaphores is fairly easy, as you will see in the example program in the next sec-tion However, even though the technique of using semaphores is simple, there are a twoproblems to be aware of: deadlock and freeing semaphore resources Deadlock can occur

if there is more than one resource whose access is controlled by semaphores For ple, if two processes require access to two non-sharable resources, one process mayobtain a semaphore lock on one resource and wait forever for the other because the otherprocess has the second resource locked, waiting for the first resource When using sema-phores it is very important to free semaphores before terminating a program

exam-An Example Program Using Semaphores

The example program in the file IPC/SEMAPHORE/semaphore.cshows how to create asemaphore set and how to access the elements of that set When using semaphores andreading through the example program semaphore.c, I encourage you to read the manpages for semget,semop, and semctl We will use a simple example in this section fortwo processes coordinating access to a single resource The resource is identified using

an arbitrary integer value The example program both reads and sets semaphores In anactual application, two or more programs would access the same semaphore set, so theymust all use the same resource value This simple example can be reused for simple semaphore requirements in your applications without your having to dig too deeply intothe full API available for using semaphores

Interprocess Communication and Network Programming

P ART III

288

The example program semaphore.cdoes the following:

• Creates a unique key and creates a semaphore

• Checks to make sure that the semaphore is created OK

• Prints out the value of the semaphore at index 0 (should be 1)

• Sets the semaphore (decrements the value of semaphore at index 0 to 0)

• Prints out the value of the semaphore at index 0 (should be 0)

• Un-sets the semaphore (increments the value of semaphore at index 0 back to 1)

• Prints out the value of the semaphore at index 0 (should be 1)

• Removes the semaphoreSetting the values for semaphores seems counter-intuitive An element of a semaphoreset is considered to be set (that is, indicating that some process is using the associatedresource) if the counter value is zero You free a semaphore (un-set it) by incrementingits value to a value of one

Figure 18.1 shows the example program’s relationship to the Linux kernel and internaldata An application program has no direct access to the data used to maintain semaphoresets; rather, an application program uses the semget,semop, and semctlsystem calls tocreate and use semaphore sets

Data for maintaining semaphores for user processes Linux system kernel System calls to create a semaphore set,

read a value of elements of semaphore set, and increment and decrement the integer values of members of the semaphore set.

The Linux kernel maintains semaphore data All use of semaphores is through system calls, and not by direct access to system data.

Use a Semaphore Set to coordinate access to a system resource

Semaphore.c example

The following code fragment creates a semaphore:

// Start by creating a semaphore:

unique_key = ftok(“.”, ‘s’);

id = semget(unique_key, 1, IPC_CREAT | IPC_EXCL | 0666);

printf(“semaphore id=%d\n”, id);

The function ftokcreates a key value based on both a directory path and a seed ter The first argument to semgetis the unique key returned from ftok The second argu-ment is the number of semaphores in the set to create; just one semaphore is needed forthis example The third argument specifies that the semaphore is created and that the cre-ation should fail if the semaphore already exists The flags (third argument) are analo-gous to the flags used in calls to opento open a file, but substituting “IPC_” for “O_”.The following code shows how to set a specified member of a semaphore set:

charac-union semun options;

options.val = 1; // specify the value semctl(id, 0, SETVAL, options); // operate on semaphore at index 0

// make sure that everything is set up OK:

if (semctl(id, 0, GETVAL, 0) == 0) { printf(“can not lock semaphore.\n”);

struct semid_ds *buf; /* buffer for IPC_STAT & IPC_SET */

ushort *array; /* array for GETALL & SETALL */

struct seminfo * buf; /* buffer for IPC_INFO */

void * pad;

};

In this example, we used a call to semctlto get the current value of the semaphore atindex zero (second argument) The first argument to this call to semctlis the semaphore

ID that was returned from the call to semget The second argument is the integer index

of the member of the semaphore set that we want to access The third argument to

semctlis the constant flag SETVAL(check the man page documentation using

man semctlto see other options) used to specify that we want to increment the value ofthe member of the semaphore set at index zero The constant SETVALis defined in theinclude file sys.sem.h The fourth argument to semctlis used to provide the data for theset operation The following code fragment prints out the value of the semaphore atindex zero:

// Print out the value of the semaphore:

i = semctl(id, 0, GETVAL, 0);

printf(“value of semaphore at index 0 is %d\n”, i);

Interprocess Communication and Network Programming

P ART III

290

Again, we have used a call to semctlto get the value of the semaphore at index zero(second argument to semctl) The third value,GETVAL, specifies that we want to get thevalue of the semaphore set member at index zero The following code sets the semaphore(decrements its value) by calling semop:

// Set the semaphore:

struct sembuf lock_it;

lock_it.sem_num = 0; // semaphore index lock_it.sem_op = -1; // operation lock_it.sem_flg = IPC_NOWAIT; // operation flags

if (semop(id, &lock_it, 1) == -1) { printf(“can not lock semaphore.\n”);

exit(1);

}

The struct sembufis defined in sys/sem.h, and has the following definition:

/* semop system calls takes an array of these */

struct sembuf { ushort sem_num; /* semaphore index in array */

short sem_op; /* semaphore operation */

short sem_flg; /* operation flags */

};

The second argument to semopis an array of sembufstructures; here we only want toperform one operation, so we created a single sembufstructure, and passed a value of 1for the third argument that is used to specify the number of commands to execute Thefollowing code un-sets the semaphore (increments its value):

// Un-set the semaphore:

lock_it.sem_num = 0;

lock_it.sem_op = 1;

lock_it.sem_flg = IPC_NOWAIT;

if (semop(id, &lock_it, 1) == -1) { printf(“could not unlock semaphore.\n”);

exit(1);

}

The following code removes a semaphore set Note that if you do not delete a phore, then you will not be able to rerun the example program without either rebootingyour Linux system, or by running it from a different directory (which creates a differentkey)

sema-// Remove the semaphore:

semctl(id, 0, IPC_RMID, 0);

The constant IPC_RMIDis defined in the include file sys/ipc.h(which includes the file

linux/ipc.h) Listing 18.1 shows the entire example semaphore program

L ISTING 18.1 EXAMPLESEMAPHOREPROGRAM

/* semaphore.c Copyright Mark Watson, 1988 Open Source Software License

int val; /* value for SETVAL */

struct semid_ds *buf; /* buffer for IPC_STAT, IPC_SET */ unsigned short int *array; /* array for GETALL, SETALL */ struct seminfo * buf; /* buffer for IPC_INFO */

};

#endif

void main() { key_t unique_key;

int id;

struct sembuf lock_it;

union semun options;

int i;

// Start by creating a semaphore:

unique_key = ftok(“.”, ‘a’); // ‘a’ can be any character // Create a new semaphore with 1 member of the set; Note that // if you want to use a semaphore created by another program // then use 0 instead of 1 for the second argument:

id = semget(unique_key, 1, IPC_CREAT | IPC_EXCL | 0666); printf(“semaphore id=%d\n”, id);

options.val = 1;

semctl(id, 0, SETVAL, options);

// make sure that everything is set up OK:

if (semctl(id, 0, GETVAL, 0) == 0) { printf(“can not lock semaphore.\n”);

exit(1);

}

// Now print out the value of the semaphore:

i = semctl(id, 0, GETVAL, 0);

printf(“value of semaphore at index 0 is %d\n”, i);

Interprocess Communication and Network Programming

P ART III

292

// Now set the semaphore:

lock_it.sem_num = 0; // semaphore index lock_it.sem_op = -1; // operation lock_it.sem_flg = IPC_NOWAIT; // operation flags

if (semop(id, &lock_it, 1) == -1) { printf(“can not lock semaphore.\n”);

exit(1);

}

// Now print out the value of the semaphore:

i = semctl(id, 0, GETVAL, 0);

printf(“value of semaphore at index 0 is %d\n”, i);

// now un-set the semaphore:

lock_it.sem_num = 0;

lock_it.sem_op = 1;

lock_it.sem_flg = IPC_NOWAIT;

if (semop(id, &lock_it, 1) == -1) { printf(“could not unlock semaphore.\n”);

exit(1);

}

// Now print out the value of the semaphore:

i = semctl(id, 0, GETVAL, 0);

printf(“value of semaphore at index 0 is %d\n”, i);

// Now remove the semaphore:

semctl(id, 0, IPC_RMID, 0);

}

Running the Semaphore Example Program

The example program for this chapter is the file semaphore.cin the directory IPC/

SEMAPHORE Change directory to IPC/SEMAPHOREon your system, use maketo build theexample program, and type semaphoreto run it You should see the following output:

markw@:/home/markw/MyDocs/LinuxBook/src/IPC/SEMAPHORES > make

cc -o semaphore semaphore.c markwcolossus:/home/markw/MyDocs/LinuxBook/src/IPC/SEMAPHORES > semaphore semaphore id=0

value of semaphore at index 0 is 1 value of semaphore at index 0 is 0 value of semaphore at index 0 is 1 markw:/home/markw/MyDocs/LinuxBook/src/IPC/SEMAPHORES >