o'reilly - sendmail - 3rd edition

Sendmail Third Edition Author : Bryan Costales, Eric Allman Publisher : Oreilly Date : December 2002 ISBN : 1-565-92839-3 Table of Contents Preface Changes Since the Previous Editio

Sendmail Third Edition

Author : Bryan Costales, Eric

Allman Publisher : Oreilly Date : December 2002 ISBN : 1-565-92839-3

Table of Contents

Preface

Changes Since the Previous Edition

Why This Book Is Necessary

History

Eric Allman Speaks

Organization

Audience and Assumptions

Unix and sendmail Versions

Conventions Used in This Handbook

Additional Sources of Information

Other Books, Other Problems

How to Contact Us

Acknowledgments

Chapter 1 Some Basics

Section 1.1 Email Basics

Section 1.2 Requests for Comments (RFCs)

Section 1.3 Email and sendmail

Section 1.4 Basic Parts of sendmail

Section 1.5 Basic Parts of a Mail Message

Section 1.6 Basic Roles of sendmail

Section 1.7 Basic Modes of sendmail

Section 1.8 The sendmail.cf File

Part I: Build and Install

Chapter 2 Build and Install sendmail

Section 2.1 Vendor Versus Compiling

Section 2.2 Obtain the Source

Section 2.3 The Build Script

Section 2.4 Building with m4

Section 2.5 Build sendmail

Section 2.6 Install sendmail

Chapter 3 Tune sendmail with Compile-Time Macros

Section 3.1 Before You Begin, a Checklist

Section 3.2 To Port, Tune, or Debug

HESIOD

HES_GETMAILHOST IDENTPROTO

PSBUFSIZ

QUEUE

QUEUESEGSIZE

REQUIRES_DIR_FSYNC SASL

SMTP

SMTPDEBUG

SMTPLINELIM

Chapter 4 Configure sendmail.cf with m4

Section 4.1 The m4 Preprocessor

Section 4.2 Configure with m4

Section 4.3 m4 Macros by Function

Chapter 5 Build and Use Companion Programs

Section 5.1 The Build Script

Part II: Administration

Chapter 6 Tune Performance

Section 6.1 Handle Deep Queues

Section 6.2 Sidestep Slow Hosts

Section 6.3 Deliver to Files

Section 6.4 Buffered File I/O

Section 6.5 Use Multiple Queues

Section 6.6 Condition the Network

Section 6.7 Tune the Kernel

Section 6.8 Pitfalls

Chapter 7 How to Handle spam

Section 7.1 The Local_check_ Rule Sets Section 7.2 How DNSBL Works

Section 7.3 Check Headers with Rule Sets Section 7.4 Relaying

Section 7.5 The access Database

Section 7.6 The Milter Library

Section 7.7 Pitfalls

Chapter 8 Test Rule Sets with -bt

Section 8.1 Overview

Section 8.2 Configuration Lines

Section 8.3 Dump a sendmail Macro or Class Section 8.4 Show an Item

Section 8.5 Complex Actions Made Simple Section 8.6 Process-Specified Addresses Section 8.7 Add Debugging for Detail

Section 8.8 Batch Rule-Set Testing

Section 8.9 Pitfalls

Chapter 9 DNS and sendmail

Section 9.1 Overview

Section 9.2 How sendmail Uses DNS

Section 9.3 Set Up MX Records

Section 9.4 How to Use nslookup

Section 9.5 Prepare for Disaster

Section 9.6 Pitfalls

Chapter 10 Maintain Security with sendmail

Section 10.1 Why root?

Section 10.2 The Environment

Section 10.3 SMTP Probes

Section 10.4 The Configuration File

Section 10.5 Permissions

Section 10.6 The Aliases File

Section 10.7 Forged Mail

Section 10.8 Security Features

Section 10.9 Support SMTP AUTH

Section 10.10 STARTTLS

Section 10.11 Other Security Information

Section 10.12 Pitfalls

Chapter 11 Manage the Queue

Section 11.1 Overview of the Queue

Section 11.2 Parts of a Queued Message Section 11.3 Using Multiple Queue Directories Section 11.4 Queue Groups (V8.12 and Above) Section 11.5 Bogus qf Files

Section 11.6 Printing the Queue

Section 11.7 How the Queue Is Processed Section 11.8 Cause Queues to Be Processed Section 11.9 Process Alternate Queues

Chapter 12 Maintain Aliases

Section 12.1 The aliases(5) File

Section 12.2 Forms of Alias Delivery

Section 12.3 Write a Delivery Agent Script Section 12.4 Special Aliases

Section 12.5 The Aliases Database

Section 12.6 Prevent Aliasing with -n

Section 12.7 Pitfalls

Chapter 13 Mailing Lists and ~/.forward

Section 13.1 Internal Mailing Lists

Section 13.2 :include: Mailing Lists

Section 13.3 Defining a Mailing List Owner Section 13.4 Exploder Mailing Lists

Section 13.5 Problems with Mailing Lists Section 13.6 Packages That Help

Section 13.7 The User's ~/.forward File

Section 13.8 Pitfalls

Chapter 14 Signals, Transactions, and Syslog

Section 14.1 Signal the Daemon

Section 14.2 Log Transactions with -X

Section 14.3 Log with syslog

Chapter 15 The sendmail Command Line

Section 15.1 Alternative argv[0] Names

Section 15.2 Command-Line Switches

Section 15.3 List of Recipient Addresses

Section 15.4 Processing the Command Line

Section 15.5 sendmail's exit( ) Status

Chapter 16 Debug sendmail with -d

Section 16.1 The Syntax of -d

Section 16.2 The Behavior of -d

Section 16.3 Interpret the Output Section 16.4 Table of All -d Categories Section 16.5 Pitfalls

Section 16.6 Reference for -d in Numerical Order -d0.1

Part III: The Configuration File

Chapter 17 Configuration File Overview

Section 17.1 Overall Syntax

Section 17.2 Comments

Section 17.3 V8 Comments

Section 17.4 Continuation Lines

Section 17.5 The V Configuration Command

Section 17.6 Pitfalls

Chapter 18 The R (Rules) Configuration Command

Section 18.1 Why Rules?

Section 18.2 The R Configuration Command

Section 18.3 Tokenizing Rules

Section 18.4 The Workspace

Section 18.5 The Behavior of a Rule

Chapter 19 The S (Rule Sets) Configuration Command

Section 19.1 The S Configuration Command

Section 19.2 The Sequence of Rule Sets

Section 19.3 The canonify Rule Set 3

Section 19.4 The final Rule Set 4

Section 19.5 The parse Rule Set 0

Section 19.6 The localaddr Rule Set 5

Section 19.7 Rule Sets 1 and 2

Chapter 20 The M (Mail Delivery Agent) Configuration Command

Section 20.1 The M Configuration Command

Section 20.2 The Symbolic Delivery Agent Name

Section 20.3 The mc Configuration Syntax

Section 20.4 Delivery Agents by Name

*file* and *include*

local and prog

mail11

ph

pop

Chapter 21 The D (Define a Macro) Configuration Command

Section 21.1 Preassigned sendmail Macros

Section 21.2 Command-Line Definitions

Section 21.3 Configuration-File Definitions

Section 21.4 Macro Names

Section 21.5 Macro Expansion: $ and $&

Section 21.6 Macro Conditionals: $?, $|, and $

Section 21.7 Macros with mc Configuration

${alg_bits}

${auth_authen} ${auth_author} ${auth_ssf}

${auth_type} $b

${bodytype} $B

$c

${cert_issuer} ${cert_md5} ${cert_subject} ${cipher}

${cipher_bits} ${client_addr} ${client_flags} ${client_name} ${client_port} ${client_resolve} ${cn_issuer} ${cn_subject} ${currHeader} $C

$d

${daemon_addr} ${daemon_family} ${daemon_flags} ${daemon_info} ${daemon_name} ${daemon_port} ${deliveryMode} ${dsn_envid} ${dsn_notify} ${dsn_ret}

$H

$i

Chapter 22 The C and F (Class Macro) Configuration Commands

Section 22.1 Class Configuration Commands

Section 22.2 Access Classes in Rules

Section 22.3 Classes with mc Configuration

Section 22.4 Internal Class Macros

Chapter 23 The K (Database-Map) Configuration Command

Section 23.1 Enable at Compile Time

Section 23.2 The K Configuration Command

Section 23.3 The K Command switches

-T

-t

-v

-z

Section 23.4 Use $( and $) in Rules

Section 23.5 Database Maps with mc Configuration Section 23.6 Pitfalls

Section 23.7 Alphabetized Database-Map Types arith

Section 24.2 Command-Line Options

Section 24.3 Configuration File Options

Section 24.4 Options in the mc File

Section 24.5 Alphabetical Table of All Options

Section 24.6 Option Argument Types

Section 24.7 Interrelating Options

Section 24.8 Pitfalls

Section 24.9 Alphabetized Options

ControlSocketName

DaemonPortOptions DataFileBufferSize

DontProbeInterfaces DontPruneRoutes

DoubleBounceAddress EightBitMode

ErrorHeader

ErrorMode

FallbackMXhost

FastSplit

MaxHopCount

MaxMessageSize

MaxMimeHeaderLength MaxQueueChildren

MaxQueueRunSize

MaxRecipientsPerMessage MaxRunnersPerQueue MeToo

Section 25.2 Header Names

Section 25.3 Header Field Contents

Section 25.4 ?flags? in Header Definitions

Section 25.5 Rules Check Header Contents

Section 25.6 Header Behavior in conf.c

Section 25.7 Headers and mc Configuration

Section 25.8 Headers by Category

Section 25.9 Forwarding with Resent Headers

Part IV: Appendixes

Appendix A The mc Configuration Macros and Directives Appendix B What's New Since V8.8

Appendix C Error Message Reference

Section C.1 Where Errors Are Printed

Section C.2 Alphabetized Error Reference

Appendix D The checkcompat( ) Cookbook

Section D.1 How checkcompat( ) Works Section D.2 Reject Bad Body Lines

Section D.3 Reject Excess 8-Bit Characters

Appendix E A Map to Tutorial Information

Section E.1 How to Use This Guide

Section E.2 The Guide

Bibliography

Requests for Comments

Publications and Postings

Preface

Changes Since the Previous Edition

Why This Book Is Necessary

History

Eric Allman Speaks

Organization

Audience and Assumptions

Unix and sendmail Versions

Conventions Used in This Handbook

Additional Sources of Information

Other Books, Other Problems

How to Contact Us

Acknowledgments

Changes Since the Previous Edition

The primary reason for this book, the third edition of sendmail, is the release of versions 8.9 through 8.12 of the sendmail program V8.12 of sendmail, in particular, differs so significantly from earlier versions that a

massive rewrite was called for

Many of you will notice that the tutorials of earlier editions have been replaced with a single introductory

chapter in this edition The tutorials were eliminated because V8.10 sendmail changed to an all m4-based compilation and configuration, and because newer versions of sendmail tend to insulate the administrator from sendmail's internals Because sendmail now places much more emphasis on the mc configuration file, it

automatically produces 99% of the rules and rule sets you will ever need Although you will need to write your own rule sets from time to time, the chapters about rule sets should prove sufficient.[1]

[1]

The Things to Try tutorial sections are also gone because they were too version-specific and impossible to

keep current Instead, configuration solutions and examples are now spread throughout this book

You might also notice some new chapters A chapter on performance tuning has been added because so many sites now seek to make mail delivery efficient A chapter on handling spam has been added because

sendmail V8.10 and above is now rich in antispam features A chapter describing the other programs

supplied with sendmail, such as vacation and makemap, has been added because those additional

programs are pivotal to the daily operation of sendmail Other chapters have been massively expanded

because sendmail has added dozens of new features, options, and macros

This edition of the sendmail book assumes you are using V8.12, the current version of the sendmail

program It follows the same general format as earlier editions of the book, but we realize this might not be the most convenient arrangement for readers who are primarily interested in what has changed since the last edition To help minimize this problem, we have added Appendix B, in which the many improvements of the intervening versions are categorized by chapter, complete with references to the appropriate sections within this book

Why This Book Is Necessary

King Gordius of Phrygia once created a knot so tangled that no one could undo it The Gordian knot stayed tangled, or so the story goes, until Alexander the Great came along and took a different approach to untying the knot With a sweep of his sword, he parted the great knot once and for all

It would be nice if the knot that is sendmail could be undone with one quick stroke of fresh insight, but, alas,

it cannot Instead, a more mundane approach must be taken, so in this book we untie the hard way, one strand at a time

But, you might ask, "Why the effort? Doesn't sendmail predate the dawn of computing time? Hasn't the time come to replace sendmail with something new, something better, something modern?" Not so Age has

brought sendmail maturity and reliability The sendmail program has withstood the test of time because it is

more than just a program, it is a philosophy: a general-purpose, internetwork mail-routing facility with the flexibility and configurability to solve the mail-routing needs of all sites large or small, complex or simple

These strengths of sendmail are also its weaknesses Configurability has bred complexity The sendmail

program is difficult to configure and even more difficult to understand Its configuration file, for example, can

be positively frightening But don't despair With this book in hand, you should be able to configure sendmail

to meet any need and bring the days of the sendmail guru to an end

History

The sendmail program was originally written by Eric Allman while he was a student and staff member at the University of California at Berkeley At the time, one campus machine (Ingres) was connected to the

ARPAnet and was home to the INGRES project where Eric was working Another machine (Ernie CoVax)

was home to the Berkeley Unix project and had recently started using the Unix to Unix Communication

Protocol (UUCP) These machines (as well as several others on campus) were connected via a low-cost network built by Eric Schmidt, called BerkNet Software existed to move mail within ARPAnet, within UUCP, and within BerkNet, but none yet existed to move mail between these three networks

A sudden increase in protocol types, coupled with the anticipation of an explosion in the number of networks,

motivated Eric Allman to write delivermail-the precursor to sendmail The delivermail program was shipped in

1979 with 4.0 and 4.1 BSD Unix Unfortunately, delivermail was not flexible enough to handle the changes in

mail-routing requirements that actually occurred Perhaps its greatest weakness was that its configuration was compiled in

In 1980, ARPAnet began converting from Network Control Protocol (NCP) to Transmission Control Protocol (TCP) This change increased the number of possible hosts from 256 to more than 1 billion Another change converted from a "flat" host-name space (such as MIT-XX) into a hierarchical namespace (such as XX.MIT.EDU) Prior to these changes, mail was transported using the File Transfer Protocol (FTP) Afterward, a new protocol was developed for transporting mail called Simple Mail Transfer Protocol (SMTP) These

developments were not instantaneous Some networks continued to run NCP years after most others

switched to TCP And SMTP underwent many revisions before finally settling into its present form

Responding to these and other changes, Eric evolved delivermail into sendmail To ensure that messages

transferred between networks would obey the conventions required by those networks, Eric took a "liberal" approach-modifying address information to conform rather than rejecting it At the time, for example, UUCP

mail often had no headers, so sendmail had to create them from scratch

The first sendmail program was shipped with 4.1c BSD (the first version of Berkeley Unix to include TCP/IP)

From that first release to the present,[2] Eric has continued to enhance sendmail, first at UC Berkeley, then at

Britton Lee, then back at UC Berkeley, then with InReference Inc., and now with Sendmail, Inc The current

major version of sendmail is V8, a major rewrite that includes many bug fixes and significant enhancements

[2]

With one long gap between 1982 and 1990

But Eric wasn't the only one working on sendmail In 1987, Lennart Lovstrand of the University of Linköping, Sweden, developed the IDA enhancements to BSD sendmail Version 5 IDA (which stands for Institutionen för Datavetenskap) injected a number of improvements into sendmail (such as support for dbm files and

separate rewriting of headers and envelopes) and fixed a number of bugs As the '90s approached, two

offspring of IDA appeared

Neil Rickert (Northern Illinois University) and Paul Pomes (The University of Illinois) took over maintenance

of IDA sendmail With contributions from around the world, their version (UIUC IDA) represents a

continuation of the work begun by Lennart Lovstrand Neil focused on fixing and enhancing the configuration

files into their current m4-based form Paul maintained the code, continually adding enhancements and fixing

bugs In general, their version was large, ambitious, and highly portable It succeeded in solving many

complex mail-routing problems

A variation on IDA sendmail was also developed by Paul Vixie (while at Digital Equipment Corporation) Called KJS (for King James sendmail), it was a more conservative outgrowth of Lennart Lovstrand's last IDA

release The focus of KJS was on code improvement rather than changes to configuration files

In addition to these major offshoots, many vendors modified sendmail to suit their needs Sun Microsystems made many modifications and enhancements to sendmail, including support for nis and nisplus maps

Hewlett-Packard also contributed many fine enhancements, including 8BITMIME support

This explosion of sendmail versions led to a great deal of confusion Solutions to problems that work for one version of sendmail failed miserably for another Even worse, configuration files were not portable, and some

features could not be shared

In 1992, Eric started creating a new version of sendmail to merge all the earlier versions V8 officially

adopted most of the good features from IDA, KJS, Sun, and HP's sendmail, and kept abreast of the latest standards from the Internet Engineering Task Force (IETF) In 1996, Eric began work on V8.8 sendmail This

release continued the trend begun with V8.7, adding many requested new features and options, and

tightening security In 1998, V8.9 was released, continuing the direction started by V8.8

In 1999, Sendmail, Inc was founded in Emeryville, California Sendmail, Inc took over maintenance and

development of the open source version of sendmail, and began work on a commercial version Sendmail,

Inc has the web site:

The first major offering from Sendmail, Inc was V8.10 sendmail, released in 2000 It was mentored by Eric

Allman, but largely written by Greg Shapiro

V8.10 and V8.11 were developed in parallel Claus Assmann added to V8.10 SMTP AUTH and STARTTLS,

as well as a number of security changes, bringing that version up to V8.11 V8.11 was released as a

commercial version because of export restrictions Shortly afterward, export restrictions were relaxed and V8.11 was released in open source form

Claus Assmann took sendmail in a somewhat new direction with V8.12, in which he added a suite of new features V8.12 was supposed to be the last of the V8 series of sendmail, but a V8.13 and perhaps a V8.14

are possible

Plans are afoot to develop a multithreaded version of sendmail that will be called V9 sendmail, but, as of this

writing, it is still in the early planning stages, and probably won't appear as an alpha release for a few years

Eric Allman Speaks

I have to admit that I'm surprised by how well sendmail has succeeded It's not because of a large marketing

organization or a deep-pockets budget I think there are three reasons

First, sendmail took the approach that it should try to accept, clean up, and deliver even very "crufty"

messages instead of rejecting them because they didn't meet some protocol I felt this was important

because I was trying to gateway UUCP to the ARPAnet At the time, the ARPAnet was small, UUCP was anarchy, and Unix mail programs generally didn't even understand headers It was harder to do, but after all, the goal was to communicate, not to be pedantic

Second, I limited myself to the routing function-I wouldn't write user agents or delivery back-ends This was a departure from the dominant thought of the time, in which routing logic, local delivery, and often the network code were incorporated directly into the user agents But it did let people incorporate their new networks quickly

Third, the sendmail configuration file was flexible enough to adapt to a rapidly changing world: the 1980s saw

the proliferation of new protocols, networks, and user agents

And, of course, it didn't hurt that it was free, available at the right time, and did what needed to be done

Configuring sendmail is complex because the world is complex It is dynamic because the world is dynamic Someday sendmail, like X11, will die-but I'm not holding my breath In the meantime, perhaps this book will

help

When I started reviewing Bryan's first-edition manuscript, I had been avoiding any major work on sendmail

But then I started reading about various petty bugs and annoyances that all seemed easy to fix So I started making small fixes, then larger ones; then I went through RFC1123 to bring the specs up-to-date, cleaned up

a bunch of 8-bit problems, and added ESMTP It would be fair to say that the first book and sendmail Version

8 fed on each other-each improving the other

Organization

We've divided this book into one introduction and four parts, each addressing a particular aspect of sendmail

Chapter 1 will be of special help to the new user It covers the basic concepts underlying mail delivery and

the roles sendmail plays in that delivery

Part I covers compilation, installation, and configuration of sendmail, and the other programs supplied with the sendmail source

Part II for more experienced users, covers general administration, including performance tuning, handling spam, rule testing, and more

Part III covers all aspects of the configuration file in detail, and includes complete reference sections

Part IV contains the appendices

Audience and Assumptions

This book is primarily intended for system administrators who also administer email But not all Unix systems are managed by administrators Many are managed by programmers, network engineers, and even

inexperienced users It is our hope that this book satisfies all of you, no matter what your level of experience The true beginner should begin with Part I, skipping ahead as needed

The beginning system administrator should probably start with Part I to learn how to build and install

sendmail, then read Part II for help in understanding how to administer sendmail Note that Part II and Part III

will reveal answers to many nagging questions that seem to be otherwise unanswered

The experienced system administrator who wants to install and manage V8 sendmail should read Part I and

Part II first to gain the needed background Then read Part III

Unix gurus and sendmail specialists should find Part III to be of value (even Eric keeps a copy on his desk)

In it, every arcane detail of sendmail is listed alphabetically For example, in Part III you'll find a single

chapter dedicated to options, with every option listed and explained

No matter what your level of expertise, the sheer size of this book forces us to assume that you are familiar with the day-to-day system workings of Unix If you aren't, you must learn Unix elsewhere

Unix and sendmail Versions

For the most part, we illustrate sendmail under BSD Unix and its variants (such as FreeBSD) Where AT&T

System V (SysV) differs (such as Sun's Solaris 2.x), we illustrate those differences

Our primary focus throughout this book is on V8.12 sendmail For completeness, and where necessary, we

also discuss V8.11 and earlier (such as BSD's version 5,[3] IDA, early Sun, Ultrix, and NeXT)

[3]

The versions jump from 5 to 8 because the managers of the BSD 4.4 Unix distribution wanted all software to

be released as version 8 Prior to that decision, the new BSD sendmail was designated Version 6 V6 survived

only the alpha and beta releases before being bumped to V8

Conventions Used in This Handbook

The following typographic conventions are used in this book:

Italic

Used for names, including pathnames, filenames, program and command names, usernames,

hostnames, machine names, and mailing-list names, as well as for mail addresses It also is used to indicate that part of a program's output is not specific For example, "error:number or file" indicates

that the error will be shown either as a number or as a filename Italic is also used to emphasize new terms and concepts when they are introduced

Constant Width

Used in examples to show the contents of files or the output from commands This includes examples from the configuration file or other files such as message files, shell scripts, or C-language program source Constant-width text is quoted only when necessary to show enclosed space; for example, the five-character "From " header

Single characters, symbolic expressions, and command-line switches are always shown in width font For instance, the o option illustrates a single character, the rule $- illustrates a symbolic expression, and -d illustrates a command-line switch

constant-Constant Bold

Used in examples to show commands or some other text that is to be typed literally by the user For example, the phrase cat /var/run/sendmail.pid means the user should type "cat /var/run/sendmail.pid" exactly as it appears in the text or example

Constant Italic

Used in examples to show variables for which a context-specific substitution should be or will be

made In the string Snum, for example, num will be a user-assigned integer

Additional Sources of Information

The source for the sendmail program comes with a document by the sendmail program's authors that is required reading Sendmail Installation and Operations Guide (located in doc/op in the source distribution)

provides installation instructions and a succinct description of the configuration file Many vendors also

provide online manuals which might reveal vendor-specific customizations not documented in this book

Also, if you have the source, see the RELEASE_NOTES file and all the */README files

Other Books, Other Problems

Two topics that are only touched upon in this book are the Domain Name System (DNS) and TCP/IP network communications At a typical site, a significant number of mail-related problems turn out to be problems with

one of these other areas rather than with sendmail

The DNS is well documented in the book DNS and BIND, 4th Edition by Paul Albitz and Cricket Liu (O'Reilly

& Associates, Inc., 2001)

The protocols used to communicate over the Internet are well documented in the book TCP/IP Network

Administration by Craig Hunt (O'Reilly & Associates, Inc., 1992)

Finally, many mail problems can be solved only by the system administrator The sendmail program runs as

root and can be installed and managed only by root The art of functioning effectively as root is superbly

covered in the UNIX System Administration Handbook by Evi Nemeth, Garth Snyder, Scott Seebass, and

Trent R Hein (Prentice Hall, 3rd Edition, 2000)