OReilly managing security with snort and IDS tools aug 2004 ISBN 0596006616

This book explains how to manage your network's security using the open source tool Snort.. This book is designed for network, system, and security administrators of large-scale enterpri

entry attempts, Managing Security with Snort

instructions on getting up and running with Snort 2.1, and how to shut down and secure workstations, servers, firewalls, routers,

sensors and other network devices.

Printed in the United States of America

Published by O'Reilly Media, Inc., 1005 Gravenstein HighwayNorth, Sebastopol, CA 95472

O'Reilly books may be purchased for educational, business, orsales promotional use Online editions are also available for

most titles (http://safari.oreilly.com) For more information,contact our corporate/institutional sales department: (800)

While every precaution has been taken in the preparation of thisbook, the publisher and authors assume no responsibility forerrors or omissions, or for damages resulting from the use ofthe information contained herein

This book explains how to manage your network's security

using the open source tool Snort The examples in this book aredesigned for use primarily on a Red Hat Linux machine Theyshould be fully functional on the latest Red Hat Enterprise Linuxversion as well as the latest Fedora release by Red Hat All

instructions were documented using the most recent Red Hatreleases, patches, and software The applications were

configured using default packages needed for a standard

installation, and each machine was secured according to thelatest errata

The instructions in this book apply to other Linux flavors, such

as SuSE, Gentoo, Debian, and most Unix variants, includingFreeBSD, OpenBSD, and Solaris Many of the applications areavailable for download as source or as precompiled binaries.Since performance is often a consideration when deploying anIDS solution, you will probably find that building the

applications from source yields the best results If you do nothave the time, desire, or need to build from source, the prebuiltpackages should work just fine and install without trouble onmost systems Consult your Linux distribution or Unix-basedoperating system for further information regarding source

compilation and installation Snort binaries are also available forthe Microsoft Windows platform, and instructions for runningSnort on a Windows platform are included

Links to the applications and their respective web sites are

provided throughout and at the end of the chapters Appendix C

also contains a compendium of all software programs and

applications referenced Check all software sites regularly forthe latest updates and information regarding their use Many ofthe programs are under active development and new versionsare posted frequently Some applications require an update with

Topics covered include:

line and GUI utilities

Packet capture and analysis using a variety of command-An introduction to the interpretation of packet headers andcontent within an IDS environment

Details on how to configure and tune your Snort IDS

installation to maximize the effectiveness and minimize thelabor involved in detecting and tracking down attacks

An in-depth look at a variety of administration tools thatassist in the management of the Snort IDS environment.Strategies for deploying an IDS in switched, high-security,and high-bandwidth environments

This book is designed for network, system, and security

administrators of large-scale enterprises as well as managers ofsmall businesses or home offices The instructions should bereadable for those with only a small amount of network andUnix experience, but also useful for experienced administratorswith a varied background in networking and system

administration To be sure, the more experienced you are, theeasier it will be to interpret the results generated by the SnortIDS

Snort can be used for a variety of applications, from acting as asimple network sniffer to an enterprise-class gateway intrusiondetection system (IDS) This book discusses the various ways touse Snort, and methods of configuring, tuning, and customizingthe application to best suit your environment Implementing anIDS solution can be a labor-intensive and sometimes

overwhelming project This book helps streamline the processes

of the initial setup and ongoing care and feeding of Snort

All the source code discussed here is freely available for

download off the Internet I have avoided any software that isclosed source, requires a license, or costs money Though linksand source code versions do change over time, every effort hasbeen made to keep listings and release numbers for each

application as up-to-date as possible If you find the URL doesnot work as listed, please check with some of the major opensource repositories: http://freshmeat.net and

to SnortCenter and Barnyard are found on the main Snort page

at http://www.snort.org

Now that you know what this book is about, here is what it'snot about This book is not a beginner's guide to packet

analysis It is intended to help you implement viable solutions

to everyday intrusion detection problems This book does notspend countless pages examining the nuances and vagaries ofevery type of fragmented packet or possible buffer overflow.Instead, it explains how to quickly capture a sampling of

hostile activity

If you are searching for a theoretical manual that provides

detailed insight into every possible security application or thatexplains how to dissect new intrusive packets, you won't find ithere This book deals with strategies and speedy

implementations using a reasonable, common-sense approach

By the end of this book, the reader will understand that a

network-based intrusion detection system is one part of a largerstrategy of defense-in-depth The book is based on the

experience of a Network Security Engineer who has both

attacked and defended very large corporate networks and

systems Whether you are looking for something to help secureyour home network, or looking for an Enterprise-class solutionthat can watch 2 Gbps of bandwidth in near-real-time, this bookwill help

This book does not make too many demands on the averagereader It is written in an informal manner and is intended formost security administrators, whether they are using Linux (oranother Unix offshoot like BSD) or Windows The main focus ofthe book will be running Snort on a Linux platform Even

beginning Linux users should have no trouble grasping the

concepts Most applicationsalong with their installation and

configurationare clearly spelled out While this book will providethe average user with the ability to get a Snort sensor up andrunning, professional deployments of any IDS solution benefitfrom a good knowledge of networking and system

administration Without this background, discrimination of what

is naughty and what is nice will be more difficult

If any of the steps explained in later chapters do not answer allyour questions, please consult the application's home page orsubscribe to its mailing list, if one is available It will be helpful

if you are familiar with Usenet newsgroups and can post

detailed questions regarding any additional use of the

applications presented here You will find that the open sourcecommunity surrounding Snort and the related applications isactive and incredibly helpful

packets flowing between machines and in and out of networks.This book also presupposes that a secure firewall is in place It

is your responsibility to ensure that your network remains safeduring the IDS installation and implementation phase Newlyinstalled systems do not survive long when exposed to the

Internet without protection.

Chapter 3

Introduces you to getting Snort up and running quicklyusing the various command-line options It discusses thevarious modes in which Snort can be used, including as asniffer and packet logger

Chapter 4

We examine how the "bad guys" attempt to probe,

penetrate, persist, propagate, and paralyze your networkand systems Methods of detecting these methods are

examined

The core of a signature-based intrusion detection systemare the rules that recognize attacks in progress One of thereal strengths of Snort is the flexibility and discrimination ofits rule sets

Chapter 8

Several mechanisms and strategies can be employed thatturn Snort from an intrusion detection system into an

intrusion prevention system These strategies are not

without their own risks, however

Chapter 9

This is perhaps the most important chapter Proper tuningand thresholding allows security administrators to minimizethe number of false positives generated by an IDS sensor,making their time spent working with Snort more efficient

Chapter 10

ACID is a popular, powerful, web-based IDS managementsystem for managing alerts generated by Snort

Chapter 11

SnortCenter makes administering multiple IDS sensorsmuch easier

Chapter 12

A wide variety of tools can help manage a Snort-based IDSdeployment Some of these solutions are more effectivethan others

Chapter 13

demand environment, this chapter will help by discussingstrategies that ensure nothing is missed by overburdenedsensors

If your intention is to deploy Snort as an IDS in a high-Appendix A

Provides the schemas for the Snort and ACID databasetables in order to aid developers in creating new tools ormodifying existing tools

Presents the default snort.conf file for reference when

reading the book and configuring sensors The commentsare actually quite good, too

Appendix C

Provides a compilation of web resources and downloadsources from throughout the book

Shows commands or other text that should be typed

literally by the user

Shows text that should be replaced with user-suppliedvalues

This icon signifies a tip, suggestion, or general note.

This icon indicates a warning or caution.

Please address comments and questions concerning this book tothe publisher:

http://www.oreilly.com/catalog/snortids

To comment or ask technical questions about this book, sendemail to:

bookquestions@oreilly.com

For more information about our books, conferences, ResourceCenters, and the O'Reilly Network, see our web site at:

http://www.oreilly.com

The authors wish to thank the people who contributed to thisproject

Kerry Cox

I owe many thanks to all the people who shared with me theirtime, talents, and experiences while patiently answering myquestions Thanks especially to all the employees at KSL,

Bonneville International, Bonneville Communications, LDS

Business College, and Deseret Management Corporation whoallowed me to install intrusion detection systems on their

servers and then critiqued the systems' performance, providing

me with feedback that assisted in many ways to make this abetter book

I would especially like to thank all the technical and

nontechnical staff with whom I work at Bonneville International,KSL, and the Deseret Management Corporation: Greg James,Roger Graves, Owen Smoot, Don Huntsman, Steve Tolman,

Edward Cheadle, Brent Cherrington, Mark Fenton, Jason

Williams, Hal Whitlock, Steve Wise, Bryan Carter, Brent Cole,Karl Hancock, Trevor Gunnell, Jamie Hall, Kevin McReynolds,Julie Hill, Jason Jones, Amy Kimball, Pat Neilson, and the manyothers whom I may have forgotten

According to Eric S Raymond, "Given enough eyeballs, all bugsare shallow." This was especially true of the assistance I

received from many friends and co-workers There are fewererrors here than there might otherwise have been thanks totheir diligence in proofing this material I am deeply indebted tothese people for the time and effort they took to verify the

accuracy of what I wrote I consider each and every one

is for the readers

I wish especially to thank the following people, who spent manyhours reviewing and critiquing the text and code of this bookbefore submissions were sent to O'Reilly I am extremely

grateful to Jason Jones for checking each chapter's syntax andtightening up the content He pointed out some crucial itemsthat made the reading flow better Our conversations to andfrom work every day helped to improve the quality of this

material I am deeply indebted to him for all his work

I wish to thank Brad Hokanson for testing the source code andinstalling numerous programs on his machines He proved thateverything shown here actually works on various operating

systems His work with encryption and wireless security wasmost valuable I want to thank Jason Williams for his help inproofing the layout and looking over the subject matter for

viability Edward Cheadle was very helpful in implementing

many of these applications in real-world scenarios His feedbackimproved much of the content

Thanks to Steve Scott for his assistance in providing detailedIDS documentation Also, I owe many thanks to Patrick S

Harper for his useful notes and explanations for performing afull source-code install His excellent paper has helped many abeginner on the road to configuring a working IDS box Thanksalso to Jamie Hall and Karl Hancock for continued feedback

from their own experiences with open source intrusion detectionsystems

I also need to thank Jason Williams again, for providing me withthe laptop on which I ran Linux Many are the nights and days

on the train I was able to write this book thanks to his donation

It proved very useful for testing Kismet and AirSnort and

setting up wireless security applications

My hat is also off to Mike Loukides for his assistance in bringing

improving the layout, content, and syntax of each chapter Ivalue his input and appreciate the trust he has placed in me Iwant to also thank the several technical reviewers who proofedthis document for potential flaws or errors I want to personallythank Edin Dizdarevic for his close scrutiny, analysis, and

commentary I very much enjoyed his German commentary andnotes on each section Thanks also to the other editors whocontributed their time and talents to making this a better book:Kevin Binsfield, Andrea Barisani, Daniel Harrison, and AdamHogan

I would especially like to thank my wife, Karen, for her

encouragement It was she who suggested I write this book andstood by me these past few months Her unwavering supporthas not gone unnoticed I have also my boys to thank for theirencouragement Kids, I'm finally done Let's play

Christopher Gerg

This book would not have been possible without the support of

my peers, friends, and family The Security Services team that Iwork with at Berbee Information Networks is the most talentedand diverse group of people I've had the privilege to work andlearn with I've learned more in the last five years than I have

up to that point in my life Paul Tatarsky, Matt Jach, Peyton

Engel, David Klann, and Joe Mondloch have shared their wit andlarge brains with me most generously I hope I'm able to repay

a fraction of the debt I owe (Assume the horse stance )

Thanks to Eric Patterson for everything

Of course, I wouldn't be able to accomplish much of anythingwithout the support of my wife, Becky, and our two crumb-

crunchers, Matthew (shorty) and Sarah (the Bunner) They

keep me sane and centered Well, centered, anyway

A special thanks to Jim Elliot for introducing me to my editor,Mike Loukides Thanks, Mike, for giving me the opportunity tostep into this project The work of John Ives, the technicalreviewer, was excellentthank you very much

This book is about building a network-based intrusion detectionsystem (NIDS) based on the open source application called

Snort Snort got a modest start as the open source project of a

software engineer names Martin Roesch (who incidentally wasthe lead engineer in the development of an IDS solution for

GTE) Snort is now a high-performance, full-featured solutionthat provides competition for some very expensive commercialsolutions (and surpasses many)

A context for the use of an NIDS solution is established by

examining the challenges confronting a network administratorwith regards to security New technologies are making it easierfor remote users and partners to access the insides of the

network, bypassing perimeter security entirely A new breed ofInternet worm is attacking from a variety of directionsthroughemail, across the network, and even across virtual private

network (VPN) connections Hacker communities are creatingtools that make attacking a network much easier This gives rise

to "script kiddies," who download an attack tool and penetrate

an organization's networkall without knowing how the tool theyare using works or the effect it will have on the target system

In the old days (two years ago or so), a firewall was most ofwhat an administrator needed to protect a network from attack

It was easy to establish where your network ended and the

Internet began Technological advances and decreasing costsfor wide area network technologies have eroded this concept of

a perimeter VPNs have all but replaced conventional dial-upmodem pools Most users have high-speed DSL or Cable Modemservice, and the VPN makes the user feel like he's sitting at hisdesk Some VPNs use an appliance that sits on the perimeter ofthe network and has the capability of controlling how the

network is used remotely While this is a boon for

telecommuters, it is a real risk for most networks A virus orworm-infected system on the user's home network suddenlyhas unfettered access to the inside of your network That high-speed highway into your network can allow rapid propagation of

an aggressive worm

Connections to business partners used to be an expensive

proposition and were only for the most well-to-do organizations.Dedicated T1 links are expensive With less expensive networkoptions (not to mention network-to-network VPN connections),this cost has decreased significantly This allows many

organizations to connect their network to yourssometimes

directly into the internal network Without real precautions inplace, security problems on the partner networks quickly

become security problems on your networkvery often

undetected until much damage is done Whether you trust yourpartner to that extent is another matter

When deploying troops in a theater of war, a general has to

consider all the ways an enemy may attack: by land (either atthe front line, or a commando raid behind the lines), by sea(surface ships or submarines), or by air (helicopters, fighters,bombers, missiles, or artillery) The general has to deploy

defenses against all potential vectors of attack He doesn't justtrust the trenches at the front line for all his security He willdeploy troops to the front line, as well as at high-value assetsbehind the lines He will deploy a variety of anti-submarine andanti-surface ship defenses He will deploy a variety of anti-airassets to protect against the various air threats This concept of

multiple overlapping defensive measures is known as defense-in-depth.

A similar system can be applied to network security Instead oftrusting the eroding value of perimeter defenses (firewalls) forall of our security, we turn to other mechanisms We configuresystems according to industry-accepted best practices (disableunnecessary services, keep software updated, run antivirus

software) We establish a system to securely aggregate oursystem logs in one place (and we monitor those logs for

anomalies) We segregate our network to control access to

important machines and to "wall-off" partner and remote

connections We utilize strong authentication and authorizationpractices And finally, we take steps to detect and prevent

intrusions (preferably attempted intrusions) on our network and

on our systems We also try to do this with limited budgets andlimited time In the real world, the general is trying to protectagainst lost real estate In the network world, the administrator

is protecting against downtime and data loss I won't beat theanalogy to death The main thing to remember is not to trust asingle component of your security framework for all your

security If you are able to, apply security as close to the thing

you are trying to secure as possible These steps will help youstop at least 80% of attacks Intrusion detection should catchthe remaining 20%.

Approaches)

Intrusion detection is simply trying to detect the signs of a

network intruder before damage is done, a service denied, ordata lost This can be done through the use of a variety of

mechanisms Properly configured systems generate system logsthat keep track of services, users, and data These logs veryoften show traces of suspicious (or downright nefarious)

activity The problem is that these logs often have a lot moreinformation in them than a security administrator is interested

in It is important to consider system log review as a basic

intrusion detection mechanism, though Many times the systemlogs show their value in a forensic analysis after the fact

The next layer of intrusion detection (and prevention) is

automated tools, commonly referred to as host-based intrusiondetection (HIDS) HIDS tools include antivirus software,

personal firewalls, NIDS installed on the individual hosts, and anew breed of software (intrusion prevention systems) that

protects system memory against buffer overflow attacks or

enforces security policies Many products are a hybrid mix ofthese solutions (a personal firewall/antivirus product, for

example)

The final layer of intrusion detection is NIDS

On a basic level, network intrusion detection is exactly what itsounds like: the process of determining when unauthorized

people are attempting to break into your network Keeping

those attackers out or extracting them from the network oncethey've gotten in is a different problem Obviously, keeping

intruders out of your network is a meaningless task if you don'tknow when they're breaking in

Detecting unauthorized connections is a good start, but it is notthe whole story Network intrusion detection systems like Snortare great at detecting attempts to login to your system, accessunprotected network shares, and things like that But there areother kinds of intrusion that are not as clear-cut as an outsiderwalking past the receptionist at the front desk and sitting down

at a computer Is a denial of service attackone that operates bysending a carefully crafted sequence of packets to a networkserver and ultimately crashing itan intrusion? No one has

literally gained access to your machine's physical resources.However, bandwidth, CPU time, and hard-drive space on yourIDS are all consumed by the attack Denial of service is

considered a successful attack because it occupies resourcesthat would have been employed somewhere else Does

someone probing your networks with port scans or pings

constitute an intrusion? Perhaps not, but it is a sign that shemay soon start doing something more hostile So we also

consider probing an intrusion, and expect our intrusion

detection system to warn us whenever things such as these

happen

Generally speaking, an intrusion detection system like Snortscans network traffic looking for suspicious activity based on thesignatures of bad packets You are probably already familiarwith tools like tcpdump and ethereal, which display all the

traffic flowing on your network within a specific subnet An

tcpdump, a packet sniffer that sniffs in the background and

does not require you to watch or analyze the traffic yourself.Tools like ethereal work well for debugging; for instance, whenyou have to look at each packet to figure out what might bewrong But on any real network, there is just too much traffic towatch for suspicious activity That is what computers are goodfor: doing a very boring job repetitively, and alerting you whensomething interesting comes along

An IDS watches the packets traversing your network and

decides whether anything is suspicious How does it know what

is suspicious? Snort bases its analysis on the signatures of badpackets: essentially, a list of descriptions of the types of packetsthat indicate the system is under attack or a successful attackhas already taken place For example, if you receive an ICMPpacket that is abnormally large, you may infer somebody is

trying the antiquated ping of death attack against a host on the

network If you see fragmented packets that are extremely

short, you may also infer that somebody is trying one of themany attacks that rely on fragmentation to sneak by firewalls.Snort (and other intrusion detection systems) comes with

thousands of signatures, based on attacks that have been

observed "in the wild." The list grows longer every day and

updates are constantly posted to the Snort web site Part of thejob (and one that is managed nicely by the tool we will soondiscuss) is keeping your signature list up-to-date

Snort and other intrusion detection systems thus provide animportant first line of defense against attacks If an intrudermanages to login to your network server, you might be able tofind the evidence in system logs, although a smart cracker

would delete your logfiles The host intrusion detection systemwatches for unauthorized activity on an individual system Ifsomeone manages to compromise the same server using a

fragmentation attack, you might be able to figure out what

happened after the fact by looking at logs, but you might

While it is too optimistic to talk about "real-time" intrusion

detection, it is extremely important that an IDS detect attacksearly, before any damage can be done, and that it send

notifications to you and to a secure database We discuss

"invisible" or stealthy methods of logging Snort's warnings andalerts to a database elsewhere If you can head off an attack,

so much the betterbut even if you cannot, an IDS might be theonly way to figure out what happened and prevent it from

happening again

Detection

The benefits of detecting an intrusion as early as possible areundeniable But it is important to deploy an IDS with realisticexpectations There are some real challenges in installing,

If you are not familiar with network sniffing tools like tcpdumpand ethereal, spend some time watching the traffic on your

network Review the contents of Chapter 2 to help you interpretthe results Only good can come from this time spent watchingand learning how things talk and move around your network.Without this background, the job of determining what is reallysomething to worry aboutas well as tuning out unneeded rulesand featuresis very difficult

Very often (and especially before tuning), when Snort sendsyou a warning that something suspicious is happening, there isnothing really serious going on Any NIDS is going to generate alot of false positives, warnings that someone or something islaunching some form of attack, when in fact nothing is

happening You may be able to minimize false positives, but youcannot entirely eliminate them Furthermore, the more falsepositives you receive, the more likely it is that Snort is missing

an actual attack or subversive intrusion attempt It is up to you

to figure out an acceptable level of risk Do you really want to

be notified about every port scan? About every unauthorizedattempt to mount a Windows share? Even on a home networkthis can quickly drive most sane administrators crazy

There is no perfect solution There's an easy way to guarantee

an attack is never goes unnoticed: flag every incoming packet

as suspicious That is obviously not realistic You won't have toworry about missing a potential attack, but the flood of falsepositives will be overwhelming At the other extreme, you couldtune out the majority of alerts and turn off most of the features

of Snort You won't have many false positives, but you'll alsomiss many of the real dangers You must find a happy mediumand decide just how many alerts you are willing to tolerate forthe sake of your network The process of reaching this

in determining the health and well-being of your systems AnIDS only provides value as a component in a defense-in-depthstrategy Do not lay all security responsibility on your IDS

installation

1.5.4 Unrealistic Expectations

When deciding to embark on a Snort installation (or any otherNIDS solution, for that matter), understand that there is somesignificant work that needs to be done on the frontend None of

it is particularly difficult, just time-consuming and detail-oriented A common misconception is that once the NIDS

sensors are deployed and initially configured, and the centralmanagement console is built and reporting, the administratorcan throw a dust cloth on it and walk away Snort is a

signature-based NIDS Signatures need to be updated

periodically to keep up with the latest exploits and attack

methods They also need constant tuning to eliminate falsepositives and allow for changes in your network environment.These tasks are not overwhelming, but not allowing time forthem greatly diminishes the value of the NIDS deployment

Snort represents a cost-effective and robust NIDS solution thatfits the needs of many organizations This book should be allyou need to get Snort installed, configured, tuned, and alertingaccurately in your environment Snort is covered from initialconfiguration to ongoing maintenance Strategies are revealed

to make Snort useful for a home office or a large corporationwith a dedicated and experienced network security staff Theapproach is one of attempting to derive reasonable approaches

to the issues at hand I try hard not to be a zealot

Snort does not stand by itself as the beginning and end of asecurity framework for an organization It is part of an overalldefense-in-depth strategy that incorporates security in all

aspects of a network Whether Snort is an important and

significant contributor relies on strong planning and an ongoingdedication to the care and feeding of your NIDS

There are a wide variety of choices in the area of intrusion

detection Digging through the propaganda generated by thevarious marketing departments is not easy Even the definition

of intrusion detection is murky, often moving from one solution

to another To cut through the noise, consider the following:

Cost

Open source software is hard to beat on price To be sure,very often such software can be more difficult to operate.Snort is one of the more mature open source packages outthere and competes with any commercial product for return

on investment There is the occasional C-level executivethat will throw out an open source solution because there is

no one to call when it breaks With mainstream acceptance

core technology Chief among these is Sourcefire, an

organization at the forefront of Snort development and

implementation Sourcefire was started by a fellow namedMartin Roesch, now the CTO (does that name sound

nonexistent A Snort instance crashing is almost unheard of

I personally have a Snort installation that watches sustained

450 Mbps of bandwidth using a cluster of six sensors Theonly time Snort is down is during a planned maintenancewindow to upgrade signatures or move to a new version.This demonstrates not only Snort's stability, but also its

computers can communicate and be used on a network

presents a real challenge The preprocessors act as

interpreters for the Snort detection engine Another real

tracks Because you can customize existing signatures orwrite your own custom rules, Snort can adapt to almost anysituation

There are a number of applications that can act as centralmonitoring and alerting consoles I talk about several,

concentrating on ACID and SnortCenter There are also anumber of community contributed scripts and plug-ins thatextend Snort's functionalityallowing syslogs to be parsedand alerted from, and another allowing the dynamic

SANS, to name two) The Snort open source community isvery active keeping signatures up to date When worms areravaging the Internet and there are constantly new

variants, there are sometimes updates multiple times a

week The Snort mailing lists are a fantastic resource for

people who are trying to run Snort or write their ownsignatures.