Detecting And Mitigating Denial Of Service Attacks _ www.bit.ly/taiho123

 Know the enemy Understand what drives the miscreants Understand their techniques  Create the security team and plan Who handles security during an event; is it the security folks; the

HOUSEKEEPING

 We value your feedback, don’t forget to complete your online session

evaluations after each session and complete the Overall Conference

Evaluation which will be available online from Friday.

 Visit the World of Solutions on Level -01!

 Please remember this is a ‘No Smoking’ venue!

 Please switch off your mobile phones!

 Please remember to wear your badge at all times including the Party!

 Do you have a question? Feel free to ask them during the Q&A section or write your question on the Question form given to you and hand it to the

Room Monitor when you see them holding up the Q&A sign.

Objectives and Assumptions

 How to detect and mitigate Denial of Services Attacks in a network.

 Explaining what kind of threats which we need to defend against.

 Explaining the various detection mechanisms which are available.

 Explaining the different mitigation techniques, how they are used and the

possible consequences of implementing them.

 The audience is assumed to consist of network architects, security officers and project managers from SP and Large Enterprise customers.

 Assumtion : The audience has a basic knowledge of routing protocols and

a good and broad understanding of various security techniques and tools

used in large networks today.

 This session is related to sessions:

Network Core Infrastructure Protection: Best Practices (BRKSEC-2013) Detecting Router Abuse (BRKSEC-2015)

Network-based Solutions for Broadband Residential Security (BRKSEC-2016) The Techtorial Mitigating DoS Attacks (TECSEC-2003) also cover all those techniques, so, if you attended the techtorial, there is no need to attend this break-out session.

 Introduction : Threat Landscape

 Six Phases of Incident Reaction process

Planning, Detection, Classification, Traceback, Reaction, Post Mortem

 Advanced Reaction Techniques

Motivation and Trends

DDoS Attacks Are Here To Stay

Symantec Internet Security

Report – March ‘06

 DoS attacks grow from 119

to 1500 per day in 2005- an increase of 1200%

 Jan06-Jun06 : Avg 6110 Dos

Attacks per day an increase

of 600% *Symantec Sept2006

 Large % of DDoS attacks are motivated by extortion

demands

 50K Average Active Bots

 Attack size is in the 2-7 Gig range

 The DoS problem is not a

100 year flood anymore!

‘Zombie' ring allegedly hit 1.5 million computers

http://www.msnbc.msn.com/id/9763824/

Dutch Internet provider XS4ALL identified the zombie network – “only a drop in the ocean."

Threat Economy: In the Past

End Value

Espionage (Corporate/

Government)

Fame Theft

Worms

Tool and Toolkit Writers

Viruses Trojans

Malware Writers

Compromise Individual Host or Application

Compromise Environment

Threat Economy: Today

Electronic IP Leakage

Machine Harvesting

Information Harvesting

Hacker/Direct Attack

Internal Theft:

Abuse of Privilege

Information Brokerage

Identity Theft

Compromised Host and Application

End Value

Financial Fraud

Commercial Sales Fraudulent Sales

Click-Through Revenue

Espionage (Corporate/ Government) Fame

Extorted Pay-Offs Theft

Denial of Service Trends

SYN AND UDP AND—

 Use of non-TCP/UDP/ICMP protocols

Get past ACLs

Increased awareness in community

 Target ISP Infrastructure

 Target Applications

SMTP reflective, VoIP

Incident Response

How do you handle a DDOS attack?

Mbehring

Incident Response Methodology

for Worms and DoS

 Traceback: Find infection vector Find ingress path

- Quarantine - Re-direction

common

Important!!

Prep the Network Create Tools Test Tools Prep Procedures Train Team

Reaction

What options do you

have to remedy?

Which option is the

best under the

circumstances?

Post Mortem

What was done?

Can anything be done to

prevent it?

How can it be less

painful in the future?

Six Phases of Incident Response

Preparation—Develop and Deploy a Solid

Security Foundation

Preparation

 Includes technical and non-technical components

 Encompasses best practices

 The hardest yet most important phase

 Without adequate preparation, you are

destined to fail

 The midst of a large attack is not the time to be

implementing foundational best practices and

processes

 Know the enemy

Understand what drives the miscreants Understand their techniques

 Create the security team and plan

Who handles security during an event; is it the security folks; the networking folks

A good operational security professional needs to be a cross between the two: silos are useless

 Harden the devices

 Prepare the tools

Network telemetry Reaction tools

Detection —How Do You Know You or

Your Customer Is Under Attack?

Detection

 It is more than just waiting for your customers to

scream or your network to crash

 What tools are available?

 What can you do today on a tight budget?

Detection —Ways to Detect

 Customer call

“The Internet is down”

 Unexplained changes in network baseline

SNMP: line/CPU overload, drops Bandwidth

Detection —Network Baselines

 NMS baselines

 Unexplained changes in link utilization

Worms can generate a lot of traffic, sudden changes

in link utilization can indicate a worm

 Unexplained changes in CPU utilization

Worm scans can affect routers/switches resulting in increased CPU both process and interrupt switched

 Unexplained syslog entries

 These are examples

Changes don’t always indicate a security event Must know what’s normal in order to identify abnormal behavior

 Classification—understand the details and

scope of the attack

Identification is not sufficient; once an attack

is identified, details matter Guides subsequent actions

 Identification and classification are often simultaneous

 Qualify and quantify the attack without jeopardizing

services availability (e.g., crashing a router):

What type of attack has been identified?

What’s the effect of the attack on the victim(s)?

What next steps are required (if any)?

 At the very least:

Source and destination address Protocol information

Port information

 Traceback—what are the sources of the attack?

How to trace to network ingress points Your Internet connection is not the only vector Understand your topology

 Traceback to network perimeter

NetFlow Backscatter Packet accounting

 Retain attack data

Use to correlate interdomain traceback Required for prosecution

Deters future attacks Clarify billing and other disputes Post mortem analysis

Reaction—Do Something to Counter the Attack

Post Mortem—Analyze the Event

Post Mortem

 The step everyone forgets

 What worked? What didn’t? How can we improve?

 Protect against repeat occurrences?

 Was the DoS attack you handled the real threat?

Or was it a smoke screen for something else that just

happened?

 What can you do to make it faster, easier, less painful

in the future?

 Metrics are important

Resources, headcount, etc.

Preparation and

Detection

Preparing your infrastructure and systems to detect and react to

DDOS attacks.

 Visibility of Infrastructure traffic

 Creation of Reaction mechanisms

 Procedures to follow during attack

 Rehearsal of DDOS attack

 Reporting and Post Mortem process’s

Visibility via Network Telemetry

SNMP: MRTG

 MRTG—the Multi Router Traffic Grapher

 Open source SNMP visualization toolset developed by

Tobi Oetiker, available from

http://people.ee.ethz.ch/~oetiker/webtools/mrtg/

 Long track-record—(in general use since 1995)

 Can be used to graph router/switch data, host

performance information from systems running SNMP

agents, etc (generates HTML w/PNG images)

 Runs on Linux, FreeBSD, Mac OS/X, Solaris, other

*NIX, Windows

 Written in Perl, has its own SNMP implementation

Example: MRTG Graphs

Source: mrtg.org

Reference Slide

RMON: Remote MONitoring

 RMON is a standard defining how remote probes or

agents relay network traffic information back to a

central console

 Cisco Network Analysis Module-2 (NAM-2), ntop

(http://www.ntop.org) are examples of RMON probes

 Most RMON probes look at raw packets via

SPAN/RSPAN

and generate statistics from observed traffic

 Mini-RMON statistics available on Cisco Catalyst

6500/NAM-2, provides detailed stats from Layer 2

access ports

NAM-2 Examples

mrtg.org

Source: Cisco Systems, Inc.

Reference Slide

 De facto logging standard for hosts, network

infrastructure devices, supported in all Cisco routers

and switches

 Choose appropriate level for each device/situation

 ACL logging dangerous due to CPU overhead

NetFlow provides more information, doesn’t max the box

 Can be used in conjunction with Anycast and

databases to provide a scalable, robust logging

infrastructure

 Different facility numbers allows for segregation of log

information based upon device type, function, other

criteria

Packet Capture

 Sometimes, there’s just no substitute for looking at the

packets on the wire

 SPAN/RSPAN/ERSPAN allow packet capture from

Cisco Catalyst switches; ip packet export allows packetcapture from routers

 Open source tools such as tcpdump, snoop, Ethereal

(http://www.ethereal.com)

 Commercial tools such as Cisco NAM-2, NAI Sniffer/

Distributed Sniffer, Wandel and Goltermann available

 Packet capture is like a wiretap

 NetFlow is like a phone bill

 This level of granularity allows NetFlow to scale

for very large amounts of traffic

 We can learn a lot from studying the phone bill

 Who’s talking to whom, over what protocols and ports, for how

long, at what speed, for what duration, etc.

 NetFlow is a form of telemetry pushed from the

routers/switches—each one can be a sensor

 Each flow defined by Seven keys

 Detecting and Analyzing Network Threats With NetFlow

http://www.cisco.com/en/US/customer/products/ps6441/products_confi guration_guide_chapter09186a008055404a.html

Core Network

Enable NetFlow

Traffic

Collector NFC, cflowd, flow-tools, Arbor

UDP NetFlow Export Packets

Application GUI Arbor, FlowScan

Export Packets

• Approximately 1500 bytes

• Typically contain 20–50

flow records

• Sent more frequently if

traffic increases on

NetFlow-enabled interfaces

PE

Creating Export Packets

NetFlow Versions

Flexible, Extensible File Export Format to Enable Easier Support of Additional Fields and Technologies; Coming Out Now Are MPLS, Multicast, and BGP Next-Hop

• Attack detection

• Application monitoring

• Billing

• Chargeback

• AS peer monitoring

• Attack detection

• BGP next-hop (v9)

• Arbor Networks

• BGP Next-hop (v9)

• Arbor Networks

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks

NetFlow in the Topology

IP Flow Switching Cache, 4456704 bytes

65527 active, 9 inactive, 2364260060 added

4143679566 ager polls, 0 flow alloc failures

Active flows timeout in 30 minutes

Inactive flows timeout in 15 seconds

last clearing of statistics never

Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)

- Flows /Sec /Flow /Pkt /Sec /Flow /Flow

SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts

Netflow Top talkers (12.3(11)T)

The NetFlow Top Talkers feature compares all of the flows and displays

information about each of the flows that have the heaviest traffic volumes

(top talkers).

ip flow-top-talkers—Enters the NetFlow Top Talkers configuration mode.

sort-by—Selects the sort order for the flows in the display output.

bytes—Sort the flows based on the numbers of bytes in each flow.

packets—Sort the flows based on the numbers of packets in each flow.

top—Specifies the number of top talkers to monitor.

match (optional)—Specifies additional criteria, such as IP addresses, port numbers, and so forth, that must be

matched in the flow to qualify as a candidate for top talker status.

show ip flow top talkers [verbose]—Displays the flows.

Router# show ip flow top 20 aggregate source-address sorted-by packets match protocol icmp

There are 6 top talkers:

IPV4 SRC-ADDR bytes pkts flows

=============== ========== ========== ==========

10.132.221.111 90440 3230 1 10.10.12.1 90440 3230 1 10.251.138.218 90440 3230 1 10.71.200.138 90384 3228 1 10.231.185.254 90384 3228 1 10.106.1.1 90356 3227 1

6 of 15 flows matched

What Is an Anomaly?

 An event or condition in the network that is identified as

a statistical abnormality when compared to typical trafficpatterns gleaned from previously collected profiles and

baselines

NetFlow-Based Traffic Characterization

and Anomaly Detection with Arbor Networks

 Most widely deployed anomaly detection system for SPs

 Uses NetFlow to quickly identify, classify, and scope DoS,

worms, etc.

 Traffic component combines NetFlow traffic characterization

with BGP

 Allows comprehensive peering analysis in real-time

 A “force multiplier” which greatly reduces reaction-times

by providing the relevant information up-front

 Can also generate its own flows from packet-capture if

NetFlow isn’t available

Network Anomaly Detection and Traffic

Characterization/Capacity Planning

Anomaly Example

Reference Slide

Anomaly Example: Detail

Reference Slide

Design Tips for NetFlow-Based Detection

 Use sampled NetFlow to reduce CPU usage on software platforms

by up to 80%

 Sampling rate is configurable

 Use sampled NetFlow for traffic capacity and network planning

 Agree on which fields of NetFlow to track

 Do not export versions 5, 7 and 9 simultaneously with version 8

 Plan NetFlow deployment in the network topology to avoid a design that

creates duplicate flows for billing

 Use a dedicated interface/VLAN for NetFlow data export (NDE)

 Monitor lost packet counter in NFC

 Check the export link bandwidth

Estimate export of 1% to 1.5% of the interface throughput

Blackhole Filtering

 Blackhole Filtering or Blackhole Routing forwards

a packet to a router’s bit bucket

Also known as “route to Null0”

 Works only on destination addresses, since it is really

part of the forwarding logic

 Forwarding ASICs are designed to work with routes to

Null0—dropping the packet with minimal to no

performance impact

 Used for years as a means to “blackhole”

unwanted packets

 Using BH will also DOS customer service as

DESTINATION is sent to Bit bucket, allows customer

servers to “recover”

Remotely Triggered Blackhole Filtering

 We will use BGP to trigger a network wide

response to an attack

 A simple static route and BGP will enable a

network-wide destination address blackhole

as fast as iBGP can update the network

 This provides a tool that can be used to respond

to security related events and forms a foundation for

other remote triggered uses

 Often referred to as RTBH

Remote Triggered Blackhole

 Configure all edge routers with static route

to Null0 (must use “reserved” network)

ip route 192.0.2.1 255.255.255.255 Null0

 Configure trigger router

Part of iBGP mesh Dedicated router recommended

ip route 192.0.2.1 255.255.255.255 Null0

Step 1: Prepare All the Routers with

Trigger

 Select a small block that will not be used for anything

other than blackhole filtering; test

Net (192.0.2.0/24) is optimal since it should not

be in use

 Put a static route with a /32 from Test-Net—

192.0.2.0/24 to Null 0 on every edge router

on the network

POP

Sinkhole Network

Peer B

Peer A IXP-W

Edge Router with Test-Net to Null0

Step 1: Prepare All the Routers with

Trigger