Cisco press cisco express forwarding may 2007 ISBN 1587052369

2635, is a Principle Engineer in the Routing Protocol Design and Architecture team at Cisco.. 13451, is a network consulting engineer for Advanced Services at Cisco in Research TriangleP

CCIE No 2635; Stacia McKee

by Nakia Stringfield - CCIE No 13451; Russ White -Publisher: Cisco Press Pub Date: April 24, 2007 Print ISBN-10: 1-58705-236-9 Print ISBN-13: 978-1-58705-236-1 Pages: 288

Cisco Express Forwarding demystifies the internal workings of

helping you to improve your troubleshooting skills for CEF- andnon-CEF-related problems

The first part of the book provides an overview of packet-switching architectures and CEF operation and advanced

features It also covers the enhanced CEF structure and generaltroubleshooting The second part of the book provides case

studies that focus on the common topics that have been

problematic for customers and those supporting Cisco networks.Full of practical examples and configurations, this book draws

on years of experience to help you keep your Cisco networksrunning efficiently

Nakia Stringfield, CCIE® No 13451, is a network consultingengineer for Advanced Services at Cisco, supporting top

financial customers with network design and applying best

practices She was formerly a senior customer support engineerfor the Routing Protocols Technical Assistance Center (TAC)

team troubleshooting issues related to CEF and routing

protocols Nakia has been with Cisco for more than six years,previously serving as a technical leader for the Architecture TACteam

Russ White, CCIE No 2635, is a Principle Engineer in the

Routing Protocol Design and Architecture team at Cisco He is amember of the IETF Routing Area Directorate, co-chair of theRouting Protocols Security Working Group in the IETF, a regularspeaker at Cisco Networkers, a member of the CCIE ContentAdvisory Group, and the coauthor of six other books about

routing and routing protocols, including Optimal Routing Designfrom Cisco Press Russ primarily works in the development ofnew features and design architectures for routing protocols.Stacia McKee is a customer support engineer and technical

(TAC) team This team focuses on providing post-sales support

of IP routing protocols, MPLS, QoS, IP multicast, and many

other Layer 3 technologies Stacia has been with Cisco for morethan six years, previously serving as a technical leader of theArchitecture TAC team and a member of the WAN/Access TACteam

Learn the key features of packet-switching architecturesUnderstand the basics of the CEF architecture and operationExamine the enhanced CEF structure, which improves

Appendix A Scalability

This book is part of the Networking Technology Series fromCisco Press®, which offers networking professionals valuableinformation for constructing efficient networks, understandingnew technologies, and building successful careers

Category: Networking

Covers: Routing and Switching

1587052369

CCIE No 2635; Stacia McKee

by Nakia Stringfield - CCIE No 13451; Russ White -Publisher: Cisco Press

Pub Date: April 24, 2007

Print ISBN-10: 1-58705-236-9 Print ISBN-13: 978-1-58705-236-1 Pages: 288

Summary

Chapter 6 Load Sharing with CEF

Benefits of Load Sharing

Load Sharing with Process Switching and Fast SwitchingComparing CEF Per-Packet and CEF Per-Destination LoadSharing

CEF Architecture and Load Sharing

CEF Load Sharing Across Parallel Paths

Per-Packet Load Sharing on Hardware-Based PlatformsCEF Per-Packet Load Sharing on the Cisco GSR PlatformCEF Load-Sharing Troubleshooting Examples

Summary

References

Chapter 7 Understanding CEF in an MPLS VPN Environment

An Internet Service Provider's Simple MPLS VPN DesignUnderstanding the CEF and MPLS VPN Relationship

CEF Considerations When Troubleshooting MPLS VPNAcross Various Platforms

information storage and retrieval system, without written

permission from the publisher, except for the inclusion of briefquotations in a review

warranty or fitness is implied

The information is provided on an "as is" basis The authors,Cisco Press, and Cisco Systems, Inc., shall have neither liabilitynor responsibility to any person or entity with respect to anyloss or damages arising from the information contained in thisbook or from the use of the discs or programs that may

The opinions expressed in this book belong to the authors andare not necessarily those of Cisco Systems, Inc

Trademark Acknowledgments

All terms mentioned in this book that are known to be

trademarks or service marks have been appropriately

capitalized Cisco Press or Cisco Systems, Inc., cannot attest tothe accuracy of this information Use of a term in this bookshould not be regarded as affecting the validity of any

trademark or service mark

Feedback Information

At Cisco Press, our goal is to create in-depth technical books ofthe highest quality and value Each book is crafted with careand precision, undergoing rigorous development that involvesthe unique expertise of members from the professional

technical community

Readers' feedback is a natural continuation of this process Ifyou have any comments regarding how we could improve thequality of this book, or otherwise alter it to better suit yourneeds, you can contact us through email at

feedback@ciscopress.com Please make sure to include thebook title and ISBN in your message

Publisher Paul Boger

Cisco

Representative

AnthonyWolfendenCisco Press

©2007 Cisco Systems, Inc All rights reserved CCVP, the Ciscologo, and the Cisco Square Bridge logo are trademarks of CiscoSystems, Inc.; Changing the Way We Work, Live, Play, andLearn is a service mark of Cisco Systems, Inc.; and Access

trademarks of Cisco Systems, Inc and/or its affiliates in theUnited States and certain other countries

All other trademarks mentioned in this document or Website arethe property of their respective owners The use of the wordpartner does not imply a partnership relationship between Ciscoand any other company (0609R)

Dedications

Nakia Stringfield:

I would like to dedicate this book to my wonderful, supportivehusband, Kwame Stringfield, and to our beautiful daughter,

Kyra Most of all, thanks go to God for favor and challengingopportunities Thanks to my parents, Robert and Annette; myfamily; my pastors; Dr Frank and JoeNell Summerfield; and myfriends for their many prayers and for believing in me

Russ White:

I would like to dedicate this book to my two daughters, Bekahand Hannah, as well as to my beautiful wife, Lori I would like

to thank God for the opportunities and skills to work on routers,routing, and books

Stacia McKee:

colleague and dearest friend, Parag Avinash Kamat (July 19,1977–August 19, 2004) May his memory live on forever I

would like to thank my wonderful husband, Michael McKee, and

my parents, Richard and Sidney Froom, for their love, patience,and support while completing this project I also thank God forall His blessings in my life

Nakia Stringfield, CCIE No 13451, is a network consulting

engineer for Advanced Services at Cisco in Research TrianglePark, North Carolina, supporting top financial customers withnetwork design and applying best practices She was formerly asenior customer support engineer for the Routing Protocols

Technical Assistance Center (TAC) team, troubleshooting issuesrelated to CEF and routing protocols Nakia has been with Ciscofor more than six years, previously serving as a technical leaderfor the Architecture TAC team She has given training courses

on CEF operation and troubleshooting for internal employees.Nakia also worked for a year with IBM Global Services LAN

Support in Research Triangle Park, North Carolina Nakia

attended North Carolina State University and completed herbachelor of science degree in electrical engineering in 1996.She also earned a master of science in computer networkingand computer engineering from North Carolina State University

in 2000

Russ White, CCIE No 2635, is a member of the Routing

Protocol Design and Architecture Team at Cisco, Research

Triangle Park, North Carolina He is a member of the InternetEngineering Task Force (IETF) Routing Area Directorate, a

cochair of the Routing Protocols Security Working Group in theIETF, a regular speaker at Networkers, a member of the CiscoCertified Internetwork Expert (CCIE) Content Advisory Group, amember of the core team developing the new Cisco Design

certification, a regular contributor to the Internet Protocol

Journal, and the coauthor of six other books about routing and routing protocols, including Optimal Routing Design, from Cisco

Press Russ primarily works in the development of new featuresand design architectures for routing protocols

Stacia McKee is a customer support engineer and technical

leader of the Routing Protocols (RP) Technical Assistance Center(TAC) team at Cisco in Research Triangle Park, North Carolina.This team focuses on providing postsales support of IP routing

of Cisco.com technical documentation, mainly in router and IOSarchitecture and IP routing protocols technologies She worksclosely with the IP Routing and IP Services groups within theCisco Network Software and Systems Technology Group

(NSSTG) on customer problems and early field trials In 2000,Stacia completed her bachelor of science degree in computerinformation systems, bachelor of science degree in businessadministration, and bachelor of arts degree in computer science

at the College of Charleston in Charleston, South Carolina

Neil Jarvis has been a software engineer in the networking

industry since 1990 He is currently employed by Cisco Systems

as a distinguished engineer, responsible for the architecture anddevelopment of switching control and data plane software,

including Cisco Express Forwarding (CEF) He was a technicalcontributor and editor of a number of IEEE 802 standards,

including 802.1 (bridging) and 802.5 (token ring) He was IEEE802.1 vice-chair for a number of years Neil graduated with amaster's degree in microelectronic systems engineering fromUMIST (Manchester, England) in 1989 and now lives with hiswife in Edinburgh, Scotland

LJ Wobker, CCIE No 5020, holds a bachelor of science

degree in computer science from North Carolina State

University in Raleigh, North Carolina He started his networkingcareer running cables as a college intern in the Cisco ResearchTriangle Park TAC lab and has worked in TAC, Advanced

Services, and software development For the last five years, LJhas been a technical marketing engineer, supporting the Cisco

12000 and CRS-1 series routers

This book would not have been possible without the help ofmany people whose various comments and suggestions helped

to formulate this project First, we would like to give specialrecognition to Richard Froom for providing crucial direction andvaluable feedback for this book We also want to thank thetechnical reviewers for this book, Neil Jarvis and LJ Wobker.Finally, we want to thank Brett Bartow, Chris Cleveland, andDayna Isley, as well as the other people at Cisco Press, forworking with us, keeping us on track, and getting this bookpublished

Icons Used in This Book

The conventions used to present command syntax in this bookare the same conventions used in the IOS Command Reference.The Command Reference describes these conventions as

follows:

Boldface indicates commands and keywords that are

entered literally as shown In actual configuration examplesand output (not general command syntax), boldface

Square brackets [ ] indicate optional elements

Braces { } indicate a required choice

Braces within brackets [{ }] indicate a required choice

within an optional element

How does a router switch a packet? What is the difference

between routing a packet and switching a packet? What is thisCEF feature that is referred to in Cisco documentation and

commonly found in Cisco IOS commands? This book answersthese questions through comprehensive discussions of CiscoExpress Forwarding (CEF)

CEF is a term used to describe one of the mechanisms used byCisco IOS routers and Cisco Catalyst switches to forward

packets Other packet-switching mechanisms include processswitching and fast switching CEF is found in almost all CiscoIOS routers and Catalyst switches However, documentation ofthe topic is scarce From a technical support perspective, CEF is

a widely misunderstood topic whose implementation varies

significantly on multiple Cisco platforms Cisco engineers, Ciscopartners, and customers need material on CEF to properly

deploy, maintain, and troubleshoot their networks

CEF offers the following benefits:

Improved performance— CEF is less CPU-intensive than

fast-switching route caching More CPU processing powercan be dedicated to Layer 3 services such as quality of

caching schemes

This book addresses common misconceptions about CEF andpacket switching across various platforms The goal is to helpend users understand CEF and know how to troubleshoot,

regardless of whether a CEF or another problem is occurring inthe network Little information collectively addresses theseconcerns because CEF is proprietary This book helps you

understand CEF better by using the following methods:

Explaining CEF basics

Supplying troubleshooting scenarios that enhance yourability to recognize common mistakes

Providing best practices for configuration

The focus audience of this book is networking professionals whorequire an understanding of Cisco packet-forwarding

architecture and who are tasked with troubleshooting routingand switching issues in a Cisco network environment This book

is an invaluable guide for those seeking to gain an

understanding of how CEF works and how to troubleshoot CEFissues on various hardware platforms

Although this book could be read from cover to cover, it is

designed to be flexible and allows you to easily move betweenchapters and sections of chapters to cover just the material thatyou need to troubleshoot an immediate problem or to

understand a concept

Cisco Express Forwarding is divided into two parts The first

part of the book provides an overview of packet-switching

architectures and CEF operation and advanced features It alsocovers the enhanced CEF structure and general troubleshooting.The second part of the book focuses on particular case studies.Because CEF is a widely misunderstood technology, the casestudies focus on a list of the common topics that have been

problematic for customers and those supporting Cisco networks.The case studies review and expand on material from the

previous parts of the book and provide more in-depth analysis

of real networking topologies and troubleshooting steps

Part I, "Understanding, Configuring, and Troubleshooting CEF"includes the following chapters:

Chapter 1, "Introduction to Packet-Switching

Architectures"— This chapter explains packet-switching

architecture and terminology It also explains utilization ofmemory and buffers

Chapter 2, "Understanding Cisco Express

Forwarding"— This chapter deals with the basics of CEF

architecture and operation It defines CEF terminology andhistory

Part II, "CEF Case Studies," deals with special CEF case studiescovering the following common scenarios:

Chapter 5, "Understanding Packet Switching on the Cisco Catalyst 6500 Supervisor 720"— This chapter

Although this book provides solid information for software

handling and hardware handling, it does not provide a detaileddescription of implementation on all Cisco platforms and relatedtechnologies Hardware design changes rapidly, and packet

handling on one platform could easily consume the entire book.This book does not address Parallel Express Forwarding (PXF),which is used on devices such as Cisco 10000 series routers,Cisco 7600 series Optical Service Modules (OSMs), and Cisco

7300 series routers PXF leverages a combination of parallelprocessing and pipelining techniques to the CEF algorithms forfaster throughput and optimal flexibility through ASIC

technology Because PXF is highly dependent on the platformand specific ASIC technology, it is not covered in this book

Hardware switching will continue to be optimized for

performance advantages Introduction of distributed CEF (dCEF)

on Cisco 7500 series routers was a start down this path yearsago to offload packet switching from the central processor tothe Versatile Interface Processor (VIP) line card Then

progression occurred to hardware-based localized switching onCisco 6500s with Distributed Forwarding Cards (DFCs),

FlexWans, and OSMs

Cisco recently introduced IOS Software Modularity, which

provides subsystem In-service Software Upgrades and ProcessFault Containment to the Cisco Catalyst 6500 series switches

As you continue to learn more about Cisco Express Forwarding,you may find the following resources helpful:

Bollapragada, V., R White, and C Murphy, Inside Cisco IOS Software Architecture, Indianapolis, Indiana: Cisco Press;

2000 Provides a detailed treatment of Cisco 7500 routersand Cisco 7200 routers

Cisco, Parallel Express Forwarding on the Cisco 10000

www.cisco.com/en/US/products/hw/routers/ps133/products_white_paper09186a008008902a.shtmlCisco, Cisco 7600 Series Router Q & A,

www.cisco.com/en/US/products/hw/routers/ps368/products_qanda_item09186a008017a32b.shtmlCisco, PXF Information for Cisco 7304 Routers,

www.cisco.com/en/US/products/hw/routers/ps352/prod_maintenance_guide09186a008057410a.htmlCisco, Cisco Catalyst 6500 Series Switches with IOS

Part I: Understanding, Configuring, and Troubleshooting CEF

This chapter begins with a discussion of the terms routing and switching and provides you with the background needed to

understand the differences between the two The chapter thencovers the physical pieces and parts of a router and discussesthe brains, Cisco IOS Software You then learn how the pieceswork together to switch a packet

Routing and Switching

The networking industry uses many terms and concepts to

describe switching and routing; because a good number of

them have overlapping meanings, deciphering the terminologycan be confusing Does a router route or switch? What's thedifference between Layer 3 switching and routing? What's Layer

7 switching, and who cares? Let's examine what happens to a

cannot transmit at the same time If, however, some physicaldevice between them allows them to transmit at the same time,they are in separate collision domains

The broadcast domain is the set of hosts that can communicate

simply by sending Layer 2 (or link-layer) broadcasts If Host Atransmits a broadcast packet to all the hosts that are locallyattached, and Host B receives it, these two hosts are in the

same broadcast domain

Broadcast and Collision Domains

Bridging breaks up the collision domain, but not the broadcastdomain In fact, traditional switching and bridging are the samething technically The primary difference is that in most

switched environments, each device connected to the network

is in a separate collision domain

Looking at the format of a typical data packet, what is changedwhen the packet crosses a switch? Not a single thing, as Figure

segment asking for the MAC address of the host with the IP

address 192.168.1.2 (this is called an Address Resolution

Protocol (ARP) request) Because Host B is in the same

broadcast domain as Host A, Host A can be certain that Host Bwill receive this broadcast and answer with the correct MACaddress to send packets to

Broadcast and Collision Domains in Routing

Routers not only break the collision domain, but they also breakthe broadcast domain, as Figure 1-3 illustrates

Figure 1-3 Routing

It cannot broadcast an address resolution packet to discoverHost B's address, so it has to use some other method to figureout how to reach this destination How does Host A know this?Note that after each IP address in Figure 1-3, there is also a/24; this number indicates the prefix length, or the number ofbits that are set in the subnet mask Host A can use this

information to determine that Host B is not in the same

broadcast domain (not on the same segment), and Host A mustuse an intervening router to reach the destination, as Figure 1-

directing its packets toward the intervening router Host A

5 shows

Figure 1-5 Packet Flow Through a Router

Host A puts the router's MAC address on the packet, so the

router accepts the packet off the network The router examinesthe destination IP address and determines what the next closerhop should be by consulting a routing table (in this case, it isHost B itself), and replaces the MAC address with the correctMAC address for the next hop The router then transmits thepacket back onto a different segment, which is in a differentbroadcast domain

Part I: Understanding, Configuring, and Troubleshooting CEF

This chapter begins with a discussion of the terms routing and switching and provides you with the background needed to

understand the differences between the two The chapter thencovers the physical pieces and parts of a router and discussesthe brains, Cisco IOS Software You then learn how the pieceswork together to switch a packet

Routing and Switching

The networking industry uses many terms and concepts to

describe switching and routing; because a good number of

them have overlapping meanings, deciphering the terminologycan be confusing Does a router route or switch? What's thedifference between Layer 3 switching and routing? What's Layer

7 switching, and who cares? Let's examine what happens to a

cannot transmit at the same time If, however, some physicaldevice between them allows them to transmit at the same time,they are in separate collision domains

The broadcast domain is the set of hosts that can communicate

simply by sending Layer 2 (or link-layer) broadcasts If Host Atransmits a broadcast packet to all the hosts that are locallyattached, and Host B receives it, these two hosts are in the

same broadcast domain

Broadcast and Collision Domains

Bridging breaks up the collision domain, but not the broadcastdomain In fact, traditional switching and bridging are the samething technically The primary difference is that in most

switched environments, each device connected to the network

is in a separate collision domain

Looking at the format of a typical data packet, what is changedwhen the packet crosses a switch? Not a single thing, as Figure

segment asking for the MAC address of the host with the IP

address 192.168.1.2 (this is called an Address Resolution

Protocol (ARP) request) Because Host B is in the same

broadcast domain as Host A, Host A can be certain that Host Bwill receive this broadcast and answer with the correct MACaddress to send packets to

Broadcast and Collision Domains in Routing

Routers not only break the collision domain, but they also breakthe broadcast domain, as Figure 1-3 illustrates

Figure 1-3 Routing

It cannot broadcast an address resolution packet to discoverHost B's address, so it has to use some other method to figureout how to reach this destination How does Host A know this?Note that after each IP address in Figure 1-3, there is also a/24; this number indicates the prefix length, or the number ofbits that are set in the subnet mask Host A can use this

information to determine that Host B is not in the same

broadcast domain (not on the same segment), and Host A mustuse an intervening router to reach the destination, as Figure 1-

directing its packets toward the intervening router Host A

5 shows

Figure 1-5 Packet Flow Through a Router

Host A puts the router's MAC address on the packet, so the

router accepts the packet off the network The router examinesthe destination IP address and determines what the next closerhop should be by consulting a routing table (in this case, it isHost B itself), and replaces the MAC address with the correctMAC address for the next hop The router then transmits thepacket back onto a different segment, which is in a differentbroadcast domain