Cisco networkers 2009 session BRKDCT 2840 data center networking taking risk away from layer two interconnects DDU

Spanning Tree Isolation:DCI Aggregation Access VSL VSL MEC MEC BPDU Filtering enabled to Filter any BPDUs config-ifspanning-tree bpdufilter enable With MEC VSS/vPC at the DCI Layer Appea

Trang 1

Minimizing the Risks with Enterprise

Multi-Site Data Center L2 Connectivity

BRKDCT-2840

Trang 2

Goals of this Session…

 Present alternatives for interconnecting multiple Data

Center locations

 Present tested methods in production for minimizing the risks associated with meeting these connectivity

requirements.

Trang 3

Session Agenda

 Data Center Interconnection – Common Scenarios

and Terms

 Dark Fiber / DWDM Solutions

 Label Based Solutions

 IP Based Solutions

 Encryption

 Recommended Designs for Optimizing Traffic Flows

 EoMPLS and VPLS Stability Testing (Reference

Material)

 Q & A

Trang 4

Layer 2 / 3 Clusters

Use Cases Risks Solution Types

Trang 5

Layer 2 / 3 Clusters

 Intra-Cluster node communications

Flow Types Traditionally Layer2 Communications on Private and/or Public interfaces IPv4 and/or IPv6 possible depending on clustering package used Ability to prioritize interfaces

 Client Access to Cluster

DNS/Active Directory resolution by clients Shared Virtual IP for service discovery Caching issues can inhibit Layer3 clustering Client application can have logic to re-establish connections

 Quorum considerations to avoid split-brain

Additional cluster nodes at alternate sites to achieve a majority node set (MNS) Possible extensions such as ping-groups (Linux-HA) to have a quorum mechanism without a member node

Shoot The Other Node In The Head topologies to resolve conflicts (STONITH)

 Mechanisms to facilitate service restoration in another location

VMware Site Recovery Manager (SRM) is one example Microsoft Server 2008 Layer 3 Clustering is another

Trang 6

Some Layer 2 Use Cases

 Extending Operating System / File System clusters

 Extending Database clusters

 Virtual machine mobility

 Physical machine mobility

 Legacy devices/apps with embedded IP addressing

 Time to deployment and operational reasons

 Extend DC to solve power/heat/space limitations

Trang 7

Layer 2 Risks

 Flooding of packets between data center‟s

 Rapid Spanning Tree (RSTP) is not easily scalable and risk grows as diameter grows

 RSTP has no domain isolation – issue in single DC can propagate

 First hop resolution and inbound service selection can

cause verbose inter-data center traffic

 In general Cisco recommends L3 routing for

geographically diverse locations

 This session focuses on making limited L2 connectivity

as stable as possible

Trang 8

Layer 2 Solution Types

No STP isolation between sites

Cost rises, still nothing to offer STP isolation

regions

STP domain concept Fundamental change requiring large time investment Operational differences and MST database management

Virtual Switching System L2TPv3 for point to point (possible STP isolation issues) EoMPLS for point to point (possible STP isolation issues)

Trang 9

Session Agenda

and Terms

 Encryption

Material)

 Q & A

Trang 10

Dark Fiber / DWDM Solutions

Trang 11

Layer 2 Prerequisites for All Options

 This session assumes a fairly detailed knowledge of

Spanning Tree Protocol

 Items we leverage in this solution:

802.1w 802.1s Port Fast BPDU Filter BPDU Guard Root Guard Loop Guard Bridge Assurance (Catalyst 6500, Nexus 5000 and 7000)

•

Trang 12

Layer 2 Extension Without Tunnels/Tags (vPC/VSS)

 6500 with Virtual Switching System cluster (Supported distances at 80km Dark Fiber)

 Nexus 7000 with Virtual Port-Channels (Supported distances at

10km Dark Fiber)

 All traffic flows to a vPC/VSS member node

 Hub-and-spoke topology from a layer 2 perspective

 Dedicated links to vPC/VSS members from each datacenter

aggregation switch

 Can consume lamdas or fiber strands quickly

 Data plane rate limiting in L2 still needs protection

 STP domains are not isolated unless we BPDU filter at all

vPC/VSS aggregation switches

Trang 13

Layer 2 Extension Using VSS over Dark Fiber - 2 Sites

ECMP

Trang 14

Layer 2 Extension Using VSS, vPC over

Dark Fiber – Multi Site

Trang 15

Layer 2 Extension Using VSS, vPC over

Dark Fiber – Multi Site

Trang 16

Spanning Tree Isolation:

DCI

Aggregation

Access VSL

VSL

MEC

BPDU Filtering enabled to Filter any BPDUs

(config-if)spanning-tree bpdufilter enable

With MEC (VSS/vPC) at the DCI Layer Appears as a Single switch and removes the need for Spanning Tree

Spanning Tree Considerations

With MEC (VSS or vPC) there is no Single Link or Device Failure will cause Data Center Isolation

High Availability:

With VSS + MEC, vPC, We Achieve Physical Device and Network Redundancy yet logically Bundling the device as 1 Destination for the

Avoiding Loops:

With MEC (VSS/vPC) there is no need for Spanning tree, no ports are Blocked Packets Are Load-balanced Across the Etherchannel Links achieving efficient Link Utilization

Multipath:

Trang 17

12 Lambda/24 Strand Example

4 Additional Lambda/8 Strands per new DC

L2 Service Only from Provider

Data Center #3

Trang 18

Data Center #3

BPDU Filtering

BPDU Filtering BPDU Filtering

Trang 19

switch virtual domain 100 switch 2

! interface port-channel 1 switch virtual link 1 interface port-channel 2 switch virtual link 2

!

interface tenGigabitEthernet 5/4 channel-group 2 mode on

interface tenGigabitEthernet 5/5 channel-group 2 mode on

! Router eigrp 10 nsf

Trang 20

5) Create vPC Peer Link

1) Add port-channel 20 to vPC 20*

Trang 21

Session Agenda

and Terms

 Encryption

Material)

 Q & A

Trang 22

MPLS Label Solutions

Trang 23

EoMPLS (Ethernet over MPLS)

 Encapsulates Ethernet frames inside MPLS packets to pass layer

3 network

 EoMPLS has routing separation from metro core devices providing connectivity – CE flapping routes wont propogate inside MPLS

 Point to point links between locations

EoMPLS Is a Pseudo-Wire

MPLS

Trang 24

Virtual Private LAN Service (VPLS)

 VPLS defines an architecture allows MPLS networks offer Layer 2 multipoint Ethernet Services

 Metro Core emulates an IEEE Ethernet bridge (virtual)

 Virtual Bridges linked with EoMPLS Pseudo Wires

VPLS Is an Architecture

MPLS

Trang 25

VPLS Components

N-PE

MPLS Core

Port or VLAN mode

Mesh of LSP between N-PEs

N-PE

Pseudo Wires within LSP

Virtual Switch Interface (VSI) terminates PW and provides Ethernet bridge function

Targeted LDP between PEs to

exchange VC labels for Pseudo

can be a switch or

router Red VFI

Yellow VFI

Trang 26

Virtual Forwarding Instance (VFI)

 IOS Representation of Virtual Switch Interface

 Flooding / Forwarding

MAC table instances per customer (port/vlan) for each PE VFI will participate in learning and forwarding process

Associate ports to MAC, flood unknowns to all other ports

 Address Learning / Aging

LDP enhanced with additional MAC List TLV (label withdrawal) MAC timers refreshed with incoming frames

 Loop Prevention

Create full-mesh of Pseudo Wire VCs (EoMPLS) Unidirectional LSP carries VCs between pair of N-PE Per VPLS Uses “split horizon” concepts to prevent loops

Trang 27

VPLS Details

 This session shows use cases for VPLS

 For more technical details, TECOPT-2100

Deploying Next Generation Carrier Ethernet: Services,

Architectures and Operations, or TECRST-3001

Layer 2 Virtual Private Networks - Converged IP/MPLS Network

 This Data Center Interconnect solution uses some

facets of MPLS, but not a full MP-BGP with multi VRF

type implementation

Trang 28

Self-Managed MPLS Core

Direct Attachment Configuration

 CEs are all part of same VPLS instance (VCID = 56)

CE router connects using VLAN 100 over sub-interface

CE2 PE3

VLAN100

Trang 29

interface GigabitEthernet 1/3.100 encapsulation dot1q 100

ip address 192.168.20.2

interface GigabitEthernet 2/0.100 encapsulation dot1q 100

ip address 192.168.20.3

Direct Attachment CE Router

Configuration

 CE routers sub-interface on same VLAN

Can also be just port based (NO VLAN)

CE2 VLAN100

VLAN100

Subnet 192.168.20.0/24

interface GigabitEthernet 2/1.100

encapsulation dot1q 100

ip address 192.168.20.1

Trang 30

l2 vfi VPLS-A manual vpn id 56

neighbor 2.2.2.2 encapsulation mpls neighbor 1.1.1.1 encapsulation mpls

l2 vfi VPLS-A manual vpn id 56

neighbor 1.1.1.1 encapsulation mpls neighbor 3.3.3.3 encapsulation mpls

l2 vfi VPLS-A manual

vpn id 56

neighbor 2.2.2.2 encapsulation mpls

neighbor 3.3.3.3 encapsulation mpls

MPLS Core

Direct Attachment VFI Configuration

 Create the Pseudo

Wires between N-PE

routers

CE2 PE3

VLAN100

Trang 31

MPLS Core

Direct Attachment CE Router

(VLAN Based)

 Configured on the CE facing interface

CE2 PE3

VLAN100

Interface GigabitEthernet3/0 switchport

switchport mode trunk switchport trunk encapsulation dot1q switchport trunk allowed vlan 100

! Interface vlan 100

no ip address xconnect vfi VPLS-A

! vlan 100 state active

This command associates the

VLAN with the VPLS instance

VLAN100 = VCID 56

Trang 32

Calculating Core MTU Requirements

* MPLS Header Size)

 Edge MTU is the MTU configured in the CE-facing PE interface

 Examples (all in Bytes):

1526 1522

Total

4 2

1500

EoMPLS VLAN Mode

4 2

1500

EoMPLS Port Mode

MPLS Header

MPLS Stack Edge

18 14

Transport

Trang 33

End to End VPLS and EoMPLS Design

Layer 3 Core Intranet

L2 Links (GE or 10GE) L3 Links (GE or 10GE)

VPLS / EoMPLS Domain

Po2 Po1

WAgg2

EAgg1

EAgg2 WMC1

WMC2

EMC1

EMC2

Trang 34

Access to Aggregation Connections

 Rapid PVST is existing protocol, and no

desire to force a change

 Aggregation switches are root for all

 HSRP tested for first hop redundancy

from server (more later)

Server Farm

Agg

Access

Trang 35

Layer 3 Aggregation and Core

Connections

Core to Enterprise Core

connected to DC Core

are to Metro Core switches

which are Ethernet over

MPLS links

peering the DC Cores in

each location in a

point-to-point scenario

DC Core

Layer 3 Enterprise Core

Agg

Bidirectional forwarding detection (bfd) interval 100 min_rx 100 multiplier 3

IF dual supervisor modules, need non-stop forwarding (NSF)

under routing process

Trang 36

EoMPLS / VPLS Infrastructure

EoMPLS and VPLS xconnects

DWDM service between data centers

Layer 2, are storm-control limited for

broadcasts and multicasts to 1% (protect

data plane)

MPLS links for the MPLS tagging

Metro Core Metro Core

Trang 37

Metro Switch Interconnectivity

L3 Links (10GE)

IGP Routing Process connecting MPLS PE’s

Ten3/0/0

Ten4/0/0

Ten3/0/0

Ten3/0/0 Ten3/0/0

Ten4/0/0

Ten4/0/0 Ten4/0/0

- Link debounce timers

Trang 38

Metro Switch Interconnectivity (EIGRP)

ip authentication mode eigrp 5 md5

ip authentication key-chain eigrp 5 password

logging event link-status

load-interval 30

udld port disable

mls qos trust dscp

mpls ip

Trang 39

Metro Switch Interconnectivity (EIGRP)

ip authentication mode eigrp 5 md5

ip authentication key-chain eigrp 5 password

load-interval 30

udld port disable

mls qos trust dscp

mpls ip

Trang 40

Metro Switch Interconnectivity (OSPF)

ip ospf network point-to-point

timers lsa arrival 0

timers pacing flood 15

network 192.168.0.0 0.0.255.255 area 0

Trang 41

Metro Switch Interconnectivity (OSPF)

ip ospf network point-to-point

timers lsa arrival 0

timers pacing flood 15

network 192.168.0.0 0.0.255.255 area 0

Trang 42

Metro Switch Example IP Addressing

 Loopbacks and WAN links use 192.168.0.0 addressing

 WestMetroCore1

Loopback0 IP Address = 192.168.255.250/32 Interface Te3/0/0 IP Address = 192.168.1.1/30 Interface Te4/0/0 IP Address = 192.168.1.9/30

 WestMetroCore2

Trang 43

Metro Switch Example IP Addressing

(Cont.)

 EastMetroCore1

 EastMetroCore2

Trang 44

Metro Switch Routing

 No Dynamic routing between the Metro Core switches and other

Data Center switches

 router eigrp 5 (the routing instance for the MPLS domain, LDP

passes on this)

passive-interface default

no passive-interface TenGigabitEthernet3/0/0

no passive-interface TenGigabitEthernet4/0/0 network 192.168.0.0 0.0.255.255

no auto-summary Nsf

 Enable NSF for LDP, mpls ldp graceful-restart global configuration

Trang 45

Metro Switch Interconnectivity

EastMetroCore1#sh ip route Gateway of last resort is 10.98.128.1 to network 0.0.0.0

D 192.168.255.250

[90/128512] via 192.168.1.1, 3w0d, TenGigabitEthernet3/0/0 192.168.1.0/30 is subnetted, 4 subnets

D 192.168.1.8 [90/768] via 192.168.1.1, 3w0d, TenGigabitEthernet3/0/0

C 192.168.1.12 is directly connected, TenGigabitEthernet4/0/0

C 192.168.1.0 is directly connected, TenGigabitEthernet3/0/0

D 192.168.1.4 [90/768] via 192.168.1.14, 3w0d, TenGigabitEthernet4/0/0 S* 0.0.0.0/0 [1/0] via 10.98.128.1

Trang 46

EoMPLS for Layer3

EoMPLS

METRO CORE

PW – Pseudo Wires

Trang 47

EoMPLS for Layer3 Configuration

Trang 48

EoMPLS for Layer3 Configuration (Cont.)

Trang 49

Trang 50

l2 vfi vlan3700 manual

Trang 51

interface Vlan3700

no ip address load-interval 30 xconnect vfi vlan3700

VLAN 3700

Trang 52

VPLS for Layer2 Configuration

Trang 53

Spanning Tree

 Spanning-Tree BPDU(s) will NOT traverse

between the Data Centers – It isn‟t needed

(and blocked) with VPLS

 We still need to control data plane layer 2

events (i.e limit the traffic)

 Since enterprises want dual N-PE devices, and VPLS blocks BPDU‟s, we require method to

block within a local DC

Trang 54

DC Core

 Without layer 2 link between Metro Switches there is a loop Each side has a “U” shape with Metro and Agg switches Broadcast storms.

Broadcast, Multicast, Unknown Unicast

Trang 55

DC Core

Server Farm

Single L2 MST Bridge

Root Bridge in West DC

for all VLANs that Go

Between Data Centers

Root Bridge in East DC for all VLANs that Go Between Data Centers

Trang 56

 MST (802.1s) chosen to present Metro Cores as single

bridge

 Red Layer 2 link is access port channel with a VLAN that

represents the MST0 instance to make the MST group

 MST bridge priority set to 0 (Metro Core will be root of

Inter-DC VLANs)

 Spanning tree root-guard enabled on Metro Cores toward

aggregation switches (protects in case the red MST link

fails)

 Only inter-DC VLANs allowed on trunks to/from

aggregation switches

 Set spanning-tree vlan cost to set the priorities on the agg

switches links to metro core – will allow us to put some

VLANs on upper Metro Core, some on Lower by default

Single L2 MST Bridge

Định dạng
Số trang	140
Dung lượng	2,71 MB