NCAR design proposal

Cisco Systems Advanced Services National Center for Atmospheric Research Campus Design Review And Best Practices Recommendation Version 1.2 Corporate Headquarters Cisco Systems, Inc.

Cisco Systems Advanced Services

National Center for Atmospheric Research Campus Design Review

And Best Practices Recommendation

Version 1.2

Corporate Headquarters

Cisco Systems, Inc

170 West Tasman Drive

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY

The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15

of the FCC rules These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense

The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy If it is not installed in accordance with Cisco’s installation instructions, it may cause interference with radio and television reception This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules These specifications are designed to provide reasonable protection against such interference in a residential installation However, there is no guarantee that interference will not occur in a particular installation

You can determine whether your equipment is causing interference by turning it off If the interference stops, it was probably caused by the Cisco equipment or one of its peripheral devices If the equipment causes interference to radio or television reception, try to correct the interference by using one or more of the following measures:

Turn the television or radio antenna until the interference stops

Move the equipment to one side or the other of the television or radio

Move the equipment farther away from the television or radio

Plug the equipment into an outlet that is on a different circuit from the television or radio (That is, make certain the equipment and the television or radio are on circuits controlled by different circuit breakers or fuses.)

Modifications to this product not authorized by Cisco Systems, Inc could void the FCC approval and negate your authority to operate the product

The following third-party software may be included with your product and will be subject to the software license agreement:

CiscoWorks software and documentation are based in part on HP OpenView under license from the Hewlett-Packard Company HP OpenView is a trademark of the Hewlett-Packard Company Copyright  1992, 1993 Hewlett-Packard Company

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version

of the UNIX operating system All rights reserved Copyright  1981, Regents of the University of California

Network Time Protocol (NTP) Copyright  1992, David L Mills The University of Delaware makes no representations about the suitability of this software for any purpose Point-to-Point Protocol Copyright  1989, Carnegie-Mellon University All rights reserved The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission

The Cisco implementation of TN3270 is an adaptation of the TN3270, curses, and termcap programs developed by the University of California, Berkeley (UCB) as part of the UCB’s public domain version of the UNIX operating system All rights reserved Copyright  1981-1988, Regents of the University of California

Cisco incorporates Fastmac and TrueView software and the RingRunner chip in some Token Ring products Fastmac software is licensed to Cisco by Madge Networks Limited, and the RingRunner chip is licensed to Cisco by Madge NV Fastmac, RingRunner, and TrueView are trademarks and in some jurisdictions registered trademarks of Madge Networks Limited Copyright  1995, Madge Networks Limited All rights reserved

Xremote is a trademark of Network Computing Devices, Inc Copyright  1989, Network Computing Devices, Inc., Mountain View, California NCD makes no representations about the suitability of this software for any purpose

The X Window System is a trademark of the X Consortium, Cambridge, Massachusetts All rights reserved

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PRACTICAL PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE

PRACTICE

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS

SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES

AccessPath, AtmDirector, Browse with Me, CCDE, CCIP, CCSI, CD-PAC, CiscoLink, the Cisco NetWorks logo, the Cisco Powered Network logo, Cisco Systems Networking Academy,

Fast Step, Follow Me Browsing, FormShare, FrameShare, GigaStack, IGX, Internet Quotient, IP/VC, iQ Breakthrough, iQ Expertise, iQ FastTrack, the iQ logo, iQ Net Readiness

Scorecard, MGX, the Networkers logo, Packet, RateMUX, ScriptBuilder, ScriptShare, SlideCast, SMARTnet, TransPath, Unity, Voice LAN, Wavelength Router, and WebViewer are

trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, Discover All That’s Possible, and Empowering the Internet Generation, are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert Logo, Cisco IOS, the Cisco IOS logo, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Enterprise/Solver, EtherChannel, EtherSwitch, FastHub, FastSwitch, IOS, IP/TV, LightStream, MICA, Network Registrar, PIX, Post-Routing, Pre-Routing, Registrar, StrataView Plus, Stratm, SwitchProbe, TeleRouter, and VCO are registered trademarks of Cisco Systems, Inc and/or its affiliates in the U.S and certain other countries

All other trademarks mentioned in this document or Web site are the property of their respective owners The use of the word partner does not imply a partnership relationship between Cisco and any other company (0105R)

INTELLECTUAL PROPERTY RIGHTS:

THIS DOCUMENT CONTAINS VALUABLE TRADE SECRETS AND CONFIDENTIAL INFORMATION OF CISCO SYSTEMS, INC AND IT’S SUPPLIERS, AND SHALL NOT

BE DISCLOSED TO ANY PERSON, ORGANIZATION, OR ENTITY UNLESS SUCH DISCLOSURE IS SUBJECT TO THE PROVISIONS OF A WRITTEN NON-DISCLOSURE AND PROPRIETARY RIGHTS AGREEMENT OR INTELLECTUAL PROPERTY LICENSE AGREEMENT APPROVED BY CISCO SYSTEMS, INC THE DISTRIBUTION OF THIS DOCUMENT DOES NOT GRANT ANY LICENSE IN OR RIGHTS, IN WHOLE OR IN PART, TO THE CONTENT, THE PRODUCT(S), TECHNOLOGY OF

INTELLECTUAL PROPERTY DESCRIBED HEREIN

Design Review Template

Document Control

Author: Hazim Dahir

Advanced Services – Central Engineering

History

Table 1 Revision History

1.0 20-Feb-2003 Draft, Released First release

1.2 24-Apr-2003 Final Updated Diagrams and added Configuration observations

Review

Table 2 Revision Review

Tsegerada Beyen and Niels Brunsgaard

Change Forecast: High

This document will be kept under revision control

A printed copy of this document is considered uncontrolled

About The Document

Document Purpose

This network design review document is intended to provide an overall assessment of the design aspects of the network and select operational functions The comments presented in this document are a result of information learned about the network from customer-documentation as well as the weekly discussions

This assessment is part of the Performance Engineering and Optimization services provided by the

Central Engineering Team This service will give a best practice assessment of the network as a complete system It uses data collected about individual devices or interfaces to generate an assessment of “Campus

Best Practices” This assessment would consider network Availability, Scalability, Convergence, Modularity, Hierarchical Design and other network stability aspects

Business Profile

Understanding the business goals of a company or institution is very important when analyzing a network design The goal of a good network design is to empower users in meeting company objectives The network should provide an acceptable level of performance and reliability while not wasting capital and other resources in the process of over-engineering the network Nor should a network be under-engineered such that it fails to meet the service levels necessary to meet the business objectives Many design

decisions are a result of thoughtful risk/benefit analysis

The National Center for Atmospheric Research, NCAR, was established in 1960 to serve as a focus for research on atmospheric and related science problems and is recognized for its scientific contributions to our understanding of the earth system, including climate change, changes in atmospheric composition, Earth-Sun interactions, weather formation and forecasting, and the impacts of all of these components on human societies

With two major sites in Boulder, I.M Pei's Mesa Laboratory and a newer Foothills Laboratory, NCAR's research is conducted in several principal disciplinary areas: atmospheric chemistry; mesoscale and microscale meteorology; solar and solar-terrestrial physics; and climate and the linking of climate with other environmental systems Focused contributions are also made to national scientific initiatives There are multi-disciplinary and cross-disciplinary efforts aimed at the development of a coupled climate system model which will simulate the complex interrelations between climate, weather, the sun, and the biosphere and oceans Research on the societal interactions with atmospheric processes is an integral part of NCAR's program

About The Document

Current Topology

ml-mr-c1-g s

ml-y2k-c1-as ml-mr-c1-as

fl4-2060-c1-gs

uv-18-c1-es

ps-3018-c1-es

es

ps-1027a-c1-ps-2008-c1-es ps-3018-c1-ts

cg2-mr-c1-gs

cg1-2010-c4-es cg1-3036-c1-gs

cg2-mr-c1-ts cg1-2036-c1-gs

cg-voipr

jef-126-c1-as

jef -126-c1-ts jef-126-c1-es

cg1-2010-c2-es

cg1-3010-c2-es cg1-2010-c1-es

cg1-3010-c1-es

cg1-2010-c3-e s

Pe arl Stre e t UNAVCO

A TM links Gigabit Ethernet links

A TM

Sw it ch

POTS

gin

Current Design Overview

The current design spreads over three major campuses The largest and most populated are Mesa and Foothills Center Green utilizes an L2 switch as an aggregation point

The existing design facilitates for several VLANs to span multiple switches as well as multiple sites Although this is not recommended, at NCAR this does not present any immediate problems or issues The single homing of switches to the perspective core switch and the absence of a Spanning Tree loop at the core provide for a stable environment

At the current time, NCAR is satisfied with the current “availability” model and hope to improve it in the future For example, the Mesa site, acts as a transport site for all traffic exchanged between CenterGreen

About The Document

and Foothills A total failure of the “ml-mr-c1-gs” switch will isolate all three major sites Relying on internal redundancy (Dual-Supervisor and HA feature) helps reduce the chance of that type of failure.

Overview of Recommendations

Executive Summary

The advent of high-speed L3 switches has moved modern Enterprise/Campus Network Design away from the flat L2 vlan-based model Cisco’s current Campus Reference Design Model, commonly known as the Multilayer Model, features high-powered L3 switches placed in key areas of the Enterprise Network

This document concentrates on key design concepts required for mission critical networks The most important ones are:

- Hierarchical Network Model: Characterization of traffic flow

- Modularity: Network made up of distinct network blocks

- Scalability: Allow network to grow without major changes or redesign

- High Availability: Internal, External, and path redundancy

- Predictability: Traffic Flows, delays, bounds, fail-over paths are predictable

- Simplicity: Satisfy network requirements with the least amount of effort or Hardware

The following sections attempt to describe two design improvement approaches:

1 Short Term Design Enhancement

2 Long Term AS Proposed Design

Short Term Design Enhancements

Foothills-CenterGreen: The Radio Link

NCAR is testing a Radio Link (TeraBeam) for possible deployment into the production environment to connect Foothills with CenterGreen If we allow the Radio Link to act a trunk, then we are creating an environment that is STP dependent for convergence That in return will force one of the Core links to be in the “Blocking” state Reliability and Utilization common sense force us to “block” the Radio Link

All Links (including the Radio Link) are better utilized in an L3 Core With the three major sites

participating, this would be a full mesh Mesa will no longer be the only link between Foothills and CenterGreen

Mixed Core (L2 + L3)

The presence of several campus-wide VLANs requires Trunking (ISL or dot1Q) in the Core Those VLANs would be handled by two trunks connecting the three sites This is exactly how all traffic is handled today

Other VLANs that are unique to an L2 switch or to one of the sites will be cleared from the L2 trunk and can be routed via a separate L3 connection This is best described by the following diagram

By adding a routing engine to the CenterGreen switch, three unique VLANs can be configured to represent the L3 core The other L2 will only carry traffic for the VLANs requiring campus-wide configuration (all other VLANs must be cleared from the trunk)

An important decision needs to be taken here regarding the Active gateway(s) for the ‘L2” VLANs Any two routers in any of the three sites can handle that requirement We can also consider M-HSRP and have one router active for half the VLANs and another router active for the other half (Point of Discussion: This document will updated accordingly)

Error! Reference source not found

Mixed Core Design

- The L3 Core consists of the three independent point-to-point VLANs X, Y, and Z

- The L2 trunk illustrated by the blue lines will carry VLANs that need site-wide accessibility

- VLANs X, Y, and Z to be cleared from the dot1Q trunk

- All Site-specific VLANs to be cleared from the dot1Q trunks

Hierarchical (MultiLayer) Network

Design: An Overview

Overview

The hierarchical three tiered campus design has become the preferred architecture for most networks The three tiered architecture is comprised of an access layer that directly connect network users by means of switches, normally placed in wiring closets positioned throughout the campus Access layer switches are also connected to

a distribution layer The distribution layer sites will have a number of access switches connected to it The number of access switches connected to the distribution layer is often determined by geographic proximity, such

as all the access switches in a building homing into one distribution site for that building The distribution sites normally consist of switches with layer 2 and layer 3 functions The distribution layer switches are usually deployed in pairs for system redundancy The distribution layer switches are then connected to a core layer switch The core switches are also usually deployed in pairs for redundancy and may support layer 3 as well as layer 2 functions An overall campus design such as this might have a numerical profile of 8000 users connected

to 40 access switches that are then connected to 4 distribution sites that are finally connected to 1 core site