Student Guide - IES-421 Sun Fire™ 15K Server Maintenance and Administration - Slide

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.Sun, Sun Microsystems, the Sun Logo, OpenBoot, Solaris, Solaris Operating Environment, Sun

Sun Fire™ 15K Server Administration

ES-421

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Sun, Sun Microsystems, the Sun Logo, OpenBoot, Solaris, Solaris Operating Environment, SunATM, SunFastEthernet, Sun Fire, Sun Fireplane interconnect, Sun Management Center, Sun Quad FastEthernet, SunSolve, SunSolve Online, SunSpectrum, Sun StorEdge, A1000, Sun StorEdge A5000, Sun StorEdge D240, Sun StorEdge D1000, Sun StorEdge L200, Sun StorEdge L400, Sun StorEdge L700, Sun StorEdge L3500, Sun StorEdge L11000 Tape Library, Sun StorEdge Multipack, Sun StorEdge T3, Sun StorEdge UniPack, and SunVTS are trademarks or registered trademarks of Sun Microsystems, Inc in the U.S and other countries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc in the U.S and other countries Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

UNIX is a registered trademark in the U.S and other countries, exclusively licensed through X/Open Company, Ltd.

The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc for its users and licensees Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.

U.S Government approval might be required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY

OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BE USED FOR REFERENCE PURPOSES IN CONJUNCTION WITH THE ILT COURSE THE MANUAL IS NOT A STANDALONE TRAINING TOOL USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOT RECOMMENDED.

Sun, Sun Microsystems, the Sun Logo, OpenBoot, Solaris, Solaris Operating Environment, SunATM, SunFastEthernet, Sun Fire, Sun Fireplane interconnect, Sun Management Center, Sun Quad FastEthernet, SunSolve, SunSolve Online, SunSpectrum, Sun StorEdge, A1000, Sun StorEdge A5000, Sun StorEdge D240, Sun StorEdge D1000, Sun StorEdge L200, Sun StorEdge L400, Sun StorEdge L700, Sun StorEdge L3500, Sun StorEdge L11000 Tape Library, Sun StorEdge Multipack, Sun StorEdge T3, Sun StorEdge UniPack, and SunVTS sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc aux Etats-Unis et dans d’autres pays.

Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc aux Etats-Unis et dans d’autres pays Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.

UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.

L’interfaces d’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc pour ses utilisateurs et licenciés Sun reconnaît les efforts de pionniers de Xerox pour larecherche et le développement du concept des interfaces d’utilisation visuelle ou graphique pour l’industrie de l’informatique Sun détient une licence non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licence couvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui en outre se conforment aux licences écrites de Sun.

L’accord du gouvernement américain est requis avant l’exportation du produit.

LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

CE MANUEL DE RÉFÉRENCE DOIT ÊTRE UTILISÉ DANS LE CADRE D’UN COURS DE

FORMATION DIRIGÉ PAR UN INSTRUCTEUR (ILT) IL NE S’AGIT PAS D’UN OUTIL DE

FORMATION INDÉPENDANT NOUS VOUS DÉCONSEILLONS DE L’UTILISER DANS LE CADRE D’UNE

AUTO-FORMATION.

About This Course

Course Goals

Upon completion of this course, you should be able to:

• Describe the Sun Fire 15K server physical architecture

• Configure the Sun Fire 15K server platform and

domains

• Perform platform administration on the Sun Fire 15Kserver

Course Map

Sun Fire Architecture

Sun Fire 15K Platform Configuration

Introducing the Sun Fire 15K Server

Sun Fire 15K Server Architecture

Sun Fire 15K System

Controller Architecture

Configuring the Sun fire 15K Platform

Configuring the

Multipathing I/O

Management

Dynamic Reconfiguration

Troubleshooting

Sun Fire 15K Server Administration and Troubleshooting

Sun Management Center 3.0 on the

Sun Fire 15K Servers

Sun Fire 15K Domains

Sun Fire 15K Server

Topics Not Covered

• Solaris administration – Covered in SA-118:

Fundamentals of Solaris™ 8 Operating Environment for System Administrators

• Storage administration – Covered in ES-255: Sun™

Storage System Administration

• Sun Fire 3800, 4800, 4810 and 6800 server

administration – Covered in ES-420: Sun Fire™

Workgroup/Enterprise Server Administration

How to Use the Icons

Additional resourcesDemonstration

DiscussionNote

CautionCaution - Electrical

Typographical Conventions

• Courier is used for the names of commands, files,

directories, programming code, programmingconstructs, and on-screen computer output

• Courier bold is used for characters and numbersthat you type, and for each line of programming codethat is referenced in a textual description

• Courier italic is used for variables and line placeholders that are replaced with a real name orvalue

command-• Courier italic bold is used to represent variableswhose values are to be entered by the student as part of

an activity

Typographical Conventions

• Palatino italic is used for book titles, new words or

terms, or words that are emphasized

Module 1

Introducing the Sun Fire™ 15K Server

• List the functional goals of the Sun Fire™ 15K server

• Describe how the Sun Fire 15K compares with the otherSun Fire™ servers and their predecessors

• Describe the key features of the Sun Fire 15K server

• Describe some of the advanced reliability, availability,and serviceability (RAS) features of the Sun Fire 15Kserver

• Which Sun Fire server models are available?

• What are the key features of the Sun Fire 15K server?

• How does the Sun Fire 15K server differ from other

members of the Sun Fire family?

Introducing the Sun Fire™ 15K Server

Scaling Between the Sun Enterprise 10000

and the Sun Fire 15K Servers

System bisection bandwidth (GB/s)

Memory capacity (GB)

0% 25% 50% 75% 100%

✖

Axis tick marks:

The Sun Fire Systems

Sun

Sun

Sun

Sun Fire™ 6800 cabinet

6 CPU/memory boards

4 I/O boards

Sun Fire 15K cabinet

18 CPU/memory boards

18 I/O boards

CPU/memory board

3 CPU/memory boards

Sun Fire™ 3800 rack mount

2 CPU/memory boards

2 I/O boards

65"

56"

PCI I/O board

CPU/Mem boards (9 more

(4 I/O in back)

I/O CPU/Mem

boards

CPU/Mem boards

I/O

P

P M M P

P M M

I I

System Configurations

Sun Fire™

3800 Server

Sun Fire™

48x0 Server

Sun Fire 4810 Server

Sun Fire™

6800 Server

Sun Fire 15K Server

Gbytes

576 Gbytes

PCI and hot-swap cPCI

swap PCI

Hot-PCI slots per

Desk- mount

Rack-Sun Fire™

6800 cabinet

Sun Fire 15K cabinet Room in

Fire™

48x0 Server

Fire 4810 Server

Fire™

6800 Server

Fire 15K Server

* The additional processors are provided through the

MaxCPU board The MaxCPU board installs in place of an I/O board and provides two additional CPUs (but no

additional memory) The total of 106 processors assumes the installation of 17 MaxCPU boards.

Sun Fire™

3800 Server

Sun Fire™

48x0 Server

Sun Fire 4810 Server

Sun Fire™

6800 Server

Sun Fire 15K Server

System Cabinet – Front View

4 fan trays (7 fans each) Top cap and extension

Slot 0 (top) CPU boards

4 fan trays (7 fans each)

Circuit breakers 4K watt AC

to 48 VDC power supplies (3 per side)

75.5 in (1917.7 mm) high

Slot 1(bottom) boards

Remote power control to I/O expansion

(hsPCI, link, MaxCPU)

Air inlet (DVD ROM, tape drive,

System Cabinet – Back View

4 fan trays (7 fans each) Top cap and extension

Slot 0 (top) CPU boards

4 fan trays (7 fans each)

Circuit breakers

4K watt AC

to 48 VDC power supplies (3 per side)

75.5 in (1917.7 mm) high

Slot 1(bottom) boards

Remote power control to I/O expansion

(hsPCI, link, MaxCPU)

Air inlet (DVD ROM, tape drive,

Sun Fire 15K Server I/O Support

• The Sun Fire 15K server supports standard hot-swapPCI (hsPCI) cards

• Four PCI cards are supported by each of the 18 possiblePCI assemblies

• The PCI boards are installed in special hsPCI cassettes

• You can power on and power off and remove each

hsPCI cassette individually

• You can remove and replace an entire Slot 1 board

though Dynamic Reconfiguration

• The Sun Fire 15K server supports Interdomain

Networking (IDN)

Supported I/O Adapters

• Sun Quad FastEthernet™ PCI adapter

• Sun™ Gigabit Ethernet PCI adapter 2.0 (GEM)

• Sun™ GigaSwift Ethernet UTP PCI adapter (Cassini)

• DualChannel Differential UltraSCSI PCI Host adapter(UD2S)

• SunATM™ PCI adapter 4.0 (155 Mbps with multimodefiber interface)

• SunATM™ PCI adapter 4.0 (155 Mbps with CAT 5UTPinterface)

Supported I/O Adapters

• SunATM™ PCI adapter 4.0 (622 Mbps with multimodefiber interface)

• Dual Fibre Channel PCI Network adapter (Crystal+)

• Single Fibre Channel PCI Network adapter (Amber)

• Single-ended SCSI and Fast Ethernet Combination

adapter (Fresh Choice)

• Sun™ Dual Fast Ethernet and Dual Ultra-SCSI PCI

adapter (Cauldron)

Supported Disk Arrays

• Sun StorEdge™ 99XX

• Sun StorEdge™ T3 (bootable)

• Sun StorEdge™ A5200

• Sun StorEdge™ S1 (bootable)

• Sun StorEdge™ D1000

Supported Backup Devices

• Sun StorEdge™ L9

• Sun StorEdge™ L180

• StorEdge™ L700

Sun Fire Server Advanced RAS Features

• Hardware redundancy

• Redundant system clock (auto-failover)

• Power component and power grid redundancy

• Enhanced fault detection self-diagnostics

• Concurrent maintenance

• Online CPU and memory upgrade

• Improved serviceability

• Cluster support

Sun Fire 15K Server Specific RAS Features

• Automatic system reconfiguration

• Enhanced use of ECC and error recovery in the

UltraSPARC III+ CPUs

• Internal Ethernet network to monitor all system

boards, connecting through the Slot 1 boards

• Improved operational and platform management

Check Your Progress

• List the functional goals of the Sun Fire 15K server

• Describe how the Sun Fire 15K compares with the otherSun Fire servers and their predecessors

• Describe the key features of the Sun Fire 15K server

• Describe some of the advanced reliability, availability,and serviceability (RAS) features of the Sun Fire 15Kserver

Module 2

Sun Fire™ 15K Server Architecture

• Describe the Sun Fire™ 15K server architecture

• Identify field-replaceable units (FRUs) by location andfunction

• Power on and power off the Sun Fire 15K platform

• Where are the components located?

• What status indicators are associated with each

component?

Architecture Overview

System controller OpenBoot PROM Environment (OE) Applications

Platform hardware

Solaris™

Operating

environment

The Sun Fire 15K Bus Architecture

The Sun Fire 15K has a number of internal buses The two

major buses are the address bus and the data bus

• Address Bus:

• Each board set has a snoopy address bus, with acoherency bandwidth of 150 million snoops persecond

• The 18x18 centerplane address and responsecrossbars have a peak bandwidth of 1.3 billionrequests and 1.3 billion responses per second

The Sun Fire 15K Bus Architecture

• Each expander board provides a 3x3 crossbarbetween its centerplane port and two system boards

• The 18x18 centerplane data crossbar has a totalbandwidth of 43 Gbytes per second, with a 4.8-Gbytes per second port to each of the 18 board sets

Interconnect Levels

C/M C/M C/M C/M

C/M C/M C/M C/M I/O I/O

I/O I/O

Center plane

Level 0

The Address Interconnect

Address repeater Address repeater

System address controller L2 address repeater

18 x 18 address crossbars and 18 x 18 response crossbar

Snoopy coherency

Point-to-point coherency

Proc Proc Proc Proc controllerPCI controllerLink

Slot-0 board type:

4 CPUs/4 memory units

Slot-1 board type:

2 PCI controllers

Data Interconnect

0.4 GB/s

Link controller

Data switch

Memory

Dual proc data switch

PCI card

Slot-0 board type:

4 CPUs/4 Memory units

Slot-1 board types:

2 PCI controllers

1 PCI /1 link controller

2 CPUs

“GB/s” numbers are peak band- widths at each part

Optics card

Optics card

0.8 GB/s each way

Memory

Dual proc data switch

Memory Coherency

The Sun Fire 15K supports two levels of memory coherence:

• Snoopy Coherency:

Implemented at the board set level When a device

on the bus needs data, it broadcasts a request to thebus Every device on the bus has review the request,

to determine whether the requested data may be intheir cache

• Scalable Shared Memory (SSM):

Implemented between board sets When a device in

a difference board set needs data, an intermediatecoherence directory cache (CDC) is used to maintaincache coherency

System Components

Control board set

Bulk power

Sun Fireplane

System board set (9 each per side)

Fan trays

Fan trays (4 each)

Fan backplane Sun Fireplane

Power centerplane Fan backplane

Address and data ASICs

Address and

Slot 0 boards:

Slot 1 boards:

4 CPUs and Ecaches

32 DIMMs

4 PCI cards in hot-swap cassettes

Board Sets

System control carrier plate

System carrier plate and expander

Expander board

hsPCI

Sun Fire link

System control peripheral board (DVD ROM, tape

Fan Tray (8 each) (upper two trays and lower

Lower fan backplane

Power centerplane

MaxCPU CPU/memory

trays not shown for clarity)

(8 each) Centerplane support board

Control board Control

System board set

board set

Power centerplane Logic centerplane

System Board Set

A system board set is a combination of three boards, an

expander board, a Slot 0 board, and a Slot 1 board

• One type of Slot 0 board, the system board, is used

across the Sun Fire server family

• There are three types of the Slot 1 boards, all of whichare unique to the Sun Fire 15K server:

• The hsPCI board

• The Sun Fire™ link board

• The MaxCPU board

System Board Set Example

2.4 GB/s

4.8 GB/s

To centerplane Address crossbar

System address

controller

Data switch

Memory Processor

and Ecache

Memory Processor

and Ecache Data path

controller

Memory

Processor and Ecache

Memory Processor

and Ecache

Bootbus controller

Bootbus controller Slot 0

Data switch

Data path controller

PCI bridge

PCI bridge

Bootbus controller

Address repeater

2.4 GB/s

2.4 GB/s

2.4 GB/s

4.8 GB/s

2.4 GB/s 2.4 GB/s

1.2 GB/s

Address repeater

System data interface

Bootbus

controller

2.4 GB/s

CPU/memory board Expander board

PCI board

Slot-1 Dual processor

data switch