Student Guide - Sun StorageTek 6000 Installation and Configuration

Course objectives: After completing this course, you will be able to: Recognize Sun 6000 modular product line components Install Common Array Manager software Configure Sun 6000 modular

Trang 1

Sun Storage 6540 product overview xiii

Sun Storage 6140 product overview xiii

Sun Storage CSM200 expansion module overview xiii

Sun Storage hardware installation xiv

Sun Storage Common Array Manager xiv

Array configuration using Sun Storage Common Array Manager xv

Storage Domains xv

Integrated data services: Snapshot xvi

Integrated data services: Volume Copy xvi

Integrated data services: Remote Replication xvi

Monitoring performance and dynamic features xvi

Problem determination xvii

M am tamm" • g e s rage array xvn th to

SSCS and Command Line Interface xvii

Preface: About tbis course 1

Course goals 1

Sun Storage 6x80 product overview 3

Objectives 3

Sun Storage modular disk family positioning 4

The mid-range family 4

Compare the Sun Storage 6140, 6540, and 6x80 arrays 5

High performance computing with the 6x80 6

6x80 controller module overview 7

6x80 controller module: Front view 8

6x80 controller module power-fan canister 9

Interconnect-battery canister 11

6x80 controller module: Power distribution 15

6x80 controller: Inside view 17

Cache DIMM memory 17

USB persistent cache 18

Host cards 19

Controller base board 19

6x80 controller module: Back view 20

Host ports 21

Drive ports 21

Other ports 22

Controller LEDs 23

6x80 summary 28

Knowledge check 31

Sun Storage 6540 Product Overview 33

Objectives 33

Recycle

~-Adobe PostSaipt

Trang 2

Controller Module 37

Power-fan canister 39

Interconnect-battery canister 42

Power distribution and battery system 46

6540 controller canister highlights 48

6540 controller canister 49

6540 4Gb/s host interface ports 51

6540 4Gb/s disk expansion ports 52

6540 drive channels and loop switches 54

Dual 10/100 Base-T Ethernet ports with EEPROM 55

Serial port connector 56

Seven segment display 57

Controller service indicators 59

6540 summary 62

Knowledge check 64

Sun Storage 6140 product overview •.••.••.••.•.••.••.••.••.••.••.••.••.••.••.• 67 Objectives 67

Sun Storage 6140 product overview 68

Compare the Sun StorEdgeTM 6130 and the Sun Storage 6140 Arrays 69

Hardware components of the Sun Storage 6140 70

Hardware overview 72

Controller module 72

DACstore 75

Back view of controller module 82

6140 controller module details 83

The 6140 controller canister 84

Battery 91

The power-fan canister 93

Controller architecture 95

6140 summary 95

Knowledge check 97

Sun Storage CSM200 expansion module overview 99

Objectives 99

Sun Storage CSM200 expansion module overview 100

Hardware overview 100

Hardware components of the Sun Storage 6x80 and 6540 100

CSM200 expansion module 101

CSM200 expansion module- Front view 102

DACstore 104

CSM200 expansion module - Back view 111

Architecture overview 119

Trang 3

~-Knowledge check 121

Sun Storage 6000 hardware installation 123

Objectives 123

Overview of the installation process 124

Cabling procedures 125

Cable types 125

Recommended cabling practices 127

Cabling for redundancy- Top-down-bottom-up 128

Cabling for performance 129

Hot-adding an expansion module 131

Cabling summary 134

Recommended cabling practices for the 6x80 135

Recommended cabling practices for the 6540 and 6140 137

Considerations for drive channel speed 145

Proper power procedures 146

Turning on the power 146

Turning off the power 148

Set the controller IP addresses 149

Configuring dynamic IP addressing 149

Configuring static IP addressing 149

Serial port service interface 150

Serial port recovery interface procedure 151

Use the hardware compatibility matrix to verify SAN components 152

Attach the host interface cables 153

Host cabling for redundancy 153

Connecting data hosts directly 154

Connecting data hosts through an external FC switch 154

Hardware installation summary 155

Knowledge check 156

Sun Storage Common Array Manager 159

Objectives 159

What is Sun Storage Common Array Manager? 160

The CAM interface 162

SMI-S overview 162

Software components 164

Sun Storage Management host software 164

CAM management methods 165

Out-of-band management method 165

In-band management method 167

Sun Storage data host software 169

Host Bus Adapter (HBA): Compatibility and configuration 170

Recycle

Trang 4

Firmware and NVSRAM files 176

Sun Storage Common Array Manager navigation 177

Common Array Manager banner 178

Common Array Manager's navigation tree 179

Common Array Manager's content area 180

Additional navigation aids 180

Administration functions and parameters 182

Accessing the managrnent software 182

Auto Service Request (ASR) 182

Initial Common Array Manager configuration 183

Configure IP addressing 184

Naming an array 184

Configuring the array password 185

Setting the array time 185

Default host type 185

Adding additional users 186

Setting module IDs 186

Common Array Manager summary 186

Knowledge check 188

Array configuration using Sun Storage Common Array Manager 189

Objectives 189

Common Array Manager configuration components 190

Creating a volume with Common Array Manager 192

Storage profiles 193

Storage pools 197

Volumes 197

Volume configuration preparation 198

Volume parameters 199

Virtual Disks 20 1 Administration functions and parameters 202

Auto Service Request (ASR) 202

Array name 203

Default host type 204

Hot spares 204

Storage array cache settings 205

Disk Scrubbing 206

Failover alert delay 206

Array time 206

Manage passwords 207

Array configuration summary 207

Knowledge check 209

Trang 5

~-What are Storage Domains? 212

Storage Domains benefits (pre-sales) 213

Storage Domains benefits (technical) 214

Storage Domain terminology 215

Steps for creating a Storage Domain 218

How Storage Domains work 220

What the host sees 222

What the storage array sees 223

Storage Domains - How many domains are required? 224

LUNS - How do you number these LUNS? 225

Summary of creating Storage Domains 225

Storage Domains summary 226

Knowledge check 227

Integrated data services - Snapshot •.••.•.••.••.••.••.••.••.••.••.••.••.••.••.• 231 Objectives 231

Data services overview 232

Snapshot 233

Snapshot terminology 233

Snapshot - Benefits 23 7 Pre-Sales benefits 237

Technical benefits 238

How does Snapshot work? 239

Examples of how Snapshot works 241

Disabling and recreating 249

Snapshot considerations 250

Creating Snapshots 251

Creating a Snapshot 251

Calculating Reserve Volume capacity 253

Creating a Snapshot 254

Snapshot summary 254

Knowledge check 255

Integrated data services - Volume Copy ••.••.••.•.••.••.••.••.••.••.••.••.••.• 257 Objectives 257

Volume Copy overview 258

Volume Copy terminology 259

Volume Copy- Benefits (pre-sales) 261

Volume Copy- Benefits (technical) 263

How Volume Copy works 264

Factors affecting Volume Copy 265

Volume Copy states 265

Volume Copy- Read/write restrictions 267

Recycle

~-Adobe PostSaipt

Trang 6

Recopying a volume 268

Stopping a Volume Copy 269

Removing Copy Pairs 270

Changing Copy priority 270

Volume permissions 271

Volume Copy compatibility with other data services 271

Storage domains 272

Snapshot 272

Remote Replication 273

Configuring a Volume Copy 274

Configuring a Volume Copy with Common Array Manager 274

Enabling the Volume Copy feature 275

Creating a Volume Copy 276

Recopying a Volume Copy 277

Changing the copy priority 277

Stopping a Volume Copy 278

Removing Copy Pairs 279

Volume permissions 280

Volume Copy sununary 280

Knowledge check 282

Integrated data services - Remote Replication 283

Objectives 283

Remote Replication overview 284

Benefits of Remote Replication 285

Remote Replication terminology 286

Sununary of Remote Replication modes 294

Technical features of Remote Replication 295

Remote replication distances 296

Configuring data replication with CAM 297

Activating and deactivating data replication 298

Disabling data replication 299

Configuring the hardware for data replication 300

Setup the hardware 300

Creating replication sets 302

What happens when an error occurs? 305

Suspend and resume 307

Role reversal 309

Changing replication modes 310

Testing replication sets 311

Removing a mirror relationship 311

Remote Replication sununary 312

Trang 7

~-Objectives 315

First principle of storage array performance 316

40/30/30 rule 317

Context for performance tuning 318

Analyzing I/0 characteristics 319

Factors that affect storage array performance 320

Cabling 320

Choosing a disk type 322

Selecting a RAID level 323

Number of spindles in a v-disk 326

Calculating an optimal segment size 329

Cache parameters 331

Read Caching Pre-fetch enabled 332

Enabling write caching and enabling write caching with mirroring 332

Number of volumes in a virtual disk 333

Choosing an optimal volume modification priority 333

Setting array-wide global parameters 334

Performance Monitor 336

The Performance Monitor pages 336

Fine tuning 336

Performance and dynamic features summary 339

Knowledge check 340

Problem determination 343

Objectives 343

Problem determination 344

Utilizing the tools available for problem determination 344

Visual Cues 344

Compatibility matrix 345

Problems and recovery 345

Service Advisor 346

Collect support data through the command line 348

Support Data bundle 348

Fault Management Service (FMS) 350

Alarms 351

FRU- Field Replaceable Units 356

Events 357

Array administration 3 58 Health administration 359

Notification 360

Activity log 361

Problem determination summary 362

Recycle

Trang 8

Objectives 365

Dynamic volume expansion (DVE) 366

Disk scrubbing 367

Installing baseline firmware 368

Upgrading to 7.xx firmware 369

Command line firmware upgrade utility 369

Maintaining the storage array summary 370

Knowledge check 371

SSCS and Command Line Interface 373

Objectives 373

Sun Storage Common Array Manager CLI (SSCS) 374

Features 374

Benefits 3 7 4 Usage 374

Other useful information to collect 375

Other command line interface tools 376

Fault Management Service (ras_admin) 376

Command Service Module (csmservice) 376

Collect support data 377

Service command line 378

SSCS and CLI summary 380

Knowledge check 381

Appendix A 383

Glossary of acronyms 383

References 383

Appendices •.••.••.••.••.••.••.••.••.••.••.••.••••.••.••.••.••.••.••.••.••.• 389 Knowledge check solutions ••.••.••.••.••.••.••.••.••.••.••.••.••.••.••.••.•• 391 Sun Storage 6x80 product overview 391

Sun Storage CSM200 expansion module overview 398

Sun Storage 6000 hardware installation 399

Sun Storage Common Array Manager 402

Array configuration using Sun Storage Common Array Manager 403

Storage Domains 405

Integrated data services: Snapshot 408

Integrated data services: Volume Copy 409

Integrated data services: Remote Replication 410

Monitoring performance and dynamic features 412

Maintaining the storage array 414

Problem determination 415

Trang 9

~-© Please

Recycle

Trang 11

~ +sun ml oro~ystems

Sun Mit~) · ~ ~ uu~ In~

IJ'S RMO) l §;i

This technical training course contains information about the operation and

management of the Sun Storage 6000 modular product line of storage arrays The basic objective of this course is to familiarize individuals with the essential concepts associated with the configuration of the Common Array Manager (CAM) software and 6000 disk storage arrays

The information contained herein is derived from end-user publications and engineering data It reflects the latest information available at the time of printing but will not include modifications if they occurred after the date of publication In all cases, if there is discrepancy between this information and official publications issued by Sun, then Sun official publications should take precedence

be covered

Trang 12

Course length:

Approximately 3 days in length

Course objectives:

After completing this course, you will be able to:

Recognize Sun 6000 modular product line components Install Common Array Manager software

Configure Sun 6000 modular product line components using Common Array Manager

Explain the data services available in Common Array Manager Perform basic problem determination functions on Sun 6000 modular product line components

Course topics:

Sun Storage 6x80 Product Overview Sun Storage 6540 Product Overview Sun Storage 6140 Product Overview Sun Storage CSM200 Expansion Module Overview Sun Storage 6000 Hardware Installation

Sun Storage Common Array Manager Array Configuring using Sun Storage Common Array Manager Storage Domains

Integrated Data Service: Snapshot Integrated Data Service: Volume Copy Integrated Data Service: Volume Replication Monitoring Performance and Dynamic Functions Problem Determination

Maintaining the Storage Array SSCS and Command Line Interface

Trang 13

Course outline:

Sun Storage 6x80 product overview

Objectives: After completing this module, you will be able to:

Describe how the key features of 6x80 storage systems address the needs of high performance computing environments

• Identify the canisters of the 6x80 controller module

• Identify the upgradable components inside the controller canister Identify the LEDs of 6x80 controller modules

Sun Storage 6540 product overview

• Describe the Sun Storage 6540 key features Identify the hardware components of the 6540 controller module

• Describe the functionality of the 6540 components

• Interpret LEDs for proper parts replacement

Sun Storage 6140 product overview

• Provide an overview of the Sun Storage 6140

• Identify the hardware components of the 6140 Describe the functionality of the 6140 controller module

• Interpret LEDs for proper parts replacement

Sun Storage CSM200 expansion module overview

Describe the Sun Storage Common Storage Module (CSM)200 expansion module key features

• Identify the hardware components of the CSM200 expansion module

Sun Storage 6000 Product Line Installation and Configuration Course Description

xiii

Trang 14

Describe the functionality of the CSM200 expansion module Interpret LEDs for proper parts replacement

Sun Storage hardware installation

Knowledge Objectives: After completing this module, you will be able to: List the basic steps for installing the Sun Storage 6x80, 6540 and 6140 Describe proper cabling techniques and methodologies

List the basic steps of hot-adding CSM200 expansion modules to a 6x80,

6540 and 6140 Perform the proper power sequence for the 6x80, 6540 and 6140 storage array

Describe procedure to set static IP addresses for the 6x80, 6540 and 6140 Skill Objectives:

Cable a storage array Set a static IP address for each controller in the 6540

Sun Storage Common Array Manager

Knowledge Objectives: After completing this module, you will be able to: Describe the functionality of Common Array Manager (CAM) Differentiate management and data host install

Describe the management methods used by CAM Explain the function of a multi-path driver

Describe logging into and navigating within CAM List initial CAM configuration steps

Skill Objectives:

Install CAM Register Devices (manual and auto discovery) Set up Alert Notification

Trang 15

Array configuration using Sun Storage Common Array Manager

Knowledge Objectives: After completing this module, you will be able to:

• Describe how to provision the storage array with Common Array Manager

• Describe additional provisioning components and how they relate to volume creation

Describe the profile parameters that are selected when creating a volume Skill Objectives:

• Set Module ID's

• Enable media background media scan

Create Storage Profiles

• Create Storage Pools

• Configure available storage capacity into volumes

Select appropriate volume parameters (RAID level, cache settings, segment size, etc.)

• Configure a global hot spare

• Access the volumes on the storage array from the host

Storage Domains

Knowledge Objectives: After completing this module, you will be able to:

• Explain the benefits of Storage Domains

• Define Storage Domain terminology

Describe the functionality of Storage Domains

• Calculate Storage Domain usage

Skill Objectives:

• Create Hosts and/or Host Groups

Map volumes to Hosts and /or Host Groups

XV

Trang 16

Integrated data services: Snapshot

Knowledge Objectives: After completing this module, you will be able to: List the benefits and application of Snapshot

Explain how Snapshot is implemented Skill Objectives:

Create and use Snapshot volumes

Integrated data services: Volume Copy

Knowledge Objectives: After completing this module, you will be able to: Describe the benefits and application of Volume Copy

Explain how Volume Copy is implemented Explain the functions that can be performed on a Copy Pair Skill Objectives:

Create and use Volume Copy volumes

Integrated data services: Remote Replication

Knowledge Objectives: After completing this module, you should be able to: Describe the benefits and applications of Remote Replication

Explain how Replication is implemented Differentiate between synchronous and asynchronous replication modes Skill Objectives:

Create and use Replication volumes

Monitoring performance and dynamic features

Knowledge Objectives: After completing this module, you will be able to: List the factors that inlluence storage array performance

Trang 17

Recognize how dynamic functions impact performance

• Explain the data presented by the CAM built-in Performance Monitor Skill Objectives:

• Dynamically modify volumes using the following dynamic features:

Dynamic RAID Migration (DRM) Dynamic Volume Expansion (DYE) Dynamic Capacity Expansion (DCE) Dynamic Segment Sizing (DSS) Defragmentation

Problem determination

Describe the tools in CAM to analyze storage array problems

• Explain how to use the service advisor to solve problems

Maintaining the storage array

Describe Dynamic Volume Expansion

• Explain the benefits of disk scrubbing

• Describe the process to install baseline firmware

SSCS and Command Line Interface

• Utilize the SSCS to export and import the configuration

• Use the fault management command line tools (FMS)

xvii

Trang 19

Preface: About this course

Course goals

Upon completion of this course, you will be able to:

• Describe the features, functions and terminology of the Sun Storage 6000 Module series

• Describe the customer benefits and requirements to migrate to or use Sun Storage series arrays

• Describe the architecture of Sun Storage 6000 series arrays

• Install Sun Storage 6000 array hardware

• Install Common Array Manager storage management software

• Configure CAM-based storage systems

• Attach production hosts to Sun Storage 6000 series arrays

• Configure and use Snapshots

• Configure and use Volume Copies

• Configure and use a Replication set

• Common Array Manager (CAM)

• Diagnose basic problems using available tools

• Use common commands in the SSCS command line interface

Preface: About this course

I

Trang 21

Sun Storage 6x80 product overview

Objectives

Upon completion of this module, you will be able to:

Describe how the key features of 6x80 storage systems address the needs of high performance computing environments

Identify the canisters of the 6x80 controller module Identify the upgradable components inside the controller canister Identify the LEDs of 6x80 controller modules

1-3

Trang 22

Sun Storage modular disk family positioning

SAI~t r lc lt :,< (Op t iOna l)

8i:p Tray - CSM ;200 ; 4Gbls end-tO-elld

Conhgma li oo- 8 · 1£GB caclle { mlt 1 al re l ease }

Mgt SVV - CAM

SANtridty { opt i ooa l ) Exp Tra y - CS M 200 ; 4Gbls end - to - end Co11 fi gu r a t ion- u p to 16GB cache

s T6 1 40 Mgt SW - CAM

SANt r ic i ty opt i ona l

Exp Tra)' - CS M 200 ; 4Gbls end - to - end

Co nfi guration - 4GB cache (8)( 4Gb p orts }

2G B each~ (4-.: 4Gb p ons)

Figure 1-2 Flexline series to 6000 modular series

Trang 23

Compare the Sun Storage 6140, 6540, and 6x80

Table 1-1 Module 6000 product line comparison

Controller

On CPU On CPU Dedicated Dedicated Dedicated ZIP

1/2/4 Gb/s; 2 1/2/4 Gb/s; 4 1/2/4 Gb/s; 4 1/2/4 Gb/s; 8 1/2/4 Gb/s; per controller per controller per controller per controller 2/4/8Gb/s;

8 per controller 2/4Gb/s; 2 per 2/4Gb/s; 2 per 2/4Gb/s; 4 per 4Gb/s; 8 per 4Gb/s; 8 per controller controller controller controller controller 1GB per 2GB per 2/4/8GB per 8GB per 8/16 per controller controller controller controller controller 10/100; 2 per 10/100; 2 per 10/100; 2 per 101100/1 000; 10/100/1000; 2 controller controller controller 2per per controller

supported redundant) redundant)

Sun Storage 6x80 product overview 1-5

Trang 24

Sustained 1/0 rate 5,789 9,000 22,000 -30,000 -45,000 (disk writes)

High performance computing with the 6x80

The 6x80 controller module provides the power and speed demanded by high performance computing (HPC) environments that store and use vast amounts of data for high-bandwidth programs and complex application processing

The controller used in the 6x80 controller module is very sophisticated and uses state-of-the-art XBB2 (RAIDCore 2) architecture These factors enable the 6x80

to use fast cache memory, USB-based drives for persistent cache storage, 4Gb/s

FC host and drive ports, high-speed busses and multiple processing elements to optimize resource utilization

All CAM-managed enterprise-class controllers run similar firmware This unique implementation creates a lower total cost of ownership (TCO) and higher return

on investment (ROI) by enabling common features and functionality, centralized management, a consistent interface and reduced training and support costs

The 6x80 controller's high speed, dedicated XOR engine generates RAID parity with no performance penalty, enabling this compute-intensive task to be handled efficiently and effortlessly A separate processor focuses on data movement control, allowing setup and control instructions to be processed and dispatched independent of data

Trang 25

Two 6x80 controllers are integrated into one controller module, and combined with one or more expansion trays, create a fully featured storage system The dual controllers are fully redundant

The 6x80 controller module supports up to 16-1, 2, or 4Gb/s FC host connections and 16 4Gb/s Fibre Channel (FC) drive connections using Fibre Channel-

Switched Loop (FC-SW) protocols giving the 6x80 access to a total of 448 FC/SATA n drives Extensive compatibility and the ability to auto-negotiate host connectivity speeds result in mjnjmal or no impact on storage networks,

protecting customers' existing infrastructure investment

There are several configuration possibilities for the 6x80 controller module based

on host ports 4 and 8Gb/s FC and 1 OGb/s iSCSI as available This flexibility makes it easy for customers to purchase exactly the storage system they need

6x80 controller module overview

F r o n t

B a c k

Figure 1-3 Front and back views of 6x80 controller module

The CAM-managed 6x80 controller module is a cabinet-mounted storage system that directs and manages the 110 activity between a host and the volumes

configured on the storage system

The 6x80 shares many of the characteristics of other members of the managed storage line: it is a 4U module that fits the standard 19-inch (48.3 em) wide cabinet, has virtually the same-sized canisters in it and uses many of the same LED indicators

CAM-The 6x80 is unique, though, in some respects For example, the 6x80 power-fan canisters do not contain chargers for the batteries These have been moved into the Interconnection Battery canister with the batteries themselves

Copyrisbt2008 SIDl Microsystems, Inc AlllliptsReserved SIDlSenices, JUDe2009, R.mlion3.0

Trang 26

From the back of the controller module, it is very easy to distinguish between the 6x80 and all other CAM-managed storage systems It has eight host and eight drive ports This gives the 6x80 greater availability for host/SAN attachments, as well as expansion tray attachments In fact, the 6x80 can support more drives than any other CAM-managed array: up to 448 drives (16 fully configured SBOD expansion trays) ofFC and/or SATA II drives

lnterco nn eot B atlell'Y

Figure 1-4 6x80 canisters

The chassis dwign has · five

rep l ace-able ca n isters~

- 2 controlle r carn i sle.rs

- 2 power -f an can i sters

- 1 lnterooil1nect bait ell}' ~n i &ter

The 6x80 controller module has five FRU canisters which can be replaced site:

on-• Two controller canisters

• Two power-fan canisters (also referred to as controller support modules)

• One interconnect battery canister

The 6x80 controller module does not have a mid-plane It is designed so that all

of the canisters interconnect with one another through the interconnect canister

The 6x80 controller module has two identical controllers The controller canisters install from the rear of the module The top controller is controller A, and the bottom controller is controller B All connections to the hosts and drives are through the controllers Controllers A and B are inverted 180 degrees from each other The reason for this is so power connections are on the outside of each controller, which makes power cables more manageable

6x80 controller module: Front view

From the front of the controller module, the power-fan canisters and interconnect battery canister are accessible All of these canisters are field replaceable, which makes the 6x80 easy and fast to service

Trang 27

6x80 controller module power-fan canister

F or cont r o ll er 6

Powe r~ Fa n Canusters

Figure 1-5 6x80 power-fan canisters

Fo r cont r ol l er A

The power-fan canister (controller support module), as the name suggests,

provides power and cooling to the storage system Each power-fan canister contains:

• A 525-watt power supply that provides power to the controllers by

converting incoming AC voltage to 12-V DC

• Two cooling fans that provide redundant cooling even if either power supply fails

• A thermal sensor that prevents power supplies from overheating Under normal operating conditions with an ambient air temperature of 40° F to 104° F (5° C to 40° C), the cooling fans maintain the correct operating temperature inside the module

Factors that can cause power supplies to overheat:

• An unusually high room temperature

• A fan failure

• Defective circuitry in the power supply

• A blocked air vent

• A failure in other devices installed in the cabinet

If the internal temperature rises above 158° F (70° C), one or both power supplies automatically shuts down, and Common Array Manager software reports the error Critical event notifications also are issued if event monitoring is enabled and event notification is configured

Trang 28

A black connector at the rear of the power-fan canister connects it to its respective controller The power-fan canister on the left has the connector at the bottom and therefore, connects to controller B The power-tan canister on the right has the connector on the top and connects to controller A

Figure 1-6 Power-fan canister connection to controller

Power-fan canister LEOs

Figure 1-7 Power-fan canister LEDs

Information about the condition of the power supplies and fans is conveyed by LEDs on the front of each power-fan canister However, the LEDs are only visible if the front cover of the 6x80 controller module is removed

Typically, a one-to-one relationship exists between the Service Action Required (SAR) and Service Action Allowed (SAA) LEDs However, there are exceptions

An example is if both power-fan canisters have a fault, one due to a power fault, and the other due to a fim fault The power-fan canister with the power fault should be removed and replaced first

Trang 29

If the power-fan canister with the fan fault is removed first, the storage system is left without power This causes the storage system to shut down and possibly lose data In this scenario, the power-fan canister with the fan fault would have the SAR LED on and the SAA LED off, indicating a problem but that the canister should not be removed

Interconnect-battery canister

••• · •

~ $ & ~t :_·,

c _ =o= r

Figure 1-8 6x80 interconnect-battery canister

The interconnect-battery canister (ICC) acts like a mid-plane for the controller status lines, power distribution lines and drive channels, as well as storing the batteries that transfer data in cache to the on-board USB-based flash drives in the event of a power failure

Note -If there is an unexpected power outage, data in cache is transferred to based persistent cache flash drives These drives are considered long-term storage, so extended power outages are no longer a concern to maintaining data in cache

USB-However, there is no true mid-plane or backplane in the 6x80 The battery canister simply acts as a way for the different canisters of the controller module to interact (and access the battery packs) To do so, the ICC board connects to all components of the ICC, as well as to both controller canisters If the ICC is not present, the other canisters might still function (depending on what the problem is), so in the strictest sense of the term, the ICC is not a true mid-plane

interconnect-The interconnect-battery canister contains:

• A removable "mid-plane" that provides cross-coupled signal connection between the controller canisters The control output from each controller canister is connected to the control input of the alternate controller canister

Trang 30

• Two battery backup (BBU) packs Each battery backup pack is sealed and contains 18 cells of lithium ion batteries Each battery pack is dedicated to one controller

• An audible alarm that provides a warning of potentially serious problems with the controller module The 6x80 module may be shipped with the alarm enabled or disabled, depending on OEM specifications If it is enabled, the alarm can be muted with the mute button on the front of the canister or by using the Common Array Manager

• Front bezel LEDs that are visible through the front cover including summary LEDs and an audible alarm for the entire controller module, as well as LEDs specific to the ICC

Caution-Never remove the interconnect-battery canister unless directed to do so by

a Customer Support representative Removing the interconnect-battery canister after either a controller or a power-fan canister has been removed could result in loss of data access

Interconnect-battery canister LEOs

e u

n-eeds needs

at!enooo atten~on

Figure 1-9 Interconnect-battery canister LEOs

The power and locate LEDs, as well as the audible alarm, are general indicators for the entire controller module, not specifically for the interconnect-battery canister The Service Action Allowed (SAA), Service Action Required (SAR) and battery LEOs, however, are specifically for the interconnect-battery canister itself Both global and ICC LEDs can be seen through the front cover

In the unlikely event that an interconnect-battery canister must be replaced (e.g., due to a bent pin or as a last resort to resolve a problem), the Common Array Manager will provide details about the procedure Data access is limited to controller A when the interconnect-battery canister is removed as ICC removal automatically suspends controller B

Trang 31

Correct preparation for removing the interconnect-battery canister must be followed Removal and replacement steps are performed in this order:

1 Place controller B ofiline so that host failover software can detect the oftline controller and re-route all 110 to controller A

2 Thrn on the SAA LED for the interconnect-battery canister using the Common Array Manager

3 Remove and replace the interconnect-battery canister

4 Thrn off the SAA LED using CAM

5 Place controller B online and rebalance volumes

Smart battery backup pack

Battery

pack

Battery

pack

Figure 1-10 6x80 interconnect-battery canister with battery backup packs

The battery backup packs (BBUs}-also referred to as the contoller's internal UP8-in the 6x80 are called Smart BBUs because they have more capabilities

than normal battery packs Each BBU contains battery cells, a charger, a battery

"Gas Gauge" chip, a discharge load and control logic As with traditional battery llllits, the controller firmware monitors the charge level, battery status and the health of the battery pack (e.g., the number of times the batteries have been charged and the temperature of the cells) It uses "learn cycles" to perform this monitoring, and the information gathered allows the firmware to determine exactly how long a BBU can hold the cache "up" in case of a power outage and what action to take if it cannot

Trang 32

To calibrate the battery gas gauge, a learn cycle has to be implemented During a learn cycle, a fully charged BBU goes through a controlled discharge to a

predetermined threshold into the discharge load, goes through a rest period and then fully recharges The learn cycle interval is scheduled in weeks, so that the start time for each learn cycle occurs on the same day of the week, at the same time of day

Because Smart BBUs monitor for voltage, it is no longer necessary to set the date

in CAM when batteries are replaced Batteries function and allow data to write to cache as long as a minimum power level is maintained (normally 20 minutes' worth}

It is expected for battery capacity to remain above the minimum application capacity during learn cycles and for write caching to continue normally However,

if the capacity unexpectedly falls below the minimum application capacity during the learn cycle, write caching is disabled for all volumes, except those that have the "cache without batteries" attribute enabled, and an alert is generated Service Advisor shows that the BBU needs replacing

Even though data in cache is stored in the USB-based persistent cache drives if a power failure occurs, it is still important to maintain the BBUs Not only do they provide power to cache memory for approximately 30 minutes, long enough to transfer the data in cache to the USB persistent cache drives, but they also keep one set of fans in the power-fan canisters running so the controller module does not overheat

Each BBU is dedicated to one controller Therefore, it is important that BOTH battery packs are charged and functional If one BBU fails, the other will not spare for it

Trang 33

6x80 controller module: Power distribution

Controller A ( top')

E nclos lft'e F ront

Figure t-tl Power distribution in controller module

As mentioned above, the 6x80 module does not have a mid-plane or backplane Therefore, all the canisters in the module are interconnected with power passing through the interconnect-battery canister All power flows through the

controllers Power will continue to flow through a controller canister to the other canisters even if the controller itself becomes inoperable

The power from the left power-fan canister is distributed by controller B, and power from the right power-fan canister is distributed by controller A Both controllers must be in place in order to provide redundant power to its partner controller

Service Advisor procedures must be followed carefully if there are multiple failures in a controller module For example, if the power supply connected to controller A fails and controller B fails, removing controller B before replacing the failed power-tan canister will cause controller A to lose power, resulting in a total loss of data access This occurs because power distribution is through the controller physically connected to the power-fan canister

Caution -Because there is an interdependency between the power-fan, battery and controller canisters, follow Service Advisor procedures when removing any of these canisters

Copyrisbt2008 SIDl Microsystems, Inc AlllliptsReserved SIDlSenices, JUDe2009, Rmlion3.0

Trang 34

Quick check

Cont r olle r A (lopl

Ccontr oll a r 8 Powar

Sup ply F ~ il uro

Question: Two failures exist on the 6x80 controller module: The left power supply has

failed and one of the right fans has failed Which LEDs will be on for the failed components? Which component should be replaced first?

Trang 35

6x80 controller: Inside view

Controller cache and the persistent cache USB flash drives are in the 6x80 controllers, and host "daughter" cards are changeable Therefore, it is worth taking a look inside the controller to see where these are located

lhe

Ms

Processor

Memory

Figure 1-12 6x80 controller inside view

Cache DIMM memory

Controller cache is an usually large set of physical memory chips dedicated to 110 operations between controller and hosts and between controller and drives Cache DIMM memory is used exclusively for host 110, while processor memory is used for RAID application code and data, the underlying OS, and so on

Cache is used even for volumes that do not have any write caching enabled An incoming write operation results in two independent operations: one from the host-side buffers to cache and another from the drive-side buffers The responses from these operations are not sent to the host until the drive-side write operation completes and the data has been written to the drive(s)

Each cache DIMM slot in the 6x80 can accommodate either a 1 or 2GB module However, all DIMMs must be the same size (all 1GB or all 2GB) The 6x80 currently supports either 8GB or 16GB cache (total for redundant controllers) DIMM slots must be populated in a certain order: either all slots (1 through 8) or slots 2, 4, 5 and 7 Any other configurations are invalid and will trigger an error

Trang 36

It is important to note that DIMM memory is inserted by the manufacturer and is not aFRU

[

Figure 1-13 Fully configured DIMM slots: 1GB in each slot

1 0

Figure 1-14 Alternate DIMM configuration: DIMMs in slots 2, 4, 5, 7

USB persistent cache

Cache memory is random access memory (RAM), so if there is a power failure, the contents of cache are lost-including all the "dirty'' data (writes not yet

written to drives) There is no way for an application using the storage system to tell exactly how many writes are lost in a case like this, and consequently, recovery can be very difficult or impossible

To combat this problem, the 6x80 controller has persistent USB-based cache backup devices which store cache contents for an indefinite length of time As long as the controller has a battery with enough capacity to let it write the full contents of cache to the persistent cache backup device, cache contents are not lost during a power failure When the backup to persistent cache completes, the controller firmware turns off the batteries Unlike battery-backed cache, the batteries are enabled on a power outage, even if there is no dirty data in cache

Trang 37

The 6x80 uses USB-based modules for persistent cache They are 4GB each, and each controller base board has four module slots (a total of 16GB) An important fact to keep in mind is that the flash module capacity needs to be equal to the DIM:M cache capacity since the persistent cache modules are used to store cache

in times of prolonged power outages

Host cards

The 6x80 can be configured with various types of daughter host cards so the controller module can match an enterprise's specific needs Because the 6x80 is scalable, it has room for two host cards giving customers room to grow Model numbers are based on the number and type of host cards in the controller

canister.Just like cache modules, though, host cards are not FRUs The host cards are inserted by the manufacturer based on Sun's specifications

Figure 1-15 6x80 model and host card configurations

Controller base board

The controller's base board is also worth a closer look It boasts a fast, dedicated ZIP chip which performs both XOR (RAID 5) calculations and p+q (RAID 6)

calculations There is a separate 2.8GHz LV Xeon CPU for each controller, and each controller can accommodate up to 1GB dedicated memory for CPU

processes Between the ZIP chip and the host and drive ports, there are two 2GB/s PCI-Express busses, which gives the 6x80 exceptional bandwidth

The busses for host card connections are 4GB/s PCI-Express busses, with one bus for each FC chip One FC chip is dedicated to the local controller ports, while the other is dedicated to interconnecting with the alternate controller This makes host-side 110 transfer extremely fast with 1GB bandwidth per host port

Trang 38

The busses for drive channels are also 4GB/s PCI-Express busses There is one bus dedicated to each 4Gb/s FC "switch" chip (loop switch), which is, in tum,

directly connected to the XOR ZIP chip Additionally, each loop switch has a dedicated connection to the alternate controller making multi-pathing and failover very efficient

There are two busses dedicated to cache mirroring These busses are 2GB/s Express busses Each connection type has two PCI-E busses dedicated to it,

PCI-maldng the 6x80 an extremely fast and efficient controller

DM

Figure 1-16 6x80 controller architecture

6x80 controller module: Back view

Hoo t p o rt s

Figure 1-17 Host and drive ports on 6x80 controller

Trang 39

The 6x80 controllers has eight drive ports and the capacity for eight host ports Both host-side and drive-side ports are 4Gb/s FC

of the link

Link speed is limited to link speeds supported by the Small Form-factor

Pluggable (SFP) transceiver on that channel The controllers enter into negotiation at controller boot-up or when the controller detects a link-up event after a previous link-down event If the auto-negotiation process fails, the

auto-controllers consider the link to be down until negotiation is attempted again

independently

Sun Storage 6x80 product overview 1-21

Trang 40

Drive channelsnoops operate at 4Gb/s and only attach to 4Gb/s-capable expansion trays The total number of drives supported by the 6x80 is 256 (16 fully populated CSM 200, 16-slot expansion trays)

4<Gb/s

FC C p

XO R

ASIC

Figure 1-18 Each loop switch connects to two drive-side ports

When cabling expansion trays to the controllers, it is important that expansion trays are cabled to both controller A and B to ensure redundancy

Other ports

SeMI pon Power o Mt ,

On/off switc h

Figure 1-19 Other connections on 6x80 controller

Each controller in the 6x80 controller module has two RJ-45 10/100/1000 Base-T Ethernet ports: one for out-of-band management; the other for field service personnel or service diagnostics By having a separate Ethernet connection for service, the customer's LAN is not exposed to unknown connections

Định dạng
Số trang	435
Dung lượng	47,19 MB