Advanced Computer Architecture - Lecture 41: Networks and clusters

Advanced Computer Architecture - Lecture 41: Networks and clusters. This lecture will cover the following: a simple network; network topology; internetworking; omega interconnection topology; distributed switch topologies; ring network; multistage interconnection topology; crossbar switch topology;...

CS 704

Advanced Computer Architecture

Lecture 41

Networks and Clusters

(Networks: Interconnection and Topology )

Prof Dr M Ashraf Chughtai

Recap: I/O Systems and Storages

Last time we concluded our discussion on the storage I/Os and communication I/Os

Here, we noticed that the dependability,

reliability, availability of the storage I/Os

mostly influence the overall performance of

computer systems

Dependability is the quality of delivered

service such that confidence can be placed on this service; and measured by quantifying the transitions between service accomplishment and service interruption

Recap: Dependability, reliability and Availability

The dependability is measured in terms of the reliability and availability of a module

The reliability of a module is the measure of

the continuous service accomplishment or the measure of the time to failure , from a reference initial instant

The availability of a module is the measure of the service accomplishment with respect to

the swinging between the accomplishment and interruption states

Recap: I/O and Storage Systems

The storages are interfaced with the processor using channel and backplane and network

very high dependability, but are more

vulnerable to the reliability, so to improve the availability and performance of network ……

Recap: Network Attached Storages

… attached storage system, disk arrays are introduced

Here, the data is stripped across a set of disks which makes the collection appears to the

software as a single large disk

The throughput of disk arrays is improved due many small disk drives having high bandwidth The drawback to an array with more devices is that dependability of the device increases,

hence, the reliability decreases; as generally N devices have 1/N reliability

Recap: Redundant Arrays of Disks

The dependability of disk array is improved by

adding redundant disks to the array to tolerate

faults

Such a disk array is called Redundant Array of Inexpensive Disk – RAID

There exist several different approaches to

include redundant disks in the disk array

These approaches are usually classified by

numerical value which identifies the RAID level

Recap: Redundant Arrays of Disks

RAID 0 is the disk array without any redundant disk, but employs the stripping of data across

a set of disks

RAID 1 or disk Mirror array is one where each disk is fully duplicated onto its "shadow“

RAID 3 or Bit-Interleaved Parity Disk employs

a parity disk for each group of data; the parity computed across recovery group to protect

against hard disk failures

Recap: Redundant Arrays of Disks

RAID 4 or Block Interleaved Parity and the

RAID 5 or Block Interleaved Distributed Parity,

both use the same ratio of data disk to parity disk as RAID 3, but they access data

differently

In RAID 4 level, the parity disk is associated to each data block, identical to it is associated to each data group in RAID 3, so it supports a

mixture of both the small and large reads and writes

Recap: Redundant Arrays of Disks

In RAID 5 level, the parity disk is associated to each data block

The data blocks are distributed among

different disks in each row;

i.e., the stripped data units are not located in the same disk

This allows simultaneous read and write of

more than one block

Interconnection Networks

Till now the focus of our studies has been the architecture of the components of a single

computer and their performance

Now today and in the following a few lectures

we will talk about how to connect computers together forming network of computers

The formation of a generic interconnection

network is depicted here

Link Interconnection Network

Interconnection Networks

The coordinated use of interconnected

computers in a machine room is referred to as the cluster

The connection of two or more

interconnec-tion networks is called Internetworking

The typical example of Internetworking is the

Interconnection Networks

Depending on the number of nodes and their proximity or nearness the interconnections are designated as:

Local Area Network-LAN: Hundreds of

computer distributed in a building within a

distance of up to a few kilometers

Wide Area Network-WAN: Interconnection of

thousands of computers distributed

throughout the world at a maximum distance

of thousands of kilometer – Automatic Teller

Machine (ATM) is a typical example

SAN is basically the cluster

However, the Moor’s Law have contracted the definition of network to an extent that it

defines the interconnection of components

within a single computer

In order to discuss the complexities and ….

Networks Communication Model

… performance of networks, let us consider a simple interconnection model of two

computers and understand the implications of network parameters

The communication model depicted here

shows that two machines are connected via

two unidirectional wires with a FIFO (queue) at the end to hold the data

Here, each machine wants receive a word or message from the other

Networks Communication Model

The machine A to get data from B, it sends a request to B, which responds by sending a

reply along with the data

In order to send a request and reply a

message contains extra information beyond data, as shown in the example message format

Networks Message Format

Here, a 1 bit header specifies the message as a

request (header=0 ) or reply (header=1)

The request carries the address of the data

word and the reply the data word

Address Header [1 bit] Payload [32 bits]

Data

Header [1 bit] Payload [32 bits]

0

1

Networks Interconnection Software

Interconnection networks involve software to establish communication

For the simple network considered here, the

software is invoked to translate the request

and reply messages

The network software :

– cooperate with the operating system to

distinguish between the processes on the other networks

– protect the processes running on networks

Networks Interconnection Software

– Ensures reliable delivery of message, i.e., to

ensure that the message is neither distorted

nor lost in transit

It is worth mentioning here that reliability

the message format is modified by adding

an error detection code (checksum or CRC) and using 2-bit header as shown here

Networks Interconnection Software

This information is calculated at the end and is added to the message;

sending-then at the receiving-end this message is

checked; and the receiver sends an

ack-nowledgment if the message passes the test

Header [2 bits] Payload [32 bits] Trailer [4 bits] (Checksum)

Data

00: Request 01: Reply 10: Acknowledge Request 11: Acknowledge Reply

Networks Interconnection Software

Furthermore, to ensure reliable deliver of

message, the sender activates a timer each time a message is sent;

The sender copies the data into an operating

system buffer to resend the message if

acknowledgement doesn’t arrive by time the timer expires, as it is presumed to be lost

At the receiving end the message is copied into the operating system buffer

The checksum is checked, if it passes the test

acknowledgment is sent other wise the message

is deleted from the buffer

Networks Interconnection Protocol

So far we have been talking about the

ack-nowledgment - protocol for reliable cation on a simple network

communi-However, there are many more issues of

reliable communication; e.g.

 Two machines from two different

manufacturers might be using different order within a word (Big Endean or Little

byte-Endean) – so the software must have to

reverse the order accordingly

Networks Interconnection Protocol

 The duplicate delivery of message should be guarded against the late delivery of the

original message, if it was stuck in the

network

 The order or the sequence of the message

should not change; so sequence number

should be included in the message

 It must also work when the receiver’s FIFO is full; so some feedback mechanism be

incorporated

Networks Performance Model

Having discussed the issues of protection,

reliability and network protocols, let us

understand the performance model of

Networks Performance Model

These parameters are:

 Bandwidth: the maximum rate at which the

network can propagate information

 Time of Flight: time of the first bit of the

message from time departed to the time it

arrives at the receiver

 Transmission Time: The time of the message

to pass through the network not including the time of flight; in other words, it is ………

Networks Performance Model

… the time between the first and the last bit of the message arrives at the receiver

 Transport Latency: The sum of time of flight

and transmission time

 Sender Overhead: time for the processor to

inject the message into the network, including both the hardware and software components

 Receiver Overhead: the time for the receiver

processor to pull the message from the ……

Networks Performance Model

…… interconnection network, including both the hardware and software

components

Based on the network performance

parameters, the

Total Latency of a message =

Sender overhead + time to flight +

(message size / bandwidth) +

Receiver overhead

Interconnection Network Media

Hierarchy

Just as the memory hierarchy, there is

hierarchy of media to interconnect computer

The interconnect media varies in cost,

performance and reliability based on the

maximum distance between nodes

The three most popular media are:

– Twisted Pair (of Copper wire)

– Coaxial Cable

– Fiber Optics

Twisted Pair of Copper Wire

 Two insulated copper wires, about 1mm thick

twisted together to reduce the electrical

interference

 The original twisted pair telephone line gives the data transfer rate of a few mega-bits per sec and

is referred to as Level-1 or Category-1 UTP

(Unshielded Twisted Pair)

 Level 3 or Cat-3 UTP is good for 10M bits/sec

Ethernet and Cat-5 for 100M bits/sec and up to

1000 M bits/sec when distance is limited to 100 meters

(Insert fig 8.11 (a) pp 802)

Coaxial Cable

 Coaxial Cable is a stiff single copper wire

surrounded by insulating material which is

covered by cylindrical sheath woven as a

braided mesh

 A 50 ohm base-band coaxial cable can deliver

10 M bits/sec over a kilometer

 Coaxial cable can deliver higher rate over a

few kilometers and offers high bandwidth and good noise immunity

(Insert fig 8.11 (b) pp 802)

 Fiber Optics contains a glass fiber core

surrounded by cladding to confine light,

which is covered by a protecting buffer

(Insert fig 8.11 (c) pp 802)

Fiber Optics

 A light source (LED or laser) and a light

detector (photo diode) are employed as

transmitter and receiver

 As we know that light bends or refracts at

interfaces and can spread slowly as it travels down the cable

 However, if the diameter of the cable is equal

to or less than one wavelength, then it is

transferred into straight line (here the angle of refraction is more than the critical angle and total reflection takes palace)

Fiber Optics

 Fiber Optic Cables are of two forms: mode and Multimode Fiber:

Single- Multimode Fiber:

– It uses inexpensive light source with wavelength

larger than that of light, and

– offers wider dispersion where some wave

frequencies have different propagation velocities

– Its dispersion is therefore limited to a few

hundred meters at 1000 M bit/sec or up to a few kilo-meters at 100 M bit/sec

– The drawbacks of single mode fiber are:

 It is more difficult to attach connecters

 It is less reliable and more expensive and has

restrictions on the degree it can be bent

 The cost, distance and bandwidth are affected by the power of light source

Interconnection Networks

So far we have discussed connecting two

computers over private lines

However, interconnecting hundreds of

computers is more interesting and challenging The bus-based LAN or Ethernet is the simplest way to interconnect more than two computers sharing a single media

Bus Based Networks Interconnection

Here, the processors and memory units are

connected through a “bus”

It is Simple and cost-effective for small-scale multiprocessors

However, the bus bandwidth limits the number

of processors

The bus-based interconnect is more

challenging also as it requires coordination

and arbitration as more than one computer

may need the same media simultaneously

we have to go for distributed arbitration

However, as the arbitration works on the

principle: “Look before you leap”; but looking first doesn’t guarantee success; as

if two nodes get hold of the media and transmit simultaneously, it leads to collision

Interconnection Networks

So to avoid collision, different techniques such

as collision detection and token passing are

used

(If time allows explain these techniques)

Alternative to sharing media is to use switching

Switch have a dedicated line which it provides in turns to all destinations

Switching allow point-to-point communication

much faster than the shared media

Switches are also called data switching

exchanges, multistage interconnection networks

or interface message processor (IMPs)

Network Topology

With this much discussion regarding sharing

of media using buses and switches, let us

discuss the topologies used to construct

computer networks

There exist numerous topologies of SANs,

LANs and WANs, however, the most popular switch-based topologies are classified as: at present are:

 Centralized Switch Topologies

 Distributed Switch Topologies

Network Topology

Today we will be talking about

 The basic Centralized Switch Topologies as:

Crossbar Switch Topology

A crossbar switch is a non-blocking switch that facilitate unidirectional interconnection of all

the inputs (any processor) to any output to the other processor

The interconnection of 2x2 crossbar switch are shown here [Fig 8.13 c]

As you can see that 2 nodes A, B can pass

information equally to outputs C and D

i.e., here no connections block any connection between other processor or memory units

Crossbar Switch Topology

The organization of a crossbar topology for nodes (processors) is shown here

8-[Fig 8.13 a]

Note that a crossbar uses n 2 switches where n

is the number of processors

Here, the links are unidirectional, i.e., the data comes in at one (left) link and goes out at other (right) link

Crossbar Switch Topology

The routing depends on the style of addressing

In source-based routing where message

specifies the path to the destination, the

message includes the sequence of out-bound arcs to teach the destination

Thus, once an outgoing link (arc) is picked, the portion of the routing sequence is dropped

from the packet

Crossbar Switch Topology

In destination-based routing the message

simply contains the destination address; and

a program running in the switch decides from routing table which port to take for a given

address

A crossbar switch offers low latency and high throughput

Multistage Interconnection Topology

An intermediate class of networks which lies between crossbar and bus based networks

Built from small (e.g., 2x2 crossbar) switch

nodes, with a regular interconnection pattern

MAC/VU-Advanced

Computer Architecture Lecture 41 Networks and Clusters (1) 47