Advanced Computer Architecture - Lecture 37: Multiprocessors

Advanced Computer Architecture - Lecture 37: Multiprocessors. This lecture will cover the following: performance of multiprocessors with symmetric shared-memory, distributed shared memory; synchronization in parallel architecture; hardware supplied synchronization instructions;...

CS 704

Advanced Computer Architecture

Lecture 37

Multiprocessors

(Performance and Synchronization)

Prof Dr M Ashraf Chughtai

Today’s Topics

Recap:

Performance of Multiprocessors with

– Symmetric Shared-Memory

– Distributed Shared Memory

Synchronization in Parallel Architecture

Conclusion

Recap: Cache Coherence Problem

So far

So far we have discussed the sharing of

caches for multi-processing in the:

 symmetric shared-memory architecture

 Distributed shared memory architecture

We have studied cache coherence problem

in symmetric and distributed

shared-memory multiprocessors; and have noticed that this problem is indeed performance-

critical

Recap: Multiprocessor cache Coherence

Last time we also studied the cache

coherence protocols , which use different techniques to track the sharing status and maintain coherence without performance degrading

These protocols are classified as:

Snooping Protocols

Directory-Based Protocols

These protocols are implemented using a FSM controller

Recap: Snooping Protocols

Snooping protocols employ write invalidate

and write broadcast techniques

Here, the block of memory is in one of the three states, and each cached-block tracks these three states; and

the controller responds to the read/write

request for a block of memory or cached

block, both from the processor and from

the bus

Recap: Implementation Complications of snoopy protocols

Exclusive or Invalid

races, interventions and invalidation have been observed in the implementation of snoopy

protocols; and

to overcome these complications number of

variations in the FSM controller have been

suggested

Protocol and Illinois Protocol

Recap: Variations in snoopy protocols

These variations resulted in four (4) states FSM controller

E xclusive, S hared and I nvalid

– The sates of Barkley Protocol are: Owned-

Exclusive, Owned -Sheared, Shared and

Invalid; and of

– Illinois Protocol are: Private Dirty , Private

clean, shared and Invalid

Recap: Directory based Protocols

The larger multiprocessor systems employ distributed shared-memory , i.e., a separate memory per processor is provided

Here, the Cache Coherency is achieved

using non-cached pages or directory

containing information for every block in

memory

The directory-based protocol tracks state of every block in every cache and finds the …

Recap: Directory Based Protocol

…… caches having copies of block being dirty or clean

The directory-based protocol tracks state

of every block in every cache and finds the caches having copies of block being dirty

or clean

Similar to the Snoopy Protocol, the

directory-based protocol are implemented

by FSM having three states: Shared,

Uncached and Exclusive

Recap: Directory-based Protocol

Recap: Directory Based Protocols

These protocols involve three processors

or nodes, namely: local, home and remote nodes

– Local node originates the request

– Home node stores the memory location

of an address

– Remote node holds a copy of a cache

block, whether exclusive or shared

Recap: Directory-based Protocol

messages such as: read misses, write

misses, invalidates or data fetch requests

state and to satisfy requests

block; and indicates an action that updates the sharing set

Example: Working of Finite State Machine Controller

Now are going to discuss the state

transition and messages generated by FSM controller in each state to implement the

directory-based protocols.

We consider an example distributed memory multiprocessor having two

shared-processors P1 and P2 where each

processor has its own cache, memory and directory

Example: Working of Finite State Machine Controller

Here, if

the required data is not in the cache and is available in memory associated with the

the state machine is said to be in Uncached state; and

transition to other states is caused by

messages such as: read miss, write miss, invalidates and data fetch request

Example: Dealing with read/write misses

A1 and A2 map to the same cache block

P1 P2 Bus Directory Memory step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Example: Working of Finite State Machine Controller

Let us assume that the initially the cache states

are Uncached (i.e., the block of data is in memory); and at the first step P1 write 10 to address A1 ,

here the following three activities take place

1.The bus action is write miss and the

processor P1 places the address A1 on the bus;

2.the data value reply message is sent to the controller, P1 is inserted in the directory

sharer-set {P1}; and

Example: Working of Finite State Machine Controller

3. the state transition from Uncached to

exclusive takes place – these operations are shown here in red color

step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Example: Working of Finite State Machine Controller

At Step 2 – P1 reads A1; CPU read HITs

occurs, hence the FSM Stays in exclusive

state

step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Example: Working of Finite State Machine Controller

At Step 3: P2 reads A1

Uncached state; the controller states of P1 and P2 change from Uncached to Shared

write-back is asserted and the state changes from

exclusive to Shared; and

address A1, into P1 and P2 caches at A1; and both P1 and P2 controllers are inserted in sharer-set

{P1,P2}

Example: Working of FSM Controller

P2: Write 20 to A1

A1 and A2 map to the same cache block

step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Excl A1 10 DaRp P1 A1 0 P1: Read A1 Excl A1 10

P2: Read A1 Shar A1 RdMs P2 A1

Shar. A1 10 Ftch P1 A1 10 10

Shar A1 10 DaRp P2 A1 10 A1 Shar {P1,P2} 10

10 10

Example: Working of Finite State Machine Controller

At Step 4: P2 write 20 to A2

find a remote write, so the state of the controller changes from shared to Invalid

bus and changes the state from shared to

exclusive and writes value 20 to A1

containing {P2}

Example: working of FSM controller

Step 4

P2: Write 20 to A1

A1 and A2 map to the same cache block

step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Example: Working of Finite State Machine Controller

A1 is in Uncached state; the sharer-set is empty and value 20 is placed in the memory

and value 40

Example Cont’d

P2: Write 20 to A1

A1 and A2 map to the same cache block

step State Addr Value State Addr Value Action Proc Addr Value Addr State {Procs} Value

Excl. A2 40 DaRp P2 A2 0 A2 Excl {P2} 0

Processor 1 Processor 2 Interconnect Directory Memory

A1

Performance of Multiprocessors

Symmetric Shared-Memory Architecture

In bus-based multiprocessor using an invalidation protocols, several phenomenon combine to

determine performance:

– Overall cache performance is combination of the behavior of the Uniprocessor cache miss-traffic

and the traffic caused by the communication due

to invalidation and subsequent cache miss

– Changing processor count, cache size and block size effect these two components of miss rate

Performance of Multiprocessors

Symmetric Shared-Memory Architecture Cont’d

– The misses arising from inter-processor

communication, called coherence misses , can be from two sources:

– True Sharing

– False sharing

– True Sharing: The so-called true sharing

misses arise from communication of data through cache-coherence mechanism

Performance of Multiprocessors

Symmetric Shared-Memory Architecture Cont’d

Explanation – True Sharing:

 The first write by processor to a shred cache-block caused an invalidation to establish ownership of that block

 When another processor attempts to read

modified word, a miss occurs and the resultant

block is transferred

 Both the misses are classified as true-sharing

misses, as they arise from the sharing of data

Performance of Multiprocessors

Symmetric Shared-Memory Architecture Cont’d

– False Sharing: it arise from the use of base coherence algorithm with a single valid bit per cache block

invalidation-Explanation:

 False sharing occurs when a block is invalidated and a subsequent reference causes a miss i.e.,

 the word being written and the word read are

different and the invalidation does not cause a

new value to be communicated, but only causes

an extra cache miss

Performance of Multiprocessors

Symmetric Shared-Memory Architecture Cont’d

– Here, the block is shared but no word in the block

is shared and the miss would not occur is the

block size were a single word

Example of True and False Sharing:

 Considering the previous example, assume the

words A1 and A2 are in the same cache block,

which is in the shared state in the caches of P1

and P2

 Let us identify the true-sharing miss and false

sharing miss for the following sequence of events

A2 was invalidated by the write of A1 in P1, but the value of A1 is not used in P2

Event 3: P1 Write A1 – is false sharing miss ; since the block containing A1 is marked shared due to

read in P2; but P2 did not read A1

Performance of Multiprocessors

Symmetric Shared-Memory Architecture Cont’d

Explanation Cont’d:

Event 4: P2 Write A2 – is false sharing miss ; since

the block containing A2 is marked shared due to

read in P2 (event 2); but P2 did not Write A2

Event 5: P1 Read A2 – is true sharing miss ; since the value being read by P2 was written by P2 (in event 4)

Performance of Multiprocessors

Distributed Shared-Memory Architecture

The performance of directory-based

multiprocessors depends on many of the same factors (such as processor count, cache size and block size etc.) that influence the performance of bus-based multiprocessor

In addition, the location of requested data item

which depends on both the initial allocation and sharing pattern also influence the performance of distributed shared-memory architecture

Performance of Multiprocessors

Distributed Shared-Memory Architecture Cont’d

Here, the distribution of memory requests between local memory and remote memory is key to the

performance, because it affects both the

consumption of both global bandwidth and latency seen by the requests

This can be visualized from these figures

Here the cache misses are separated into the local and remote requests

(Fig 6.31 – 6.33 pp 585 - 587)

Performance of Multiprocessors

Distributed Shared-Memory Architecture Cont’d

The graphs for data miss rate vs processor count

obtained using benchmarks FFT, LU, Barnes and

by the changes in processor count, except for

Ocean where the miss rate rises at 64 processors

Note that this rise is the result of increase in the local misses which is due to mapping conflicts

and increase in the remote misses resulting from coherence misses

(Fig 6.31 pp 585)

Performance of Multiprocessors

Distributed Shared-Memory Architecture Cont’d

The graphs for data miss rate vs cache size,

obtained using same benchmarks, show that miss rate decrease as cache size grow

Note that there is a steady decrease in the local

miss rate while the decline in the remote miss rate depend on coherence misses

In all cases shown here, the decrease in the local miss rate is larger than the decrease in the remote miss rate

Performance of Multiprocessors

Distributed Shared-Memory Architecture Cont’d

The graphs for data miss rate vs block size,

obtained using same benchmarks, show that miss rate decrease as block size increases

(Fig 6.33pp 586)

Why Synchronization?

need to know when it is safe for different

processes to use shared data

synchronization mechanisms

These mechanisms are built with user-level

software routines that rely on the hardware

supplied synchronization instructions

For small multiprocessors Uninterruptable

instruction are used to fetch and update memory which is referred to as the atomic operation

For large scale multiprocessors, synchronization can be a bottleneck

Several techniques have been proposed to reduce contention and latency of synchronization

Here, we will examine the hardware primitives to implement synchronization and then construct

synchronization routines

Hardware Primitives: Uninterruptable Instructions

The basic requirement to implement

synchronization in a multiprocessor is the set of hardware primitives with the ability to

atomically read and modify a memory

location,

– i.e., read and modify are performed in one

step

One typical operation that interchanges a

value in a register for a value in memory is referred to as Atomic exchange

Hardware Primitives: Uninterruptable Instructions

There are number of other atomic primitives that can be used to implement synchronization

The key property of these atomic primitives is that they read and update a memory value atomically

The other such operations used in many old

multiprocessors is Test-and-Set and

fetch-and-increment etc.

Now let us understand how the atomic operation work?

Hardware Primitives: Uninterruptable Instructions

Atomic Exchange: To see how we can use this

primitive to build synchronization, let us assume

we want to build a simple lock where

0 indicates that lock is free; and

1 indicates that lock is unavailable

To implement synchronization, a processor

tries to set the lock by exchange of 1, which is

in the register, with the memory address

corresponding to the lock

The value returned from the exchange

instruction is 1 if some other processor had

Hardware Primitives: Uninterruptable Instructions

The synchronization is locked and unavailable if some other processor had already claimed

access; otherwise the value returned is 0

In the later case, where the value returned is 0,

the value is changed to 1, preventing any

competing exchange from also retrieving 0

Example:

simultaneously

exchange first and returns 0, and the second …

Hardware Primitives: Uninterruptable Instructions

… processor will return 1 when it does the

exchange

Test-and-set: tests a value and sets it if the value passes the test

Fetch-and-increment: it returns the value of a

memory location and atomically increments it

is indivisible

Uninterruptable Instructions … Cont’d

Implementing a single atomic instruction in

hardware is complex and is hard to have read &

write in one instruction; therefore

In the recent multiprocessor pair of instructions is used – the two instructions are:

Here, the second instruction returns a value

from which it can be deduced as if the

instruction were executed as atomic

Uninterruptable Instructions … Cont’d

Note that

– Load linked (LL) returns the initial value

– Store conditional (SC) returns 1 if it succeeds (no other store to same memory location since proceeding load) and 0 otherwise

These instructions are used in sequence:

– If the contents of memory location, specified by the LL

are changed before the before the SC to the same

address occurs, then the SC fails

– The store conditional returns a value 1 or 0 indicating

whether the SC was successful or not

Uninterruptable Instructions … Cont’d

Let us consider an example program

segment showing implementation of atomic exchange on memory location specified by the contents of register R1

Example doing atomic swap with LL & SC:

try: MOV R3,R4 ; mov exchange value

ll R2,0(R1) ; load linked

sc R3,0(R1) ; store conditional beqz R3,try ; branch store fails (R3 = 0) mov R4,R2 ; put load value in R4

At the end of this sequence, the contents of R4 and memory location specified by R1 have been atomically exchanged