Advanced Computer Architecture - Lecture 30: Memory hierarchy design

Advanced Computer Architecture - Lecture 30: Memory hierarchy design. This lecture will cover the following: cache performance enhancement; reducing miss rate; classification of cache misses; reducing cache miss rate; way prediction and pseudo-associativity; compiler optimization;...

CS 704

Advanced Computer Architecture

Lecture 30

Memory Hierarchy Design

Cache Performance Enhancement

(Reducing Miss Rate)

Today’s Topics

Recap: Reducing Miss Penalty

Classification of Cache Misses

Reducing Cache Miss Rate

Summary

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 2

Recap: Improving Cache Performance

─ The miss penalty

─ The miss rate

─ The miss Penalty or miss rate via

Parallelism

─ The time to hit in the cache

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 4

Recap: Reducing Miss Penalty

– Multilevel Caches

– Critical Word first and Early Restart

– Priority to Read Misses Over writes

– Merging Write Buffers

– Victim Caches

Recap: Reducing Miss Penalty

‘Multi level caches’

‘The more the merrier

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 6

Recap: Reducing Miss Penalty

“ Critical Word First and Early Restart’ intolerance

Reduces miss-penalty

Recap: Reducing Miss Penalty

‘Priority to read miss over the write miss’

Favoritism

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 8

Recap: Reducing Miss Penalty

‘Merging write-buffer,’

Acquaintance

Victim cache

Salvage

Recap: Reducing Miss Penalty

Reduces miss penalty

Multi level caches

Reduces miss rate

Cache-misses

Methods to reduce the miss rate

Cache Misses - Classification

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 12

Cache Misses - Classification

Conflict Miss

Many blocks map to the same address or set

Cache Misses - Classification

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 14

Cache Misses - Classification

Reducing Miss Rate

1 Larger Block Size

2 Larger Caches

3 Higher Associativity

4 Way Prediction and Pseudo-associativity

5 Compiler Optimization

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 16

1: Larger Block Size

Reduce the miss rate

Spatial locality

Larger block have maximum number of data

or instructions

1: Larger Block Size

In small cache , larger blocks may increase

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 18

1: Larger Block Size

1: Larger Block Size

Assumption

80 clock cycles of over head

Delivers 16 bytes every 2 clock cycle

Hit time of 1 clock cycle

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 20

1: Larger Block Size: Solution

Average memory access time

= Hit time + Miss Rate x Miss Penalty

4KB cache, the miss rate = 7.24%

Miss penalty =

80 +4 = 84 clocks

1: Larger Block Size: Solution

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 22

1: Larger Block Size: Solution

Copy table 5.18 pp 428

1: Larger Block Size: Solution

– Latency and bandwidth of the lower level memory

– High latency and high bandwidth

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 24

2: Large Cache Size

Reduce Capacity misses

In 2001

2 nd – level and 3 rd -level caches

Drawback

Longer hit time

Higher cost (access time)

3: Higher Associativity

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 28

4: Way Prediction and

Pseudo-associativity

Fast hit time of direct-mapped caches

Lower conflict misses of set-associative caches

Reduces the conflict misses

4: Way Prediction and Pseudo-associativity

Way Prediction

Block in a set

Steps

1 Extra bits

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 30

4: Way Prediction and Pseudo-associativity

2-way prediction 4-way prediction

2 Multiplexer

Single tag

4: Way Prediction and Pseudo-associativity

 Other blocks for matches in subsequent

clock cycles

 Alpha 21264

 Latency of 1 clock cycle

 3 clock cycles

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 32

4: Way Prediction and Pseudo-associativity

Pseudo-associative

Column associative caches

Miss

“Pseudo-set”

4: Way Prediction and Pseudo-associativity

Pseudo Associative caches

Performance

“Slower hit”

5: Compiler Optimization

Instruction

Code optimization

– Reordering the Procedures

– Using Cache-line Alignment

The code-line alignment method:

– Decreases cache miss

– Entry point is at the beginning of a cache

block

5: Compiler Optimization: Loop Interchange

• program having nested loops that

access data in non-sequential

order for j (0100) and in

sequential order for i (05000)

5: Compiler Optimization:

Using Blocking

‘Row major order’ (row-by-row)

‘Column major order’

Iteration for matrix multiplication

Whereas,

can hold one N x N matrix and

one row of N elements ,

then the full matrix Z and one ( i th )

row of Y can stay in the cache

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 46

5: Compiler Optimization:

Using Blocking

B is chosen such that one row of B and

one B x B matrix can fit in in cache.

This ensures that the y and z blocks are

resident on cache

Let us have a look on to the modified code which shows that the two inner loops now compute in steps of size B (blocking

factor) rather than the full N x N size of

arrays X and Z

5: Compiler Optimization:

Using Blocking

The way-prediction techniques checks a

section of cache for hit and then on miss it checks the rest of the cache

The final technique – loop interchange and blocking , is a software approach to

optimize the cache performance

Next time we will talk about the way to

enhance performance by having processor

Example: Avg Memory Access

Time vs Miss Rate

Example: assume CCT = 1.10 for 2-way, 1.12 for 4-way, 1.14 for 8-way vs CCT

direct mapped

Allah Hafiz