Advanced Computer Architecture - Lecture 33: Memory hierarchy design

Advanced Computer Architecture - Lecture 33: Memory hierarchy design. This lecture will cover the following: virtual memory system; virtual memory address translation; virtual memory performance; protection of multiple processes sharing memory; VM address translation concept; fast address translation;...

CS 704

Advanced Computer Architecture

Lecture 33

Memory Hierarchy Design

(Virtual Memory System)

Prof Dr M Ashraf Chughtai

Today’s Topics

Recap: Main memory and Virtual memory Design

Virtual Memory Address Translation Virtual Memory Performance

Protection of multiple processes

Main memory organization

Organized using banks of memory arrays

Dual Inline Memory Modules - DIMMs

Fast page mode

Synchronous

Double Data Rate DRAMs

Recap: Memory Hierarchy

Recap: Main Memory Performance

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 4

Fast page mode

Synchronous DRAM (SDRAM)

Double Data Rate (DDR) DRAM

latency and bandwidth

concern of caches

bandwidth

Inputs/outputs and multiprocessors

Wider Main Memory

Simple Interleaved Memory

Independent Memory Banks

Recap: Virtual Memory

Recap: Virtual Memory … Cont’d

Recap: Virtual Memory … Cont’d

Fix-sized fragment

Variable-sized fragment

Contiguous pages in virtual memory

Physically available on the main memory

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 8

Recap: Virtual Memory Cont’d

Protection and Relocation

Protection

Relocation

Recap: Cache verses Virtual memory

– Page fault or address fault

– CPU produces virtual address

– Mapping of a virtual address

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 10

Recap: Cache verses Virtual memory … Cont’d

– Replacement

– The size of processor address

– Secondary storage

Recap: Cache verses Virtual memory … Cont’d

– The page replacement strategies

– FIFO – First –in-First Out

– LRU – Least recently Used

– Approximation to LRU

• Bit

• Resets the reference bit

• Page with a reference bit

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 12

Recap: Cache verses Virtual memory … Cont’d

– VM Write strategies may be:

– Write Back

– Write Through

Write through is impossible because:

• Too long access to disk

• The write buffer

• The I/O system

Recap: Virtual Memory operation

– The CPU generates the Virtual Address

– Lookup table

– Location of the page or segment

– Virtual addresses to physical addresses

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 14

Recap: Virtual Memory operation Cont’d

– page fault

 The OS has full control over placement

 OS exception handler is invoked

 current process suspends the data is to the

main memory by the OS

 The contents of the page table are updated

VM Address Translation Concept

Assume that Virtual Address space V

comprises a set of N pages

VM Address Translation Concept

Assuming n-bit virtual address, m-bit

physical address and p-bit page offset, the virtual and physical address limits and the page size can be expressed as

VM Address Translation Concept page offset

page number

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 18

n–1 p p–1 0

virtual page number page offset virtual address

physical page number page offset physical address

0 p–1

address translation mechanism

p m–1

VM Address Translation Concept

Page Table

Memory resident page table

(physical page or disk address)

1 0 0 0

Page Table Operation: 3 steps

1: Translation

2: Computing Physical Address

Address Translation via Page Table

virtual page number (VPN) page offset

virtual address

physical page number (PPN) page offset

physical address

0 p–1

p m–1

page table base register

if valid=0 then page not in memory

valid access physical page number (PPN)

VPN acts as

table index

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 22

Page entry Table

Page Table Operation

3: Checking Protection

Simple Memory System Example

Addressing

– 14-bit virtual addresses

– 12-bit physical address

– Page size = 64 bytes

13 12 11 10 9 8 7 6 5 4 3 2 1 0

11 10 9 8 7 6 5 4 3 2 1 0

VPO

PPO PPN

VPN

(Virtual Page Number) (Virtual Page Offset)

(Physical Page Number) (Physical Page Offset)

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 24

Simple Memory System Page

Fast Address Translation

large and in the main memory

miss penalty

one memory access to obtain the physical address

second to get the data

Miss penalty can be reduced

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 26

miss

Fast Translation with a TLB

Fast Address Translation Cont’d

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 28

Virtual Address Physical Address Dirty Ref Valid Access

Address Translation with a TLB

virtual address virtual page number page offset

physical address

valid tag physical page number

valid tag data tag index byte offset

Fast Address Translation Cont’d

– Fully associative placement policy

– Violation against protection information in

the TLB

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 30

Fast Address Translation Cont’d

– The physical address

– The page offset

– A full physical address

Fast Address Translation Cont’d

Address Translation Example

virtual address of 64 bits

– Physical Address: 41 bits

– TLB – direct mapped with 256 entries

– First Level Caches: direct mapped with 8KB

entries, block size 64 byte

– Second Level Cache: direct mapped 4MB

direct mapped; block size 64 bytes

Address Translation Example VA – L2

MAC/VU-Advanced

Computer Architecture Lec 33 Memory Hierarchy Design (9) 34

VM Protection Process

VM Protection Mechanisms

The address translation mechanis

Protection attribute bits

VM Protection Mechanisms

Protection mechanism

The address is said to be valid if

Base <= address <= Bound Base and Bound register

Virtual memory:

– Software-management

– Very high miss penalty

=> miss rate must be very low

Memory hierarchy organization

Modules of DRAM and SRAM

design and working of disk storages

DRAM, SRAM and Disk

Recap: Memory Hierarchy Principles Concept of Caching

Principle of Locality

MAC/VU-Advanced

Computer Architecture Lecture 27 Memory Hierarchy (3) 42

Recap: Principle of Locality

Recap: Types of Locality

Recap: Improving Cache Performance

─ The miss penalty

─ The miss rate

─ The miss Penalty or miss rate via

Parallelism

─ The time to hit in the cache

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 46

Recap: Reducing Miss Penalty

– Multilevel Caches

– Critical Word first and Early Restart

– Priority to Read Misses Over writes

– Merging Write Buffers

– Victim Caches

Recap: Reducing Miss Penalty

‘Multi level caches’

the more the merrier

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 48

Recap: Reducing Miss Penalty

“ Critical Word First and Early Restart’,

Recap: Reducing Miss Penalty

‘priority to read miss over the write miss’,

Favoritism

MAC/VU-Advanced

Computer Architecture Lecture 30 Memory Hierarchy (6) 50

Recap: Reducing Miss Penalty

‘merging write-buffer,’

acquaintance

“victim cache’

salvage

Recap: Reducing Miss Penalty

Reducing miss penalty

Reducing miss rate

Cache-misses and methods to reduce the miss rate

MAC/VU-Advanced

Computer Architecture Lecture 31 Memory Hierarchy (7) 52

Summary – Cache Optimization

– 5 methods to reduce the miss penalty

– 7 ways to reduce 3Cs

– 3 methods for reducing miss rate and miss

penalty via parallelism; and

– 4 techniques to reduce hit time

The performance of these methods is summarized here

Allah Hafiz