The Memory Hierarchyfastmove what you use heresmall↑With good locality ofreference,memorye王appears as fast asand as large as全backupbig but sloweverythinghereComputerArchitecture16
Computer Architecture The Memory Hierarchy 16 fast small big but slow move what you use here backup everything here With good locality of reference, memory appears as fast as and as large as faster per byte cheaper per byte
Memory Hierarchy:Fundamental tradeoff- Fast memory: small- Large memory: slowIdea: Memory hierarchyHard DiskMainCPUCacheMemory-(DRAM)RFLatency, cost, size,bandwidthComputerArchitecture17
Computer Architecture Memory Hierarchy • Fundamental tradeoff – Fast memory: small – Large memory: slow • Idea: Memory hierarchy • Latency, cost, size, bandwidth 17 CPU Main Memory (DRAM) RF Cache Hard Disk
Locality: One's recent past is a very good predictor of his/hernearfuture.Temporal Locality: If you just did something, it is verylikely that you will do the same thing again soon- since you are here today, there is a good chance you will behere again and again regularly: Spatial Locality: If you did something, it is very likelyyou will do something similar/related (in space)- every time I find you in this room, you are probably sittingclose to the same peopleComputerArchitecture18
Computer Architecture Locality • One’s recent past is a very good predictor of his/her near future. • Temporal Locality: If you just did something, it is very likely that you will do the same thing again soon – since you are here today, there is a good chance you will be here again and again regularly • Spatial Locality: If you did something, it is very likely you will do something similar/related (in space) – every time I find you in this room, you are probably sitting close to the same people 18
Memory Locality: A "typical" program has a lot of locality in memoryreferences- typical programs are composed of"loops'. Temporal: A program tends to reference the samememory location many times and all within a smallwindow of timeSpatial: A program tends to reference a cluster ofmemory locations at a time-most notable examples:.1.instruction memory references. 2.array/data structure referencesComputerArchitecture19
Computer Architecture Memory Locality • A “typical” program has a lot of locality in memory references – typical programs are composed of “loops” • Temporal: A program tends to reference the same memory location many times and all within a small window of time • Spatial: A program tends to reference a cluster of memory locations at a time – most notable examples: • 1. instruction memory references • 2. array/data structure references 19
Caching Basics: Exploit Temporal Locality. Idea: Store recently accessed data in automaticallymanaged fast memory (called cache) Anticipation: the data will be accessed again soonTemporal locality principleRecently accessed data will be again accessed in the near福future. This is what Maurice Wilkes had in mind:Wilkes,“Slave Memories and Dynamic Storage Allocation,"IEEETrans. On Electronic Computers, 1965.“The use is discussed of a fast core memory of, say 32000 words asa slavetoa slower core memoryof,say,one millionwordsin suchawaythat in practical casestheeffectiveaccesstime isnearerthat ofthefast memorythanthat of the slow memory.ComputerArchitecture20
Computer Architecture Caching Basics: Exploit Temporal Locality • Idea: Store recently accessed data in automatically managed fast memory (called cache) • Anticipation: the data will be accessed again soon • Temporal locality principle – Recently accessed data will be again accessed in the near future – This is what Maurice Wilkes had in mind: • Wilkes, “Slave Memories and Dynamic Storage Allocation, ” IEEE Trans. On Electronic Computers, 1965. • “The use is discussed of a fast core memory of, say 32000 words as a slave to a slower core memory of, say, one million words in such a way that in practical cases the effective access time is nearer that of the fast memory than that of the slow memory.” 20