MemoryHierarchyTechnologies Caches use SRAM for speed and technologycompatibilityFast (typical access times of 0.5 to 2.5 nsec)Low density (6 transistor cells), higher power, expensive ($2000to$5000perGB in 2008)Static: content will last"forever" (as long as power is left on) Main memory uses DRAM for size (density)Slower (typicalaccess times of 50 to 70 nsec)High density (1 transistor cells), lower power, cheaper ($20 to $75per GB in 2008)Dynamic: needs to be “"refreshed" regularly (~ every 8 ms)consumes1%to2%oftheactivecyclesoftheDRAMAddressesdividedinto2halves (rowand column)-RASorRowAccessStrobetriggeringtherowdecoder- CAS or Column Access Strobe triggering the column selectorCS314Chapter5A.7HaojinZhu, SJTU,2017
CS314 Chapter 5A.7 Haojin Zhu, SJTU, 2017 Memory Hierarchy Technologies ❑ Caches use SRAM for speed and technology compatibility Fast (typical access times of 0.5 to 2.5 nsec) Low density (6 transistor cells), higher power, expensive ($2000 to $5000 per GB in 2008) Static: content will last “forever” (as long as power is left on) ❑ Main memory uses DRAM for size (density) Slower (typical access times of 50 to 70 nsec) High density (1 transistor cells), lower power, cheaper ($20 to $75 per GB in 2008) Dynamic: needs to be “refreshed” regularly (~ every 8 ms) - consumes1% to 2% of the active cycles of the DRAM Addresses divided into 2 halves (row and column) - RAS or Row Access Strobe triggering the row decoder - CAS or Column Access Strobe triggering the column selector
The Memory Hierarchy: Why Does it Work? Temporal Locality (locality in time)Ifa memory locationis referencedthen it will tend tobereferencedagainsoon=Keepmostrecentlyaccesseddataitemsclosertotheprocessor Spatial Locality (locality in space)Ifa memory locationis referenced, thelocations with nearbyaddresseswilltendtobereferencedsoonMoveblocksconsistingofcontiguouswordsclosertotheprocessorCS314Chapter5A.8HaojinZhu, SJTU, 2017
CS314 Chapter 5A.8 Haojin Zhu, SJTU, 2017 The Memory Hierarchy: Why Does it Work? ❑ Temporal Locality (locality in time) If a memory location is referenced then it will tend to be referenced again soon Keep most recently accessed data items closer to the processor ❑ Spatial Locality (locality in space) If a memory location is referenced, the locations with nearby addresses will tend to be referenced soon Move blocks consisting of contiguous words closer to the processor
The Memory Hierarchy: Terminology Block (or line): the minimum unit of information that ispresent (or not) in a cache Hit Rate: the fraction of memory accesses found in a levelof the memory hierarchyHit Time:Timeto accessthatlevel whichconsistsofTimeto accesstheblock+ Timeto determinehit/missMiss Rate:the fraction of memory accesses not found in alevel of the memory hierarchy = 1 - (Hit Rate)MissPenalty:TimetoreplaceablockinthatlevelwiththecorrespondingblockfromalowerlevelwhichconsistsofTime to access the block in the lower level + Time to transmit that blocktothelevelthatexperiencedthemiss+Timetoinserttheblockinthatlevel+TimetopasstheblocktothereguestorHit Time << Miss PenaltyCS314Chapter5A.9HaojinZhu, SJTU, 2017
CS314 Chapter 5A.9 Haojin Zhu, SJTU, 2017 The Memory Hierarchy: Terminology ❑ Block (or line): the minimum unit of information that is present (or not) in a cache ❑ Hit Rate: the fraction of memory accesses found in a level of the memory hierarchy Hit Time: Time to access that level which consists of Time to access the block + Time to determine hit/miss ❑ Miss Rate: the fraction of memory accesses not found in a level of the memory hierarchy 1 - (Hit Rate) Miss Penalty: Time to replace a block in that level with the corresponding block from a lower level which consists of Time to access the block in the lower level + Time to transmit that block to the level that experienced the miss + Time to insert the block in that level + Time to pass the block to the requestor Hit Time << Miss Penalty
Characteristics of the Memory HierarchyProcessorInclusive-↑ 4-8 bytes (word)what is in L1$is a subsetofL1$Increasingwhat is in L2$distance8-32bytes (block)is a subset offrom theL2$what is in MMprocessorthat is a+1to4blocksin accesssubset of is inMain MemorytimeSMI1,024+bytes(disksector=page)Secondary Memory(Relative)sizeofthememory ateachlevelCS314Chapter5A.10HaojinZhu,SJTU,2017
CS314 Chapter 5A.10 Haojin Zhu, SJTU, 2017 Characteristics of the Memory Hierarchy Increasing distance from the processor in access time L1$ L2$ Main Memory Secondary Memory Processor (Relative) size of the memory at each level Inclusive– what is in L1$ is a subset of what is in L2$ is a subset of what is in MM that is a subset of is in SM 4-8 bytes (word) 1 to 4 blocks 1,024+ bytes (disk sector = page) 8-32 bytes (block)
Howis theHierarchy Managed?registersmemorybycompiler(programmer?)口 cache <> main memorybythecachecontrollerhardware main memory disksbytheoperatingsystem (virtual memory)virtual to physical address mapping assisted by the hardware(TLB)bytheprogrammer (files)CS314Chapter5A.11HaojinZhu, SJTU,2017
CS314 Chapter 5A.11 Haojin Zhu, SJTU, 2017 How is the Hierarchy Managed? ❑ registers memory by compiler (programmer?) ❑ cache main memory by the cache controller hardware ❑ main memory disks by the operating system (virtual memory) virtual to physical address mapping assisted by the hardware (TLB) by the programmer (files)