Kepler Memory HierarchyRegisterSpillstolocalmemoryCachesShared memoryL1cacheL2cacheConstantcacheTexturecacheGlobalmemory
Kepler Memory Hierarchy Register Spills to local memory Caches Shared memory L1 cache L2 cache Constant cache Texture cache Global memory
Kepler/Fermi Memory HierarchySM-0SM-1SM-NRegistersRegistersRegistersAS?V?OP0fastkeplerL1&L1&L1&cTEXcCTEXTEXSMEMSMEMSMEM小个lowL2TGlobal Memory
Kepler/Fermi Memory Hierarchy L2 Global Memory Registers C SM-0 L1& SMEM TEX Registers C SM-1 L1& SMEM TEX Registers C SM-N L1& SMEM TEX low fast kepler
Wrong View To OptimizationTryall the optimizationmethods inbookOptimizationisendlessEfficiencyLoW
Wrong View To Optimization Try all the optimization methods in book Optimization is endless
General Optimization Strategies:MeasurementFindout the limitingfactorinkernelperformanceMemorybandwidthbound(memoryoptimization)Instructionthroughputbound(instructionoptimization)Latencybound(configurationoptimization)Measureeffectivememory/instructionthroughput:NViDIA VisualProfiler
General Optimization Strategies: Measurement Find out the limiting factor in kernel performance Memory bandwidth bound (memory optimization) Instruction throughput bound (instruction optimization) Latency bound (configuration optimization) Measure effective memory/instruction throughput: NVIDIA Visual Profiler
ResolvedFind LimiterMemoryInstructionLatencyboundboundboundCompareto EffectiveCompareto EffectiveValueGB/sValueinst/sMemoryInstructionConfigurationoptimizationoptimizationoptimizationDone!
Resolved Find Limiter Compare to Effective Value GB/s Memory optimization Compare to Effective Value inst/s Instruction optimization Configuration optimization Memory bound Instruction bound Latency bound Done! << << ~ ~