– To provide you with a framework for further studies on – Thinking about the problems of parallel programming – Discussing your work with others – Approaching complex parallel programming problems – Using or building useful tools and environments

• Study increasingly sophisticated parallel merge kernels • Observe the combined effects of data - dependent execution and a lack of data parallelism on GPU algorithm design

• To learn to regularize irregular data with – Limiting variations with clamping – Sorting – Transposition • To learn to write a high-performance SpMV kernel based on JDS transposed format • To learn the key techniques for compacting input data in parallel sparse methods for reduced consumption of memory bandwidth – Better utilization of on-chip memory – Fewer bytes transferred to on-chip memory – Better utilization of global memory – Challenge: retaining regularity

电子科技大学：《GPU并行编程 GPU Parallel Programming》课程教学资源（课件讲稿）Lecture 07 JOINT CUDA-MPI PROGRAMMING

Prefix Sum A Work-inefficient Scan Kernel A Work-Efficient Parallel Scan Kernel More on Parallel Scan

parallel histogram Data Racing condition privatized histogram kernel

Warps and SIMD performance impact of control divergence Parallel reduction Memory parallelism

Memory access efficiency Tiled Matrix Multiplication Tiled Matrix Multiplication Kernel Handling Boundary Conditions in Tiling Tiled Kernel for Arbitrary Matrix Dimensions

Multidimensional Kernel Configuration Color-to-Greyscale Image Processing Example Blur Image Processing Example

Introduction to Heterogeneous Parallel Computing CUDA C vs. CUDA Libs vs. OpenACC Memory Allocation and Data Movement API Functions Data Parallelism and Threads