Interleaved Partitioning (Iteration 2) r o g r a mm i n g m as s i e a Thread 0 Thread 1 Thread 2 Thread 3 1 1 0 2 0 0 a-d e-h m-p q-t U-X y-Z 电子科妓女学 O
11 Interleaved Partitioning (Iteration 2) …
parallel histogram Data Racing condition privatized histogram kernel 电子种越女学 O
12 parallel histogram Data Racing condition privatized histogram kernel
Objective To understand data races in parallel computing Data races can occur when performing read-modify-write operations Data races can cause errors that are hard to reproduce Atomic operations are designed to eliminate such data races Atomic operation concepts Types of atomic operations in CUDA Intrinsic functions A basic histogram kernel 电子科妓女学 O
13 Objective – To understand data races in parallel computing – Data races can occur when performing read-modify-write operations – Data races can cause errors that are hard to reproduce – Atomic operations are designed to eliminate such data races – Atomic operation concepts – Types of atomic operations in CUDA – Intrinsic functions – A basic histogram kernel
Data Race in Parallel Thread Execution thread1:Old←Mem[x] thread2: old←Mem[x] New←Old+1 New←Old+1 Mem[x]←New Mem[x]←New Old and New are per-thread register variables. Question 1:If Mem[x]was initially 0,what would the value of Mem[x]be after threads 1 and 2 have completed? Question 2:What does each thread get in their Old variable? Unfortunately,the answers may vary according to the relative execution timing between the two threads,which is referred to as a data race. race condition,where the outcome of two or more simultaneous update operations varies depending on the relative timing of the operations involved. 电子科妓女学
14 Data Race in Parallel Thread Execution Old and New are per-thread register variables. Question 1: If Mem[x] was initially 0, what would the value of Mem[x] be after threads 1 and 2 have completed? Question 2: What does each thread get in their Old variable? Unfortunately, the answers may vary according to the relative execution timing between the two threads, which is referred to as a data race. race condition, where the outcome of two or more simultaneous update operations varies depending on the relative timing of the operations involved. thread1: thread2: Old Mem[x] New Old + 1 Mem[x] New Old Mem[x] New Old + 1 Mem[x] New