Parallel Programming Model What programmer uses in coding applications? Specifies communication and synchronization Communication primitives exposed to user-level realizes the programming mode Ex Uniprocessor, Multiprogramming, Data parallel message-passing, shared-address-space
Parallel Programming Model • What programmer uses in coding applications? – Specifies communication and synchronization – Communication primitives exposed to user-level realizes the programming model – Ex) Uniprocessor, Multiprogramming, Data parallel, message-passing, shared-address-space 6
Aspects of Parallel Processing ③ Algorithm developer Application developer ang mode System programmer Interconnection Network Memory Memory Memory Memory Multiprocessors Multiprocessors Multiprocessors Multiprocessors Architecture designer
Aspects of Parallel Processing 7 Algorithm developer Application developer Interconnection Network Memory P P P P Memory P P P P Memory P P P P Memory P P P P Multiprocessors Multiprocessors Multiprocessors Multiprocessors Parallel computing model Parallel programming model System programmer Architecture designer 3 4 2 1 Middleware
Parallel computing models PRAM Parallel Random Access memory A set of p processors Global shared memory Each processor can access any memory location in one time step Globally synchronized Executing same program in lockstep
Parallel Computing Models • PRAM – Parallel Random Access Memory – A set of p processors – Global shared memory • Each processor can access any memory location in one time step – Globally synchronized • Executing same program in lockstep 8
lustration of pram Single program executed in MIMD mode CLK Each processor has P1 P2 P3 P a unique index p Shared memory P processors connected to a single shared memory
Illustration of PRAM 9 P1 P2 P3 Pp Shared Memory CLK P processors connected to a single shared memory Each processor has a unique index. Single program executed in MIMD mode
Features Model architecture Synchronized RAM with common clock, but not SIMD operation MIMD No local memory in each RAN One global shared memory single address space architecture Synchronization, communication, parallelism overhead are zero
Features • Model architecture – Synchronized RAM with common clock, but not SIMD operation: MIMD – No local memory in each RAM – One global shared memory • single address space architecture – Synchronization, communication, parallelism overhead are zero. 10