Computer ArchitectureMultiprocessorsComputerArchitecture
Computer Architecture Computer Architecture Mul0processors
Multiprocessors andIssues in MultiprocessingComputerArchitecture2
Computer Architecture Multiprocessors and Issues in Multiprocessing 2
Flynn's Taxonomy of ComputersMike Flynn,“Very High-Speed Computing Systems,"Proc.of IEEE,1966SISD: Single instruction operates on single data elementSIMD: Single instruction operates on multiple data elements- Array processor-VectorprocessorMISD:Multiple instructions operate on single data element- Closestform:systolicarrayprocessor,streamingprocessorMIMD: Multiple instructions operate on multiple dataelements (multiple instructionstreams)-Multiprocessor-MultithreadedprocessorComputerArchitecture
Computer Architecture Flynn’s Taxonomy of Computers • Mike Flynn, “Very High-Speed Computing Systems, ” Proc. of IEEE, 1966 • SISD: Single instruction operates on single data element • SIMD: Single instruction operates on multiple data elements – Array processor – Vector processor • MISD: Multiple instructions operate on single data element – Closest form: systolic array processor, streaming processor • MIMD: Multiple instructions operate on multiple data elements (multiple instruction streams) – Multiprocessor – Multithreaded processor 3
Why Parallel Computers?Parallelism: Doing multiple things at a timeThings: instructions, operations, tasks. Main Goal- Improve performance (Execution time or task throughput)ExecutiontimeofaprogramgovernedbyAmdahl'sLaw: Other Goals-Reduce power consumption(4N units at freq F/4) consume less power than (N units at freq F)Why?- Improve cost efficiency and scalability, reduce complexityHardertodesignasingleunitthatperformsaswellasNsimplerunits- Improve dependability: Redundant execution in spaceComputerArchitecture
Computer Architecture Why Parallel Computers? • Parallelism: Doing multiple things at a time • Things: instructions, operations, tasks • Main Goal – Improve performance (Execution time or task throughput) • Execution time of a program governed by Amdahl’s Law • Other Goals – Reduce power consumption • (4N units at freq F/4) consume less power than (N units at freq F) • Why? – Improve cost efficiency and scalability, reduce complexity • Harder to design a single unit that performs as well as N simpler units – Improve dependability: Redundant execution in space 4
Types of Parallelism and How to Exploit Them.InstructionLevelParallelism- Different instructions within a stream canbe executed in parallel-Pipelining,out-of-orderexecution,speculative execution,VLIw-Dataflow·DataParallelism- Different pieces of data can be operated on in parallel-SIMD:Vectorprocessing,arrayprocessing- Systolic arrays, streamingprocessorsTaskLevelParallelism- Different"tasks/threads”can beexecuted inparallel- Multithreading- Multiprocessing (multi-core)ComputerArchitecture
Computer Architecture Types of Parallelism and How to Exploit Them • Instruction Level Parallelism – Different instructions within a stream can be executed in parallel – Pipelining, out-of-order execution, speculative execution, VLIW – Dataflow • Data Parallelism – Different pieces of data can be operated on in parallel – SIMD: Vector processing, array processing – Systolic arrays, streaming processors • Task Level Parallelism – Different “tasks/threads” can be executed in parallel – Multithreading – Multiprocessing (multi-core) 5