中国科学技术大学计算机科学与技术系Universityof Scienceand Technologyof ChinaDEPARTMENTOFCOMPUTERSCIENCETNAbstract- Generally speaking, parallel computing deals with theparallel computer architectures, parallel algorithms andparallel programming. In this lecture we will discussbriefly them separately. In part I, we will discuss thecontemporary parallel computer system architectures andmemory access models, parallel system interconnectionsand parallel system performance evaluation. In part IIwe will discuss the parallel computational models, thedesign methods, techniques and methodology of parallelalgorithms, as well as some parallel numerical algorithms.In part III, we will discuss the parallel programmingmodels,shared-memory, message-passing and dataparallelprogramming, as well as parallel programming environmentand tools
Abstract ▪ Generally speaking, parallel computing deals with the parallel computer architectures, parallel algorithms and parallel programming. In this lecture we will discuss briefly them separately. In part I, we will discuss the contemporary parallel computer system architectures and memory access models, parallel system interconnections and parallel system performance evaluation. In part II, we will discuss the parallel computational models, the design methods, techniques and methodology of parallel algorithms, as well as some parallel numerical algorithms. In part III, we will discuss the parallel programming models,shared-memory, message-passing and dataparallel programming, as well as parallel programming environment and tools
中国科学技术大学计算机科学与技术系JniversityofScienceandTechnologyof ChinaDEPARTMENTCOMPLSIENCEmPart I : Parallel Computer SystemsHardware Platform for ParallelComputing: System Architectures and Models System Interconnections Performance Evaluation
Part I : Parallel Computer Systems Hardware Platform for Parallel Computing: ▪ System Architectures and Models ▪ System Interconnections ▪ Performance Evaluation
中国科学技术大学计算机科学与技术系JniversityofScience andTechnologyof ChinaDEPARTNENTCOMPERSCIENCEANDTEOHNOLFSystem Architectures and Models- Parallel Computer System ArchitecturesPVP:ParallelVectorProcessorsSMP:SymmetricMultiprocessorsMPP:MassivelyParallelProcessors- DSM : Distributed Shared MemoryCOW:ClusterOf Workstations Parallel Computer Memory Access ModelsUMA:UniformMemoryAccessNUMA:Non-UniformMemoryAccessCOMA:Cache-OnlyMemoryAccessNORMA:NO-Remote Memory Access
System Architectures and Models ▪ Parallel Computer System Architectures ▪ PVP : Parallel Vector Processors ▪ SMP : Symmetric Multiprocessors ▪ MPP : Massively Parallel Processors ▪ DSM : Distributed Shared Memory ▪ COW : Cluster Of Workstations ▪ Parallel Computer Memory Access Models ▪ UMA : Uniform Memory Access ▪ NUMA : Non-Uniform Memory Access ▪ COMA : Cache-Only Memory Access ▪ NORMA : NO-Remote Memory Access
中国科学技术大学计算机科学与技术系Jniversityof Scienceand TechnologyofChinaDEPARTVENTCCMSCIENCEANSystem InterconnectionsNetworkEnvironments Intra-node Interconnections(Buses,Switches)Inter-nodeInterconnections(SAN) Inter-systemInterconnections(LAN,MAN, WAN) Interconnection TopologiesStatic-ConnectionNetworks(LARC.MCTCHCCcC- Dynamic-Connection Networks (Buses, Crossbar, MIN)Wide-BandNetworks- FDDI(FiberDistributedDataInterface)FE/GE(FastEthernet/GigabitEthernet)ATM(AsynchronousTransferMode)ScI(ScalableCoherenceInterface)
System Interconnections ▪ Network Environments ▪ Intra-node Interconnections( Buses , Switches ) ▪ Inter-node Interconnections( SAN ) ▪ Inter-system Interconnections( LAN , MAN , WAN ) ▪ Interconnection Topologies ▪ Static-Connection Networks( LA,RC,MC,TC,HC,CCC) ▪ Dynamic-Connection Networks (Buses, Crossbar, MIN) ▪ Wide-Band Networks ▪ FDDI( Fiber Distributed Data Interface ) ▪ FE/GE( Fast Ethernet / Gigabit Ethernet ) ▪ ATM( Asynchronous Transfer Mode ) ▪ SCI( Scalable Coherence Interface )
中国科学技术大学计算机科学与技术系Universityof Scienceand Technologyof ChinaDEPARTVENTCOMPUTERSCEOHNOLOOFPerformance Evaluation-SpeedupofSystemsAmdahl'sLaw- Gustafson's LawSunandNi'sLawScalabilityof Systems.Iso-efficiencyIso-speedAverage LatencyPerformanceof Systems:BenchmarksLINPACKSPECPARKBENCHNASetc
Performance Evaluation ▪ Speed up of Systems ▪ Amdahl’s Law ▪ Gustafson’s Law ▪ Sun and Ni’s Law ▪ Scalability of Systems ▪ Iso-efficiency ▪ Iso-speed ▪ Average Latency ▪ Performance of Systems : Benchmarks ▪ LINPACK ▪ SPEC ▪ PARKBENCH ▪ NAS etc