CUDA-based cluster Each node contains N GPUs GPU0 GPU N GPU O GPU N PCle PCle PCle 景 CPUO CPU M CPUO CPU M Host Memory Host Memory 电子件越女学 UaiversityofEectrieScincean Tecolg China
6 CUDA-based cluster – Each node contains N GPUs 6 … … GPU 0 GPU N P CIe P CIe CPU 0 CPU M Host Memory … … GPU 0 GPU N P CIe P CIe CPU 0 CPU M Host Memory
MPI Model Many processes distributed in a cluster Node Node Node Node Each process computes part of the output Processes communicate with each other Processes can synchronize 电子件越女学 O
7 MPI Model – Many processes distributed in a cluster – Each process computes part of the output – Processes communicate with each other – Processes can synchronize © David Kirk/NVIDIA and Wen-mei W. Hwu, 2007-2012 ECE408/CS483, University of Illinois, Urbana-Champaign 7 Node Node Node Node