MPI CART SUB Variable c Type In/Out Description Name old comm MPI Comm Input Cartesian Communicator handle belongs int Input Array of size dims specifying whether a dimension belongs to new comm new comm MPI Comm Output Cartesian Communicator handle
MPI_CART_SUB Variable Name C Type In/Out Description old_comm MPI_Comm Input Cartesian Communicator handle belongs int * Input Array of size ndims specifying whether a dimension belongs to new_comm new_comm MPI_Comm Output Cartesian Communicator handle
MPI CART SUB For a 2D Cartesian grid, create subgrids of rows and columns Create Cartesian topology for processes Create 2D Cartesian topology for processes MPI Cart create(MPI COMM WORLD, ndim, dims, period, reorder &comm2D) MPI Comm rank(comm2D, &id2D) MPI Cart coords(comm2D, id2D, ndim, coords2D) Create 1D row subgrids * it belongs[01=0 belongs[1]=1: this dimension belongs to subgrid MPI Cart sub(comm2D, belongs, &commrow) / Create 1D column subgrids * belongs[0]=1: /this dimension belongs to subgrid * belongs(1=0; MPI Cart sub(comm2D, belongs, &commcol
MPI_CART_SUB • For a 2D Cartesian grid, create subgrids of rows and columns. Create Cartesian topology for processes. /* Create 2D Cartesian topology for processes */ MPI_Cart_create(MPI_COMM_WORLD, ndim, dims, period, reorder, &comm2D); MPI_Comm_rank(comm2D, &id2D); MPI_Cart_coords(comm2D, id2D, ndim, coords2D); /* Create 1D row subgrids */ belongs[0] = 0; belongs[1] = 1; ! this dimension belongs to subgrid MPI_Cart_sub(comm2D, belongs, &commrow); /* Create 1D column subgrids */ belongs[0] = 1; /* this dimension belongs to subgrid */ belongs[1] = 0; MPI_Cart_sub(comm2D, belongs, &commcol);
MPI CART SUB Shown in Figure 8. 4 (a below is a 3-by-2 Cartesian topology Figure 8.4(b)shows the resulting row subgrids, while Figure 8. 4( c)shows the corresponding column subgrids. In black, the first row of numbers in each cell lists the 2D Cartesian coordinate index pair i i and the associated rank number on the second row and in green, are shown the 1D subgrid Cartesian coordinates and the subgrid rank number(in parentheses). Their order is counted relative to their respective subgrid communicator groups
MPI_CART_SUB • Shown in Figure 8.4 (a) below is a 3-by-2 Cartesian topology. Figure 8.4 (b) shows the resulting row subgrids, while Figure 8.4 (c) shows the corresponding column subgrids. In black, the first row of numbers in each cell lists the 2D Cartesian coordinate index pair "i,j" and the associated rank number. On the second row, and in green, are shown the 1D subgrid Cartesian coordinates and the subgrid rank number (in parentheses). Their order is counted relative to their respective subgrid communicator groups
MPI CART SUB 0,0(0)0,1(1) 0.0(0)0,1(1) 00(0)01(1) 0(0) 1,0(2)11(3) 1,0(2)11(3) 10(2)11(3) 0(0)1(1) 20(4)21(5) 20(4)21(5) 20(4)21(5) 2(2 Figure 8.4a) Figure 8. 4 Figure 8.4(c) 2D Cartesian ( b 3 RoW 2 Column Grid Subgrids Subgrids
MPI_CART_SUB 0,0(0) 0,1(1) 0,0(0) 0(0) 0,1(1) 1(1) 0,0(0) 0(0) 0,1(1) 0(0) 1,0(2) 1,1(3) 1,0(2) 0(0) 1,1(3) 1(1) 1,0(2) 1(1) 1,1(3) 1(1) 2,0(4) 2,1(5) 2,0(4) 0(0) 2,1(5) 1(1) 2,0(4) 2(2) 2,1(5) 2(2) Figure 8.4 (a). 2D Cartesian Grid Figure 8.4 (b). 3 Row Subgrids Figure 8.4 (c). 2 Column Subgrids
MPI CART SUB Note MPI CART SUB is a collective routine It must be called by all processes in old comm MPL_CART_SUB generated subgrid communicators are derived from Cartesian grid created with MPI CART CREATE Full length of each dimension of the original Cartesian grid is used in the subgrids Each subgrid has a corresponding communicator. It inherits properties of the parent Cartesian grid; it remains a Cartesian grid It returns the communicator to which the calling process belongs There is a comparable MPl COMM SPLIT to perform similar function MPI CARTDIM GET and MP CaRT GET can be used to acquire structural information of a grid (such as dimension, size periodicity)
MPI_CART_SUB • Note: – MPI_CART_SUB is a collective routine. It must be called by all processes in old_comm. – MPI_CART_SUB generated subgrid communicators are derived from Cartesian grid created with MPI_CART_CREATE. – Full length of each dimension of the original Cartesian grid is used in the subgrids. – Each subgrid has a corresponding communicator. It inherits properties of the parent Cartesian grid; it remains a Cartesian grid. – It returns the communicator to which the calling process belongs. – There is a comparable MPI_COMM_SPLIT to perform similar function. – MPI_CARTDIM_GET and MPI_CART_GET can be used to acquire structural information of a grid (such as dimension, size, periodicity)