2.9 Implementation Issues 21 2.9.1 Independence of Basic Runtime Routines 2.9.2 Interaction with Signals.. 2.10 Examples 3 3 Point-to-Point Communication 3.2 Blocking Send and Receive Operations..................... 3.2.1 Blocking Send............................. 3.22 3.2.3 Message Envelope...· 2844578 3.2.4 Blocking Receive 3.2.5 Keturn Status 3.2.6 Passing MPI_STATUS_IGNORE for Status.. 。。。，。。。。。。，，。 3.3 Data Type Matching and Data Conversion. 3.3.1 Type Matching Rules Type MPI_CHARACTER. 3.3.2 Data Conversion·· 45 3.4 Communication Modes 3.5 Semantics of Point-to-Point Communication 3.6 Buffer Allocation and Usage 3.6.1 Model Implementation of Buffered mode 3.7 Nonblocking Communication 3.7.1 Communication request obiects 3.7.2 Communication Initiation 3.7.3 Communication Completion 82 3.7.4 Semantics of Nonblocking Communications 3.75 Multiple Completions 3.7.6 Non-destructive Test of status 3.8 Probe and Cancel. 38 1 Probe 3.8.2 Matching Probe 3.8.3 Matched Receives 3.8.4 Cancel 3.9 Persistent Communication Requests 3 10 Send Receive 3.11 Null Processes 80 4 Datatypes 好 4.1 Derived Datatypes 4.1.1 Type Constructors with Explicit Addresses.... ，，，。，，。。 4.1.2 Datatype Constructors.. 855 4.1.3 Subarray Datatype Constructor ... 4.1.4 Distributed Array Datatype Constructor. 4.1.5 Address and Size Functions.. 101 4.1.6 Lower-Bound and Upper-Bound Markers 104 4.1.7 Extent and Bounds of Datatypes 106 4.1.8 True Extent of Datatypes 108
2.9 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.9.1 Independence of Basic Runtime Routines . . . . . . . . . . . . . . . 22 2.9.2 Interaction with Signals . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.10 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3 Point-to-Point Communication 23 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.2 Blocking Send and Receive Operations . . . . . . . . . . . . . . . . . . . . . 24 3.2.1 Blocking Send . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.2.2 Message Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2.3 Message Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.4 Blocking Receive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2.5 Return Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2.6 Passing MPI_STATUS_IGNORE for Status . . . . . . . . . . . . . . . . 32 3.3 Data Type Matching and Data Conversion . . . . . . . . . . . . . . . . . . 33 3.3.1 Type Matching Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Type MPI_CHARACTER . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.2 Data Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.4 Communication Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.5 Semantics of Point-to-Point Communication . . . . . . . . . . . . . . . . . . 40 3.6 Buffer Allocation and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.6.1 Model Implementation of Buffered Mode . . . . . . . . . . . . . . . . 46 3.7 Nonblocking Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.7.1 Communication Request Objects . . . . . . . . . . . . . . . . . . . . 48 3.7.2 Communication Initiation . . . . . . . . . . . . . . . . . . . . . . . . 48 3.7.3 Communication Completion . . . . . . . . . . . . . . . . . . . . . . . 52 3.7.4 Semantics of Nonblocking Communications . . . . . . . . . . . . . . 56 3.7.5 Multiple Completions . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.7.6 Non-destructive Test of status . . . . . . . . . . . . . . . . . . . . . . 63 3.8 Probe and Cancel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.8.1 Probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.8.2 Matching Probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.8.3 Matched Receives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.8.4 Cancel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.9 Persistent Communication Requests . . . . . . . . . . . . . . . . . . . . . . 73 3.10 Send-Receive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.11 Null Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 4 Datatypes 83 4.1 Derived Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.1.1 Type Constructors with Explicit Addresses . . . . . . . . . . . . . . 85 4.1.2 Datatype Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.1.3 Subarray Datatype Constructor . . . . . . . . . . . . . . . . . . . . . 94 4.1.4 Distributed Array Datatype Constructor . . . . . . . . . . . . . . . . 96 4.1.5 Address and Size Functions . . . . . . . . . . . . . . . . . . . . . . . 101 4.1.6 Lower-Bound and Upper-Bound Markers . . . . . . . . . . . . . . . 104 4.1.7 Extent and Bounds of Datatypes . . . . . . . . . . . . . . . . . . . . 106 4.1.8 True Extent of Datatypes . . . . . . . . . . . . . . . . . . . . . . . . 108 vi
4.1.9 Commit and Free 109 111 ral Data 1 4.1.12 Correct Use of Addr 1 4 1 13 Dec odin 4.1.14Ex Canonical MPI_PACK and MPI_UNPACK 138 5 Collective Communication 141 51 Introduction and Overview 141 5.2 Communicator Argument 144 521 Specifics for Intracommunicator Collective Operations 144 52.2 Applying Collective Operations to Intercommunicators 145 5.23 ecifics for Intercommunicator Collective Operations 146 5.3 Barrier synchronization 147 5.4 Broadcast 4 5.4.1 Example using MPI_BCAST 5.5 Gather 14 5.5.1 Examples using MPl GATtHER.MPI GATHERV 5.6 Scatter 15 5.6.1 Examples using MPl sCatTER.MPI SCATTERV 16 5.7 Gather-to-all 5.7.1 Example using MPI_ALLGATHER 5.8 All-to-All Scatter/Gather 5.9 Global Reduction Operations 173 17 500 Predefined reduction on 170 502 nd Reductions 17 504 MINLOC and MAxlOC 170 595 Use r-Defined Reduction Op 1Q e of User-defined Reduce 18 506 All-Roduc 187 ocal Reduction 5.10 Reduco 5.10.1 MPI REDUCE SCATTER BLOCK 190 5.10.2 MPI_REDUCE_SCATTER 191 5.11Sc 5..1nc 193 5.112 xclusive 990 飞T11.1 MPI_IBCAST 5.123 g 20 5.12.5 Nonbloc king Gather-to-all
4.1.9 Commit and Free . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.1.10 Duplicating a Datatype . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.1.11 Use of General Datatypes in Communication . . . . . . . . . . . . . 111 4.1.12 Correct Use of Addresses . . . . . . . . . . . . . . . . . . . . . . . . 115 4.1.13 Decoding a Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.1.14 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 4.2 Pack and Unpack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 4.3 Canonical MPI_PACK and MPI_UNPACK . . . . . . . . . . . . . . . . . . . 138 5 Collective Communication 141 5.1 Introduction and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 5.2 Communicator Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 5.2.1 Specifics for Intracommunicator Collective Operations . . . . . . . . 144 5.2.2 Applying Collective Operations to Intercommunicators . . . . . . . . 145 5.2.3 Specifics for Intercommunicator Collective Operations . . . . . . . . 146 5.3 Barrier Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 5.4 Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 5.4.1 Example using MPI_BCAST . . . . . . . . . . . . . . . . . . . . . . . 149 5.5 Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 5.5.1 Examples using MPI_GATHER, MPI_GATHERV . . . . . . . . . . . . 152 5.6 Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.6.1 Examples using MPI_SCATTER, MPI_SCATTERV . . . . . . . . . . 162 5.7 Gather-to-all . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 5.7.1 Example using MPI_ALLGATHER . . . . . . . . . . . . . . . . . . . . 167 5.8 All-to-All Scatter/Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 5.9 Global Reduction Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 173 5.9.1 Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 5.9.2 Predefined Reduction Operations . . . . . . . . . . . . . . . . . . . . 176 5.9.3 Signed Characters and Reductions . . . . . . . . . . . . . . . . . . . 178 5.9.4 MINLOC and MAXLOC . . . . . . . . . . . . . . . . . . . . . . . . 179 5.9.5 User-Defined Reduction Operations . . . . . . . . . . . . . . . . . . 183 Example of User-defined Reduce . . . . . . . . . . . . . . . . . . . . 186 5.9.6 All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 5.9.7 Process-Local Reduction . . . . . . . . . . . . . . . . . . . . . . . . . 189 5.10 Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 5.10.1 MPI_REDUCE_SCATTER_BLOCK . . . . . . . . . . . . . . . . . . . 190 5.10.2 MPI_REDUCE_SCATTER . . . . . . . . . . . . . . . . . . . . . . . . 191 5.11 Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 5.11.1 Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 5.11.2 Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 5.11.3 Example using MPI_SCAN . . . . . . . . . . . . . . . . . . . . . . . . 195 5.12 Nonblocking Collective Operations . . . . . . . . . . . . . . . . . . . . . . . 196 5.12.1 Nonblocking Barrier Synchronization . . . . . . . . . . . . . . . . . . 198 5.12.2 Nonblocking Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . 199 Example using MPI_IBCAST . . . . . . . . . . . . . . . . . . . . . . 199 5.12.3 Nonblocking Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 5.12.4 Nonblocking Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 5.12.5 Nonblocking Gather-to-all . . . . . . . . . . . . . . . . . . . . . . . . 204 vii
5.12.6 Nonblocking All-to-All Scatter/Gather... 200 5.12.7 Nonblocking Reduce 209 5.12 8 Nonblocking AllReduce 210 5.12.9 Nonblocking Reduce-Scatter with Equal Blocks 211 5 12 10Nonblocking Reduce Scatter 212 5 12 11 Nonblocking Inclusive Scan 213 5.12.12 Nonblocking Exclusive Scan 214 5.13 Correctness 214 6 Groups,Contexts,Communicators,and Caching 6.1 Introduction.· 6.1.1 Features Needed to Support Libraries 6.1.2 MPI's Support for Libraries 62 Basic Concepts................ 6.2.1 Groups······· 226 6.2.2 Contexts 226 6.2.3 Intra-Communicators 227 6.2.4 Predefined Intra-Communicators 227 6.3 Group Management·· 228 6.31 Grou Accessors 228 6.3.2 Group Constructors 230 633 Groun Destructors 235 6.4 Communicator Management 235 6.4.1 Communicator accessors 235 6.42 Communicator Constructors 237 6.43 Communicator Destructors 248 6.4.4 Communicator Info. 248 6.5 Motivating Examples. 250 6.5.1 Current practice 25 6.5.2 Current Practice #2 6.5.3 (Approximate)Current Practice #3 251 6.5.4 Example #4 252 6.5.5 Library Example #1 253 6.5.6 Library Example #2 255 6.6 Inter-Communication 661 Inter-communicator Accessors 6.6.2 Inter-communicator Operations 260 6.6.3 Inter-Communication Ex Example 1:Three-Gro Example 2:Three-Gro p“Ring 264 6.7 265 6.7.1 Functionality 266 6.7.2 267 6.7.3 Windows 272 6.7.4 Dataty 275 6.7.5 Class for Invalid Keyval 279 6.7.6 279 6.8 Naming 2 viii
5.12.6 Nonblocking All-to-All Scatter/Gather . . . . . . . . . . . . . . . . . 206 5.12.7 Nonblocking Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 5.12.8 Nonblocking All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . . 210 5.12.9 Nonblocking Reduce-Scatter with Equal Blocks . . . . . . . . . . . . 211 5.12.10 Nonblocking Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . . . 212 5.12.11 Nonblocking Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . . 213 5.12.12 Nonblocking Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . . 214 5.13 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 6 Groups, Contexts, Communicators, and Caching 223 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 6.1.1 Features Needed to Support Libraries . . . . . . . . . . . . . . . . . 223 6.1.2 MPI’s Support for Libraries . . . . . . . . . . . . . . . . . . . . . . . 224 6.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 6.2.1 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 6.2.2 Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 6.2.3 Intra-Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . 227 6.2.4 Predefined Intra-Communicators . . . . . . . . . . . . . . . . . . . . 227 6.3 Group Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 6.3.1 Group Accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 6.3.2 Group Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 6.3.3 Group Destructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 6.4 Communicator Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 6.4.1 Communicator Accessors . . . . . . . . . . . . . . . . . . . . . . . . 235 6.4.2 Communicator Constructors . . . . . . . . . . . . . . . . . . . . . . . 237 6.4.3 Communicator Destructors . . . . . . . . . . . . . . . . . . . . . . . 248 6.4.4 Communicator Info . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 6.5 Motivating Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 6.5.1 Current Practice #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 6.5.2 Current Practice #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 6.5.3 (Approximate) Current Practice #3 . . . . . . . . . . . . . . . . . . 251 6.5.4 Example #4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 6.5.5 Library Example #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 6.5.6 Library Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 6.6 Inter-Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 6.6.1 Inter-communicator Accessors . . . . . . . . . . . . . . . . . . . . . . 259 6.6.2 Inter-communicator Operations . . . . . . . . . . . . . . . . . . . . . 260 6.6.3 Inter-Communication Examples . . . . . . . . . . . . . . . . . . . . . 263 Example 1: Three-Group “Pipeline” . . . . . . . . . . . . . . . . . . 263 Example 2: Three-Group “Ring” . . . . . . . . . . . . . . . . . . . . 264 6.7 Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 6.7.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 6.7.2 Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 6.7.3 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 6.7.4 Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 6.7.5 Error Class for Invalid Keyval . . . . . . . . . . . . . . . . . . . . . . 279 6.7.6 Attributes Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 6.8 Naming Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 viii
6.9 Formalizing the Loosely Synchronous Model 285 6 9 1 Basic Statem 285 6.9.2 Models of Execution 286 Static Comn nicator Allocation 286 unicator Allocation The General Case 7 Process Topologies 289 7.1 Introduction 289 72 Virtual Topologies 290 7.3 Embedding in MPl 290 290 75 Topology Constructors 292 751 Cartesian Constructor 292 75.2 Cartesian Convenience Function:MPI_DIMS_CREATE 292 753 Graph Constructor 294 754 Distributed Graph Constructor 296 755 Topology inguiry functions 30 75 6 Cartesian Shift Coordinates 310 7.5.7 Partitioning of cartesian structures 311 7.5.8 Low-Level Topology Functions 219 7.6 Neighborhood Collective Communication 4 7.6.1 Neighborhood Gather 5 760 Neighbor Alltoall 8 Nonblocking Neighborhood Com 771 Nonblocking Neighborhood Gathe 324 779 Nonblocking Neighborhood Alltoall 326 7.8 An Application Example 329 8 MPI Environmental Management 333 8.I Implementation Information 333 8.1.1 Version Inguiries ，，。 333 8.1.2 Environmental Inquiries 334 Tag values 335 Host Rank 335 IO Rank. 335 Clock Synchronization 336 Inquire Processor Name 336 8.2 Memory Allocation. 337 8.3 Error Handling 340 8 3 1 Error Handlers for communicators 83.2 Error Handlers for Windows 343 833 Error Handlers for Files 345 8.3.4 Freeing Errorhandlers and Retrieving Error strings 346 8 4 Error Codes and Classes 347 85 Error Classes,Error Codes,and Error Handlers 350 8.6 Timers and Synchronization 354 8.7 Startup 355 ix
6.9 Formalizing the Loosely Synchronous Model . . . . . . . . . . . . . . . . . . 285 6.9.1 Basic Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 6.9.2 Models of Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 Static Communicator Allocation . . . . . . . . . . . . . . . . . . . . 286 Dynamic Communicator Allocation . . . . . . . . . . . . . . . . . . . 286 The General Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 7 Process Topologies 289 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 7.2 Virtual Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 7.3 Embedding in MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 7.4 Overview of the Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 7.5 Topology Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 7.5.1 Cartesian Constructor . . . . . . . . . . . . . . . . . . . . . . . . . . 292 7.5.2 Cartesian Convenience Function: MPI_DIMS_CREATE . . . . . . . . 292 7.5.3 Graph Constructor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 7.5.4 Distributed Graph Constructor . . . . . . . . . . . . . . . . . . . . . 296 7.5.5 Topology Inquiry Functions . . . . . . . . . . . . . . . . . . . . . . . 302 7.5.6 Cartesian Shift Coordinates . . . . . . . . . . . . . . . . . . . . . . . 310 7.5.7 Partitioning of Cartesian Structures . . . . . . . . . . . . . . . . . . 311 7.5.8 Low-Level Topology Functions . . . . . . . . . . . . . . . . . . . . . 312 7.6 Neighborhood Collective Communication . . . . . . . . . . . . . . . . . . . 314 7.6.1 Neighborhood Gather . . . . . . . . . . . . . . . . . . . . . . . . . . 315 7.6.2 Neighbor Alltoall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 7.7 Nonblocking Neighborhood Communication . . . . . . . . . . . . . . . . . . 323 7.7.1 Nonblocking Neighborhood Gather . . . . . . . . . . . . . . . . . . . 324 7.7.2 Nonblocking Neighborhood Alltoall . . . . . . . . . . . . . . . . . . . 326 7.8 An Application Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 8 MPI Environmental Management 333 8.1 Implementation Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 8.1.1 Version Inquiries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 8.1.2 Environmental Inquiries . . . . . . . . . . . . . . . . . . . . . . . . . 334 Tag Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Host Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 IO Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Clock Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 336 Inquire Processor Name . . . . . . . . . . . . . . . . . . . . . . . . . 336 8.2 Memory Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 8.3 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 8.3.1 Error Handlers for Communicators . . . . . . . . . . . . . . . . . . . 341 8.3.2 Error Handlers for Windows . . . . . . . . . . . . . . . . . . . . . . . 343 8.3.3 Error Handlers for Files . . . . . . . . . . . . . . . . . . . . . . . . . 345 8.3.4 Freeing Errorhandlers and Retrieving Error Strings . . . . . . . . . . 346 8.4 Error Codes and Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 8.5 Error Classes, Error Codes, and Error Handlers . . . . . . . . . . . . . . . . 350 8.6 Timers and Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 8.7 Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 ix
8.7.1 Allowing User Functions at Process Termination 361 8.7.2 Determining Whether MPI Has Finished.. 361 8.8 Portable MPI Process Startup... 362 9 The Info Object 366 10 Process Creation and Management 371 27 10.2 The Dyr amic Process Model 372 372 1022 The Runtime Env en 2 10.3 Proc 27 es in MP 7 and Establishin mication 27 g Multiple Executables and ablish ing Communication d Keys 3 wn _COMM_SPAWN 10.4 Establish ame sses,Ports,and All Tha r Ro tines 386 Clie Name Publishi Key alues 。。。。。。，。。。 10.4.6 Client/Se rver Examples imple 上xample- Completely Portable Ocean/Atmosphere- Relies on Name Pu blishing Simple Client-Server Example 10.5 Other Functionality 。。。 10. Universe Size 397 10.5.2 Singleton MPI_INIT 10.5.3 MPI_APPNUM . ，。。，。。。，。。。，。。 10.5.4 Releasing Connection 397 10.5.5 Another Way to Establish MPI Communication............ 3 11 One-Sided Communications 40, 1111nt 0 11 2 1 Window Cre 112.2 Window at An. 112.3 Window emor 405 M 407 11.2.4 Windoy Dyna cally Attached Memory 112.5w 413 1126 Windo Attributes 415 cation Call Get
8.7.1 Allowing User Functions at Process Termination . . . . . . . . . . . 361 8.7.2 Determining Whether MPI Has Finished . . . . . . . . . . . . . . . . 361 8.8 Portable MPI Process Startup . . . . . . . . . . . . . . . . . . . . . . . . . . 362 9 The Info Object 365 10 Process Creation and Management 371 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 10.2 The Dynamic Process Model . . . . . . . . . . . . . . . . . . . . . . . . . . 372 10.2.1 Starting Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 10.2.2 The Runtime Environment . . . . . . . . . . . . . . . . . . . . . . . 372 10.3 Process Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 10.3.1 Processes in MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 10.3.2 Starting Processes and Establishing Communication . . . . . . . . . 374 10.3.3 Starting Multiple Executables and Establishing Communication . . 379 10.3.4 Reserved Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 10.3.5 Spawn Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Manager-worker Example Using MPI_COMM_SPAWN . . . . . . . . 383 10.4 Establishing Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 385 10.4.1 Names, Addresses, Ports, and All That . . . . . . . . . . . . . . . . 385 10.4.2 Server Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 10.4.3 Client Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 10.4.4 Name Publishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 10.4.5 Reserved Key Values . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 10.4.6 Client/Server Examples . . . . . . . . . . . . . . . . . . . . . . . . . 392 Simplest Example — Completely Portable. . . . . . . . . . . . . . . 392 Ocean/Atmosphere — Relies on Name Publishing . . . . . . . . . . 393 Simple Client-Server Example . . . . . . . . . . . . . . . . . . . . . . 393 10.5 Other Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 10.5.1 Universe Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 10.5.2 Singleton MPI_INIT . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 10.5.3 MPI_APPNUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 10.5.4 Releasing Connections . . . . . . . . . . . . . . . . . . . . . . . . . . 397 10.5.5 Another Way to Establish MPI Communication . . . . . . . . . . . . 399 11 One-Sided Communications 401 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 11.2 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 11.2.1 Window Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 11.2.2 Window That Allocates Memory . . . . . . . . . . . . . . . . . . . . 405 11.2.3 Window That Allocates Shared Memory . . . . . . . . . . . . . . . . 407 11.2.4 Window of Dynamically Attached Memory . . . . . . . . . . . . . . 410 11.2.5 Window Destruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 11.2.6 Window Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414 11.2.7 Window Info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 11.3 Communication Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 11.3.1 Put . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 11.3.2 Get . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 x