《高等数值分析（高性能计算/并行计算）》课程教学资源（参考资料）OpenMP Application Program Interface Version 4.0

OpenMP OpenMP Application Program Interface Version 4.0-July 2013 Copyright 1997-2013 OpenMP Architecture Review Board. Permission to copy without fee all or part of this material is granted. provided the OpenMP Architecture Review Board copyright notice and the title of this document appear.Notice is given that copying is by permission of OpenMP Architecture Review Board

OpenMP Application Program Interface Version 4.0 - July 2013 Copyright © 1997-2013 OpenMP Architecture Review Board. Permission to copy without fee all or part of this material is granted, provided the OpenMP Architecture Review Board copyright notice and the title of this document appear. Notice is given that copying is by permission of OpenMP Architecture Review Board

CONTENTS 1.Introduction...................................1 1.1 Scope …1 1.2 Glossary............................................2 1.2.1 Threading Concepts..............................2 1.2.2 OpenMP Language Terminology .2 1.2.3 Synchronization Terminology ……8 1.2.4 Tasking Terminology 8 1.2.5 Data Terminology.................................10 1.2.6 Implementation Terminology …12 1.3 Execution Model ·。。4…44…。…。…。…。。。。…。4。……。。。444 14 1.4 Memory Model...........17 1.4.1 Structure of the OpenMP Memory Model...............17 1.4.2 Device Data Environments …18 1.4.3 The Flush Operation 19 1.4.4 OpenMP Memory Consistency......................20 1.5 OpenMP Compliance…… .21 1.6 Normative References ….22 1.7 Organization of this document 23 2.Directives....................25 2.1 Directive Format...........26 2.1.1 Fixed Source Form Directives .27 2.1.2 Free Source Form Directives........................28 2.1.3 Stand-Alone Directives............................31 2.2 Conditional Compilation .32 2.2.1 Fixed Source Form Conditional Compilation Sentinels ..32

i CONTENTS 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Threading Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 OpenMP Language Terminology . . . . . . . . . . . . . . . . . . . . . 2 1.2.3 Synchronization Terminology . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2.4 Tasking Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2.5 Data Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.6 Implementation Terminology . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3 Execution Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4 Memory Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4.1 Structure of the OpenMP Memory Model . . . . . . . . . . . . . . . 17 1.4.2 Device Data Environments . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.4.3 The Flush Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.4 OpenMP Memory Consistency . . . . . . . . . . . . . . . . . . . . . . 20 1.5 OpenMP Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.6 Normative References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.7 Organization of this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2. Directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1 Directive Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.1.1 Fixed Source Form Directives . . . . . . . . . . . . . . . . . . . . . . . 27 2.1.2 Free Source Form Directives . . . . . . . . . . . . . . . . . . . . . . . . 28 2.1.3 Stand-Alone Directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2 Conditional Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2.1 Fixed Source Form Conditional Compilation Sentinels . . . . 32

2.2.2 Free Source Form Conditional Compilation Sentinel ..33 2.3 Internal Control Variables 。。。。。。。。。。。。。。。。。。。。。。。。。。。 34 2.3.1 ICV Descriptions .35 2.3.2 ICV Initialization .36 2.3.3 Modifying and Retrieving ICV Values 37 2.3.4 How ICVs are Scoped 39 2.3.5 ICV Override Relationships 40 2.4 Array Sections ....42 2.5 parallel Construct .44 2.5.1 Determining the Number of Threads for a parallel Region ....... 47 2.5.2 Controlling OpenMP Thread Affinity …….49 2.6 Canonical Loop Form 51 2.7 Worksharing Constructs 。。。”，。。，。。。。。。。。…。。。。。。。。。。。。。 53 2.7.1 Loop Construct .53 2.7.2 sections Construct 60 2.7.3 single Construct 63 2.7.4 workshare Construct ，。。。。。。+。年，+。。。。。，，。，。。。。 65 2.8 SIMD Constructs .68 2.8.1 simd construct 68 2.8.2 declare simd construct 72 2.8.3 Loop SIMD construct gg。。。。。。。。g。。。。。。。。。tg。。。g。。 76 2.9 Device Constructs .77 2.9.1 target data Construct .77 2.9.2 target Construct 79 2.9.3 target update Construct .81 2.9.4 declare target Directive 83 2.9.5 teams Construct .86 2.9.6 distribute Construct 88 2.9.7 distribute simd Construct 91 ii OpenMP API Version 4.0-July 2013

ii OpenMP API • Version 4.0 - July 2013 2.2.2 Free Source Form Conditional Compilation Sentinel . . . . . . 33 2.3 Internal Control Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.3.1 ICV Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3.2 ICV Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.3 Modifying and Retrieving ICV Values . . . . . . . . . . . . . . . . . . 37 2.3.4 How ICVs are Scoped . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.3.5 ICV Override Relationships . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.4 Array Sections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.5 parallel Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.5.1 Determining the Number of Threads for a parallel Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 2.5.2 Controlling OpenMP Thread Affinity . . . . . . . . . . . . . . . . . . . 49 2.6 Canonical Loop Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.7 Worksharing Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.7.1 Loop Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.7.2 sections Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 2.7.3 single Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 2.7.4 workshare Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 2.8 SIMD Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 2.8.1 simd construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 2.8.2 declare simd construct . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 2.8.3 Loop SIMD construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 2.9 Device Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 2.9.1 target data Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 2.9.2 target Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 2.9.3 target update Construct . . . . . . . . . . . . . . . . . . . . . . . . . 81 2.9.4 declare target Directive . . . . . . . . . . . . . . . . . . . . . . . . . 83 2.9.5 teams Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 2.9.6 distribute Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 2.9.7 distribute simd Construct . . . . . . . . . . . . . . . . . . . . . . . 91

2.9.8 Distribute Parallel Loop Construct 92 2.9.9 Distribute Parallel Loop SIMD Construct .94 2.10 Combined Constructs .95 2.10.1 Parallel Loop Construct............................95 2.10.2 parallel sections Construct 97 2.10.3 parallel workshare Construct .99 2.10.4 Parallel Loop SIMD Construct 100 2.10.5 target teams construct 101 2.10.6 teams distribute Construct .102 2.10.7 teams distribute simd Construct 104 2.10.8 target teams distribute Construct 105 2.10.9 target teams distribute simd Construct 106 2.10.10 Teams Distribute Parallel Loop Construct 107 2.10.11 Target Teams Distribute Parallel Loop Construct 109 2.10.12 Teams Distribute Parallel Loop SIMD Construct 110 2.10.13 Target Teams Distribute Parallel Loop SIMD Construct....111 2.11 Tasking Constructs........ .113 2.11.1 task Construct 113 2.11.2 taskyield Construct 。。。。。。。。。。。4。。。。。。。4 117 2.11.3 Task Scheduling.................................118 2.12 Master and Synchronization Constructs..................... 120 2.12.1 master Construct 120 2.12.2 critical Construct 122 2.12.3 barrier Construct 123 2.12.4 taskwait Construct 125 2.12.5 taskgroup Construct 126 2.12.6 atomic Construct 。。。。。。。。。。4。。。。。。。。。。。。。。。。。…… 127 2.12.7 flush Construct 134 2.12.8 ordered Construct 138 2.13 Cancellation Constructs 140

iii 2.9.8 Distribute Parallel Loop Construct . . . . . . . . . . . . . . . . . . . . 92 2.9.9 Distribute Parallel Loop SIMD Construct . . . . . . . . . . . . . . . 94 2.10 Combined Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 2.10.1 Parallel Loop Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 2.10.2 parallel sections Construct . . . . . . . . . . . . . . . . . . . . . 97 2.10.3 parallel workshare Construct . . . . . . . . . . . . . . . . . . . . 99 2.10.4 Parallel Loop SIMD Construct . . . . . . . . . . . . . . . . . . . . . . . 100 2.10.5 target teams construct . . . . . . . . . . . . . . . . . . . . . . . . . . 101 2.10.6 teams distribute Construct . . . . . . . . . . . . . . . . . . . . . . 102 2.10.7 teams distribute simd Construct . . . . . . . . . . . . . . . . . 104 2.10.8 target teams distribute Construct . . . . . . . . . . . . . . . 105 2.10.9 target teams distribute simd Construct . . . . . . . . . . 106 2.10.10 Teams Distribute Parallel Loop Construct . . . . . . . . . . . . . . 107 2.10.11 Target Teams Distribute Parallel Loop Construct . . . . . . . . . 109 2.10.12 Teams Distribute Parallel Loop SIMD Construct . . . . . . . . . 110 2.10.13 Target Teams Distribute Parallel Loop SIMD Construct . . . . 111 2.11 Tasking Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 2.11.1 task Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 2.11.2 taskyield Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 2.11.3 Task Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 2.12 Master and Synchronization Constructs . . . . . . . . . . . . . . . . . . . . . . 120 2.12.1 master Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 2.12.2 critical Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 2.12.3 barrier Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 2.12.4 taskwait Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 2.12.5 taskgroup Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 2.12.6 atomic Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 2.12.7 flush Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 2.12.8 ordered Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 2.13 Cancellation Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

2.13.1 cancel Construct ….140 2.13.2 cancellation point Construct … 143 2.14 Data Environment ,146 2.14.1 Data-sharing Attribute Rules 2.14.2 threadprivate Directive .150 2.14.3 Data-Sharing Attribute Clauses 155 2.14.4 Data Copying Clauses .173 2.14.5 map Clause ....177 2.15 declare reduction Directive ......180 2.16 Nesting of Regions 186 3.Runtime Library Routines....................................187 3.1 Runtime Library Definitions …188 3.2 Execution Environment Routines 。，。。。。。。。。。。。。。。。。。。。。 189 3.2.1 omp set num threads .189 3.2.2 omp_get_num_threads ,191 3.2.3 omp_get_max_threads .192 3.2.4 omp_get_thread_num 。年+。。。。。年+。。。。。，，。。。，。。+ 193 3.2.5 omp get num procs ,195 3.2.6 omp_in_parallel ,196 3.2.7 omp_set_dynamic .197 3.2.8 omp_get_dynamic 。g年g。gg。。。。。。。。g。。g。。。。。。。。。。 198 3.2.9 omp_get_cancellation 199 3.2.10 omp_set_nested 200 3.2.11 omp_get_nested 201 3.2.12 omp set schedule 。g。。。。。。。。。g。。。。。。。。g。。。。。。 203 3.2.13 omp_get_schedule............................. 205 3.2.14 omp_get_thread_limit 206 3.2.15 omp_set_max_active_levels 207 3.2.16 omp_get_max_active_levels 209 iv OpenMP API.Version 4.0-July 2013

iv OpenMP API • Version 4.0 - July 2013 2.13.1 cancel Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 2.13.2 cancellation point Construct . . . . . . . . . . . . . . . . . . . . 143 2.14 Data Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 2.14.1 Data-sharing Attribute Rules . . . . . . . . . . . . . . . . . . . . . . . . 146 2.14.2 threadprivate Directive . . . . . . . . . . . . . . . . . . . . . . . . . 150 2.14.3 Data-Sharing Attribute Clauses . . . . . . . . . . . . . . . . . . . . . . 155 2.14.4 Data Copying Clauses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 2.14.5 map Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 2.15 declare reduction Directive . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 2.16 Nesting of Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 3. Runtime Library Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 3.1 Runtime Library Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 3.2 Execution Environment Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 3.2.1 omp_set_num_threads . . . . . . . . . . . . . . . . . . . . . . . . . . 189 3.2.2 omp_get_num_threads . . . . . . . . . . . . . . . . . . . . . . . . . . 191 3.2.3 omp_get_max_threads . . . . . . . . . . . . . . . . . . . . . . . . . . 192 3.2.4 omp_get_thread_num . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 3.2.5 omp_get_num_procs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 3.2.6 omp_in_parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 3.2.7 omp_set_dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 3.2.8 omp_get_dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 3.2.9 omp_get_cancellation . . . . . . . . . . . . . . . . . . . . . . . . . 199 3.2.10 omp_set_nested . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 3.2.11 omp_get_nested . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 3.2.12 omp_set_schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 3.2.13 omp_get_schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 3.2.14 omp_get_thread_limit . . . . . . . . . . . . . . . . . . . . . . . . . 206 3.2.15 omp_set_max_active_levels . . . . . . . . . . . . . . . . . . . . 207 3.2.16 omp_get_max_active_levels . . . . . . . . . . . . . . . . . . . . 209