xii CONTENTS 7.2 Examples of lock-free data structures 184 Writing a thread-safe stack without locks 184"Stopping those pesky leaks:managing memory in lock-free data structures 188 Detecting nodes that can't be reclaimed using hazard pointers 193 Detecting nodes in use with reference counting 200"Applying the memory model to the lock-free stack 205 Writing a thread-safe queue without locks 209 7.3 Guidelines for writing lock-free data structures 221 Guideline:use std::memoryorder seq_cst for prototyping 221 Guideline:use a lock-free memory reclamation scheme 221 Guideline:watch out for the ABA problem 222 Guideline:identify busy-wait loops and help the other thread 222 7.4 Summary 223 Designing concurrent code 224 8.1 Techniques for dividing work between threads 225 Dividing data between threads before processing begins 226 Dividing data recursively 227"Dividing work by task type 231 8.2 Factors affecting the performance of concurrent code 233 How many processors?234Data contention and cache ping-pong235·False sharing237·How close is your data？238·Oversubscription and excessive task switching 239 8.3 Designing data structures for multithreaded performance 239 Dividing array elements for complex operations 240 Data access patterns in other data structures 242 8.4 Additional considerations when designing for concurrency 243 Exception safety in parallel algorithms 243 Scalability and Amdahl's law 250Hiding latency with multiple threads 252 Improving responsiveness with concurrency 253 8.5 Designing concurrent code in practice 255 A parallel implementation of std::for_each 255"A parallel implementation of std::find 257"A parallel implementation of std::partial sum 263 8.6 Summary 272
xii CONTENTS 7.2 Examples of lock-free data structures 184 Writing a thread-safe stack without locks 184 ■ Stopping those pesky leaks: managing memory in lock-free data structures 188 Detecting nodes that can’t be reclaimed using hazard pointers 193 Detecting nodes in use with reference counting 200 ■ Applying the memory model to the lock-free stack 205 ■ Writing a thread-safe queue without locks 209 7.3 Guidelines for writing lock-free data structures 221 Guideline: use std::memory_order_seq_cst for prototyping 221 Guideline: use a lock-free memory reclamation scheme 221 Guideline: watch out for the ABA problem 222 Guideline: identify busy-wait loops and help the other thread 222 7.4 Summary 223 8 Designing concurrent code 224 8.1 Techniques for dividing work between threads 225 Dividing data between threads before processing begins 226 Dividing data recursively 227 ■ Dividing work by task type 231 8.2 Factors affecting the performance of concurrent code 233 How many processors? 234 ■ Data contention and cache ping-pong 235 ■ False sharing 237 ■ How close is your data? 238 ■ Oversubscription and excessive task switching 239 8.3 Designing data structures for multithreaded performance 239 Dividing array elements for complex operations 240 Data access patterns in other data structures 242 8.4 Additional considerations when designing for concurrency 243 Exception safety in parallel algorithms 243 ■ Scalability and Amdahl’s law 250 ■ Hiding latency with multiple threads 252 Improving responsiveness with concurrency 253 8.5 Designing concurrent code in practice 255 A parallel implementation of std::for_each 255 ■ A parallel implementation of std::find 257 ■ A parallel implementation of std::partial_sum 263 8.6 Summary 272
CONTENTS xi进 Advanced thread management 273 9.1 Thread pools 274 The simplest possible thread pool 274"Waiting for tasks submitted to a thread pool 276"Tasks that wait for other tasks 280 Avoiding contention on the work queue 283 Work stealing 284 9.2 Interrupting threads 289 Launching and interrupting another thread 289.Detecting that a thread has been interrupted 291"Interrupting a condition variable wait 291 Interrupting a wait on std::condition_variable any 294"Interrupting other blocking calls 296 Handling interruptions 297 Interrupting background tasks on application exit 298 9.3 Summary 299 10 Testing and debugging multithreaded applications 300 10.1 Types of concurrency-related bugs 301 Unwanted blocking 301 Race conditions 302 10.2 Techniques for locating concurrency-related bugs 303 Reviewing code to locate potential bugs 303 Locating concurrency-related bugs by testing 305 Designing for testability 307"Multithreaded testing techniques 308 Structuring multithreaded test code 311 Testing the performance of multithreaded code 314 10.3 Summary 314 appendix A Brief reference for some C++11 language features 315 appendix B Brief comparison of concurrency libraries 340 appendix C A message-passing framework and complete ATM example 342 appendix D C++Thread Library reference 360 resources 487 index 489
CONTENTS xiii 9 Advanced thread management 273 9.1 Thread pools 274 The simplest possible thread pool 274 ■ Waiting for tasks submitted to a thread pool 276 ■ Tasks that wait for other tasks 280 ■ Avoiding contention on the work queue 283 Work stealing 284 9.2 Interrupting threads 289 Launching and interrupting another thread 289 ■ Detecting that a thread has been interrupted 291 ■ Interrupting a condition variable wait 291 ■ Interrupting a wait on std::condition_variable_any 294 ■ Interrupting other blocking calls 296 ■ Handling interruptions 297 Interrupting background tasks on application exit 298 9.3 Summary 299 10 Testing and debugging multithreaded applications 300 10.1 Types of concurrency-related bugs 301 Unwanted blocking 301 ■ Race conditions 302 10.2 Techniques for locating concurrency-related bugs 303 Reviewing code to locate potential bugs 303 Locating concurrency-related bugs by testing 305 Designing for testability 307 ■ Multithreaded testing techniques 308 ■ Structuring multithreaded test code 311 Testing the performance of multithreaded code 314 10.3 Summary 314 appendix A Brief reference for some C++11 language features 315 appendix B Brief comparison of concurrency libraries 340 appendix C A message-passing framework and complete ATM example 342 appendix D C++ Thread Library reference 360 resources 487 index 489
preface I encountered the concept of multithreaded code while working at my first job after I left college.We were writing a data processing application that had to populate a data- base with incoming data records.There was a lot of data,but each record was inde- pendent and required a reasonable amount of processing before it could be inserted into the database.To take full advantage of the power of our 10-CPU UltraSPARC,we ran the code in multiple threads,each thread processing its own set of incoming records.We wrote the code in C++,using POSIX threads,and made a fair number of mistakes-multithreading was new to all of us-but we got there in the end.It was also while working on this project that I first became aware of the C++Standards Commit- tee and the freshly published C++Standard. I have had a keen interest in multithreading and concurrency ever since.Where others saw it as difficult,complex,and a source of problems,I saw it as a powerful tool that could enable your code to take advantage of the available hardware to run faster. Later on I would learn how it could be used to improve the responsiveness and perfor- mance of applications even on single-core hardware,by using multiple threads to hide the latency of time-consuming operations such as I/O.I also learned how it worked at the OS level and how Intel CPUs handled task switching. Meanwhile,my interest in C++brought me in contact with the ACCU and then the C++Standards panel at BSI,as well as Boost.I followed the initial development of the Boost Thread Library with interest,and when it was abandoned by the original developer,I jumped at the chance to get involved.I have been the primary developer and maintainer of the Boost Thread Library ever since. XV
xv preface I encountered the concept of multithreaded code while working at my first job after I left college. We were writing a data processing application that had to populate a database with incoming data records. There was a lot of data, but each record was independent and required a reasonable amount of processing before it could be inserted into the database. To take full advantage of the power of our 10-CPU UltraSPARC, we ran the code in multiple threads, each thread processing its own set of incoming records. We wrote the code in C++, using POSIX threads, and made a fair number of mistakes—multithreading was new to all of us—but we got there in the end. It was also while working on this project that I first became aware of the C++ Standards Committee and the freshly published C++ Standard. I have had a keen interest in multithreading and concurrency ever since. Where others saw it as difficult, complex, and a source of problems, I saw it as a powerful tool that could enable your code to take advantage of the available hardware to run faster. Later on I would learn how it could be used to improve the responsiveness and performance of applications even on single-core hardware, by using multiple threads to hide the latency of time-consuming operations such as I/O. I also learned how it worked at the OS level and how Intel CPUs handled task switching. Meanwhile, my interest in C++ brought me in contact with the ACCU and then the C++ Standards panel at BSI, as well as Boost. I followed the initial development of the Boost Thread Library with interest, and when it was abandoned by the original developer, I jumped at the chance to get involved. I have been the primary developer and maintainer of the Boost Thread Library ever since
xvi PREFACE As the work of the C++Standards Committee shifted from fixing defects in the exist- ing standard to writing proposals for the next standard(named C++Ox in the hope that it would be finished by 2009,and now officially C++11,because it was finally pub- lished in 2011),I got more involved with BSI and started drafting proposals of my own. Once it became clear that multithreading was on the agenda,I jumped in with both feet and authored or coauthored many of the multithreading and concurrency- related proposals that shaped this part of the new standard.I feel privileged to have had the opportunity to combine two of my major computer-related interests-C++ and multithreading-in this way. This book draws on all my experience with both C++and multithreading and aims to teach other C++developers how to use the C++11 Thread Library safely and effi- ciently.I also hope to impart some of my enthusiasm for the subject along the way
xvi PREFACE As the work of the C++ Standards Committee shifted from fixing defects in the existing standard to writing proposals for the next standard (named C++0x in the hope that it would be finished by 2009, and now officially C++11, because it was finally published in 2011), I got more involved with BSI and started drafting proposals of my own. Once it became clear that multithreading was on the agenda, I jumped in with both feet and authored or coauthored many of the multithreading and concurrencyrelated proposals that shaped this part of the new standard. I feel privileged to have had the opportunity to combine two of my major computer-related interests—C++ and multithreading—in this way. This book draws on all my experience with both C++ and multithreading and aims to teach other C++ developers how to use the C++11 Thread Library safely and efficiently. I also hope to impart some of my enthusiasm for the subject along the way
acknowledgments I will start by saying a big "Thank you"to my wife,Kim,for all the love and support she has given me while writing this book.It has occupied a significant part of my spare time for the last four years,and without her patience,support,and understanding,I couldn't have managed it. Second,I would like to thank the team at Manning who have made this book possi- ble:Marjan Bace,publisher;Michael Stephens,associate publisher;Cynthia Kane,my development editor;Karen Tegtmeyer,review editor;Linda Recktenwald,my copy- editor;Katie Tennant,my proofreader;and Mary Piergies,the production manager. Without their efforts you would not be reading this book right now. I would also like to thank the other members of the C++Standards Committee who wrote committee papers on the multithreading facilities:Andrei Alexandrescu, Pete Becker,Bob Blainer,Hans Boehm,Beman Dawes,Lawrence Crowl,Peter Dimov, Jeff Garland,Kevlin Henney,Howard Hinnant,Ben Hutchings,Jan Kristofferson,Doug Lea,Paul McKenney,Nick McLaren,Clark Nelson,Bill Pugh,Raul Silvera,Herb Sutter, Detlef Vollmann,and Michael Wong,plus all those who commented on the papers,dis- cussed them at the committee meetings,and otherwise helped shaped the multithread- ing and concurrency support in C++11. Finally,I would like to thank the following people,whose suggestions have greatly improved this book:Dr.Jamie Allsop,Peter Dimov,Howard Hinnant,Rick Molloy, Jonathan Wakely,and Dr.Russel Winder,with special thanks to Russel for his detailed reviews and to Jonathan who,as technical proofreader,painstakingly checked all the content for outright errors in the final manuscript during production.(Any remaining xvii
xvii acknowledgments I will start by saying a big “Thank you” to my wife, Kim, for all the love and support she has given me while writing this book. It has occupied a significant part of my spare time for the last four years, and without her patience, support, and understanding, I couldn’t have managed it. Second, I would like to thank the team at Manning who have made this book possible: Marjan Bace, publisher; Michael Stephens, associate publisher; Cynthia Kane, my development editor; Karen Tegtmeyer, review editor; Linda Recktenwald, my copyeditor; Katie Tennant, my proofreader; and Mary Piergies, the production manager. Without their efforts you would not be reading this book right now. I would also like to thank the other members of the C++ Standards Committee who wrote committee papers on the multithreading facilities: Andrei Alexandrescu, Pete Becker, Bob Blainer, Hans Boehm, Beman Dawes, Lawrence Crowl, Peter Dimov, Jeff Garland, Kevlin Henney, Howard Hinnant, Ben Hutchings, Jan Kristofferson, Doug Lea, Paul McKenney, Nick McLaren, Clark Nelson, Bill Pugh, Raul Silvera, Herb Sutter, Detlef Vollmann, and Michael Wong, plus all those who commented on the papers, discussed them at the committee meetings, and otherwise helped shaped the multithreading and concurrency support in C++11. Finally, I would like to thank the following people, whose suggestions have greatly improved this book: Dr. Jamie Allsop, Peter Dimov, Howard Hinnant, Rick Molloy, Jonathan Wakely, and Dr. Russel Winder, with special thanks to Russel for his detailed reviews and to Jonathan who, as technical proofreader, painstakingly checked all the content for outright errors in the final manuscript during production. (Any remaining