12345678910CRS0302010099 First printing.October 1999 Dedication To my mother,Rivka Bulka and to the memory of my father Yacov Bulka,survivor of the Auschwitz and.which was his D.B D.M Preface If you conducted an informal survey of software developers on the issue ofC++performance,you would undoubtedly find that the apoorcocanguag application domain was ruled by plain C and occasionally even assembly language ad the development projects plunged in headfirst.Some time later,software solutions implemented in C++began ng out.In h tmal,to put it gently.E speed was not up for negotiation speed was top priority.Since networking sofware is pretty low on the so hain. he loge nu ers of applicatior ns we y up t higher level applications. was not unique .All around early adopters ofC++had difficultie s with the resulting nt pa igm.we blamed it on Cthe dominant efor the expression of the paradigm. Even ompilers were still ess tially in th r infancy L:L con cern was s rooted in the perception that C++ cannot match the performance delivered by its C nas ha m s de grams.We've seen it done in practice empt to sha exp e many wn purs nt docu
vi 1 2 3 4 5 6 7 8 9 10 —CRS—03 02 01 00 99 First printing, October 1999 Dedication To my mother, Rivka Bulka and to the memory of my father Yacov Bulka, survivor of the Auschwitz concentration camp. They could not take away his kindness, compassion and optimism, which was his ultimate triumph. He passed away during the writing of this book. D.B To Ruth, the love of my life, who made time for me to write this. To the boys, Austin, Alex, and Steve, who missed their dad for a while. To my parents, Mom and Dad, who have always loved and supported me D.M. Preface If you conducted an informal survey of software developers on the issue of C++ performance, you would undoubtedly find that the vast majority of them view performance issues as the Achilles’ heel of an otherwise fine language. We have heard it repeatedly ever since C++ burst on the corporate scene: C++ is a poor choice for implementing performance-critical applications. In the mind of developers, this particular application domain was ruled by plain C and, occasionally, even assembly language. As part of that software community we had the opportunity to watch that myth develop and gather steam. Years ago, we participated in the wave that embraced C++ with enthusiasm. All around us, many development projects plunged in headfirst. Some time later, software solutions implemented in C++ began rolling out. Their performance was typically less than optimal, to put it gently. Enthusiasm over C++ in performance-critical domains has cooled. We were in the business of supplying networking software whose execution speed was not up for negotiation—speed was top priority. Since networking software is pretty low on the software food-chain, its performance is crucial. Large numbers of applications were going to sit on top of it and depend on it. Poor performance in the low levels ripples all the way up to higher level applications. Our experience was not unique. All around, early adopters of C++ had difficulties with the resulting performance of their C++ code. Instead of attributing the difficulties to the steep learning curve of the new object-oriented software development paradigm, we blamed it on C++, the dominant language for the expression of the paradigm. Even though C++ compilers were still essentially in their infancy, the language was branded as inherently slow. This belief spread quickly and is now widely accepted as fact. Software organizations that passed on C++ frequently pointed to performance as their key concern. That concern was rooted in the perception that C++ cannot match the performance delivered by its C counterpart. Consequently, C++ has had little success penetrating software domains that view performance as top priority: operating system kernels, device drivers, networking systems (routers, gateways, protocol stacks), and more. We have spent years dissecting large systems of C and C++ code trying to squeeze every ounce of performance out of them. It is through our experience of slugging it out in the trenches that we have come to appreciate the potential of C++ to produce highly efficient programs. We’ve seen it done in practice. This book is our attempt to share that experience and document the many lessons we have learned in our own pursuit of C++ efficiency. Writing efficient C++ is not trivial, nor is it rocket science. It takes the
understanding of some performance principles,as well as informationonC+performance traps and pitfalls The 80-20 rule is an important principle in the world of software construction.We adopt it in the writing of this book as well:20%of all performance bugs will show up%of the time.We therefore chose to d unts the mo This h in those perte ues tha 、i set of all r ssible performance bus and their solutions:hence we will not cover what we consider esoteric and rare performance pitfalls. oubtedly biased b of server-side The profile of performance problem.Generic performance principles transcend distinct domains,and apply equally well in omains oth than networ ting software We do not delve into the asymptotic complexity of algorithms,data structures and the latest anc focus on simple.p ractical evervday coding and design principles that vield large performance improvements We point out common de sign and cod ing prac ces that l of sube(and not so subtle)performance principles So how do we parate myth from reality?Is C+ performance truly inferior to that of C?It is our pears to be the same thing the Cp oganeneal aster.How er,we also claim that the apparent similarity of the two programs typically is based on thei tion is that the speed cfcouad hjust acpbeperfomnce,but yidsareprfomanc We w ould like to thank .The toughest part was getting Julia Sime made a si ificant contribution to the early the one who pointed out to u rence our opinions ough t they ares Many thanks to the reviewers hired by Addison-Wesley,their feedback was extremely valuable. Heather Kreger Kath n Britton ruth willenborg David wisler Bala raiaraman Don "Spike' Washburn,and Nils Brubaker. Last but not least,we would like to thank our wives,Cynthia Powers Bulka and Ruth Washington Mayhew
vii understanding of some performance principles, as well as information on C++ performance traps and pitfalls. The 80-20 rule is an important principle in the world of software construction. We adopt it in the writing of this book as well: 20% of all performance bugs will show up 80% of the time. We therefore chose to concentrate our efforts where it counts the most. We are interested in those performance issues that arise frequently in industrial code and have significant impact. This book is not an exhaustive discussion of the set of all possible performance bugs and their solutions; hence, we will not cover what we consider esoteric and rare performance pitfalls. Our point of view is undoubtedly biased by our practical experience as programmers of server-side, performance-critical communications software. This bias impacts the book in several ways: • The profile of performance issues that we encounter in practice may be slightly different in nature than those found in scientific computing, database applications, and other domains. That’s not a problem. Generic performance principles transcend distinct domains, and apply equally well in domains other than networking software. • At times, we invented contrived examples to drive a point home, although we tried to minimize this. We have made enough coding mistakes in the past to have a sizable collection of samples taken from real production-level code that we have worked on. Our expertise was earned the hard way—by learning from our own mistakes as well as those of our colleagues. As much as possible, we illustrated our points with real code samples. • We do not delve into the asymptotic complexity of algorithms, data structures, and the latest and greatest techniques for accessing, sorting, searching, and compressing data. These are important topics, but they have been extensively covered elsewhere [Knu73, BR95, KP74]. Instead, we focus on simple, practical, everyday coding and design principles that yield large performance improvements. We point out common design and coding practices that lead to poor performance, whether it be through the unwitting use of language features that carry high hidden costs or through violating any number of subtle (and not so subtle) performance principles. So how do we separate myth from reality? Is C++ performance truly inferior to that of C? It is our contention that the common perception of inferior C++ performance is invalid. We concede that in general, when comparing a C program to a C++ version of what appears to be the same thing, the C program is generally faster. However, we also claim that the apparent similarity of the two programs typically is based on their data handling functionality, not their correctness, robustness, or ease of maintenance. Our contention is that when C programs are brought up to the level of C++ programs in these regards, the speed differences disappear, or the C++ versions are faster. Thus C++ is inherently neither slower nor faster. It could be either, depending on how it is used and what is required from it. It’s the way it is used that matters: If used properly, C++ can yield software systems exhibiting not just acceptable performance, but yield superior software performance. We would like to thank the many people who contributed to this work. The toughest part was getting started and it was our editor, Marina Lang, who was instrumental in getting this project off the ground. Julia Sime made a significant contribution to the early draft and Yomtov Meged contributed many valuable suggestions as well. He also was the one who pointed out to us the subtle difference between our opinions and the absolute truth. Although those two notions may coincide at times, they are still distinct. Many thanks to the reviewers hired by Addison-Wesley; their feedback was extremely valuable. Thanks also to our friends and colleagues who reviewed portions of the manuscript. They are, in no particular order, Cyndy Ross, Art Francis, Scott Snyder, Tricia York, Michael Fraenkel, Carol Jones, Heather Kreger, Kathryn Britton, Ruth Willenborg, David Wisler, Bala Rajaraman, Don “Spike” Washburn, and Nils Brubaker. Last but not least, we would like to thank our wives, Cynthia Powers Bulka and Ruth Washington Mayhew
Introduction In the days of assembler lan estimated the tion speed 器0 fragment co r of clock cycles their execution would er was trivially one-to-one.The assembler code was con bler.The code. s ta g.It re moet ght chCstatement translates C++has shattered this nice linear relationship between the number of source level statements and compiler-gener mbly statemn co ereas the co st of C sta nents is largely uniform.the whereas another can generate 300.Implementing high-performance Ccod has nexpected demand on programmers:the need to navigate through a performance mne d,trying to st cenerate large overhead and know how to design around them These are thatCand assembler eq moK puryaq weod mok jo Mou uonnax a ouu apo Lsu ospe y duo++l The task ng( programmer migrating h skills that are specife toC +and that trar nd the s performance t lkely to by hi en overhead, e to stumble upo a are hot going en many xamples of poor that were rooed in since moved into the mainstream.However.and reuse seldomg ency.In mathemati It would be painf to reduce ever theorem back to basic akes sense to leverage sp ecial circumstances and to take e desigr it is acceptable under some circumstances to place ce than reuse. hen you mpie s are tha future it might be reused.Some performance problems inO design are due to putting the emphasison the wrong place at the ng th em they have,not on Roots of Software Inefficiency the root of all nerf vil.Even elimin ed overhead the cas se,then e awesome performance due to the lack of silent overhead.Additional factors affect software performance in viii
viii Introduction In the days of assembler language programming, experienced programmers estimated the execution speed of their source code by counting the number of assembly language instructions. On some architectures, such as RISC, most assembler instructions executed in one clock cycle each. Other architectures featured wide variations in instruction to instruction execution speed, but experienced programmers were able to develop a good feel for average instruction latency. If you knew how many instructions your code fragment contained, you could estimate with accuracy the number of clock cycles their execution would consume. The mapping from source code to assembler was trivially one-to-one. The assembler code was the source code. On the ladder of programming languages, C is one step higher than assembler language. C source code is not identical to the corresponding compiler-generated assembler code. It is the compiler’s task to bridge the gap from source code to assembler. The mapping of source-to-assembler code is no longer the one-toone identity mapping. It remains, however, a linear relationship: Each source level statement in C corresponds to a small number of assembler instructions. If you estimate that each C statement translates into five to eight assembler instructions, chances are you will be in the ballpark. C++ has shattered this nice linear relationship between the number of source level statements and compiler-generated assembly statement count. Whereas the cost of C statements is largely uniform, the cost of C++ statements fluctuates wildly. One C++ statement can generate three assembler instructions, whereas another can generate 300. Implementing high-performance C++ code has placed a new and unexpected demand on programmers: the need to navigate through a performance minefield, trying to stay on a safe three-instruction-per-statement path and to avoid usage of routes that contain 300-instruction land mines. Programmers must identify language constructs likely to generate large overhead and know how to code or design around them. These are considerations that C and assembler language programmers have never had to worry about. The only exception may be the use of macros in C, but those are hardly as frequent as the invocations of constructors and destructors in C++ code. The C++ compiler might also insert code into the execution flow of your program “behind your back.” This is news to the unsuspecting C programmer migrating to C++ (which is where many of us are coming from). The task of writing efficient C++ programs requires C++ developers to acquire new performance skills that are specific to C++ and that transcend the generic software performance principles. In C programming, you are not likely to be blindsided by hidden overhead, so it is possible to stumble upon good performance in a C program. In contrast, this is unlikely to happen in C++: You are not going to achieve good performance accidentally, without knowing the pitfalls lurking about. To be fair, we have seen many examples of poor performance that were rooted in inefficient objectoriented (OO) design. The ideas of software flexibility and reuse have been promoted aggressively ever since OO moved into the mainstream. However, flexibility and reuse seldom go hand-in-hand with performance and efficiency. In mathematics, it would be painful to reduce every theorem back to basic principles. Mathematicians try to reuse results that have already been proven. Outside mathematics, however, it often makes sense to leverage special circumstances and to take shortcuts. In software design, it is acceptable under some circumstances to place higher priority on performance than reuse. When you implement the read() or write() function of a device driver, the known performance requirements are generally much more important to your software’s success than the possibility that at some point in the future it might be reused. Some performance problems in OO design are due to putting the emphasis on the wrong place at the wrong time. Programmers should focus on solving the problem they have, not on making their current solution amenable to some unidentified set of possible future requirements. Roots of Software Inefficiency Silent C++ overhead is not the root of all performance evil. Even eliminating compiler-generated overhead would not always be sufficient. If that were the case, then every C program would enjoy automatic awesome performance due to the lack of silent overhead. Additional factors affect software performance in
general and C++performance in particular.What are those factors?The first level of performance classification is given in Figure 1. Figure 1.High-level classification of software performance Software Performance Design Coding At the highest level,software efficieny is determined by the efficiency of two main ingredients m's high-level design To fix perfo ance problems at efficiencyd the program's big picture Totent this that level you must understand the pr No amount o of coding an provide shelter f a bad design to look very far into a code fragme This high-level classification can be broken down further into finer subtopics.as shown in Figure 2 Figure 2.Refinement of the design performance view Design Algorithms Data Stuctures Program Decompositior Design efficiency is broken down further into two items: rithmsad data svrman alorith it mng to ms and of data rching.sorting.comp reduced to that aspect alone is inaccurate.The efficieny of algorithms and data structures is necessary but not sufficient:By itself,it does not guarantee good overall program efficiency
ix general and C++ performance in particular. What are those factors? The first level of performance classification is given in Figure 1. Figure 1. High-level classification of software performance. At the highest level, software efficiency is determined by the efficiency of two main ingredients: • Design efficiency This involves the program’s high-level design. To fix performance problems at that level you must understand the program’s big picture. To a large extent, this item is language independent. No amount of coding efficiency can provide shelter for a bad design. • Coding efficiency Small- to medium-scale implementation issues fall into this category. Fixing performance in this category generally involves local modifications. For example, you do not need to look very far into a code fragment in order to lift a constant expression out of a loop and prevent redundant computations. The code fragment you need to understand is limited in scope to the loop body. This high-level classification can be broken down further into finer subtopics, as shown in Figure 2. Figure 2. Refinement of the design performance view. Design efficiency is broken down further into two items: • Algorithms and data structures Technically speaking, every program is an algorithm in itself. Referring to “algorithms and data structures” actually refers to the well-known subset of algorithms for accessing, searching, sorting, compressing, and otherwise manipulating large collections of data. Oftentimes performance automatically is associated with the efficiency of the algorithms and data structures used in a program, as if nothing else matters. To claim that software performance can be reduced to that aspect alone is inaccurate. The efficiency of algorithms and data structures is necessary but not sufficient: By itself, it does not guarantee good overall program efficiency
ofasingle componntAtypcWebaracts(vi API)thWeb server. ICp ere are s with respec co Coding efficiency can also be subdivided,as shown in Figure 3. Figure 3.Refinement of the coding performance view Coding Con We split up coding efficiency into four items. Language constructs C++adds power and flexibility to its c ancestor.These added benefits do not come for free some C++language constructs may produce overhead in exchange.We will e throughout the book.This topic is,by na +specm the -it ju es that way.Developing however these architectural issues cannot be ignored since they can impact ally.Wh we must bear in mind that The cost of me .There are orders of mag ndcdioe r program d Cp turns Awareness of these issues helps software performance Libraries the choice of librar s used by an imple also affect nerfo nce for may perform task faster than others Bec to the library's source ode,it is hard to tel how li exampl mnt their services.For er string,you ca sprintf(string."&d"i): or an integer-to-ASClI function call [KR88]. itoa(i,string); Which one is more efficient?Is the difference significant?
x • Program decomposition This involves decomposition of the overall task into communicating subtasks, object hierarchies, functions, data, and function flow. It is the program’s high-level design and includes component design as well as intercomponent communication. Few programs consist of a single component. A typical Web application interacts (via API) with a Web server, TCP sockets, and a database, at the very least. There are efficiency tricks and pitfalls with respect to crossing the API layer with each of those components. Coding efficiency can also be subdivided, as shown in Figure 3. Figure 3. Refinement of the coding performance view. We split up coding efficiency into four items: • Language constructs C++ adds power and flexibility to its C ancestor. These added benefits do not come for free—some C++ language constructs may produce overhead in exchange. We will discuss this issue throughout the book. This topic is, by nature, C++ specific. • System architecture System designers invest considerable effort to present the programmer with an idealistic view of the system: infinite memory, dedicated CPU, parallel thread execution, and uniform-cost memory access. Of course, none of these is true—it just feels that way. Developing software free of system architecture considerations is also convenient. To achieve high performance, however, these architectural issues cannot be ignored since they can impact performance drastically. When it comes to performance we must bear in mind that o Memory is not infinite. It is the virtual memory system that makes it appear that way. o The cost of memory access is nonuniform. There are orders of magnitude difference among cache, main memory, and disk access. o Our program does not have a dedicated CPU. We get a time slice only once in a while. o On a uniprocessor machine, parallel threads do not truly execute in parallel—they take turns. Awareness of these issues helps software performance. • Libraries The choice of libraries used by an implementation can also affect performance. For starters, some libraries may perform a task faster than others. Because you typically don’t have access to the library’s source code, it is hard to tell how library calls implement their services. For example, to convert an integer to a character string, you can choose between sprintf(string, “%d”, i); or an integer-to-ASCII function call [KR88], itoa(i, string); Which one is more efficient? Is the difference significant?