Contents ix 24 Single-Source Shortest Paths 643 24.1 The Bellman-Ford algorithm 65/ 24.2 Single-source shortest paths in directed acyclic graphs 655 24.3 Dijkstra's algorithm 658 24.4 Difference constraints and shortest paths 664 24.5 Proofs of shortest-paths properties 671 25 All-Pairs Shortest Paths 684 25.1 Shortest paths and matrix multiplication 686 25.2 The Floyd-Warshall algorithm 693 25.3 Johnson's algorithm for sparse graphs 700 26 Maximum Flow 708 26.1 Flow networks 709 26.2 The Ford-Fulkerson method 7/4 26.3 Maximum bipartite matching 732 26.4 Push-relabel algorithms 736 26.5 The relabel-to-front algorithm 748 VII Selected Topics Introduction 769 27 Multithreaded Algorithms 772 27.1 The basics of dynamic multithreading 774 27.2 Multithreaded matrix multiplication 792 27.3 Multithreaded merge sort 797 28 Matrix Operations 813 28.1 Solving systems of linear equations 813 28.2 Inverting matrices 827 28.3 Symmetric positive-definite matrices and least-squares approximation 832 29 Linear Programming 843 29.1 Standard and slack forms 850 29.2 Formulating problems as linear programs 859 29.3 The simplex algorithm 864 29.4 Duality 879 29.5 The initial basic feasible solution 886
Contents ix 24 Single-Source Shortest Paths 643 24.1 The Bellman-Ford algorithm 651 24.2 Single-source shortest paths in directed acyclic graphs 655 24.3 Dijkstra’s algorithm 658 24.4 Difference constraints and shortest paths 664 24.5 Proofs of shortest-paths properties 671 25 All-Pairs Shortest Paths 684 25.1 Shortest paths and matrix multiplication 686 25.2 The Floyd-Warshall algorithm 693 25.3 Johnson’s algorithm for sparse graphs 700 26 Maximum Flow 708 26.1 Flow networks 709 26.2 The Ford-Fulkerson method 714 26.3 Maximum bipartite matching 732 ? 26.4 Push-relabel algorithms 736 ? 26.5 The relabel-to-front algorithm 748 VII Selected Topics Introduction 769 27 Multithreaded Algorithms 772 27.1 The basics of dynamic multithreading 774 27.2 Multithreaded matrix multiplication 792 27.3 Multithreaded merge sort 797 28 Matrix Operations 813 28.1 Solving systems of linear equations 813 28.2 Inverting matrices 827 28.3 Symmetric positive-definite matrices and least-squares approximation 832 29 Linear Programming 843 29.1 Standard and slack forms 850 29.2 Formulating problems as linear programs 859 29.3 The simplex algorithm 864 29.4 Duality 879 29.5 The initial basic feasible solution 886
Contents 30 Polynomials and the FFT 898 30.1 Representing polynomials 900 30.2 The DFT and FFT 906 30.3 Efficient FFT implementations 9/5 31 Number-Theoretic Algorithms 926 31.1 Elementary number-theoretic notions 927 31.2 Greatest common divisor 933 31.3 Modular arithmetic 939 31.4 Solving modular linear equations 946 31.5 The Chinese remainder theorem 950 31.6 Powers of an element 954 31.7 The RSA public-key cryptosystem 958 *31.8 Primality testing 965 ★ 31.9 Integer factorization 975 32 String Matching 985 32.1 The naive string-matching algorithm 988 32.2 The Rabin-Karp algorithm 990 32.3 String matching with finite automata 995 32.4 The Knuth-Morris-Pratt algorithm 1002 33 Computational Geometry 1014 33.1 Line-segment properties 10/5 33.2 Determining whether any pair of segments intersects 1021 33.3 Finding the convex hull 1029 33.4 Finding the closest pair of points 1039 34 NP-Completeness 1048 34.1 Polynomial time 1053 34.2 Polynomial-time verification 106/ 34.3 NP-completeness and reducibility 1067 34.4 NP-completeness proofs 1078 34.5 NP-complete problems 1086 35 Approximation Algorithms 1106 35.1 The vertex-cover problem 1108 35.2 The traveling-salesman problem 111/ 35.3 The set-covering problem 1/17 35.4 Randomization and linear programming 1/23 35.5 The subset-sum problem //28
x Contents 30 Polynomials and the FFT 898 30.1 Representing polynomials 900 30.2 The DFT and FFT 906 30.3 Efficient FFT implementations 915 31 Number-Theoretic Algorithms 926 31.1 Elementary number-theoretic notions 927 31.2 Greatest common divisor 933 31.3 Modular arithmetic 939 31.4 Solving modular linear equations 946 31.5 The Chinese remainder theorem 950 31.6 Powers of an element 954 31.7 The RSA public-key cryptosystem 958 ? 31.8 Primality testing 965 ? 31.9 Integer factorization 975 32 String Matching 985 32.1 The naive string-matching algorithm 988 32.2 The Rabin-Karp algorithm 990 32.3 String matching with finite automata 995 ? 32.4 The Knuth-Morris-Pratt algorithm 1002 33 Computational Geometry 1014 33.1 Line-segment properties 1015 33.2 Determining whether any pair of segments intersects 1021 33.3 Finding the convex hull 1029 33.4 Finding the closest pair of points 1039 34 NP-Completeness 1048 34.1 Polynomial time 1053 34.2 Polynomial-time verification 1061 34.3 NP-completeness and reducibility 1067 34.4 NP-completeness proofs 1078 34.5 NP-complete problems 1086 35 Approximation Algorithms 1106 35.1 The vertex-cover problem 1108 35.2 The traveling-salesman problem 1111 35.3 The set-covering problem 1117 35.4 Randomization and linear programming 1123 35.5 The subset-sum problem 1128
Contents xi VIl Appendix:Mathematical Background Introduction 1143 A Summations 1145 A.1 Summation formulas and properties 1145 A.2 Bounding summations //49 B Sets,Etc.1158 B.1 Sets 1158 B.2 Relations 1/63 B.3 Functions //66 B.4 Graphs 1168 B.5 Trees 1173 C Counting and Probability 1183 C.1 Counting 1183 C.2 Probability 1189 C.3 Discrete random variables 1196 C.4 The geometric and binomial distributions 120/ ★ C.5 The tails of the binomial distribution /208 D Matrices 1217 D.1 Matrices and matrix operations 1217 D.2 Basic matrix properties /222 Bibliography 1231 Index 1251
Contents xi VIII Appendix: Mathematical Background Introduction 1143 A Summations 1145 A.1 Summation formulas and properties 1145 A.2 Bounding summations 1149 B Sets, Etc. 1158 B.1 Sets 1158 B.2 Relations 1163 B.3 Functions 1166 B.4 Graphs 1168 B.5 Trees 1173 C Counting and Probability 1183 C.1 Counting 1183 C.2 Probability 1189 C.3 Discrete random variables 1196 C.4 The geometric and binomial distributions 1201 ? C.5 The tails of the binomial distribution 1208 D Matrices 1217 D.1 Matrices and matrix operations 1217 D.2 Basic matrix properties 1222 Bibliography 1231 Index 1251
Preface Before there were computers,there were algorithms.But now that there are com- puters,there are even more algorithms,and algorithms lie at the heart of computing. This book provides a comprehensive introduction to the modern study of com- puter algorithms.It presents many algorithms and covers them in considerable depth,yet makes their design and analysis accessible to all levels of readers.We have tried to keep explanations elementary without sacrificing depth of coverage or mathematical rigor. Each chapter presents an algorithm,a design technique,an application area,or a related topic.Algorithms are described in English and in a pseudocode designed to be readable by anyone who has done a little programming.The book contains 244 figures-many with multiple parts-illustrating how the algorithms work.Since we emphasize efficiency as a design criterion,we include careful analyses of the running times of all our algorithms. The text is intended primarily for use in undergraduate or graduate courses in algorithms or data structures.Because it discusses engineering issues in algorithm design,as well as mathematical aspects,it is equally well suited for self-study by technical professionals. In this,the third edition,we have once again updated the entire book.The changes cover a broad spectrum,including new chapters,revised pseudocode,and a more active writing style. To the teacher We have designed this book to be both versatile and complete.You should find it useful for a variety of courses,from an undergraduate course in data structures up through a graduate course in algorithms.Because we have provided considerably more material than can fit in a typical one-term course,you can consider this book to be a“buffet'”or“smorgasbord'”from which you can pick and choose the material that best supports the course you wish to teach
Preface Before there were computers, there were algorithms. But now that there are computers, there are even more algorithms, and algorithms lie at the heart of computing. This book provides a comprehensive introduction to the modern study of computer algorithms. It presents many algorithms and covers them in considerable depth, yet makes their design and analysis accessible to all levels of readers. We have tried to keep explanations elementary without sacrificing depth of coverage or mathematical rigor. Each chapter presents an algorithm, a design technique, an application area, or a related topic. Algorithms are described in English and in a pseudocode designed to be readable by anyone who has done a little programming. The book contains 244 figures—many with multiple parts—illustrating how the algorithms work. Since we emphasize efficiency as a design criterion, we include careful analyses of the running times of all our algorithms. The text is intended primarily for use in undergraduate or graduate courses in algorithms or data structures. Because it discusses engineering issues in algorithm design, as well as mathematical aspects, it is equally well suited for self-study by technical professionals. In this, the third edition, we have once again updated the entire book. The changes cover a broad spectrum, including new chapters, revised pseudocode, and a more active writing style. To the teacher We have designed this book to be both versatile and complete. You should find it useful for a variety of courses, from an undergraduate course in data structures up through a graduate course in algorithms. Because we have provided considerably more material than can fit in a typical one-term course, you can consider this book to be a “buffet” or “smorgasbord” from which you can pick and choose the material that best supports the course you wish to teach
xiv Preface You should find it easy to organize your course around just the chapters you need.We have made chapters relatively self-contained,so that you need not worry about an unexpected and unnecessary dependence of one chapter on another.Each chapter presents the easier material first and the more difficult material later,with section boundaries marking natural stopping points.In an undergraduate course, you might use only the earlier sections from a chapter;in a graduate course,you might cover the entire chapter. We have included 957 exercises and 158 problems.Each section ends with exer- cises,and each chapter ends with problems.The exercises are generally short ques- tions that test basic mastery of the material.Some are simple self-check thought exercises,whereas others are more substantial and are suitable as assigned home- work.The problems are more elaborate case studies that often introduce new ma- terial;they often consist of several questions that lead the student through the steps required to arrive at a solution. Departing from our practice in previous editions of this book,we have made publicly available solutions to some,but by no means all,of the problems and ex- ercises.Our Web site,http://mitpress.mit.edu/algorithms/,links to these solutions. You will want to check this site to make sure that it does not contain the solution to an exercise or problem that you plan to assign.We expect the set of solutions that we post to grow slowly over time,so you will need to check it each time you teach the course. We have starred (*)the sections and exercises that are more suitable for graduate students than for undergraduates.A starred section is not necessarily more diffi- cult than an unstarred one,but it may require an understanding of more advanced mathematics.Likewise,starred exercises may require an advanced background or more than average creativity. To the student We hope that this textbook provides you with an enjoyable introduction to the field of algorithms.We have attempted to make every algorithm accessible and interesting.To help you when you encounter unfamiliar or difficult algorithms,we describe each one in a step-by-step manner.We also provide careful explanations of the mathematics needed to understand the analysis of the algorithms.If you already have some familiarity with a topic,you will find the chapters organized so that you can skim introductory sections and proceed quickly to the more advanced material. This is a large book,and your class will probably cover only a portion of its material.We have tried,however,to make this a book that will be useful to you now as a course textbook and also later in your career as a mathematical desk reference or an engineering handbook
xiv Preface You should find it easy to organize your course around just the chapters you need. We have made chapters relatively self-contained, so that you need not worry about an unexpected and unnecessary dependence of one chapter on another. Each chapter presents the easier material first and the more difficult material later, with section boundaries marking natural stopping points. In an undergraduate course, you might use only the earlier sections from a chapter; in a graduate course, you might cover the entire chapter. We have included 957 exercises and 158 problems. Each section ends with exercises, and each chapter ends with problems. The exercises are generally short questions that test basic mastery of the material. Some are simple self-check thought exercises, whereas others are more substantial and are suitable as assigned homework. The problems are more elaborate case studies that often introduce new material; they often consist of several questions that lead the student through the steps required to arrive at a solution. Departing from our practice in previous editions of this book, we have made publicly available solutions to some, but by no means all, of the problems and exercises. Our Web site, http://mitpress.mit.edu/algorithms/, links to these solutions. You will want to check this site to make sure that it does not contain the solution to an exercise or problem that you plan to assign. We expect the set of solutions that we post to grow slowly over time, so you will need to check it each time you teach the course. We have starred (?) the sections and exercises that are more suitable for graduate students than for undergraduates. A starred section is not necessarily more diffi- cult than an unstarred one, but it may require an understanding of more advanced mathematics. Likewise, starred exercises may require an advanced background or more than average creativity. To the student We hope that this textbook provides you with an enjoyable introduction to the field of algorithms. We have attempted to make every algorithm accessible and interesting. To help you when you encounter unfamiliar or difficult algorithms, we describe each one in a step-by-step manner. We also provide careful explanations of the mathematics needed to understand the analysis of the algorithms. If you already have some familiarity with a topic, you will find the chapters organized so that you can skim introductory sections and proceed quickly to the more advanced material. This is a large book, and your class will probably cover only a portion of its material. We have tried, however, to make this a book that will be useful to you now as a course textbook and also later in your career as a mathematical desk reference or an engineering handbook