BULLETIN (New Series)OF THE AMERICAN MATHEMATICAL SOCIETY (00012 4.October 2006,Pages 439-561 Article electronically published on August 7,2006 EXPANDER GRAPHS AND THEIR APPLICATIONS SHLOMO HOORY.NATHAN LINIAL.AND AVI WIGDERSON An Overview A major consideration we had in writing this survey was to make it accessible to mathematicians as well as to computer scientists,since expander graphs,the protagonists of our story,come up in numerous and often surprising contexts in both fields. But,perhaps,we should start with a few words about graphs in general.They are,of course,one of the prime objects of study in Discrete Mathematics.However, graphs are among the most ubiquitous models of both natural and human-made structures.In the natural and social sciences they model relations among species, societies,companies,etc.In computer science,they represent networks of commu- nication,data organization,computational devices as well as the flow of computa- tion,and more.In mathematics,Cayley graphs are useful in Group Theory.Graphs carry a natural metric and are therefore useful in Geometry,and though they are "just"one-dimensional complexes,they are useful in certain parts of Topology,e.g. Knot Theory.In statistical physics,graphs can represent local connections between interacting parts of a system,as well as the dynamics of a physical process on such systems. The study of these models calls.then.for the comprehension of the significant structural properties of the relevant graphs.But are there nontrivial structural properties which are universally important?Expansion of a graph requires that it is simultaneously sparse and highly connected.Expander graphs were first de- fined by Bassalygo and Pinsker,and their existence first proved by Pinsker in the early'70s.The property of being an expander seems significant in many of these mathematical,computational and physical contexts.It is not surprising that ex- panders are useful in the design and analysis of communication networks.What is less obvious is that expanders have surprising utility in other computational settings such as in the theory of error correcting codes and the theory of pseudorandom- ness.In mathematics,we will encounter e.g.their role in the study of metric embeddings,and in particular in work around the Baum-Connes Conjecture.Ex- pansion is closely related to the convergence rates of Markov Chains,and so they play a key role in the study of Monte-Carlo algorithms in statistical mechanics and in a host of practical computational applications.The list of such interesting and fruitful connections goes on and on with so many applications we will not even Received by the editors April 28,2006,and,in revised form,May 10,2006. 2000 Mathematics Subject Classification.Primary 01-01,68-01,05-01,68Q01,94-01;Sec- ondary68Q15,68Q17,94B05.05C25,05C35,05C40,05C50,05C80.05C90,60J10.35J99,20F05 20F69.20C99. Work supported in part by grants from the Israel Science Foundation and the Israel-U.S Binational Fund. C2006 S.Hoory,N.Linial,and A.Wigderson 439
BULLETIN (New Series) OF THE AMERICAN MATHEMATICAL SOCIETY Volume 43, Number 4, October 2006, Pages 439–561 S 0273-0979(06)01126-8 Article electronically published on August 7, 2006 EXPANDER GRAPHS AND THEIR APPLICATIONS SHLOMO HOORY, NATHAN LINIAL, AND AVI WIGDERSON An Overview A major consideration we had in writing this survey was to make it accessible to mathematicians as well as to computer scientists, since expander graphs, the protagonists of our story, come up in numerous and often surprising contexts in both fields. But, perhaps, we should start with a few words about graphs in general. They are, of course, one of the prime objects of study in Discrete Mathematics. However, graphs are among the most ubiquitous models of both natural and human-made structures. In the natural and social sciences they model relations among species, societies, companies, etc. In computer science, they represent networks of communication, data organization, computational devices as well as the flow of computation, and more. In mathematics, Cayley graphs are useful in Group Theory. Graphs carry a natural metric and are therefore useful in Geometry, and though they are “just” one-dimensional complexes, they are useful in certain parts of Topology, e.g. Knot Theory. In statistical physics, graphs can represent local connections between interacting parts of a system, as well as the dynamics of a physical process on such systems. The study of these models calls, then, for the comprehension of the significant structural properties of the relevant graphs. But are there nontrivial structural properties which are universally important? Expansion of a graph requires that it is simultaneously sparse and highly connected. Expander graphs were first de- fined by Bassalygo and Pinsker, and their existence first proved by Pinsker in the early ’70s. The property of being an expander seems significant in many of these mathematical, computational and physical contexts. It is not surprising that expanders are useful in the design and analysis of communication networks. What is less obvious is that expanders have surprising utility in other computational settings such as in the theory of error correcting codes and the theory of pseudorandomness. In mathematics, we will encounter e.g. their role in the study of metric embeddings, and in particular in work around the Baum-Connes Conjecture. Expansion is closely related to the convergence rates of Markov Chains, and so they play a key role in the study of Monte-Carlo algorithms in statistical mechanics and in a host of practical computational applications. The list of such interesting and fruitful connections goes on and on with so many applications we will not even Received by the editors April 28, 2006, and, in revised form, May 10, 2006. 2000 Mathematics Subject Classification. Primary 01-01, 68-01, 05-01, 68Q01, 94-01; Secondary 68Q15, 68Q17, 94B05, 05C25, 05C35, 05C40, 05C50, 05C80, 05C90, 60J10, 35J99, 20F05, 20F69, 20C99. Work supported in part by grants from the Israel Science Foundation and the Israel-U.S. Binational Fund. c 2006 S. Hoory, N. Linial, and A. Wigderson 439
440 SHLOMO HOORY.NATHAN LINIAL.AND AVI WIGDERSON be able to mention.This universality of expanders is becoming more evident as more connections are discovered.It transpires that expansion is a fundamental mathematical concept,well deserving to be thoroughly investigated on its own. In hindsight,one reason that expanders are so ubiquitous is that their very defini- tion can be given in at least three languages:combinatorial/geometric,probabilistic and algebraic.Combinatorially,expanders are graphs which are highly connected; to disconnect a large part of the graph,one has to sever many edges.Equivalently, using the geometric notion of isoperimetry,every set of vertices has a (relatively) very large boundary.From the probabilistic viewpoint,one considers the natural random walk on a graph,in which we have a token on a vertex,that moves at every step to a random neighboring vertex,chosen uniformly and independently. Expanders are graphs for which this process converges to its limiting distribution as rapidly as possible.Algebraically,one can consider the Laplace operator on the graph and its spectrum.From this perspective,expanders are graphs in which the first positive eigenvalue(of their Laplace operator)is bounded away from zero. The study of expanders leads in different directions.There are structural prob- lems:what are the best bounds on the various expansion parameters,and how do they relate to each other and to other graph invariants?There are problems concerning explicit constructions:how to efficiently generate expanders with given parameters.These are extremely important for applications.There are algorith- mic problems-given a graph,test if it is an expander with given parameters. Finally,there is the problem of understanding the relation of expansion with other mathematical notions,and the application of expanders to practical and theoretical problems. In the past four decades,a great amount of research has been done on these topics,resulting in a wide-ranging body of knowledge.In this survey,we could not hope to cover even a fraction of it.We have tried to make the presentation as broad as possible,touching on the various research directions mentioned above. Even what we do cover is of course incomplete,and we try to give the relevant references for more comprehensive coverage.We have also tried to mention in each section related research which we are not covering at all and to reference some of this as well. The selection of material naturally reflects our interests and biases.It is rather diverse and can be read in different orders,according to the reader's taste and interests. General background material on the computer science side includes the books on Computational Complexity (specifically,complexity classes)Pap94,Sip97],on Algorithms [CLRSO1]and on Randomized Algorithms MR95],and the survey on the P versus NP problem Wig06. This article evolved from lecture notes for a course on expanders taught at the Hebrew University,Israel,in 2003 by Nati Linial and Avi Wigderson.We are greatly indebted to the scribes of the course notes:Ran Gilad-Bachrach,Danny Harnik. Boaz Barak,Udi Wieder,Eran Ofek,Erez Waisbard,Yael Vinner-Dekel,Yishai Beeri,David Statter,Eyal Bigman,Tamir Hazan,Elon Portugaly,Ariel Elbaz, Yuval Filmus,Michal Igell,Eyal Rozenman,Danny Gutfreund,and Yonatan Bilu. Also,we acknowledge that the proof that Margulis construction is an expander is taken (with slight changes)from course notes of Ravi Boppana,with Mukesh Dalal as scribe
440 SHLOMO HOORY, NATHAN LINIAL, AND AVI WIGDERSON be able to mention. This universality of expanders is becoming more evident as more connections are discovered. It transpires that expansion is a fundamental mathematical concept, well deserving to be thoroughly investigated on its own. In hindsight, one reason that expanders are so ubiquitous is that their very definition can be given in at least three languages: combinatorial/geometric, probabilistic and algebraic. Combinatorially, expanders are graphs which are highly connected; to disconnect a large part of the graph, one has to sever many edges. Equivalently, using the geometric notion of isoperimetry, every set of vertices has a (relatively) very large boundary. From the probabilistic viewpoint, one considers the natural random walk on a graph, in which we have a token on a vertex, that moves at every step to a random neighboring vertex, chosen uniformly and independently. Expanders are graphs for which this process converges to its limiting distribution as rapidly as possible. Algebraically, one can consider the Laplace operator on the graph and its spectrum. From this perspective, expanders are graphs in which the first positive eigenvalue (of their Laplace operator) is bounded away from zero. The study of expanders leads in different directions. There are structural problems: what are the best bounds on the various expansion parameters, and how do they relate to each other and to other graph invariants? There are problems concerning explicit constructions: how to efficiently generate expanders with given parameters. These are extremely important for applications. There are algorithmic problems - given a graph, test if it is an expander with given parameters. Finally, there is the problem of understanding the relation of expansion with other mathematical notions, and the application of expanders to practical and theoretical problems. In the past four decades, a great amount of research has been done on these topics, resulting in a wide-ranging body of knowledge. In this survey, we could not hope to cover even a fraction of it. We have tried to make the presentation as broad as possible, touching on the various research directions mentioned above. Even what we do cover is of course incomplete, and we try to give the relevant references for more comprehensive coverage. We have also tried to mention in each section related research which we are not covering at all and to reference some of this as well. The selection of material naturally reflects our interests and biases. It is rather diverse and can be read in different orders, according to the reader’s taste and interests. General background material on the computer science side includes the books on Computational Complexity (specifically, complexity classes) [Pap94, Sip97], on Algorithms [CLRS01] and on Randomized Algorithms [MR95], and the survey on the P versus NP problem [Wig06]. This article evolved from lecture notes for a course on expanders taught at the Hebrew University, Israel, in 2003 by Nati Linial and Avi Wigderson. We are greatly indebted to the scribes of the course notes: Ran Gilad-Bachrach, Danny Harnik, Boaz Barak, Udi Wieder, Eran Ofek, Erez Waisbard, Yael Vinner-Dekel, Yishai Beeri, David Statter, Eyal Bigman, Tamir Hazan, Elon Portugaly, Ariel Elbaz, Yuval Filmus, Michal Igell, Eyal Rozenman, Danny Gutfreund, and Yonatan Bilu. Also, we acknowledge that the proof that Margulis construction is an expander is taken (with slight changes) from course notes of Ravi Boppana, with Mukesh Dalal as scribe
EXPANDER GRAPHS AND THEIR APPLICATIONS 441 We are also grateful for the careful reading of this manuscript by Mark Goresky, Eyal Rozenman and Dave Xiao.Their many constructive comments significantly improved its presentation.Special thanks to Eyal Rozenman for his help with writing the section on Cayley graphs. Contents 1.The magical mystery tour 443 1.1.Three motivating problems 444 1.1.1.Hardness results for linear transformations 444 1.1.2. Construction of good error correcting codes 445 1.1.3. Deterministic error amplification for RP 446 1.2.Magical graphs 447 1.3.The three solutions 448 1.3.1.A super concentrator with O(n)edges 448 1.3.2.Construction of good error correcting codes 450 1.3.3. Deterministic error amplification for RP 451 2.Graph expansion and eigenvalues 452 2.1.Edge expansion and a combinatorial definition of expanders 452 2.2.Examples of expander graphs 453 2.3. Graph spectrum and an algebraic definition of expansion 453 2.4.The Expander Mixing Lemma 454 2.5.How big can the spectral gap be? 455 2.6.Four perspectives on expansion and how they compare 456 2.6.1.Extremal problems 456 2.6.2.Typical behavior 457 2.6.3.Explicit constructions 457 2.6.4.Algorithms 457 2.6.5.Comparisons 457 3.Random walks on expander graphs 458 3.1.Rapid mixing of walks 458 3.1.1.Convergence in the l1 and l2 norms 459 3.1.2.Convergence in entropy 460 3.2.Random walks resemble independent sampling 461 3.3.Applications 464 3.3.1.Efficient error reduction in probabilistic algorithms 464 3.3.2.Hardness of approximating maximum clique size 465 4.A geometric view of expander graphs 469 4.1.The classical isoperimetric problem 469 4.2.Graph isoperimetric problems 470 4.2.1.Example:The discrete cube 471 4.3.The Margulis construction 471 4.3.1.The discrete Laplacian 472 4.4.The Cheeger constant and inequality 473 4.5.Expansion and the spectral gap 474
EXPANDER GRAPHS AND THEIR APPLICATIONS 441 We are also grateful for the careful reading of this manuscript by Mark Goresky, Eyal Rozenman and Dave Xiao. Their many constructive comments significantly improved its presentation. Special thanks to Eyal Rozenman for his help with writing the section on Cayley graphs. Contents 1. The magical mystery tour 443 1.1. Three motivating problems 444 1.1.1. Hardness results for linear transformations 444 1.1.2. Construction of good error correcting codes 445 1.1.3. Deterministic error amplification for RP 446 1.2. Magical graphs 447 1.3. The three solutions 448 1.3.1. A super concentrator with O(n) edges 448 1.3.2. Construction of good error correcting codes 450 1.3.3. Deterministic error amplification for RP 451 2. Graph expansion and eigenvalues 452 2.1. Edge expansion and a combinatorial definition of expanders 452 2.2. Examples of expander graphs 453 2.3. Graph spectrum and an algebraic definition of expansion 453 2.4. The Expander Mixing Lemma 454 2.5. How big can the spectral gap be? 455 2.6. Four perspectives on expansion and how they compare 456 2.6.1. Extremal problems 456 2.6.2. Typical behavior 457 2.6.3. Explicit constructions 457 2.6.4. Algorithms 457 2.6.5. Comparisons 457 3. Random walks on expander graphs 458 3.1. Rapid mixing of walks 458 3.1.1. Convergence in the l1 and l2 norms 459 3.1.2. Convergence in entropy 460 3.2. Random walks resemble independent sampling 461 3.3. Applications 464 3.3.1. Efficient error reduction in probabilistic algorithms 464 3.3.2. Hardness of approximating maximum clique size 465 4. A geometric view of expander graphs 469 4.1. The classical isoperimetric problem 469 4.2. Graph isoperimetric problems 470 4.2.1. Example: The discrete cube 471 4.3. The Margulis construction 471 4.3.1. The discrete Laplacian 472 4.4. The Cheeger constant and inequality 473 4.5. Expansion and the spectral gap 474
442 SHLOMO HOORY.NATHAN LINIAL.AND AVI WIGDERSON 4.5.1.Large spectral gap implies high expansion 475 4.5.2.High expansion implies large spectral gap 475 4.6.Expansion of small sets 477 4.6.1.Connection with the spectral gap 477 4.6.2.Typical behavior 478 4.7.Expansion in hypergraphs? 481 5.Extremal problems on spectrum and expansion 481 5.1.The d-regular tree 482 5.1.1.The expansion of Ta 482 5.1.2.The spectrum of Td 483 5.2.The Alon-Boppana lower bound 484 5.2.1.Statement of the theorem 5.2.2.Proof I:Counting closed walks in Td 48 5.2.3.Proof II:Using spherical functions 5.2.4.Extensions of the Alon-Boppana theorem 5.3.Ramanujan graphs 488 6.Spectrum and expansion in lifts of graphs 489 6.1. Covering maps and lifts 489 6.2.Eigenvalues-old and new 490 6.3.The universal covering tree 6.3.1.Irregular Ramanujan graphs? 491 6.4.Nearly-Ramanujan graphs by way of 2-lifts 492 7.The spectrum of random graphs 493 7.1.The bulk of the spectrum 493 7.2.The extreme eigenvalues 496 7.2.1.An illustration of the trace method 7.3.Variations on a theme 6 7.3.1.Back to the irregular case 5 7.3.2.Are most regular graphs Ramanujan? 7.3.3.More on random lifts 591 7.3.4. The eigenvectors 502 8.The Margulis construction 503 8.1.A detour into harmonic analysis 504 8.1.1.Characters 604 8.2.Back to the proof 505 9.The zig-zag product 508 9.1.Introduction 508 9.2. Construction of an expander family using zig-zag 509 9.3.Definition and analysis of the zig-zag product 510 9.4.Entropy analysis 512 9.5.An application to complexity theory:SL=L 512 10.Lossless conductors and expanders 514 10.1.Conductors and lossless expanders 515 10.1.1.Conductors 515
442 SHLOMO HOORY, NATHAN LINIAL, AND AVI WIGDERSON 4.5.1. Large spectral gap implies high expansion 475 4.5.2. High expansion implies large spectral gap 475 4.6. Expansion of small sets 477 4.6.1. Connection with the spectral gap 477 4.6.2. Typical behavior 478 4.7. Expansion in hypergraphs? 481 5. Extremal problems on spectrum and expansion 481 5.1. The d-regular tree 482 5.1.1. The expansion of Td 482 5.1.2. The spectrum of Td 483 5.2. The Alon-Boppana lower bound 484 5.2.1. Statement of the theorem 484 5.2.2. Proof I: Counting closed walks in Td 484 5.2.3. Proof II: Using spherical functions 485 5.2.4. Extensions of the Alon-Boppana theorem 487 5.3. Ramanujan graphs 488 6. Spectrum and expansion in lifts of graphs 489 6.1. Covering maps and lifts 489 6.2. Eigenvalues - old and new 490 6.3. The universal covering tree 491 6.3.1. Irregular Ramanujan graphs? 491 6.4. Nearly-Ramanujan graphs by way of 2-lifts 492 7. The spectrum of random graphs 493 7.1. The bulk of the spectrum 493 7.2. The extreme eigenvalues 496 7.2.1. An illustration of the trace method 496 7.3. Variations on a theme 500 7.3.1. Back to the irregular case 500 7.3.2. Are most regular graphs Ramanujan? 501 7.3.3. More on random lifts 501 7.3.4. The eigenvectors 502 8. The Margulis construction 503 8.1. A detour into harmonic analysis 504 8.1.1. Characters 504 8.2. Back to the proof 505 9. The zig-zag product 508 9.1. Introduction 508 9.2. Construction of an expander family using zig-zag 509 9.3. Definition and analysis of the zig-zag product 510 9.4. Entropy analysis 512 9.5. An application to complexity theory: SL = L 512 10. Lossless conductors and expanders 514 10.1. Conductors and lossless expanders 515 10.1.1. Conductors 515
EXPANDER GRAPHS AND THEIR APPLICATIONS 443 10.1.2.Lossless expanders 517 10.2.The construction 517 10.2.1.The zig-zag product for conductors 518 10.2.2.Proof of the main theorem 519 10.2.3.Final comments 522 11.Cayley expander graphs 522 11.1.Representations of finite groups 525 11.1.1.Representations and irreducible representations 526 11.1.2.Schreier graphs 528 11.1.3.Kazhdan constant and expansion of Cayley graphs 529 11.2.The replacement product and semidirect product 531 11.3. Constructing expander families by iterated semidirect products 533 11.3.1.Cayley expanders from group rings 533 11.3.2.Cayley expanders from iterated wreath products 534 11.4.Expansion is not a group property 535 11.5.Hypercontractive inequalities in groups? 536 12.Error correcting codes 536 12.1.Definition of error correcting codes 537 12.2. Linear codes 538 12.3.Asymptotic bounds 538 12.3.1.Lower bounds on size:The Gilbert-Varshamov bound 538 12.3.2.Upper bounds:Sphere packing and linear programming 539 12.4. Codes from graphs 540 12.5. Efficient asymptotically good codes from lossless expanders 541 13.Metric embedding 543 13.1. Basic definitions 543 13.2. Finding the minimal l2 distortion 544 13.3.Distortion bounds via semidefinite duality 546 13.3.1. Embedding the cube into l2 546 13.3.2.Embedding expander graphs into l2 547 13.4.Algorithms for cut problems via embeddings 548 13.5.A glimpse into the bigger picture 551 About the authors 551 References 552 Index 560 1.The magical mystery tour We begin our discussion with three fundamental problems from three different domains.At first sight these problems seem to have very little to do with expander graphs,or even graph theory,but as we shall see,they can all be solved using expander graphs
EXPANDER GRAPHS AND THEIR APPLICATIONS 443 10.1.2. Lossless expanders 517 10.2. The construction 517 10.2.1. The zig-zag product for conductors 518 10.2.2. Proof of the main theorem 519 10.2.3. Final comments 522 11. Cayley expander graphs 522 11.1. Representations of finite groups 525 11.1.1. Representations and irreducible representations 526 11.1.2. Schreier graphs 528 11.1.3. Kazhdan constant and expansion of Cayley graphs 529 11.2. The replacement product and semidirect product 531 11.3. Constructing expander families by iterated semidirect products 533 11.3.1. Cayley expanders from group rings 533 11.3.2. Cayley expanders from iterated wreath products 534 11.4. Expansion is not a group property 535 11.5. Hypercontractive inequalities in groups? 536 12. Error correcting codes 536 12.1. Definition of error correcting codes 537 12.2. Linear codes 538 12.3. Asymptotic bounds 538 12.3.1. Lower bounds on size: The Gilbert-Varshamov bound 538 12.3.2. Upper bounds: Sphere packing and linear programming 539 12.4. Codes from graphs 540 12.5. Efficient asymptotically good codes from lossless expanders 541 13. Metric embedding 543 13.1. Basic definitions 543 13.2. Finding the minimal l2 distortion 544 13.3. Distortion bounds via semidefinite duality 546 13.3.1. Embedding the cube into l2 546 13.3.2. Embedding expander graphs into l2 547 13.4. Algorithms for cut problems via embeddings 548 13.5. A glimpse into the bigger picture 551 About the authors 551 References 552 Index 560 1. The magical mystery tour We begin our discussion with three fundamental problems from three different domains. At first sight these problems seem to have very little to do with expander graphs, or even graph theory, but as we shall see, they can all be solved using expander graphs