19.6 Multigrid Methods for Boundary Value Problems 871 standard tridiagonal algorithm.Given u",one solves(19.5.36)for u+1/2,substitutes on the right-hand side of(19.5.37),and then solves for u+.The key question is how to choose the iteration parameter r,the analog of a choice of timestep for an initial value problem. As usual,the goal is to minimize the spectral radius of the iteration matrix. Although it is beyond our scope to go into details here,it turns out that,for the optimal choice of r,the ADI method has the same rate of convergence as SOR. The individual iteration steps in the ADI method are much more complicated than in SOR,so the ADI method would appear to be inferior.This is in fact true if we choose the same parameter r for every iteration step.However,it is possible to 81 choose a different r for each step.If this is done optimally,then ADI is generally more efficient than SOR.We refer you to the literature [1-4]for details. Our reason for not fully implementing ADI here is that,in most applications, it has been superseded by the multigrid methods described in the next section.Our advice is to use SOR for trivial problems (e.g.,20 x 20),or for solving a larger problem once only,where ease of programming outweighs expense of computer time.Occasionally,the sparse matrix methods of 82.7 are useful for solving a set of difference equations directly.For production solution of large elliptic problems, however,multigrid is now almost always the method of choice. 9 CITED REFERENCES AND FURTHER READING: Hockney,R.W.,and Eastwood,J.W.1981,Computer Simulation Using Particles (New York: McGraw-Hill),Chapter 6. 85总6 Young,D.M.1971,Iterative Solution of Large Linear Systems (New York:Academic Press).[1] Stoer,J.,and Bulirsch,R.1980,Introduction to Numerical Analysis(New York:Springer-Verlag). 558.3-8.6.2] Varga,R.S.1962,Matrix /terative Analysis (Englewood Cliffs,NJ:Prentice-Hall).[3] Spanier,J.1967,in Mathematical Methods for Digital Computers,Volume 2(New York:Wiley). Chapter 11.[4] 19.6 Multigrid Methods for Boundary Value Problems 、之花 61 Numerical 10.621 43108 Recipes Practical multigrid methods were first introduced in the 1970s by Brandt.These (outside methods can solve elliptic PDEs discretized on N grid points in O(N)operations North The "rapid"direct elliptic solvers discussed in $19.4 solve special kinds of elliptic equations in O(N log N)operations.The numerical coefficients in these estimates are such that multigrid methods are comparable to the rapid methods in execution speed.Unlike the rapid methods,however,the multigrid methods can solve general elliptic equations with nonconstant coefficients with hardly any loss in efficiency. Even nonlinear equations can be solved with comparable speed. Unfortunately there is not a single multigrid algorithm that solves all elliptic problems.Rather there is a multigrid technique that provides the framework for solving these problems.You have to adjust the various components of the algorithm within this framework to solve your specific problem.We can only give a brief
19.6 Multigrid Methods for Boundary Value Problems 871 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). standard tridiagonal algorithm. Given un, one solves (19.5.36) for un+1/2, substitutes on the right-hand side of (19.5.37), and then solves for u n+1. The key question is how to choose the iteration parameter r, the analog of a choice of timestep for an initial value problem. As usual, the goal is to minimize the spectral radius of the iteration matrix. Although it is beyond our scope to go into details here, it turns out that, for the optimal choice of r, the ADI method has the same rate of convergence as SOR. The individual iteration steps in the ADI method are much more complicated than in SOR, so the ADI method would appear to be inferior. This is in fact true if we choose the same parameter r for every iteration step. However, it is possible to choose a different r for each step. If this is done optimally, then ADI is generally more efficient than SOR. We refer you to the literature [1-4] for details. Our reason for not fully implementing ADI here is that, in most applications, it has been superseded by the multigrid methods described in the next section. Our advice is to use SOR for trivial problems (e.g., 20 × 20), or for solving a larger problem once only, where ease of programming outweighs expense of computer time. Occasionally, the sparse matrix methods of §2.7 are useful for solving a set of difference equations directly. For production solution of large elliptic problems, however, multigrid is now almost always the method of choice. CITED REFERENCES AND FURTHER READING: Hockney, R.W., and Eastwood, J.W. 1981, Computer Simulation Using Particles (New York: McGraw-Hill), Chapter 6. Young, D.M. 1971, Iterative Solution of Large Linear Systems (New York: Academic Press). [1] Stoer, J., and Bulirsch, R. 1980, Introduction to Numerical Analysis (New York: Springer-Verlag), §§8.3–8.6. [2] Varga, R.S. 1962, Matrix Iterative Analysis (Englewood Cliffs, NJ: Prentice-Hall). [3] Spanier, J. 1967, in Mathematical Methods for Digital Computers, Volume 2 (New York: Wiley), Chapter 11. [4] 19.6 Multigrid Methods for Boundary Value Problems Practical multigrid methods were first introduced in the 1970s by Brandt. These methods can solve elliptic PDEs discretized on N grid points in O(N) operations. The “rapid” direct elliptic solvers discussed in §19.4 solve special kinds of elliptic equations in O(N log N) operations. The numerical coefficients in these estimates are such that multigrid methods are comparable to the rapid methods in execution speed. Unlike the rapid methods, however, the multigrid methods can solve general elliptic equations with nonconstant coefficients with hardly any loss in efficiency. Even nonlinear equations can be solved with comparable speed. Unfortunately there is not a single multigrid algorithm that solves all elliptic problems. Rather there is a multigrid technique that provides the framework for solving these problems. You have to adjust the various components of the algorithm within this framework to solve your specific problem. We can only give a brief
872 Chapter 19.Partial Differential Equations introduction to the subject here.In particular,we will give two sample multigrid routines,one linear and one nonlinear.By following these prototypes and by perusing the references [1-41,you should be able to develop routines to solve your own problems. There are two related,but distinct,approaches to the use ofmultigrid techniques. The first,termed"the multigrid method,"is a means for speeding up the convergence of a traditional relaxation method,as defined by you on a grid of pre-specified fineness.In this case,you need define your problem(e.g.,evaluate its source terms) only on this grid.Other,coarser,grids defined by the method can be viewed as temporary computational adjuncts. The second approach,termed(perhaps confusingly)"the full multigrid(FMG) method,"requires you to be able to define your problem on grids of various sizes (generally by discretizing the same underlying PDE into different-sized sets of finite- difference equations).In this approach,the method obtains successive solutions on 不 finer and finer grids.You can stop the solution either at a pre-specified fineness,or you can monitor the truncation error due to the discretization,quitting only when it is tolerably small. RECIPESI In this section we will first discuss the"multigrid method,"then use the concepts 器3d 2 developed to introduce the FMG method.The latter algorithm is the one that we implement in the accompanying programs. From One-Grid,through Two-Grid,to Multigrid The key idea of the multigrid method can be understood by considering the simplest case of a two-grid method.Suppose we are trying to solve the linear elliptic problem 6 Cu=f (19.6.1) where C is some linear elliptic operator and f is the source term.Discretize equation (19.6.1)on a uniform grid with mesh size h.Write the resulting set of linear algebraic equations as Chuh fh (19.6.2) 多，aoao Numerica 10.621 43106 Let ih denote some approximate solution to equation(19.6.2).We will use the symbol uh to denote the exact solution to the difference equations(19.6.2).Then the error in uh or the correction is Uh th uh (19.6.3 The residual or defect is dh=Chuh-fh (19.6.4) (Beware:some authors define residual as minus the defect,and there is not universal agreement about which of these two quantities 19.6.4 defines.)Since Ch is linear, the error satisfies ChUh =-dh (19.6.5)
872 Chapter 19. Partial Differential Equations Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). introduction to the subject here. In particular, we will give two sample multigrid routines, one linear and one nonlinear. By following these prototypes and by perusing the references [1-4], you should be able to develop routines to solve your own problems. There are two related, but distinct, approaches to the use of multigrid techniques. The first, termed “the multigrid method,” is a means for speeding up the convergence of a traditional relaxation method, as defined by you on a grid of pre-specified fineness. In this case, you need define your problem (e.g., evaluate its source terms) only on this grid. Other, coarser, grids defined by the method can be viewed as temporary computational adjuncts. The second approach, termed (perhaps confusingly) “the full multigrid (FMG) method,” requires you to be able to define your problem on grids of various sizes (generally by discretizing the same underlying PDE into different-sized sets of finitedifference equations). In this approach, the method obtains successive solutions on finer and finer grids. You can stop the solution either at a pre-specified fineness, or you can monitor the truncation error due to the discretization, quitting only when it is tolerably small. In this section we will first discuss the “multigrid method,” then use the concepts developed to introduce the FMG method. The latter algorithm is the one that we implement in the accompanying programs. From One-Grid, through Two-Grid, to Multigrid The key idea of the multigrid method can be understood by considering the simplest case of a two-grid method. Suppose we are trying to solve the linear elliptic problem Lu = f (19.6.1) where L is some linear elliptic operator and f is the source term. Discretize equation (19.6.1) on a uniform grid with mesh size h. Write the resulting set of linear algebraic equations as Lhuh = fh (19.6.2) Let uh denote some approximate solution to equation (19.6.2). We will use the symbol uh to denote the exact solution to the difference equations (19.6.2). Then the error in uh or the correction is vh = uh − uh (19.6.3) The residual or defect is dh = Lhuh − fh (19.6.4) (Beware: some authors define residual as minus the defect, and there is not universal agreement about which of these two quantities 19.6.4 defines.) Since Lh is linear, the error satisfies Lhvh = −dh (19.6.5)
19.6 Multigrid Methods for Boundary Value Problems 873 At this point we need to make an approximation to Ch in order to find uh.The classical iteration methods,such as Jacobi or Gauss-Seidel,do this by finding,at each stage,an approximate solution of the equation Chin =-dh (19.6.6) where Ch is a"simpler"operator than Ch.For example,Ch is the diagonal part of Ch for Jacobi iteration,or the lower triangle for Gauss-Seidel iteration.The next approximation is generated by 好w=h+th (19.6.7) 菲 Now consider,as an alternative,a completely different type of approximation ICAL for Ch,one in which we“coarsify”rather than“simplify.”That is,.we form some 3 appropriate approximation Ci of Ch on a coarser grid with mesh size H(we will always take H=2h,but other choices are possible).The residual equation (19.6.5) RECIPES is now approximated by 令 CHUH =-dH (19.6.8) Since C has smaller dimension,this equation will be easier to solve than equation 9」 (19.6.5).To define the defect d on the coarse grid,we need a restriction operator R that restricts dh to the coarse grid: dH Rdh (19.6.9) The restriction operator is also called the fine-to-coarse operator or the injection operator.Once we have a solution n to equation (19.6.8),we need a prolongation operator P that prolongates or interpolates the correction to the fine grid: 三 Uh =PUH (19.6.10) Numerica 10621 431 The prolongation operator is also called the coarse-to-fine operator or the inter- Recipes polation operator.Both R and P are chosen to be linear operators.Finally the approximation un can be updated: 裙ew=h十th (19.6.11) One step of this coarse-grid correction scheme is thus: Coarse-Grid Correction Compute the defect on the fine grid from (19.6.4). .Restrict the defect by (19.6.9). .Solve (19.6.8)exactly on the coarse grid for the correction .Interpolate the correction to the fine grid by (19.6.10)
19.6 Multigrid Methods for Boundary Value Problems 873 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). At this point we need to make an approximation to Lh in order to find vh. The classical iteration methods, such as Jacobi or Gauss-Seidel, do this by finding, at each stage, an approximate solution of the equation Lhvh = −dh (19.6.6) where Lh is a “simpler” operator than Lh. For example, Lh is the diagonal part of Lh for Jacobi iteration, or the lower triangle for Gauss-Seidel iteration. The next approximation is generated by unew h = uh + vh (19.6.7) Now consider, as an alternative, a completely different type of approximation for Lh, one in which we “coarsify” rather than “simplify.” That is, we form some appropriate approximation LH of Lh on a coarser grid with mesh size H (we will always take H = 2h, but other choices are possible). The residual equation (19.6.5) is now approximated by LH vH = −dH (19.6.8) Since LH has smaller dimension, this equation will be easier to solve than equation (19.6.5). To define the defect dH on the coarse grid, we need a restriction operator R that restricts dh to the coarse grid: dH = Rdh (19.6.9) The restriction operator is also called the fine-to-coarse operator or the injection operator. Once we have a solution vH to equation (19.6.8), we need a prolongation operator P that prolongates or interpolates the correction to the fine grid: vh = PvH (19.6.10) The prolongation operator is also called the coarse-to-fine operator or the interpolation operator. Both R and P are chosen to be linear operators. Finally the approximation uh can be updated: unew h = uh + vh (19.6.11) One step of this coarse-grid correction scheme is thus: Coarse-Grid Correction • Compute the defect on the fine grid from (19.6.4). • Restrict the defect by (19.6.9). • Solve (19.6.8) exactly on the coarse grid for the correction. • Interpolate the correction to the fine grid by (19.6.10).
874 Chapter 19.Partial Differential Equations Compute the next approximation by (19.6.11). Let's contrast the advantages and disadvantages ofrelaxation and the coarse-grid correction scheme.Consider the error vh expanded into a discrete Fourier series.Call the components in the lower half of the frequency spectrum the smooth components and the high-frequency components the nonsmooth components.We have seen that relaxation becomes very slowly convergent in the limit h-0,i.e.,when there are a large number of mesh points.The reason turns out to be that the smooth components are only slightly reduced in amplitude on each iteration.However,many relaxation methods reduce the amplitude of the nonsmooth components by large factors on 三 each iteration:They are good smoothing operators. For the two-grid iteration,on the other hand,components of the error with g wavelengths S 2H are not even representable on the coarse grid and so cannot be reduced to zero on this grid.But it is exactly these high-frequency components that can be reduced by relaxation on the fine grid!This leads us to combine the ideas of relaxation and coarse-grid correction: Two-Grid Iteration 影 Pre-smoothing:Compute ih by applying v>0 steps of a relaxation method to uh. 9 .Coarse-grid correction:As above,using in to give inew Post-smoothing:Compute unew by applying v2>0 steps of the relaxation method to inew a以N It is only a short step from the above two-grid method to a multigrid method. Instead of solving the coarse-grid defect equation(19.6.8)exactly,we can get an approximate solution of it by introducing an even coarser grid and using the two-grid OF SCIENTIFIC iteration method.If the convergence factor of the two-grid method is small enough, we will need only a few steps of this iteration to get a good enough approximate 6 solution.We denote the number of such iterations by y.Obviously we can apply this idea recursively down to some coarsest grid.There the solution is found easily,for example by direct matrix inversion or by iterating the relaxation scheme to convergence. One iteration of a multigrid method,from finest grid to coarser grids and back to finest grid again,is called a cycle.The exact structure of a cycle depends on the 10621 value ofy.the number of two-grid iterations at each intermediate stage.The case y=1 is called a V-cycle,while y=2 is called a W-cycle(see Figure 19.6.1).These Numerical Recipes 43106 are the most important cases in practice. Note that once more than two grids are involved,the pre-smoothing steps after (outside 腿 the first one on the finest grid need an initial approximation for the error v.This should be taken to be zero. North Smoothing,Restriction,and Prolongation Operators The most popular smoothing method,and the one you should try first,is Gauss-Seidel,since it usually leads to a good convergence rate.If we order the mesh points from I to N.then the Gauss-Seidel scheme is i=1,,N (19.6.12)
874 Chapter 19. Partial Differential Equations Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). • Compute the next approximation by (19.6.11). Let’s contrast the advantages and disadvantages of relaxation and the coarse-grid correction scheme. Consider the error vh expanded into a discrete Fourier series. Call the components in the lower half of the frequency spectrum the smooth components and the high-frequency components the nonsmooth components. We have seen that relaxation becomes very slowly convergent in the limit h → 0, i.e., when there are a large number of mesh points. The reason turns out to be that the smooth components are only slightly reduced in amplitude on each iteration. However, many relaxation methods reduce the amplitude of the nonsmooth components by large factors on each iteration: They are good smoothing operators. For the two-grid iteration, on the other hand, components of the error with wavelengths <∼ 2H are not even representable on the coarse grid and so cannot be reduced to zero on this grid. But it is exactly these high-frequency components that can be reduced by relaxation on the fine grid! This leads us to combine the ideas of relaxation and coarse-grid correction: Two-Grid Iteration • Pre-smoothing: Compute u¯h by applying ν1 ≥ 0 steps of a relaxation method to uh. • Coarse-grid correction: As above, using u¯h to give u¯new h . • Post-smoothing: Compute unew h by applying ν2 ≥ 0 steps of the relaxation method to u¯new h . It is only a short step from the above two-grid method to a multigrid method. Instead of solving the coarse-grid defect equation (19.6.8) exactly, we can get an approximate solution of it by introducing an even coarser grid and using the two-grid iteration method. If the convergence factor of the two-grid method is small enough, we will need only a few steps of this iteration to get a good enough approximate solution. We denote the number of such iterations by γ. Obviously we can apply this idea recursively down to some coarsest grid. There the solution is found easily, for example by direct matrix inversion or by iterating the relaxation scheme to convergence. One iteration of a multigrid method, from finest grid to coarser grids and back to finest grid again, is called a cycle. The exact structure of a cycle depends on the value of γ, the number of two-grid iterations at each intermediate stage. The case γ = 1 is called a V-cycle, while γ = 2 is called a W-cycle (see Figure 19.6.1). These are the most important cases in practice. Note that once more than two grids are involved, the pre-smoothing steps after the first one on the finest grid need an initial approximation for the error v. This should be taken to be zero. Smoothing, Restriction, and Prolongation Operators The most popular smoothing method, and the one you should try first, is Gauss-Seidel, since it usually leads to a good convergence rate. If we order the mesh points from 1 to N, then the Gauss-Seidel scheme is ui = − N j=1 j=i Lijuj − fi 1 Lii i = 1,...,N (19.6.12)
19.6 Multigrid Methods for Boundary Value Problems 875 ⑤ 2-grid E ⑤ ⑤ ⑨ 3-grid ⊙ © ⑨ E E E 83 旦 granted for from NUMERICAL RECIPES IN C: 19881992 ⑨ 1.200 4-grid ⑤ ⊙ server THE E E E E E (North America computer, ART Y=1 Y=2 Figure 19.6.1. Structure of multigrid cycles.S denotes smoothing,while E denotes exact solution 9 Progra on the coarsest grid.Each descending line denotes restriction (R)and each ascending line denotes prolongation (P).The finest grid is at the top level of each diagram.For the V-cycles (y 1)the E step is replaced by one 2-grid iteration each time the number of grid levels is increased by one.For the W-cycles(=2),each E step gets replaced by two 2-grid iterations. 受 to dir OF SCIENTIFIC COMPUTING (ISBN 1988-1992 where new values of u are used on the right-hand side as they become available.The exact form of the Gauss-Seidel method depends on the ordering chosen for the mesh 10-621 points.For typical second-order elliptic equations like our model problem equation (19.0.3),as differenced in equation (19.0.8),it is usually best to use red-black Fuurggoglrion Numerical Recipes 43108 ordering,making one pass through the mesh updating the"even"points(like the red squares of a checkerboard)and another pass updating the "odd"points(the black squares).When quantities are more strongly coupled along one dimension than (outside another,one should relax a whole line along that dimension simultaneously.Line North Software. relaxation for nearest-neighbor coupling involves solving a tridiagonal system,and so is still efficient.Relaxing odd and even lines on successive passes is called zebra relaxation and is usually preferred over simple line relaxation. visit website Note that SOR should not be used as a smoothing operator.The overrelaxation machine destroys the high-frequency smoothing that is so crucial for the multigrid method. A succint notation for the prolongation and restriction operators is to give their symbol.The symbol of p is found by considering vr to be 1 at some mesh point (y),zero elsewhere,and then asking for the values of Pun.The most popular prolongation operator is simple bilinear interpolation.It gives nonzero values at the 9 points (,y),(+h,y),...,(-h,y-h),where the values are 1
19.6 Multigrid Methods for Boundary Value Problems 875 Permission is granted for internet users to make one paper copy for their own personal use. Further reproduction, or any copyin Copyright (C) 1988-1992 by Cambridge University Press. Programs Copyright (C) 1988-1992 by Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) g of machinereadable files (including this one) to any server computer, is strictly prohibited. To order Numerical Recipes books or CDROMs, visit website http://www.nr.com or call 1-800-872-7423 (North America only), or send email to directcustserv@cambridge.org (outside North America). E γ = 1 γ = 2 2-grid 3-grid 4-grid S S S S S S E S S S S E S S E S S E S S S E S S E S S S S E S S S E S S Figure 19.6.1. Structure of multigrid cycles. S denotes smoothing, while E denotes exact solution on the coarsest grid. Each descending line \ denotes restriction (R) and each ascending line / denotes prolongation (P). The finest grid is at the top level of each diagram. For the V-cycles (γ = 1) the E step is replaced by one 2-grid iteration each time the number of grid levels is increased by one. For the W-cycles (γ = 2), each E step gets replaced by two 2-grid iterations. where new values of u are used on the right-hand side as they become available. The exact form of the Gauss-Seidel method depends on the ordering chosen for the mesh points. For typical second-order elliptic equations like our model problem equation (19.0.3), as differenced in equation (19.0.8), it is usually best to use red-black ordering, making one pass through the mesh updating the “even” points (like the red squares of a checkerboard) and another pass updating the “odd” points (the black squares). When quantities are more strongly coupled along one dimension than another, one should relax a whole line along that dimension simultaneously. Line relaxation for nearest-neighbor coupling involves solving a tridiagonal system, and so is still efficient. Relaxing odd and even lines on successive passes is called zebra relaxation and is usually preferred over simple line relaxation. Note that SOR should not be used as a smoothing operator. The overrelaxation destroys the high-frequency smoothing that is so crucial for the multigrid method. A succint notation for the prolongation and restriction operators is to give their symbol. The symbol of P is found by considering v H to be 1 at some mesh point (x, y), zero elsewhere, and then asking for the values of Pv H. The most popular prolongation operator is simple bilinear interpolation. It gives nonzero values at the 9 points (x, y), (x + h, y),...,(x − h, y − h), where the values are 1, 1 2 ,..., 1 4