MIest 16888 Today's Topics Multidisciplinary System Design variable linking Design Optimization(MSDO) Reduced-Basis Methods Approximation Methods Response Surface Approximations Lecture 19 Kriging 16Apml2004 Variable-Fidelity Models Karen willcox e Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIlesd Why Approximation Methods? Approximation Methods Ve have seen throughout the course the constant trade-off Recall that the analysis or simcode must be invoked between computational cost and fidelity. each time the optimizer selects a new design vector to how to (e.g CFD, FEM) Typically, hundreds( thousands)of design vectors will e analyzed throughout an optimization run from Giesing. 1998 Can use Approximation Models(Surrogate Models) for objective functions and constraints Level of MsDo If approximate models are inexpensive to evaluate, can imed MDO analyze many more design vector options without studies optimizationiteration Approximation methods provide a way to get high-fidelity Concept first introduced in structural optimization by model information throughout the optimization without the Barthelemy and Haftka, 1993 computational e Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics
1 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Lecture 19 16 April 2004 Karen Willcox Multidisciplinary System Multidisciplinary System Design Optimization (MSDO) Design Optimization (MSDO) 2 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Today’s Topics Today’s Topics • Design variable linking • Reduced-Basis Methods • Response Surface Approximations • Kriging • Variable-Fidelity Models 3 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Why Approximation Methods? Why Approximation Methods? We have seen throughout the course the constant trade-off between computational cost and fidelity. Level of MSDO Fidelity Level trade studies limited optimization/iteration full MDO empirical models intermediate fidelity (e.g. vortex lattice, beam theory) high fidelity (e.g. CFD,FEM) increasing difficulty from Giesing, 1998 can we do better? can the results be believed? how to implement? Approximation methods provide a way to get high-fidelity model information throughout the optimization without the computational expense. 4 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Approximation Methods • Recall that the analysis or simcode must be invoked each time the optimizer selects a new design vector to try • Typically, hundreds (thousands) of design vectors will be analyzed throughout an optimization run • Can use Approximation Models (Surrogate Models) for objective functions and constraints • If approximate models are inexpensive to evaluate, can analyze many more design vector options without worrying about computational resources • Concept first introduced in structural optimization by Barthelemy and Haftka, 1993
Mlesd Approximation Methods Overview E0 Mles Design Variable Linking reduce the number of Design variable linking design variables in Not all design variables may be independent Reduced-basis methods the optimization code For example, symmetry may exist in the problem simcode analysis full optimizer uses y, same number of y=TX+y Response surface methods design variables angIng simcode analysis Define simplified X Variable-fidelity methods combine high-fidelity and approximation models rxn x1 @ Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics Mesd Reduced-Basis Methods Reduced-Basis Methods Consider r feasible design vectors: x1, x2,.,x' We can now optimize J(x)by finding the optimal values for the coefficients ap Ve could consider the desired design to be a linear dimension n dimension r combination of these basis vectors do one full-order evaluation of resulting answe approach is efficient if r<< n will give the true optimum only if x* lies in the span of x) added fo basis vectors could be scalar vector generality previous designs coefficient lutions over a particular range(DoE) derived in some other way( POD) e Massachusetts Institute of Technology- Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
5 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Overview Design variable linking Reduced-basis methods • reduce the number of design variables in the optimization code • simcode analysis full order Response surface methods Kriging • same number of design variables • simcode analysis simplified Variable-fidelity methods • combine high-fidelity and approximation models 6 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Design Variable Linking Design Variable Linking • Not all design variables may be independent • For example, symmetry may exist in the problem Define: C = + ⎡ ⎤ ⎢ ⎥ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ = ⎢ ⎥ + ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎢ ⎥ ⎣ ⎦ y Tx y r ×1 n ×1 r ×n r ×1 optimizer uses y, but provides x to simcode for analysis y T x yC 7 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Methods Basis Methods Consider r feasible design vectors: x1, x2, ..., xr We could consider the desired design to be a linear combination of these basis vectors: 1 * αι = = ∑ + r i C i x x x scalar coefficient basis vector added for generality 8 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Methods Basis Methods We can now optimize J(x) by finding the optimal values for the coefficients αi. dimension n dimension r • do one full-order evaluation of resulting answer • approach is efficient if r << n • will give the true optimum only if x* lies in the span of {xi} • basis vectors could be – previous designs – solutions over a particular range (DoE) – derived in some other way (POD)
Mlesd Reduced-Basis Example 16888 Reduced-Basis Example Example using a reduced-basis approach(van der Plaats Fig 7-2 airfoil design for a unique application many airfoil shapes with known performance are Vanderplaats, G. N Numerical Optimization Techniques for Engineering Des available design variables are(x, y) coordinates at chordwise locations(n-100) use four basis airfoil shapes(low-speed airfoils)which contain the n geometry points plus two basis shapes which allow trailing edge thickness to vary rplaats, G N. Numerical Optimization Techniques for Engineering Design rplaats R&D, 1999. Figure 7-3 =6(K<n) optimize for high speed, maximum lift with a constraint on drag @ Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics Mlesd Proper Orthogonal Decomposition E0t Mlesd Proper Orthogonal Decomposition E89 Loeve decomposition, singular value decomposition en- Also known as principal components analysis, Kahunen- These optimal basis functions can be calculated by Evaluating the correlation matrix R 2. Solving the MxM eigenvalue probler p are orthog They are computed from M empirical solutions x', x2,.1 They are optimal in the sense that they minimize the error 3. Constructing the basis vectors between the original and the projected data Use components of the x eigenvector to calculate the /th max(, g=∑ POD basis vector. The th ortant is the / th e Massachusetts Institute of Technology -Prof de Weck and Prof. Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems DiMsion and Dept of Aeronautics and Astronautics
9 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Example Basis Example Example using a reduced-basis approach (van der Plaats Fig 7-2): airfoil design for a unique application. • many airfoil shapes with known performance are available • design variables are (x,y) coordinates at chordwise locations (n~100) • use four basis airfoil shapes (low-speed airfoils) which contain the n geometry points • plus two basis shapes which allow trailing edge thickness to vary • r=6 (r<<n) • optimize for high speed, maximum lift with a constraint on drag 10 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Reduced Reduced-Basis Example Basis Example 11 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Proper Orthogonal Decomposition Proper Orthogonal Decomposition 1 * αιϕ = = ∑ r i i x • The r basis vectors ϕi are orthogonal • They are computed from M empirical solutions {x1, x2,... xM } • They are optimal in the sense that they minimize the error between the original and the projected data 2 2 max ( , ) ϕ ϕ ϕ x Also known as principal components analysis, KahunenLoève decomposition, singular value decomposition 12 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Proper Orthogonal Decomposition Proper Orthogonal Decomposition These optimal basis functions can be calculated by: 1. Evaluating the correlation matrix: 2. Solving the M×M eigenvalue problem: 3. Constructing the basis vectors: i j T R ij = xx R v v i i i = λ 1 M i i j j i ϕ v = = ∑ x Use components of the jth eigenvector to calculate the jth POD basis vector. The jth eigenvalue tells us how important is the jth basis vector. Vanderplaats, G. N. Numerical Optimization Techniques for Engineering Design. Vanderplaats R&D, 1999. Figure 7-2. Vanderplaats, G. N. Numerical Optimization Techniques for Engineering Design. Vanderplaats R&D, 1999. Figure 7-3
Mlesd Approximation Methods Overview E0 Mles Response Surface Methodology 505 reduce the number of Keep the same number of design variables, but simplify the Design variable linking design variables in simcode analysis Reduced-basis methods the optimization code Create approximating functions to objective and constraints simcode analysis full Optimize using the approximations order Update approximations using current optimal solution same number of Response surface methods design variables Response surfaces are smooth even if design space is noisy Kriging simcode analysis simplified Polynomial-based modeling technique Provide compact, explicit functional relationship between combine high-fidelity and response and design variables Variable-fidelity methods approximation models Least squares is computationally inexpensive and easy to @Massachusetts Institute of Technology -Prof de Weck and Prof Willcox G Massachusetts Insttute of Technology .Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIlesd Local Approximations Local Approximations Most common are taylor series expansions ()4×)+J(x)]x+2xH(x)x+ It is expensive to update the gradient vector and Hessian matrⅸ Sx=x-x ne appre Could use the first two terms linear approximation perform several approximation cycles updating only the constant term Solve, reanalyze and repeat= sequential linear programming then update the linear term and repeat Could also include quadratic term finally, update the Hessian only when no other (-x)+(x3).x+20xH(x)x+ progress can be made If the design space is highly nonlinear, there is no 1 function guarantee that this approach will work n func function evaluat setts Institute of Technology -Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
13 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods Overview Design variable linking Reduced-basis methods • reduce the number of design variables in the optimization code • simcode analysis full order Response surface methods Kriging • same number of design variables • simcode analysis simplified Variable-fidelity methods • combine high-fidelity and approximation models 14 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Response Surface Methodology Response Surface Methodology • Keep the same number of design variables, but simplify the simcode analysis • Create approximating functions to objective and constraints • Optimize using the approximations • Update approximations using current optimal solution guess and repeat • Response surfaces are smooth even if design space is noisy • Polynomial-based modeling technique • Provide compact, explicit functional relationship between response and design variables • Least squares is computationally inexpensive and easy to implement 15 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Local Approximations Local Approximations Most common are Taylor series expansions: = + ⎡∇ ⎤ δ + δ δ + 0 ⎣ 0 ⎦ " 1 ( ) ( ) ( ) ( ) 2 T T J J J 0 x x x x x H x x 0 δ x xx = − • Could use the first two terms ⇒ linear approximation • Solve, reanalyze and repeat = sequential linear programming • Could also include quadratic term: = + ⎡∇ ⎤ δ + δ δ + 0 ⎣ 0 ⎦ " 1 ( ) ( ) ( ) ( ) 2 T T J J J 0 x x x x x H x x update requires: 1 function evaluation n function evaluations n(n+1)/2 function evaluations 16 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Local Approximations Local Approximations • It is expensive to update the gradient vector and Hessian matrix • One approach: – perform several approximation cycles updating only the constant term – then update the linear term and repeat – finally, update the Hessian only when no other progress can be made • If the design space is highly nonlinear, there is no guarantee that this approach will work
M sd Response Surface Methodology 5. 33 RSM Another approach: use what information is available to e.g. define AJ=J(x)-J(x) create the approximate Use this approximation to make a small move in the quadratic approximation A=VSx+OxHey Analyze the result precisely = new function evaluation Use the new function evaluation to improve the 6x1+6x2++dx approximation to the design spa ax → Fit a response surface Can use a quadratic or higher order surface (H1bx+H25x2+…+Hmbx) Might choose to use only some of the function evaluations +Hnox x,+hnox, dx, ++H,Ox,dx (e.g. those in a local neighborhood +H25xbx3+…+ Hr-l,dx-5 (all derivatives and entries of H are evaluated at x o) @Massachusetts Institute of Technology -Prof de Weck and Prof Willcox e Massachusetts Insttute of Technology. Prof de Weck and Prof Willcox Engineering Systems Division and Dept of Aeronautics and Astronautics Engineering Systems Division and Dept of Aeronautics and Astronautics MIest RSM RSM Assume we have evaluated the baseline plus g designs q equations, N unknowns If g<N, only some coefficients can be calculated x",x,,x→J0J,J If N, use weighted least squares could write q equations of the form(*)using Weight designs closer to current xq more heavily In general, use g2n+ 1 initial designs so an initial linear approximation can be provided If have baseline plus n designs, can fit a linear There is a total of N=ntn(n+1)/2 unknowns approximation in each direction (i.e. a hyperplane) If have an fit a quadratic or higher order surface a Massachusetts Institute of Technology -Prof de Weck and Prof Willcox etts Institute of Technology .Prof de Weck and Prof Willcox Engineening Systems Division and Dept of Aeronautics and Astronautics ngineering Systems Divsion and Dept of Aeronautics and Astronautics
17 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics Response Surface Methodology Response Surface Methodology • Another approach: use what information is available to create the approximation • Use this approximation to make a small move in the design variables • Analyze the result precisely ⇒ new function evaluation • Use the new function evaluation to improve the approximation to the design space ⇒ Fit a response surface • Can use a quadratic or higher order surface • Might choose to use only some of the function evaluations (e.g. those in a local neighborhood) 18 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM ∆ = − 0 eg J J J . . define ( ) ( ) x x ( ) T T 1 2 1 2 2 2 2 11 1 22 2 12 1 2 13 1 3 1 1 23 2 3 1, 1 ... 1 ... 2 ... ... n n nn n n n n n n n J J J J J x x x x x x H x H x H x H x x H x x H x x H x x H x x δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ δ − − ∆ =∇ + ∂ ∂ ∂ = + + + ∂ ∂ ∂ + + + + + + + + + + + x x H x quadratic approximation: (all derivatives and entries of H are evaluated at x0) (∗) 19 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM Assume we have evaluated the baseline plus q designs: 0 1 → 0 1 , ,..., , ,... q q xx x J J J We could write q equations of the form (∗) using There is a total of N=n+n(n+1)/2 unknowns: ∂ ∂ ∂ ∂ ∂ ∂ 11 12 1 2 , ,..., , , ,..., nn n J J J HH H x x x ( ) ( ) ( ) 1 0 1 0 2 0 2 0 0 0 , , , , , , q q x x x x − J J J J − − − " x x − J J − 20 © Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Engineering Systems Division and Dept. of Aeronautics and Astronautics RSM • q equations, N unknowns • If q<N, only some coefficients can be calculated • If q>N, use weighted least squares • Weight designs closer to current xq more heavily • In general, use q≥n+1 initial designs so an initial linear approximation can be provided • If have baseline plus n designs, can fit a linear approximation in each direction (i.e. a hyperplane) • If have more solutions, can fit a quadratic or higher order surface