Examples that Use Standard Algorithms the gradient at the solution)in firstorderopt, and the type of algorithm used output funcCount: 40 firstorderopt: 9. 2801e-004 lgorithm: 'medium-scale: Quas When more than one local minimum exists, the initial guess for the vector Exl, x] affects both the number of function evaluations and the value of the solution point. In the preceding example, xo is initialized to [-1, 1] The variable options can be passed to fminunc to change characteristics of the x= fminunc(@objfun, XO, options) options is a structure that contains values for termination tolerances and algorithm choices. An options structure can be created using the optimset options optimset( LargeScale,'off); n this example, we have turned off the default selection of the large-scale algorithm and so the medium-scale algorithm is used. Other options include controlling the amount of command line display during the optimization iteration, the tolerances for the termination criteria, whether a user-supplied gradient or Jacobian is to be used, and the maximum number of iterations or function evaluations. See optimset, the individual optimization functions, and Table 5, Optimization Parameters, on page 5-10 for more options and nformation Nonlinear Inequality Constrained Example If inequality constraints are added to Eq 2-1, the resulting problem can be solved by the fmincon function. For example, find x that solves minimize f(x)=e(4x1+2x5+4x,x,+2x,+1) ubject to the constraints 2-9
Examples that Use Standard Algorithms 2-9 the gradient at the solution) in firstorderopt, and the type of algorithm used in algorithm: output = iterations: 7 funcCount: 40 stepsize: 1 firstorderopt: 9.2801e-004 algorithm: 'medium-scale: Quasi-Newton line search' When more than one local minimum exists, the initial guess for the vector [x1, x2] affects both the number of function evaluations and the value of the solution point. In the preceding example, x0 is initialized to [-1,1]. The variable options can be passed to fminunc to change characteristics of the optimization algorithm, as in x = fminunc(@objfun,x0,options); options is a structure that contains values for termination tolerances and algorithm choices. An options structure can be created using the optimset function: options = optimset('LargeScale','off'); In this example, we have turned off the default selection of the large-scale algorithm and so the medium-scale algorithm is used. Other options include controlling the amount of command line display during the optimization iteration, the tolerances for the termination criteria, whether a user-supplied gradient or Jacobian is to be used, and the maximum number of iterations or function evaluations. See optimset, the individual optimization functions, and Table 5, Optimization Parameters, on page 5-10 for more options and information. Nonlinear Inequality Constrained Example If inequality constraints are added to Eq. 2-1, the resulting problem can be solved by the fmincon function. For example, find x that solves (2-2) subject to the constraints minimize x f x( ) e x1 4x1 2 2x2 2 4x1x2 2x2 = ( ) ++ ++ 1
Tutorial ≤-1.5 10 Because neither of the constraints is linear you cannot pass the constraints to fmincon at the command line Instead you can create a second M-file confun. m that returns the value at both constraints at the current x in a vector The constrained optimizer fmincon, is then invoked Because fmincon expects the constraints to be written in the form c(x)s0, you must rewrite your constraints in the form +1.5≤0 (23) 10≤0 Step 1: Write an M-file confun m for the constraints function [c, ceq]=confun(x) aints [1.5+X(1)*x(2)-X(1)·X(2) 9 Nonlinear equality constraints ceq=[ Step 2: Invoke constrained optimization routine. at the solutio ptions optimset('LargeScale, 'off )i fmincon (aobjfun, XO,[,[,[,[,[,[, aconfun, options After 38 function calls, the solution x produced with function value fval is 9.54741.0474 fval We can evaluate the constraints at the solution 1.0e-14 2-10
2 Tutorial 2-10 Because neither of the constraints is linear, you cannot pass the constraints to fmincon at the command line. Instead you can create a second M-file, confun.m, that returns the value at both constraints at the current x in a vector c. The constrained optimizer, fmincon, is then invoked. Because fmincon expects the constraints to be written in the form , you must rewrite your constraints in the form (2-3) Step 1: Write an M-file confun.m for the constraints. function [c, ceq] = confun(x) % Nonlinear inequality constraints c = [1.5 + x(1)*x(2) - x(1) - x(2); -x(1)*x(2) - 10]; % Nonlinear equality constraints ceq = []; Step 2: Invoke constrained optimization routine. x0 = [-1,1]; % Make a starting guess at the solution options = optimset('LargeScale','off'); [x, fval] = ... fmincon(@objfun,x0,[],[],[],[],[],[],@confun,options) After 38 function calls, the solution x produced with function value fval is x = -9.5474 1.0474 fval = 0.0236 We can evaluate the constraints at the solution [c,ceq] = confun(x) c= 1.0e-14 * 0.1110 x1x2 x1 – x2 – ≤ –1.5 x1x2 ≥ –10 c x( ) ≤ 0 x1x2 x1 – x2 – + 1.5 ≤ 0 x1x2 – – 10 ≤ 0
Examples that Use Standard Algorithms 0.1776 Note that both constraint values are less than or equal to zero; that is, x satisfies c(x)≤0 Constrained Example with bounds The variables in x can be restricted to certain limits by specifying simple bound constraints to the constrained optimizer function For fmincon, the command x fmincon(Gobjfun, O, [,[,[,[,lb, ub, econfun options); limits x to be within the range 1b<=x<=ub To restrict x in Eq 2-2 to be greater than zero (i. e, x120, x220), use the commands X0=[-1,1] Make a starting guess at the solution 1b=[0,0] Set lower bounds 9 No upper bounds options optimset(LargeScale, 'off); fmincon (cobjfun, X0, [,[,[],[],Ib, ub, aconfun, options) [c, ceq]= confun(x) Note that to pass in the lower bounds as the seventh argument to fmincon, you must specify values for the third through sixth arguments. In this example, we pecified [ for these arguments since there are no linear inequalities or linear After 13 function evaluations, the solution produced is foal= 8.5000 cea 2-11
Examples that Use Standard Algorithms 2-11 -0.1776 ceq = [] Note that both constraint values are less than or equal to zero; that is, x satisfies . Constrained Example with Bounds The variables in x can be restricted to certain limits by specifying simple bound constraints to the constrained optimizer function. For fmincon, the command x = fmincon(@objfun,x0,[],[],[],[],lb,ub,@confun,options); limits x to be within the range lb <= x <= ub. To restrict x in Eq. 2-2 to be greater than zero (i.e., ), use the commands x0 = [-1,1]; % Make a starting guess at the solution lb = [0,0]; % Set lower bounds ub = [ ]; % No upper bounds options = optimset('LargeScale','off'); [x,fval = ... fmincon(@objfun,x0,[],[],[],[],lb,ub,@confun,options) [c, ceq] = confun(x) Note that to pass in the lower bounds as the seventh argument to fmincon, you must specify values for the third through sixth arguments. In this example, we specified [] for these arguments since there are no linear inequalities or linear equalities. After 13 function evaluations, the solution produced is x = 0 1.5000 fval = 8.5000 c = 0 -10 ceq = c x( ) ≤ 0 x1 0 , x2 ≥ ≥ 0
2 Tutorial When lb or ub contains fewer elements than x, only the first corresponding elements in x are bounded. Alternatively, if only some of the variables are bounded then use inf in lb for unbounded below variables and inf in ub for unbounded above variables. For example 1b =[-inf 0 b=[10 inf] bounds x1 s10,0sx2(x, has no lower bound and x, has no upper bound) USing inf and-inf give better numerical results than using a very large positive number or a very large negative number to imply lack of bounds. Note that the number of function evaluations to find the solution is reduced because we further restricted the search space. Fewer function evaluations are usually taken when a problem has more constraints and bound limitations because the optimization makes better decisions regarding step size and regions of feasibility than in the unconstrained case. It is, therefore, good practice to bound and constrain problems, where possible, to promote fast convergence to a solution Constrained Example with Gradients Ordinarily the medium-scale minimization routines use numerical gradients calculated by finite-difference approximation. This procedure systematically perturbs each of the variables in order to calculate function and constraint partial derivatives. Alternatively, you can provide a function to compute partial derivatives analytically. Typically, the problem is solved more accurately and efficiently if such a function is provided Step 1: Write an M-file for the objective function and gradient f=exp(x(1))*(4*x(1)^2+2*X(2)^2+4*X(1)*X(2)+2*x(2)+1) Gradient of the objective function xp(x(1))*(4*X(1)^2+2*X(2)^2+4*X(1)*X(2)+2*X(2)+1); exp(x(1))*(4*x(1)+4*x(2)+2)]; 2-12
2 Tutorial 2-12 [] When lb or ub contains fewer elements than x, only the first corresponding elements in x are bounded. Alternatively, if only some of the variables are bounded, then use -inf in lb for unbounded below variables and inf in ub for unbounded above variables. For example, lb = [-inf 0]; ub = [10 inf]; bounds ( has no lower bound and has no upper bound). Using inf and -inf give better numerical results than using a very large positive number or a very large negative number to imply lack of bounds. Note that the number of function evaluations to find the solution is reduced because we further restricted the search space. Fewer function evaluations are usually taken when a problem has more constraints and bound limitations because the optimization makes better decisions regarding step size and regions of feasibility than in the unconstrained case. It is, therefore, good practice to bound and constrain problems, where possible, to promote fast convergence to a solution. Constrained Example with Gradients Ordinarily the medium-scale minimization routines use numerical gradients calculated by finite-difference approximation. This procedure systematically perturbs each of the variables in order to calculate function and constraint partial derivatives. Alternatively, you can provide a function to compute partial derivatives analytically. Typically, the problem is solved more accurately and efficiently if such a function is provided. To solve Eq. 2-2 using analytically determined gradients, do the following. Step 1: Write an M-file for the objective function and gradient. function [f,G] = objfungrad(x) f = exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1); % Gradient of the objective function t = exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1); G = [ t + exp(x(1)) * (8*x(1) + 4*x(2)), exp(x(1))*(4*x(1)+4*x(2)+2)]; x1 10 , 0 x2 ≤ ≤ x1 x2
Examples that Use Standard Algorithms Step 2: Write an M-file for the nonlinear constraints and the gradients of function [c, ceq, DC, DCeq]= confungrad(x) c(1)=1.5+x(1)*x(2)-x(1)-X(2); %Inequality constraints 9 Gradient of the constraints DC=[X(2)-1,-X(2); x(1)-1,-X(1)] No nonlinear equality constraints ceq=[; G contains the partial derivatives of the objective function, f, returned by obj fungrad (x), with respect to each of the elements in x e2(4x1+2x2+4x1x2+2x2+1)+e2(8x1+4x2) df (24) e(4x1+4x2+2) The columns of DC contain the partial derivatives for each respective constraint (i. e the ith column of DC is the partial derivative of the ith constraint with respect to x). So in the above example, DC is (25 Since you are providing the gradient of the objective in objfungrad. m and the gradient of the constraints in confungrad. m, you must tell fmincon that these M-files contain this additional information. Use optimset to turn the parameters Gradobj and Gradconstr to'on'in the example,s existing options options optimset(options, 'Gradobj','on','GradConstr,'on)
Examples that Use Standard Algorithms 2-13 Step 2: Write an M-file for the nonlinear constraints and the gradients of the nonlinear constraints. function [c,ceq,DC,DCeq] = confungrad(x) c(1) = 1.5 + x(1) * x(2) - x(1) - x(2); %Inequality constraints c(2) = -x(1) * x(2)-10; % Gradient of the constraints DC= [x(2)-1, -x(2); x(1)-1, -x(1)]; % No nonlinear equality constraints ceq=[]; DCeq = [ ]; G contains the partial derivatives of the objective function, f, returned by objfungrad(x), with respect to each of the elements in x: (2-4) The columns of DC contain the partial derivatives for each respective constraint (i.e., the ith column of DC is the partial derivative of the ith constraint with respect to x). So in the above example, DC is (2-5) Since you are providing the gradient of the objective in objfungrad.m and the gradient of the constraints in confungrad.m, you must tell fmincon that these M-files contain this additional information. Use optimset to turn the parameters GradObj and GradConstr to 'on' in the example’s existing options structure: options = optimset(options,'GradObj','on','GradConstr','on'); ∂f ∂x ----- e x1 4x1 2 2x2 2 4x1x2 2x2 ( ) ++ ++ 1 e x1 8x1 4x2 + ( ) + e x1 4x1 4x2 ( ) + + 2 = c1 ∂ x1 ∂ -------- c2 ∂ x1 ∂ -------- c1 ∂ x2 ∂ -------- c2 ∂ x2 ∂ -------- x2 – 1 x2 – x1 – 1 x1 – =