Topic #22 16.31 Feedback Control Deterministic lOR Optimal control and the riccati equation · Lagrange multipliers The Hamiltonian matrix and the symmetric root locus Factoids: for symmtric R au ru 2u'R aR=R Copyright 2001 by Jonathan How
Topic #22 16.31 Feedback Control Deterministic LQR • Optimal control and the Riccati equation • Lagrange multipliers • The Hamiltonian matrix and the symmetric root locus Factoids: for symmtric R ∂uTRu = 2uTR ∂u ∂Ru = R ∂u Copyright 2001 by Jonathan How. 1
Fal!2001 16.3122-1 Linear Quadratic Regulator(LQR) We have seen the solutions to the lqr problem using the symmetric root locus which defines the location of the closed-loop poles Linear full-state feedback control Would like to demonstrate from first principles that this is the optimal form of the control Deterministic Linear Quadratic Regulator Plant: i(t)=A(tac(t)+ Bu(tu(t), a(to)=ro (t)=C2(t)x(t) Cost LQR 2(t) R2(t)z(t)+u(t)Ru(t)u(t) dt+a(tt)Ptr a(tf) Where P 20, Rz(t)>0 and Ruu(t)>0 Define Rxx=CRnC2≥0 A(t)is a continuous function of time Bu(t), C2(t, Rz(t), Ruu(t) are piecewise continuous functions of time. and all are bounded Problem Statement: Find the input u(t)Vt Eto, te] to mini- mize JLQR
Fall 2001 16.31 22—1 Linear Quadratic Regulator (LQR) • We have seen the solutions to the LQR problem using the symmetric root locus which defines the location of the closed-loop poles. — Linear full-state feedback control. — Would like to demonstrate from first principles that this is the optimal form of the control. • Deterministic Linear Quadratic Regulator Plant: x˙(t) = A(t)x(t) + Bu(t)u(t), x(t0) = x0 z(t) = Cz(t)x(t) Cost: Z tf £ JLQR = zT (t)Rzz(t)z(t) + uT (t)Ruu(t)u(t) ¤ dt + x(tf )Ptfx(tf ) t0 — Where Ptf ≥ 0, Rzz(t) > 0 and Ruu(t) > 0 — Define Rxx = Cz TRzzCz ≥ 0 — A(t) is a continuous function of time. — Bu(t), Cz(t), Rzz(t), Ruu(t) are piecewise continuous functions of time, and all are bounded. • Problem Statement: Find the input u(t) ∀t ∈ [t0, tf ] to minimize JLQR
Fal!2001 16.3122-2 Note that this is the most general form of the LQR problem we rarely need this level of generality and often suppress the time dependence of the matrices Aircraft landing problem To optimize the cost, we follow the procedure of augmenting the constraints in the problem(the system dynamics)to the cost(inte- grand) to form the Hamiltonian H=2(2(01(6+()u()+X()(4a()+Bu(t) A(t ERnXI is called the Adjoint variable or Costate It is the Lagrange multiplier in the problem From Stengel (pg427), the necessary and sufficient conditions for optimality are that R×x(t)-A7(t) a(te)= pe a(tf) 3.a.=0→R1l+B2(t)=0.r=-RlB(t) Ou2=0(need to check that Ruu >0
Fall 2001 16.31 22—2 • Note that this is the most general form of the LQR problem — we rarely need this level of generality and often suppress the time dependence of the matrices. — Aircraft landing problem. • To optimize the cost, we follow the procedure of augmenting the constraints in the problem (the system dynamics) to the cost (integrand) to form the Hamiltonian: 1 ¢ H = 2 ¡ xT (t)Rxxx(t) + uT (t)Ruuu(t) + λT (t) (Ax(t) + Buu(t)) — λ(t) ∈ Rn×1 is called the Adjoint variable or Costate — It is the Lagrange multiplier in the problem. • From Stengel (pg427), the necessary and sufficient conditions for optimality are that: T 1. λ˙(t) = −∂H = −Rxxx(t) − AT λ(t) ∂x 2. λ(tf ) = Ptfx(tf ) 3. ∂H = 0 ⇒ Ruuu + Bu T λ(t)=0, so uopt = −R−1 ∂u uu Bu T λ(t) 4. ∂2 H ≥ 0 (need to check that Ruu ≥ 0) ∂u2
Fal!2001 16.31 Optimization -1 This control design problem is a constrained optimization, with the constraints being the dynamics of the system The standard way of handling the constraints in an optimization is to add them to the cost using a lagrange multiplier Results in an unconstrained optimization Example: min f(, y)=a2 +y2 subject to the constraint that c(x,y)=x+y+2=0 Figure 1: Optimization results Clearly the unconstrained minimum is at a=y=0
Fall 2001 16.31 Optimization-1 • This control design problem is a constrained optimization, with the constraints being the dynamics of the system. • The standard way of handling the constraints in an optimization is to add them to the cost using a Lagrange multiplier — Results in an unconstrained optimization. • Example: min f(x, y) = x2 + y2 subject to the constraint that c(x, y) = x + y + 2 = 0 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 x Figure 1: Optimization results y • Clearly the unconstrained minimum is at x = y = 0
Fal!2001 16.31 Optimization -2 To find the constrained minimum form the augmented cost function L,f(x,y)+λc(x,y)=x2+y2+入(x+y+2) Where X is the lagrange multiplier e Note that if the constraint is satisfied, then L= f The solution approach without constraints is to find the stationary point of f(a, y)(af ax=af ay=0) With constraints we find the stationary points of L OL alaL axya入 which gives OL 2x+入=0 a=20+X=0 OL +y+2=0 e This gives 3 equations in 3 unknowns, solve to fine The key point here is that due to the constraint, the selection of a and y during the minimization are not independent The Lagrange multiplier captures this dependency The lQr optimization follows the same path as this, but it is com- plicated by the fact that the cost involves an integration over time
Fall 2001 16.31 Optimization-2 • To find the constrained minimum, form the augmented cost function L , f(x, y) + λc(x, y) = x2 + y2 + λ(x + y + 2) — Where λ is the Lagrange multiplier • Note that if the constraint is satisfied, then L ≡ f • The solution approach without constraints is to find the stationary point of f(x, y) (∂f/∂x = ∂f/∂y = 0) — With constraints we find the stationary points of L ∂L ∂L ∂L = = = 0 ∂x ∂y ∂λ which gives ∂L ∂x = 2x + λ = 0 ∂L = 2y + λ = 0 ∂y ∂L = x + y + 2 = 0 ∂λ • This gives 3 equations in 3 unknowns, solve to find x? = y? = −1 • The key point here is that due to the constraint, the selection of x and y during the minimization are not independent — The Lagrange multiplier captures this dependency. • The LQR optimization follows the same path as this, but it is complicated by the fact that the cost involves an integration over time