School of Automation, XJTUaH[x(t),u*(t),a(t),t)x(t) =(2.10)x(to) = xoaa(t)aH[x(t),u*(t),a(t),t)ap[T,x(T)]i(t) = -^(T) =(2.11)二ax(t)ax(T)a[T,x(T)]H[x(T),u*(T), a(T), T) + (A)0aT［例]LQR问题(2.12)x(t) = A(t)x(t) + B(t)u(t), x(to) = XoJ(xo,to) =xT(T)Srx(T) +J,[xTQx +uTRu]dt (2.13)7CAIYUANLI
School of Automation, XJTU © CAI YUANLI 7 𝑥̇(𝑡) = 𝜕𝐻[𝑥(𝑡),𝑢 ∗ (𝑡),𝜆(𝑡),𝑡] 𝜕𝜆(𝑡) ，𝑥(𝑡0 ) = 𝑥0 (2.10) 𝜆̇(𝑡) = − 𝜕𝐻[𝑥(𝑡),𝑢 ∗ (𝑡),𝜆(𝑡),𝑡] 𝜕𝑥(𝑡) ，𝜆(𝑇) = 𝜕𝜑[𝑇,𝑥(𝑇)] 𝜕𝑥(𝑇) (2.11) 𝐻[𝑥(𝑇), 𝑢 ∗ (𝑇), 𝜆(𝑇), 𝑇] + 𝜕𝜑[𝑇,𝑥(𝑇)] 𝜕𝑇 = 0 (A) [例]LQR 问题 𝑥̇(𝑡) = 𝐴(𝑡)𝑥(𝑡) + 𝐵(𝑡)𝑢(𝑡)，𝑥(𝑡0 ) = 𝑥0 (2.12) 𝐽(𝑥0 ,𝑡0 ) = 1 2 𝑥 𝑇 (𝑇)𝑆𝑇𝑥(𝑇) + 1 2 ∫ [𝑥 𝑇𝑄𝑥 + 𝑢 𝑇𝑅𝑢]d𝑡 𝑇 𝑡0 (2.13)
School ofAutomation,XJTU其中，Sπ≥0,Q≥0,R>0。(对称加权矩阵）[SolutionA] HJB方程H[x(t), u(t),Jx, t] =[xTQx + uT Ru] + J*T[Ax + Bu]=> u*(t) = -R-1BTJ*(2.14)=> -Jt =↓xTQx -J+TBR-1BTJ+ + JTAx(2.15)n边界条件: J*[x(T),T] ==xT(T)Stx(T)。8CAIYUANLI
School of Automation, XJTU © CAI YUANLI 8 其中，𝑆𝑇 ≥ 0,𝑄 ≥ 0, 𝑅 > 0。（对称加权矩阵） [Solution A] HJB 方程 𝐻[𝑥(𝑡), 𝑢(𝑡),𝐽𝑥 ∗ ,𝑡] = 1 2 [𝑥 𝑇𝑄𝑥 + 𝑢 𝑇𝑅𝑢] + 𝐽𝑥 ∗𝑇 [𝐴𝑥 + 𝐵𝑢] => 𝑢 ∗ (𝑡) = −𝑅 −1𝐵 𝑇 𝐽𝑥 ∗ (2.14) => −𝐽𝑡 ∗ = 1 2 𝑥 𝑇𝑄𝑥 − 1 2 𝐽𝑥 ∗𝑇𝐵𝑅 −1𝐵 𝑇 𝐽𝑥 ∗ + 𝐽𝑥 ∗𝑇𝐴𝑥 (2.15) 边界条件：𝐽 ∗ [𝑥(𝑇), 𝑇] = 1 2 𝑥 𝑇 (𝑇)𝑆𝑇𝑥(𝑇)
School ofAutomation,XJTU设 J*[x(t),t) ==xT(t)P(t)x(t) ， P(t) = PT(t) ≥ 0 , 那 么J[x(t),t) = P(t)x(t), Jt[x(t),t) ==xT(t)P(t)x(t), 代入(2.15)式可得:P + PA+ATP- PBR-1BTP +Q = 0(2.16)Riccati矩阵微分方程。（P(T）=Sr)(2.17)u*(t) = -R-1BTP(t)x(t)9CAIYUANLI
School of Automation, XJTU © CAI YUANLI 9 设 𝐽 ∗ [𝑥(𝑡),𝑡] = 1 2 𝑥 𝑇 (𝑡)𝑃(𝑡)𝑥(𝑡) ， 𝑃(𝑡) = 𝑃 𝑇 (𝑡) ≥ 0 ， 那 么 𝐽𝑥 ∗ [𝑥(𝑡),𝑡] = 𝑃(𝑡)𝑥(𝑡)，𝐽𝑡 ∗ [𝑥(𝑡),𝑡] = 1 2 𝑥 𝑇 (𝑡)𝑃̇(𝑡)𝑥(𝑡)，代入(2.15)式可 得： 𝑃̇ + 𝑃𝐴 + 𝐴 𝑇𝑃 − 𝑃𝐵𝑅 −1𝐵 𝑇𝑃 + 𝑄 = 0 (2.16) ——Riccati 矩阵微分方程。（𝑃(𝑇) = 𝑆𝑇） 𝑢 ∗ (𝑡) = −𝑅 −1𝐵 𝑇𝑃(𝑡)𝑥(𝑡) (2.17)
School ofAutomation,XJTUJ*(xo,to) ==xT(to)P(to)x(to)(2.18)[SolutionB]极大值原理H[x(t), u(t), a(t), t] = [xTQx + uT Ru] + ^T[Ax + Bu]u*(t) = -R-1BTa(2.19)(2.20)x(t) = A(t)x(t) - B(t)R-1BTa, x(to) = xo10CAIYUANLI
School of Automation, XJTU © CAI YUANLI 10 𝐽 ∗ (𝑥0 ,𝑡0 ) = 1 2 𝑥 𝑇 (𝑡0 )𝑃(𝑡0 )𝑥(𝑡0 ) (2.18) [Solution B] 极大值原理 𝐻[𝑥(𝑡), 𝑢(𝑡), 𝜆(𝑡),𝑡] = 1 2 [𝑥 𝑇𝑄𝑥 + 𝑢 𝑇𝑅𝑢] + 𝜆 𝑇 [𝐴𝑥 + 𝐵𝑢] => 𝑢 ∗ (𝑡) = −𝑅 −1𝐵 𝑇𝜆 (2.19) 𝑥̇(𝑡) = 𝐴(𝑡)𝑥(𝑡) − 𝐵(𝑡)𝑅 −1𝐵 𝑇𝜆，𝑥(𝑡0 ) = 𝑥0 (2.20)
School of Automation, XJTUi(t) = -ATa(t) - Qx(t), ^(T) = Srx(T)(2.21)设(t) = P(t)x(t), P(t) = pT(t) ≥0, 那么P(T) = ST, 且P + PA+ATP -PBR-1BTP + Q = 0。11CAIYUANLI
School of Automation, XJTU © CAI YUANLI 11 𝜆̇(𝑡) = −𝐴 𝑇𝜆(𝑡) − 𝑄𝑥(𝑡)，𝜆(𝑇) = 𝑆𝑇𝑥(𝑇) (2.21) 设𝜆(𝑡) = 𝑃(𝑡)𝑥(𝑡)，𝑃(𝑡) = 𝑃 𝑇 (𝑡) ≥ 0，那么𝑃(𝑇) = 𝑆𝑇，且 𝑃̇ + 𝑃𝐴 + 𝐴 𝑇𝑃 − 𝑃𝐵𝑅 −1𝐵 𝑇𝑃 + 𝑄 = 0