Linear–quadratic regulator

For a continuous-time linear system, defined on ${\displaystyle t\in [t_{0},t_{1}]}$, described by:

${\displaystyle {\dot {x}}=Ax+Bu}$

where ${\displaystyle x\in \mathbb {R} ^{n}}$ (that is, ${\displaystyle x}$ is an ${\displaystyle n}$-dimensional real-valued vector) is the state of the system and ${\displaystyle u\in \mathbb {R} ^{m}}$ is the control input. Given a quadratic cost function for the system, defined as:

${\displaystyle J=x^{T}(t_{1})F(t_{1})x(t_{1})+\int \limits _{t_{0}}^{t_{1}}\left(x^{T}Qx+u^{T}Ru+2x^{T}Nu\right)dt}$

the feedback control law that minimizes the value of the cost is:

${\displaystyle u=-Kx\,}$

where ${\displaystyle K}$ is given by:

${\displaystyle K=R^{-1}(B^{T}P(t)+N^{T})\,}$

and ${\displaystyle P}$ is found by solving the continuous time Riccati differential equation:

${\displaystyle A^{T}P(t)+P(t)A-(P(t)B+N)R^{-1}(B^{T}P(t)+N^{T})+Q=-{\dot {P}}(t)\,}$

with the boundary condition:

${\displaystyle P(t_{1})=F(t_{1}).}$

The first order conditions for J_min are:

1) State equation

${\displaystyle {\dot {x}}=Ax+Bu}$

2) Co-state equation

${\displaystyle -{\dot {\lambda }}=Qx+Nu+A^{T}\lambda }$

3) Stationary equation

${\displaystyle 0=Ru+N^{T}x+B^{T}\lambda }$

4) Boundary conditions

${\displaystyle x(t_{0})=x_{0}}$

and ${\displaystyle \lambda (t_{1})=F(t_{1})x(t_{1})}$

For a continuous-time linear system described by:

${\displaystyle {\dot {x}}=Ax+Bu}$

with a cost function defined as:

${\displaystyle J=\int _{0}^{\infty }\left(x^{T}Qx+u^{T}Ru+2x^{T}Nu\right)dt}$

the feedback control law that minimizes the value of the cost is:

${\displaystyle u=-Kx\,}$

where ${\displaystyle K}$ is given by:

${\displaystyle K=R^{-1}(B^{T}P+N^{T})\,}$

and ${\displaystyle P}$ is found by solving the continuous time algebraic Riccati equation:

${\displaystyle A^{T}P+PA-(PB+N)R^{-1}(B^{T}P+N^{T})+Q=0\,}$

This can be also written as:

${\displaystyle {\mathcal {A}}^{T}P+P{\mathcal {A}}-PBR^{-1}B^{T}P+{\mathcal {Q}}=0\,}$

with

${\displaystyle {\mathcal {A}}=A-BR^{-1}N^{T}\qquad {\mathcal {Q}}=Q-NR^{-1}N^{T}\,}$

For a discrete-time linear system described by: [2]

${\displaystyle x_{k+1}=Ax_{k}+Bu_{k}\,}$

with a performance index defined as:

${\displaystyle J=x_{H_{p}}^{T}Q_{H_{p}}x_{H_{p}}+\sum \limits _{k=0}^{H_{p}-1}\left(x_{k}^{T}Qx_{k}+u_{k}^{T}Ru_{k}+2x_{k}^{T}Nu_{k}\right)}$, where ${\displaystyle H_{p}}$ is the time horizon

the optimal control sequence minimizing the performance index is given by:

${\displaystyle u_{k}=-F_{k}x_{k}\,}$

where:

${\displaystyle F_{k}=(R+B^{T}P_{k+1}B)^{-1}(B^{T}P_{k+1}A+N^{T})\,}$

and ${\displaystyle P_{k}}$ is found iteratively backwards in time by the dynamic Riccati equation:

${\displaystyle P_{k-1}=A^{T}P_{k}A-(A^{T}P_{k}B+N)\left(R+B^{T}P_{k}B\right)^{-1}(B^{T}P_{k}A+N^{T})+Q}$

from terminal condition ${\displaystyle P_{H_{p}}=Q_{H_{p}}}$.[3] Note that ${\displaystyle u_{H_{p}}}$ is not defined, since ${\displaystyle x}$ is driven to its final state ${\displaystyle x_{H_{p}}}$ by ${\displaystyle Ax_{H_{p}-1}+Bu_{H_{p}-1}}$.

For a discrete-time linear system described by:

${\displaystyle x_{k+1}=Ax_{k}+Bu_{k}\,}$

with a performance index defined as:

${\displaystyle J=\sum \limits _{k=0}^{\infty }\left(x_{k}^{T}Qx_{k}+u_{k}^{T}Ru_{k}+2x_{k}^{T}Nu_{k}\right)}$

the optimal control sequence minimizing the performance index is given by:

${\displaystyle u_{k}=-Fx_{k}\,}$

where:

${\displaystyle F=(R+B^{T}PB)^{-1}(B^{T}PA+N^{T})\,}$

and ${\displaystyle P}$ is the unique positive definite solution to the discrete time algebraic Riccati equation (DARE):

${\displaystyle P=A^{T}PA-(A^{T}PB+N)\left(R+B^{T}PB\right)^{-1}(B^{T}PA+N^{T})+Q}$.

This can be also written as:

${\displaystyle P={\mathcal {A}}^{T}P{\mathcal {A}}-{\mathcal {A}}^{T}PB\left(R+B^{T}PB\right)^{-1}B^{T}P{\mathcal {A}}+{\mathcal {Q}}}$

with:

${\displaystyle {\mathcal {A}}=A-BR^{-1}N^{T}\qquad {\mathcal {Q}}=Q-NR^{-1}N^{T}}$.

Note that one way to solve the algebraic Riccati equation is by iterating the dynamic Riccati equation of the finite-horizon case until it converges.

If the state equation is quadratic then the problem is known as the quadratic-quadratic regulator (QQR). The Al'Brekht algorithm can be applied to reduce this problem to one that can be solved efficiently using tensor based linear solvers.[6]

If the state equation is polynomial then the problem is known as the polynomial-quadratic regulator (PQR). Again, the Al'Brekht algorithm can be applied to reduce this problem to a large linear one which can be solved with a generalization of the Bartels-Stewart algorithm; this is feasible provided that the degree of the polynomial is not too high.[7]

Model predictive control and linear-quadratic regulators are two types of optimal control methods that have distinct approaches for setting the optimization costs. In particular, when the LQR is run repeatedly with a receding horizon, it becomes a form of model predictive control (MPC). In general, however, MPC does not rely on any assumptions regarding linearity of the system.