Pontryagin's maximum principle

For set ${\displaystyle {\mathcal {U}}}$ and functions

${\displaystyle \Psi$ :\mathbb {R} ^{n}\to \mathbb {R} } ,

${\displaystyle H:\mathbb {R} ^{n}\times {\mathcal {U}}\times \mathbb {R} ^{n}\times \mathbb {R} \to \mathbb {R} }$,

${\displaystyle L:\mathbb {R} ^{n}\times {\mathcal {U}}\to \mathbb {R} }$,

${\displaystyle f:\mathbb {R} ^{n}\times {\mathcal {U}}\to \mathbb {R} ^{n}}$,

we use the following notation:

${\displaystyle \Psi _{T}(x(T))=\left.{\frac {\partial \Psi (x)}{\partial T}}\right|_{x=x(T)}\,}$,

${\displaystyle \Psi _{x}(x(T))={\begin{bmatrix}\left.{\frac {\partial \Psi (x)}{\partial x_{1}}}\right|_{x=x(T)}&\cdots &\left.{\frac {\partial \Psi (x)}{\partial x_{n}}}\right|_{x=x(T)}\end{bmatrix}}}$,

${\displaystyle H_{x}(x^{*},u^{*},\lambda ^{*},t)={\begin{bmatrix}\left.{\frac {\partial H}{\partial x_{1}}}\right|_{x=x^{*},u=u^{*},\lambda =\lambda ^{*}}&\cdots &\left.{\frac {\partial H}{\partial x_{n}}}\right|_{x=x^{*},u=u^{*},\lambda =\lambda ^{*}}\end{bmatrix}}}$,

${\displaystyle L_{x}(x^{*},u^{*})={\begin{bmatrix}\left.{\frac {\partial L}{\partial x_{1}}}\right|_{x=x^{*},u=u^{*}}&\cdots &\left.{\frac {\partial L}{\partial x_{n}}}\right|_{x=x^{*},u=u^{*}}\end{bmatrix}}}$,

${\displaystyle f_{x}(x^{*},u^{*})={\begin{bmatrix}\left.{\frac {\partial f_{1}}{\partial x_{1}}}\right|_{x=x^{*},u=u^{*}}&\cdots &\left.{\frac {\partial f_{1}}{\partial x_{n}}}\right|_{x=x^{*},u=u^{*}}\\\vdots &\ddots &\vdots \\\left.{\frac {\partial f_{n}}{\partial x_{1}}}\right|_{x=x^{*},u=u^{*}}&\ldots &\left.{\frac {\partial f_{n}}{\partial x_{n}}}\right|_{x=x^{*},u=u^{*}}\end{bmatrix}}}$.

Here the necessary conditions are shown for minimization of a functional.

Consider an n-dimensional dynamical system, with state variable ${\displaystyle x\in \mathbb {R} ^{n}}$, and control variable ${\displaystyle u\in {\mathcal {U}}}$, where ${\displaystyle {\mathcal {U}}}$ is the set of admissible controls. The evolution of the system is determined by the state and the control, according to the differential equation ${\displaystyle {\dot {x}}=f(x,u)}$. Let the system's initial state be ${\displaystyle x_{0}}$ and let the system's evolution be controlled over the time-period with values ${\displaystyle t\in [0,T]}$. The latter is determined by the following differential equation:

${\displaystyle {\dot {x}}=f(x,u),\quad x(0)=x_{0},\quad u(t)\in {\mathcal {U}},\quad t\in [0,T]}$

The control trajectory ${\displaystyle u:[0,T]\to {\mathcal {U}}}$ is to be chosen according to an objective. The objective is a functional ${\displaystyle J}$ defined by

${\displaystyle J=\Psi (x(T))+\int _{0}^{T}L{\big (}x(t),u(t){\big )}\,dt}$,

where ${\displaystyle L(x,u)}$ can be interpreted as the rate of cost for exerting control ${\displaystyle u}$ in state ${\displaystyle x}$, and ${\displaystyle \Psi (x)}$ can be interpreted as the cost for ending up at state ${\displaystyle x}$. The specific choice of ${\displaystyle L,\Psi }$ depends on the application.

The constraints on the system dynamics can be adjoined to the Lagrangian ${\displaystyle L}$ by introducing time-varying Lagrange multiplier vector ${\displaystyle \lambda }$, whose elements are called the costates of the system. This motivates the construction of the Hamiltonian ${\displaystyle H}$ defined for all ${\displaystyle t\in [0,T]}$ by:

${\displaystyle H{\big (}x(t),u(t),\lambda (t),t{\big )}=\lambda ^{\rm {T}}(t)\cdot f{\big (}x(t),u(t){\big )}+L{\big (}x(t),u(t){\big )}}$

where ${\displaystyle \lambda ^{\rm {T}}}$ is the transpose of ${\displaystyle \lambda }$.

Pontryagin's minimum principle states that the optimal state trajectory ${\displaystyle x^{*}}$, optimal control ${\displaystyle u^{*}}$, and corresponding Lagrange multiplier vector ${\displaystyle \lambda ^{*}}$ must minimize the Hamiltonian ${\displaystyle H}$ so that

${\displaystyle H{\big (}x^{*}(t),u^{*}(t),\lambda ^{*}(t),t{\big )}\leq H{\big (}x(t),u,\lambda (t),t{\big )}}$

(1)

for all time ${\displaystyle t\in [0,T]}$ and for all permissible control inputs ${\displaystyle u\in {\mathcal {U}}}$. Here, the trajectory of the Lagrangian multiplier vector ${\displaystyle \lambda }$ is the solution to the costate equation and its terminal conditions:

${\displaystyle -{\dot {\lambda }}^{\rm {T}}(t)=H_{x}{\big (}x^{*}(t),u^{*}(t),\lambda (t),t{\big )}=\lambda ^{\rm {T}}(t)\cdot f_{x}{\big (}x^{*}(t),u^{*}(t){\big )}+L_{x}{\big (}x^{*}(t),u^{*}(t){\big )}}$

(2)

${\displaystyle \lambda ^{\rm {T}}(T)=\Psi _{x}(x(T))}$

(3)

If ${\displaystyle x(T)}$ is fixed, then these three conditions in (1)-(3) are the necessary conditions for an optimal control.

If the final state ${\displaystyle x(T)}$ is not fixed (i.e., its differential variation is not zero), there is an additional condition

${\displaystyle \Psi _{T}(x(T))+H(T)=0}$

(4)

These four conditions in (1)-(4) are the necessary conditions for an optimal control.

• Lagrange multipliers on Banach spaces, Lagrangian method in calculus of variations

• Geering, H. P. (2007). Optimal Control with Engineering Applications. Springer. ISBN 978-3-540-69437-3.
• Kirk, D. E. (1970). Optimal Control Theory: An Introduction. Prentice Hall. ISBN 0-486-43484-2.
• Lee, E. B.; Markus, L. (1967). Foundations of Optimal Control Theory. New York: Wiley.
• Seierstad, Atle; Sydsæter, Knut (1987). Optimal Control Theory with Economic Applications. Amsterdam: North-Holland. ISBN 0-444-87923-4.

• "Pontryagin maximum principle", Encyclopedia of Mathematics, EMS Press, 2001 [1994]