Functional derivative

Suppose ${\displaystyle B}$ is a Banach space and ${\displaystyle F}$ is a functional defined on ${\displaystyle B}$. The differential of ${\displaystyle F}$ at a point ${\displaystyle \rho \in B}$ is the linear functional ${\displaystyle \delta F[\rho ,\cdot ]}$ on ${\displaystyle B}$ defined[2] by the condition that, for all ${\displaystyle \phi \in B}$,

$${\displaystyle F[\rho +\phi ]-F[\rho ]=\delta F[\rho$$ ;\phi ]+\epsilon \cdot \|\phi \|}

where ${\displaystyle \epsilon }$ is a real number that depends on ${\displaystyle \|\phi \|}$ in such a way that ${\displaystyle \epsilon \to 0}$ as ${\displaystyle \|\phi \|\to 0}$. This means that ${\displaystyle \delta F[\rho ,\cdot ]}$ is the Fréchet derivative of ${\displaystyle F}$ at ${\displaystyle \rho }$.

However, this notion of functional differential is so strong it may not exist,[3] and in those cases a weaker notion, like the Gateaux derivative is preferred. In many practical cases, the functional differential is defined[4] as the directional derivative

$${\displaystyle {\begin{aligned}\delta F[\rho ,\phi ]&=\lim _{\varepsilon \to 0}{\frac {F[\rho +\varepsilon \phi ]-F[\rho ]}{\varepsilon }}\\&=\left[{\frac {d}{d\varepsilon }}F[\rho +\varepsilon \phi ]\right]_{\varepsilon =0}.\end{aligned}}}$$

Note that this notion of the functional differential can even be defined without a norm.

In many applications, the domain of the functional ${\displaystyle F}$ is a space of differentiable functions ${\displaystyle \rho }$ defined on some space ${\displaystyle \Omega }$ and ${\displaystyle F}$ is of the form

$${\displaystyle F[\rho ]=\int _{\Omega }L(x,\rho (x),D\rho (x))\,dx}$$

for some function ${\displaystyle L(x,\rho (x),D\rho (x))}$ that may depend on ${\displaystyle x}$, the value ${\displaystyle \rho (x)}$ and the derivative ${\displaystyle D\rho (x)}$. If this is the case and, moreover, ${\displaystyle \delta F[\rho ,\phi ]}$ can be written as the integral of ${\displaystyle \phi }$ times another function (denoted δF/δρ)

$${\displaystyle \delta F[\rho ,\phi ]=\int _{\Omega }{\frac {\delta F}{\delta \rho }}(x)\ \phi (x)\ dx}$$

then this function δF/δρ is called the functional derivative of F at ρ.[5][6] If ${\displaystyle F}$ is restricted to only certain functions ${\displaystyle \rho }$ (for example, if there are some boundary conditions imposed) then ${\displaystyle \phi }$ is restricted to functions such that ${\displaystyle \rho +\epsilon \phi }$ continues to satisfy these conditions.

Heuristically, ${\displaystyle \phi }$ is the change in ${\displaystyle \rho }$, so we 'formally' have ${\displaystyle \phi =\delta \rho }$, and then this is similar in form to the total differential of a function ${\displaystyle F(\rho _{1},\rho _{2},\dots ,\rho _{n})}$,

$${\displaystyle dF=\sum _{i=1}^{n}{\frac {\partial F}{\partial \rho _{i}}}\ d\rho _{i},}$$

where ${\displaystyle \rho _{1},\rho _{2},\dots ,\rho _{n}}$ are independent variables. Comparing the last two equations, the functional derivative ${\displaystyle \delta F/\delta \rho (x)}$ has a role similar to that of the partial derivative ${\displaystyle \partial F/\partial \rho _{i}}$, where the variable of integration ${\displaystyle x}$ is like a continuous version of the summation index ${\displaystyle i}$.[7] One thinks of δF/δρ as the gradient of F at the point ρ, so the value δF/δρ(x) measures how much the functional F will change if the function ρ is changed at the point x. Hence the formula

$${\displaystyle \int {\frac {\delta F}{\delta \rho }}(x)\phi (x)\;dx}$$

is regarded as the directional derivative at point ρ in the direction of ϕ. This is analogous to vector calculus, where the inner product of a vector ${\displaystyle v}$ with the gradient gives the directional derivative in the direction of ${\displaystyle v}$.

Given a functional

$${\displaystyle F[\rho ]=\int f({\boldsymbol {r}},\rho ({\boldsymbol {r}}),\nabla \rho ({\boldsymbol {r}}))\,d{\boldsymbol {r}},}$$

and a function ϕ(r) that vanishes on the boundary of the region of integration, from a previous section Definition,

$${\displaystyle {\begin{aligned}\int {\frac {\delta F}{\delta \rho ({\boldsymbol {r}})}}\,\phi ({\boldsymbol {r}})\,d{\boldsymbol {r}}&=\left[{\frac {d}{d\varepsilon }}\int f({\boldsymbol {r}},\rho +\varepsilon \phi ,\nabla \rho +\varepsilon \nabla \phi )\,d{\boldsymbol {r}}\right]_{\varepsilon =0}\\&=\int \left({\frac {\partial f}{\partial \rho }}\,\phi +{\frac {\partial f}{\partial \nabla \rho }}\cdot \nabla \phi \right)d{\boldsymbol {r}}\\&=\int \left[{\frac {\partial f}{\partial \rho }}\,\phi +\nabla \cdot \left({\frac {\partial f}{\partial \nabla \rho }}\,\phi \right)-\left(\nabla \cdot {\frac {\partial f}{\partial \nabla \rho }}\right)\phi \right]d{\boldsymbol {r}}\\&=\int \left[{\frac {\partial f}{\partial \rho }}\,\phi -\left(\nabla \cdot {\frac {\partial f}{\partial \nabla \rho }}\right)\phi \right]d{\boldsymbol {r}}\\&=\int \left({\frac {\partial f}{\partial \rho }}-\nabla \cdot {\frac {\partial f}{\partial \nabla \rho }}\right)\phi ({\boldsymbol {r}})\ d{\boldsymbol {r}}\,.\end{aligned}}}$$

The second line is obtained using the total derivative, where ∂f /∂∇'ρ is a derivative of a scalar with respect to a vector.[Note 4]

The third line was obtained by use of a product rule for divergence. The fourth line was obtained using the divergence theorem and the condition that ϕ = 0 on the boundary of the region of integration. Since ϕ is also an arbitrary function, applying the fundamental lemma of calculus of variations to the last line, the functional derivative is

$${\displaystyle {\frac {\delta F}{\delta \rho ({\boldsymbol {r}})}}={\frac {\partial f}{\partial \rho }}-\nabla \cdot {\frac {\partial f}{\partial \nabla \rho }}}$$

where ρ = ρ(r) and f = f (r, ρ, ∇ρ). This formula is for the case of the functional form given by F[ρ] at the beginning of this section. For other functional forms, the definition of the functional derivative can be used as the starting point for its determination. (See the example Coulomb potential energy functional.)

The above equation for the functional derivative can be generalized to the case that includes higher dimensions and higher order derivatives. The functional would be,

$${\displaystyle F[\rho ({\boldsymbol {r}})]=\int f({\boldsymbol {r}},\rho ({\boldsymbol {r}}),\nabla \rho ({\boldsymbol {r}}),\nabla ^{(2)}\rho ({\boldsymbol {r}}),\dots ,\nabla ^{(N)}\rho ({\boldsymbol {r}}))\,d{\boldsymbol {r}},}$$

where the vector r ∈ Rⁿ, and ∇⁽ⁱ⁾ is a tensor whose nⁱ components are partial derivative operators of order i,

$${\displaystyle \left[\nabla ^{(i)}\right]_{\alpha _{1}\alpha _{2}\cdots \alpha _{i}}={\frac {\partial ^{\,i}}{\partial r_{\alpha _{1}}\partial r_{\alpha _{2}}\cdots \partial r_{\alpha _{i}}}}\qquad \qquad {\text{where}}\quad \alpha _{1},\alpha _{2},\cdots ,\alpha _{i}=1,2,\cdots ,n\ .}$$

[Note 5]

An analogous application of the definition of the functional derivative yields

$${\displaystyle {\begin{aligned}{\frac {\delta F[\rho ]}{\delta \rho }}&{}={\frac {\partial f}{\partial \rho }}-\nabla \cdot {\frac {\partial f}{\partial (\nabla \rho )}}+\nabla ^{(2)}\cdot {\frac {\partial f}{\partial \left(\nabla ^{(2)}\rho \right)}}+\dots +(-1)^{N}\nabla ^{(N)}\cdot {\frac {\partial f}{\partial \left(\nabla ^{(N)}\rho \right)}}\\&{}={\frac {\partial f}{\partial \rho }}+\sum _{i=1}^{N}(-1)^{i}\nabla ^{(i)}\cdot {\frac {\partial f}{\partial \left(\nabla ^{(i)}\rho \right)}}\ .\end{aligned}}}$$

In the last two equations, the nⁱ components of the tensor ${\displaystyle {\frac {\partial f}{\partial \left(\nabla ^{(i)}\rho \right)}}}$ are partial derivatives of f with respect to partial derivatives of ρ,

$${\displaystyle \left[{\frac {\partial f}{\partial \left(\nabla ^{(i)}\rho \right)}}\right]_{\alpha _{1}\alpha _{2}\cdots \alpha _{i}}={\frac {\partial f}{\partial \rho _{\alpha _{1}\alpha _{2}\cdots \alpha _{i}}}}\qquad \qquad {\text{where}}\quad \rho _{\alpha _{1}\alpha _{2}\cdots \alpha _{i}}\equiv {\frac {\partial ^{\,i}\rho }{\partial r_{\alpha _{1}}\,\partial r_{\alpha _{2}}\cdots \partial r_{\alpha _{i}}}}\ ,}$$

and the tensor scalar product is,

$${\displaystyle \nabla ^{(i)}\cdot {\frac {\partial f}{\partial \left(\nabla ^{(i)}\rho \right)}}=\sum _{\alpha _{1},\alpha _{2},\cdots ,\alpha _{i}=1}^{n}\ {\frac {\partial ^{\,i}}{\partial r_{\alpha _{1}}\,\partial r_{\alpha _{2}}\cdots \partial r_{\alpha _{i}}}}\ {\frac {\partial f}{\partial \rho _{\alpha _{1}\alpha _{2}\cdots \alpha _{i}}}}\ .}$$

[Note 6]

The Thomas–Fermi model of 1927 used a kinetic energy functional for a noninteracting uniform electron gas in a first attempt of density-functional theory of electronic structure:

$${\displaystyle T_{\mathrm {TF} }[\rho ]=C_{\mathrm {F} }\int \rho ^{5/3}(\mathbf {r} )\,d\mathbf {r} \,.}$$

Since the integrand of T_TF[ρ] does not involve derivatives of ρ(r), the functional derivative of T_TF[ρ] is,[12]

$${\displaystyle {\begin{aligned}{\frac {\delta T_{\mathrm {TF} }}{\delta \rho ({\boldsymbol {r}})}}&=C_{\mathrm {F} }{\frac {\partial \rho ^{5/3}(\mathbf {r} )}{\partial \rho (\mathbf {r} )}}\\&={\frac {5}{3}}C_{\mathrm {F} }\rho ^{2/3}(\mathbf {r} )\,.\end{aligned}}}$$

For the electron-nucleus potential, Thomas and Fermi employed the Coulomb potential energy functional

$${\displaystyle V[\rho ]=\int {\frac {\rho ({\boldsymbol {r}})}{|{\boldsymbol {r}}|}}\ d{\boldsymbol {r}}.}$$

Applying the definition of functional derivative,

$${\displaystyle {\begin{aligned}\int {\frac {\delta V}{\delta \rho ({\boldsymbol {r}})}}\ \phi ({\boldsymbol {r}})\ d{\boldsymbol {r}}&{}=\left[{\frac {d}{d\varepsilon }}\int {\frac {\rho ({\boldsymbol {r}})+\varepsilon \phi ({\boldsymbol {r}})}{|{\boldsymbol {r}}|}}\ d{\boldsymbol {r}}\right]_{\varepsilon =0}\\&{}=\int {\frac {1}{|{\boldsymbol {r}}|}}\,\phi ({\boldsymbol {r}})\ d{\boldsymbol {r}}\,.\end{aligned}}}$$

So,

$${\displaystyle {\frac {\delta V}{\delta \rho ({\boldsymbol {r}})}}={\frac {1}{|{\boldsymbol {r}}|}}\ .}$$

For the classical part of the electron-electron interaction, Thomas and Fermi employed the Coulomb potential energy functional

$${\displaystyle J[\rho ]={\frac {1}{2}}\iint {\frac {\rho (\mathbf {r} )\rho (\mathbf {r} ')}{|\mathbf {r} -\mathbf {r} '|}}\,d\mathbf {r} d\mathbf {r} '\,.}$$

From the definition of the functional derivative,

$${\displaystyle {\begin{aligned}\int {\frac {\delta J}{\delta \rho ({\boldsymbol {r}})}}\phi ({\boldsymbol {r}})d{\boldsymbol {r}}&{}=\left[{\frac {d\ }{d\epsilon }}\,J[\rho +\epsilon \phi ]\right]_{\epsilon =0}\\&{}=\left[{\frac {d\ }{d\epsilon }}\,\left({\frac {1}{2}}\iint {\frac {[\rho ({\boldsymbol {r}})+\epsilon \phi ({\boldsymbol {r}})]\,[\rho ({\boldsymbol {r}}')+\epsilon \phi ({\boldsymbol {r}}')]}{|{\boldsymbol {r}}-{\boldsymbol {r}}'|}}\,d{\boldsymbol {r}}d{\boldsymbol {r}}'\right)\right]_{\epsilon =0}\\&{}={\frac {1}{2}}\iint {\frac {\rho ({\boldsymbol {r}}')\phi ({\boldsymbol {r}})}{|{\boldsymbol {r}}-{\boldsymbol {r}}'|}}\,d{\boldsymbol {r}}d{\boldsymbol {r}}'+{\frac {1}{2}}\iint {\frac {\rho ({\boldsymbol {r}})\phi ({\boldsymbol {r}}')}{|{\boldsymbol {r}}-{\boldsymbol {r}}'|}}\,d{\boldsymbol {r}}d{\boldsymbol {r}}'\\\end{aligned}}}$$

The first and second terms on the right hand side of the last equation are equal, since r and r′ in the second term can be interchanged without changing the value of the integral. Therefore,

$${\displaystyle \int {\frac {\delta J}{\delta \rho ({\boldsymbol {r}})}}\phi ({\boldsymbol {r}})d{\boldsymbol {r}}=\int \left(\int {\frac {\rho ({\boldsymbol {r}}')}{|{\boldsymbol {r}}-{\boldsymbol {r}}'|}}d{\boldsymbol {r}}'\right)\phi ({\boldsymbol {r}})d{\boldsymbol {r}}}$$

and the functional derivative of the electron-electron coulomb potential energy functional J[ρ] is,[13]

$${\displaystyle {\frac {\delta J}{\delta \rho ({\boldsymbol {r}})}}=\int {\frac {\rho ({\boldsymbol {r}}')}{|{\boldsymbol {r}}-{\boldsymbol {r}}'|}}d{\boldsymbol {r}}'\,.}$$

The second functional derivative is

$${\displaystyle {\frac {\delta ^{2}J[\rho ]}{\delta \rho (\mathbf {r} ')\delta \rho (\mathbf {r} )}}={\frac {\partial }{\partial \rho (\mathbf {r} ')}}\left({\frac {\rho (\mathbf {r} ')}{|\mathbf {r} -\mathbf {r} '|}}\right)={\frac {1}{|\mathbf {r} -\mathbf {r} '|}}.}$$

In 1935 von Weizsäcker proposed to add a gradient correction to the Thomas-Fermi kinetic energy functional to make it better suit a molecular electron cloud:

$${\displaystyle T_{\mathrm {W} }[\rho ]={\frac {1}{8}}\int {\frac {\nabla \rho (\mathbf {r} )\cdot \nabla \rho (\mathbf {r} )}{\rho (\mathbf {r} )}}d\mathbf {r} =\int t_{\mathrm {W} }\ d\mathbf {r} \,,}$$

where

$${\displaystyle t_{\mathrm {W} }\equiv {\frac {1}{8}}{\frac {\nabla \rho \cdot \nabla \rho }{\rho }}\qquad {\text{and}}\ \ \rho =\rho ({\boldsymbol {r}})\ .}$$

Using a previously derived formula for the functional derivative,

$${\displaystyle {\begin{aligned}{\frac {\delta T_{\mathrm {W} }}{\delta \rho ({\boldsymbol {r}})}}&={\frac {\partial t_{\mathrm {W} }}{\partial \rho }}-\nabla \cdot {\frac {\partial t_{\mathrm {W} }}{\partial \nabla \rho }}\\&=-{\frac {1}{8}}{\frac {\nabla \rho \cdot \nabla \rho }{\rho ^{2}}}-\left({\frac {1}{4}}{\frac {\nabla ^{2}\rho }{\rho }}-{\frac {1}{4}}{\frac {\nabla \rho \cdot \nabla \rho }{\rho ^{2}}}\right)\qquad {\text{where}}\ \ \nabla ^{2}=\nabla \cdot \nabla \ ,\end{aligned}}}$$

and the result is,[14]

$${\displaystyle {\frac {\delta T_{\mathrm {W} }}{\delta \rho ({\boldsymbol {r}})}}=\ \ \,{\frac {1}{8}}{\frac {\nabla \rho \cdot \nabla \rho }{\rho ^{2}}}-{\frac {1}{4}}{\frac {\nabla ^{2}\rho }{\rho }}\ .}$$

The entropy of a discrete random variable is a functional of the probability mass function.

$${\displaystyle H[p(x)]=-\sum _{x}p(x)\log p(x)}$$

Thus,

$${\displaystyle {\begin{aligned}\sum _{x}{\frac {\delta H}{\delta p(x)}}\,\phi (x)&{}=\left[{\frac {d}{d\epsilon }}H[p(x)+\epsilon \phi (x)]\right]_{\epsilon =0}\\&{}=\left[-\,{\frac {d}{d\varepsilon }}\sum _{x}\,[p(x)+\varepsilon \phi (x)]\ \log[p(x)+\varepsilon \phi (x)]\right]_{\varepsilon =0}\\&{}=-\sum _{x}\,[1+\log p(x)]\ \phi (x)\,.\end{aligned}}}$$

Thus,

$${\displaystyle {\frac {\delta H}{\delta p(x)}}=-1-\log p(x).}$$

Let

$${\displaystyle F[\varphi (x)]=e^{\int \varphi (x)g(x)dx}.}$$

Using the delta function as a test function,

$${\displaystyle {\begin{aligned}{\frac {\delta F[\varphi (x)]}{\delta \varphi (y)}}&{}=\lim _{\varepsilon \to 0}{\frac {F[\varphi (x)+\varepsilon \delta (x-y)]-F[\varphi (x)]}{\varepsilon }}\\&{}=\lim _{\varepsilon \to 0}{\frac {e^{\int (\varphi (x)+\varepsilon \delta (x-y))g(x)dx}-e^{\int \varphi (x)g(x)dx}}{\varepsilon }}\\&{}=e^{\int \varphi (x)g(x)dx}\lim _{\varepsilon \to 0}{\frac {e^{\varepsilon \int \delta (x-y)g(x)dx}-1}{\varepsilon }}\\&{}=e^{\int \varphi (x)g(x)dx}\lim _{\varepsilon \to 0}{\frac {e^{\varepsilon g(y)}-1}{\varepsilon }}\\&{}=e^{\int \varphi (x)g(x)dx}g(y).\end{aligned}}}$$

Thus,

$${\displaystyle {\frac {\delta F[\varphi (x)]}{\delta \varphi (y)}}=g(y)F[\varphi (x)].}$$

This is particularly useful in calculating the correlation functions from the partition function in quantum field theory.

A function can be written in the form of an integral like a functional. For example,

$${\displaystyle \rho ({\boldsymbol {r}})=F[\rho ]=\int \rho ({\boldsymbol {r}}')\delta ({\boldsymbol {r}}-{\boldsymbol {r}}')\,d{\boldsymbol {r}}'.}$$

Since the integrand does not depend on derivatives of ρ, the functional derivative of ρ(r) is,

$${\displaystyle {\begin{aligned}{\frac {\delta \rho ({\boldsymbol {r}})}{\delta \rho ({\boldsymbol {r}}')}}\equiv {\frac {\delta F}{\delta \rho ({\boldsymbol {r}}')}}&={\frac {\partial \ \ }{\partial \rho ({\boldsymbol {r}}')}}\,[\rho ({\boldsymbol {r}}')\delta ({\boldsymbol {r}}-{\boldsymbol {r}}')]\\&=\delta ({\boldsymbol {r}}-{\boldsymbol {r}}').\end{aligned}}}$$

The functional derivative of the iterated function ${\displaystyle f(f(x))}$ is given by:

$${\displaystyle {\frac {\delta f(f(x))}{\delta f(y)}}=f'(f(x))\delta (x-y)+\delta (f(x)-y)}$$

and

$${\displaystyle {\frac {\delta f(f(f(x)))}{\delta f(y)}}=f'(f(f(x))(f'(f(x))\delta (x-y)+\delta (f(x)-y))+\delta (f(f(x))-y)}$$

In general:

$${\displaystyle {\frac {\delta f^{N}(x)}{\delta f(y)}}=f'(f^{N-1}(x)){\frac {\delta f^{N-1}(x)}{\delta f(y)}}+\delta (f^{N-1}(x)-y)}$$

Putting in N = 0 gives:

$${\displaystyle {\frac {\delta f^{-1}(x)}{\delta f(y)}}=-{\frac {\delta (f^{-1}(x)-y)}{f'(f^{-1}(x))}}}$$