Convex conjugate

See this article for example.

Let F denote a cumulative distribution function of a random variable X. Then (integrating by parts),

$${\displaystyle f(x):=\int _{-\infty }^{x}F(u)\,du=\operatorname {E} \left[\max(0,x-X)\right]=x-\operatorname {E} \left[\min(x,X)\right]}$$

has the convex conjugate

$${\displaystyle f^{*}(p)=\int _{0}^{p}F^{-1}(q)\,dq=(p-1)F^{-1}(p)+\operatorname {E} \left[\min(F^{-1}(p),X)\right]=pF^{-1}(p)-\operatorname {E} \left[\max(0,F^{-1}(p)-X)\right].}$$

A particular interpretation has the transform

$${\displaystyle f^{\text{inc}}(x):=\arg \sup _{t}t\cdot x-\int _{0}^{1}\max\{t-f(u),0\}\,du,}$$

as this is a nondecreasing rearrangement of the initial function f; in particular, ${\displaystyle f^{\text{inc}}=f}$ for f nondecreasing.

Declare that ${\displaystyle f\leq g}$ if and only if ${\displaystyle f(x)\leq g(x)}$ for all ${\displaystyle x.}$ Then convex-conjugation is order-reversing, which by definition means that if ${\displaystyle f\leq g}$ then ${\displaystyle f^{*}\geq g^{*}.}$

For a family of functions ${\displaystyle \left(f_{\alpha }\right)_{\alpha }}$ it follows from the fact that supremums may be interchanged that

${\displaystyle \left(\inf _{\alpha }f_{\alpha }\right)^{*}(x^{*})=\sup _{\alpha }f_{\alpha }^{*}(x^{*}),}$

and from the max–min inequality that

${\displaystyle \left(\sup _{\alpha }f_{\alpha }\right)^{*}(x^{*})\leq \inf _{\alpha }f_{\alpha }^{*}(x^{*}).}$

The convex conjugate of a function is always lower semi-continuous. The biconjugate ${\displaystyle f^{**}}$ (the convex conjugate of the convex conjugate) is also the closed convex hull, i.e. the largest lower semi-continuous convex function with ${\displaystyle f^{**}\leq f.}$ For proper functions ${\displaystyle f,}$

${\displaystyle f=f^{**}}$ if and only if ${\displaystyle f}$ is convex and lower semi-continuous, by the Fenchel–Moreau theorem.

For any function f and its convex conjugate f *, Fenchel's inequality (also known as the Fenchel–Young inequality) holds for every ${\displaystyle x\in X}$ and ${\displaystyle p\in X^{*}}$:

${\displaystyle \left\langle p,x\right\rangle \leq f(x)+f^{*}(p).}$

Furthermore, the equality holds only when ${\displaystyle p\in \partial f(x)}$. The proof follows from the definition of convex conjugate: ${\displaystyle f^{*}(p)=\sup _{\tilde {x}}\left\{\langle p,{\tilde {x}}\rangle -f({\tilde {x}})\right\}\geq \langle p,x\rangle -f(x).}$

For two functions ${\displaystyle f_{0}}$ and ${\displaystyle f_{1}}$ and a number ${\displaystyle 0\leq \lambda \leq 1}$ the convexity relation

${\displaystyle \left((1-\lambda )f_{0}+\lambda f_{1}\right)^{*}\leq (1-\lambda )f_{0}^{*}+\lambda f_{1}^{*}}$

holds. The ${\displaystyle {*}}$ operation is a convex mapping itself.

The infimal convolution (or epi-sum) of two functions ${\displaystyle f}$ and ${\displaystyle g}$ is defined as

${\displaystyle \left(f\operatorname {\Box } g\right)(x)=\inf \left\{f(x-y)+g(y)\mid y\in \mathbb {R} ^{n}\right\}.}$

Let ${\displaystyle f_{1},\ldots ,f_{m}}$ be proper, convex and lower semicontinuous functions on ${\displaystyle \mathbb {R} ^{n}.}$ Then the infimal convolution is convex and lower semicontinuous (but not necessarily proper),[2] and satisfies

${\displaystyle \left(f_{1}\operatorname {\Box } \cdots \operatorname {\Box } f_{m}\right)^{*}=f_{1}^{*}+\cdots +f_{m}^{*}.}$

The infimal convolution of two functions has a geometric interpretation: The (strict) epigraph of the infimal convolution of two functions is the Minkowski sum of the (strict) epigraphs of those functions.[3]

If the function ${\displaystyle f}$ is differentiable, then its derivative is the maximizing argument in the computation of the convex conjugate:

${\displaystyle f^{\prime }(x)=x^{*}(x):=\arg \sup _{x^{*}}{\langle x,x^{*}\rangle }-f^{*}\left(x^{*}\right)}$ and

${\displaystyle f^{{*}\prime }\left(x^{*}\right)=x\left(x^{*}\right):=\arg \sup _{x}{\langle x,x^{*}\rangle }-f(x);}$

hence

${\displaystyle x=\nabla f^{*}\left(\nabla f(x)\right),}$

${\displaystyle x^{*}=\nabla f\left(\nabla f^{*}\left(x^{*}\right)\right),}$

and moreover

${\displaystyle f^{\prime \prime }(x)\cdot f^{{*}\prime \prime }\left(x^{*}(x)\right)=1,}$

${\displaystyle f^{{*}\prime \prime }\left(x^{*}\right)\cdot f^{\prime \prime }\left(x(x^{*})\right)=1.}$

If for some ${\displaystyle \gamma >0,}$ ${\displaystyle g(x)=\alpha +\beta x+\gamma \cdot f\left(\lambda x+\delta \right)}$, then

${\displaystyle g^{*}\left(x^{*}\right)=-\alpha -\delta {\frac {x^{*}-\beta }{\lambda }}+\gamma \cdot f^{*}\left({\frac {x^{*}-\beta }{\lambda \gamma }}\right).}$

Let ${\displaystyle A:X\to Y}$ be a bounded linear operator. For any convex function ${\displaystyle f}$ on ${\displaystyle X,}$

${\displaystyle \left(Af\right)^{*}=f^{*}A^{*}}$

where

${\displaystyle (Af)(y)=\inf\{f(x):x\in X,Ax=y\}}$

is the preimage of ${\displaystyle f}$ with respect to ${\displaystyle A}$ and ${\displaystyle A^{*}}$ is the adjoint operator of ${\displaystyle A.}$[4]

A closed convex function ${\displaystyle f}$ is symmetric with respect to a given set ${\displaystyle G}$ of orthogonal linear transformations,

${\displaystyle f(Ax)=f(x)}$ for all ${\displaystyle x}$ and all ${\displaystyle A\in G}$

if and only if its convex conjugate ${\displaystyle f^{*}}$ is symmetric with respect to ${\displaystyle G.}$