Hilbert projection theorem

Proof that a minimum point ${\displaystyle y}$ exists

Let ${\displaystyle \delta$ :=\inf _{c\in C}\|x-c\|} be the distance between ${\displaystyle x}$ and ${\displaystyle C,}$ ${\displaystyle \left(c_{n}\right)_{n=1}^{\infty }}$ a sequence in ${\displaystyle C}$ such that the distance squared between ${\displaystyle x}$ and ${\displaystyle c_{n}}$ is less than or equal to ${\displaystyle \delta ^{2}+1/n.}$ Let ${\displaystyle n}$ and ${\displaystyle m}$ be two integers, then the following equalities are true:

and

Therefore

(This equation is the same as the formula ${\displaystyle a^{2}=2b^{2}+2c^{2}-4M_{a}^{2}}$ for the length ${\displaystyle M_{a}}$ of a median in a triangle with sides of length ${\displaystyle a,b,}$ and ${\displaystyle c,}$ where specifically, the triangle's vertices are ${\displaystyle x,c_{m},c_{n}}$).

By giving an upper bound to the first two terms of the equality and by noticing that the middle of ${\displaystyle c_{n}}$ and ${\displaystyle c_{m}}$ belong to ${\displaystyle C}$ and has therefore a distance greater than or equal to ${\displaystyle \delta }$ from ${\displaystyle x,}$ it follows that:

The last inequality proves that ${\displaystyle \left(c_{n}\right)_{n=1}^{\infty }}$ is a Cauchy sequence. Since ${\displaystyle C}$ is complete, the sequence is therefore convergent to a point ${\displaystyle m\in C,}$ whose distance from ${\displaystyle x}$ is minimal. ${\displaystyle \blacksquare }$

Proof that ${\displaystyle m}$ is unique

Let ${\displaystyle m_{1}}$ and ${\displaystyle m_{2}}$ be two minimum points. Then:

Since ${\displaystyle {\frac {m_{1}+m_{2}}{2}}}$ belongs to ${\displaystyle C,}$ we have ${\displaystyle \left\|{\frac {m_{1}+m_{2}}{2}}-x\right\|^{2}\geq \delta ^{2}}$ and therefore

Hence ${\displaystyle m_{1}=m_{2},}$ which proves uniqueness. ${\displaystyle \blacksquare }$

Proof of characterization of minimum point when ${\displaystyle C}$ is a closed vector subspace

Assume that ${\displaystyle C}$ is a closed vector subspace of ${\displaystyle H.}$ It must be shown the minimizer ${\displaystyle m}$ is the unique element in ${\displaystyle C}$ such that ${\displaystyle \langle m-x,c\rangle =0}$ for every ${\displaystyle c\in C.}$

Proof that the condition is sufficient: Let ${\displaystyle z\in C}$ be such that ${\displaystyle \langle z-x,c\rangle =0}$ for all ${\displaystyle c\in C.}$ If ${\displaystyle c\in C}$ then ${\displaystyle c-z\in C}$ and so

which implies that ${\displaystyle \|z-x\|^{2}\leq \|c-x\|^{2}.}$ Because ${\displaystyle c\in C}$ was arbitrary, this proves that ${\displaystyle \|z-x\|=\inf _{c\in C}\|c-x\|}$ and so ${\displaystyle z}$ is a minimum point.

Proof that the condition is necessary: Let ${\displaystyle m\in C}$ be the minimum point. Let ${\displaystyle c\in C}$ and ${\displaystyle t\in \mathbb {R} .}$ Because ${\displaystyle m+tc\in C,}$ the minimality of ${\displaystyle m}$ guarantees that ${\displaystyle \|m-x\|\leq \|(m+tc)-x\|.}$ Thus

is always non-negative and ${\displaystyle \langle m-x,c\rangle }$ must be a real number. If ${\displaystyle \langle m-x,c\rangle \neq 0}$ then the map ${\displaystyle f(t):=2t\langle m-x,c\rangle +t^{2}\|c\|^{2}}$ has a minimum at ${\displaystyle t_{0}:=-{\frac {\langle m-x,c\rangle }{\|c\|^{2}}}}$ and moreover, ${\displaystyle f\left(t_{0}\right)<0,}$ which is a contradiction. Thus ${\displaystyle \langle m-x,c\rangle =0.}$ ${\displaystyle \blacksquare }$

Proof

Let ${\displaystyle C}$ be as described in this theorem and let

This theorem will follow from the following lemmas.

Lemma 2 and Lemma 1 together prove that there exists some ${\displaystyle c\in C}$ such that ${\displaystyle \|c\|=d.}$ Lemma 1 can be used to prove uniqueness as follows.

Suppose ${\displaystyle b\in C}$ is such that ${\displaystyle \|b\|=d}$ and denote the sequence by ${\displaystyle \left(c_{n}\right)_{n=1}^{\infty }}$ so that the subsequence ${\displaystyle \left(c_{2n}\right)_{n=1}^{\infty }}$ of even indices is the constant sequence ${\displaystyle c,c,c,\ldots }$ while the subsequence ${\displaystyle \left(c_{2n-1}\right)_{n=1}^{\infty }}$ of odd indices is the constant sequence ${\displaystyle b,b,b,\ldots .}$

Because ${\displaystyle \left\|c_{n}\right\|=d}$ for every ${\displaystyle n\in \mathbb {N} ,}$ ${\displaystyle \lim _{n\to \infty }\left\|c_{n}\right\|=\lim _{n\to \infty }d=d}$ in ${\displaystyle \mathbb {R} ,}$ which shows that the sequence ${\displaystyle \left(c_{n}\right)_{n=1}^{\infty }}$ satisfies the hypotheses of Lemma 1. Lemma 1 guarantees the existence of some ${\displaystyle x\in C}$ such that ${\displaystyle \lim _{n\to \infty }c_{n}=x}$ in ${\displaystyle H.}$ Because ${\displaystyle \left(c_{n}\right)_{n=1}^{\infty }}$ converges to ${\displaystyle x,}$ so do all of its subsequences. In particular, the subsequence ${\displaystyle c,c,c,\ldots }$ converges to ${\displaystyle x,}$ which implies that ${\displaystyle x=c}$ (because limits in ${\displaystyle H}$ are unique and this constant subsequence also converges to ${\displaystyle c}$). Similarly, ${\displaystyle x=b}$ because the subsequence ${\displaystyle b,b,b,\ldots }$ converges to both ${\displaystyle x}$ and ${\displaystyle b.}$ Thus ${\displaystyle b=c,}$ which proves the theorem. ${\displaystyle \blacksquare }$