Runge's phenomenon

Runge's phenomenon is the consequence of two properties of this problem.

• The magnitude of the n-th order derivatives of this particular function grows quickly when n increases.
• The equidistance between points leads to a Lebesgue constant that increases quickly when n increases.

The phenomenon is graphically obvious because both properties combine to increase the magnitude of the oscillations.

The error between the generating function and the interpolating polynomial of order n is given by

${\displaystyle f(x)-P_{n}(x)={\frac {f^{(n+1)}(\xi )}{(n+1)!}}\prod _{i=0}^{n}(x-x_{i})}$

for some ${\displaystyle \xi }$ in (−1, 1). Thus,

${\displaystyle \max _{-1\leq x\leq 1}|f(x)-P_{n}(x)|\leq \max _{-1\leq x\leq 1}{\frac {\left|f^{(n+1)}(x)\right|}{(n+1)!}}\max _{-1\leq x\leq 1}\prod _{i=0}^{n}|x-x_{i}|}$.

Denote by ${\displaystyle w_{n}(x)}$ the nodal function

${\displaystyle w_{n}(x)=(x-x_{0})(x-x_{1})\cdots (x-x_{n})}$

and let ${\displaystyle W_{n}}$ be the maximum of the magnitude of the ${\displaystyle w_{n}}$ function:

${\displaystyle W_{n}=\max _{-1\leq x\leq 1}|w_{n}(x)|}$.

It is elementary to prove that with equidistant nodes

${\displaystyle W_{n}\leq n!h^{n+1}}$

where ${\displaystyle h=2/n}$ is the step size.

Moreover, assume that the (n+1)-th derivative of ${\displaystyle f}$ is bounded, i.e.

${\displaystyle \max _{-1\leq x\leq 1}|f^{(n+1)}(x)|\leq M_{n+1}}$.

Therefore,

${\displaystyle \max _{-1\leq x\leq 1}|f(x)-P_{n}(x)|\leq M_{n+1}{\frac {h^{n+1}}{(n+1)}}}$.

But the magnitude of the (n+1)-th derivative of Runge's function increases when n increases, since ${\displaystyle M_{n+1}\leq (n+1)!5^{n+1}}$. The consequence is that the resulting upper bound, ${\displaystyle (10/n)^{n+1}n!}$, tends to infinity when n tends to infinity.

Although often used to explain the Runge phenomenon, the fact that the upper bound of the error goes to infinity does not necessarily imply, of course, that the error itself also diverges with n.

The oscillation can be minimized by using nodes that are distributed more densely towards the edges of the interval, specifically, with asymptotic density (on the interval [−1, 1]) given by the formula[3] ${\displaystyle 1/{\sqrt {1-x^{2}}}}$. A standard example of such a set of nodes is Chebyshev nodes, for which the maximum error in approximating the Runge function is guaranteed to diminish with increasing polynomial order.

When equidistant samples must be used because resampling on well-behaved sets of nodes is not feasible, the S-Runge algorithm can be considered.[4] In this approach, the original set of nodes is mapped on the set of Chebyshev nodes, providing a stable polynomial reconstruction. The peculiarity of this method is that there is no need of resampling at the mapped nodes, which are also called fake nodes. A Python implementation of this procedure can be found here.

The problem can be avoided by using spline curves which are piecewise polynomials. When trying to decrease the interpolation error one can increase the number of polynomial pieces which are used to construct the spline instead of increasing the degree of the polynomials used.

One can also fit a polynomial of higher degree (for instance, with ${\displaystyle n}$ points use a polynomial of order ${\displaystyle N=n^{2}}$ instead of ${\displaystyle n+1}$), and fit an interpolating polynomial whose first (or second) derivative has minimal ${\displaystyle L^{2}}$ norm.

A similar approach is to minimize a constrained version of the ${\displaystyle L^{p}}$ distance between the polynomial's ${\displaystyle m}$-th derivative and the mean value of its ${\displaystyle m}$-th derivative. Explicitly, to minimize

${\displaystyle V_{p}=\int _{a}^{b}\left|{\frac {\mathrm {d} ^{m}P_{N}(x)}{\mathrm {d} x^{m}}}-{\frac {1}{b-a}}\int _{a}^{b}{\frac {\mathrm {d} ^{m}P_{N}(z)}{\mathrm {d} z^{m}}}\mathrm {d} z\right|^{p}\mathrm {d} x-\sum _{i=1}^{n}\lambda _{i}\,\left(P_{N}(x_{i})-f(x_{i})\right),}$

where ${\displaystyle N\geq n-1}$ and ${\displaystyle m<N}$, with respect to the polynomial coefficients and the Lagrange multipliers, ${\displaystyle \lambda _{i}}$. When ${\displaystyle N=n-1}$, the constraint equations generated by the Lagrange multipliers reduce ${\displaystyle P_{N}(x)}$ to the minimum polynomial that passes through all ${\displaystyle n}$ points. At the opposite end, ${\displaystyle \lim _{N\to \infty }P_{N}(x)}$ will approach a form very similar to a piecewise polynomials approximation. When ${\displaystyle m=1}$, in particular, ${\displaystyle \lim _{N\to \infty }P_{N}(x)}$ approaches the linear piecewise polynomials, i.e. connecting the interpolation points with straight lines.

The role played by ${\displaystyle p}$ in the process of minimizing ${\displaystyle V_{p}}$ is to control the importance of the size of the fluctuations away from the mean value. The larger ${\displaystyle p}$ is, the more large fluctuations are penalized compared to small ones. The greatest advantage of the Euclidean norm, ${\displaystyle p=2}$, is that it allows for analytic solutions and it guarantees that ${\displaystyle V_{p}}$ will only have a single minimum. When ${\displaystyle p\neq 2}$ there can be multiple minima in ${\displaystyle V_{p}}$, making it difficult to ensure that a particular minimum found will be the global minimum instead of a local one.

Another method is fitting a polynomial of lower degree using the method of least squares. Generally, when using ${\displaystyle m}$ equidistant points, if ${\displaystyle N<2{\sqrt {m}}}$ then least squares approximation ${\displaystyle P_{N}(x)}$ is well-conditioned.[5]

Using Bernstein polynomials, one can uniformly approximate every continuous function in a closed interval, although this method is rather computationally expensive.

This method proposes to optimally stack a dense distribution of constraints of the type P″(x) = 0 on nodes positioned externally near the endpoints of each side of the interpolation interval, where P"(x) is the second derivative of the interpolation polynomial. Those constraints are called External Fake Constraints as they do not belong to the interpolation interval and they do not match the behaviour of the Runge function. The method has demonstrated that it has a better interpolation performance than Piecewise polynomials (splines) to mitigate the Runge phenomenon.[6]