Root-finding algorithms

The simplest root-finding algorithm is the bisection method. Let f be a continuous function, for which one knows an interval [a, b] such that f(a) and f(b) have opposite signs (a bracket). Let c = (a +b)/2 be the middle of the interval (the midpoint or the point that bisects the interval). Then either f(a) and f(c), or f(c) and f(b) have opposite signs, and one has divided by two the size of the interval. Although the bisection method is robust, it gains one and only one bit of accuracy with each iteration. Therefore, the number of function evaluations required for finding an ε-approximate root is ${\displaystyle \log _{2}{\frac {b-a}{\varepsilon }}}$. Other methods, under appropriate conditions, can gain accuracy faster.

The false position method, also called the regula falsi method, is similar to the bisection method, but instead of using bisection search's middle of the interval it uses the x-intercept of the line that connects the plotted function values at the endpoints of the interval, that is

${\displaystyle c={\frac {af(b)-bf(a)}{f(b)-f(a)}}.}$

False position is similar to the secant method, except that, instead of retaining the last two points, it makes sure to keep one point on either side of the root. The false position method can be faster than the bisection method and will never diverge like the secant method; however, it may fail to converge in some naive implementations due to roundoff errors that may lead to a wrong sign for f(c); typically, this may occur if the rate of variation of f is large in the neighborhood of the root.

The ITP method is the only known method to bracket the root with the same worst case guarantees of the bisection method while guaranteeing a superlinear convergence to the root of smooth functions as the secant method. It is also the only known method guaranteed to outperform the bisection method on the average for any continuous distribution on the location of the root (see ITP Method#Analysis). It does so by keeping track of both the bracketing interval as well as the minmax interval in which any point therein converges as fast as the bisection method. The construction of the queried point c follows three steps: interpolation (similar to the regula falsi), truncation (adjusting the regula falsi similar to Regula falsi § Improvements in regula falsi) and then projection onto the minmax interval. The combination of these steps produces a simultaneously minmax optimal method with guarantees similar to interpolation based methods for smooth functions, and, in practice will outperform both the bisection method and interpolation based methods under both smooth and non-smooth functions.

Newton's method assumes the function f to have a continuous derivative. Newton's method may not converge if started too far away from a root. However, when it does converge, it is faster than the bisection method, and is usually quadratic. Newton's method is also important because it readily generalizes to higher-dimensional problems. Newton-like methods with higher orders of convergence are the Householder's methods. The first one after Newton's method is Halley's method with cubic order of convergence.

Replacing the derivative in Newton's method with a finite difference, we get the secant method. This method does not require the computation (nor the existence) of a derivative, but the price is slower convergence (the order is approximately 1.6 (golden ratio)). A generalization of the secant method in higher dimensions is Broyden's method.

If we use a polynomial fit to remove the quadratic part of the finite difference used in the Secant method, so that it better approximates the derivative, we obtain Steffensen's method, which has quadratic convergence, and whose behavior (both good and bad) is essentially the same as Newton's method but does not require a derivative.

We can use the fixed-point iteration to find the root of a function. Given a function ${\displaystyle f(x)}$ which we have set to zero to find the root (${\displaystyle f(x)=0}$), we rewrite the equation in terms of ${\displaystyle x}$ so that ${\displaystyle f(x)=0}$ becomes ${\displaystyle x=g(x)}$ (note, there are often many ${\displaystyle g(x)}$ functions for each ${\displaystyle f(x)=0}$ function). Next, we relabel each side of the equation as ${\displaystyle x_{n+1}=g(x_{n})}$ so that we can perform the iteration. Next, we pick a value for ${\displaystyle x_{1}}$ and perform the iteration until it converges towards a root of the function. If the iteration converges, it will converge to a root. The iteration will only converge if ${\displaystyle |g'(root)|<1}$.

As an example of converting ${\displaystyle f(x)=0}$ to ${\displaystyle x=g(x)}$, if given the function ${\displaystyle f(x)=x^{2}+x-1}$, we will rewrite it as one of the following equations.

${\displaystyle x_{n+1}=(1/x_{n})-1}$,

${\displaystyle x_{n+1}=1/(x_{n}+1)}$,

${\displaystyle x_{n+1}=1-x_{n}^{2}}$,

${\displaystyle x_{n+1}=x_{n}^{2}+2x_{n}-1}$, or

${\displaystyle x_{n+1}=\pm {\sqrt {1-x_{n}}}}$.

The appearance of complex values in interpolation methods can be avoided by interpolating the inverse of f, resulting in the inverse quadratic interpolation method. Again, convergence is asymptotically faster than the secant method, but inverse quadratic interpolation often behaves poorly when the iterates are not close to the root.

Brent's method is a combination of the bisection method, the secant method and inverse quadratic interpolation. At every iteration, Brent's method decides which method out of these three is likely to do best, and proceeds by doing a step according to that method. This gives a robust and fast method, which therefore enjoys considerable popularity.

Ridders' method is a hybrid method that uses the value of function at the midpoint of the interval to perform an exponential interpolation to the root. This gives a fast convergence with a guaranteed convergence of at most twice the number of iterations as the bisection method.