Deterministic global optimization

Neumaier[1] classified global optimization methods in four categories, depending on their degree of rigour with which they approach the optimum, as follows:

• An incomplete method uses clever intuitive heuristics for searching but has no safeguards if the search gets stuck in a local minimum
• An asymptotically complete method reaches a global minimum with certainty or at least with probability one if allowed to run indefinitely long, but has no means to know when a global minimizer has been found.
• A complete method reaches a global minimum with certainty, assuming exact computations and indefinitely long run time, and knows after a finite time that an approximate global minimizer has been found (to within prescribed tolerances).
• A rigorous method reaches a global minimum with certainty and within given tolerances even in the presence of rounding errors, except in near-degenerate cases, where the tolerances may be exceeded.

Deterministic global optimization methods typically belong to the last two categories. Note that building a rigorous piece of software is extremely difficult as the process requires that all the dependencies are also coded rigorously.

Deterministic global optimization methods require ways to rigorously bound function values over regions of space. One could say that a main difference between deterministic and non-deterministic methods in this context is that the former perform calculations over regions of the solution space, whereas the latter perform calculations on single points. This is either done by exploiting particular functional forms (e.g. McCormick relaxations[2]), or using interval analysis in order to work with more general functional forms. In any case bounding is required, which is why deterministic global optimization methods cannot give a rigorous result when working with black-box code, unless that code is explicitly written to also return function bounds. For this reason, it is common for problems in deterministic global optimization to be represented using a computational graph, as it is straightforward to overload all operators such that the resulting function values or derivatives yield interval (rather than scalar) results.

Zero-order methods consist of methods which make use of zero-order interval arithmetic.[3] A representative example is interval bisection.

First-order methods consist of methods which make use of first-order information, e.g., interval gradients or interval slopes.

Second-order methods make use of second-order information, usually eigenvalue bounds derived from interval Hessian matrices. One of the most general second-order methodologies for handling problems of general type is the αΒΒ algorithm.

• ANTIGONE: Algorithms for coNTinuous / Integer Global Optimization of Nonlinear Equations).[4] It is a proprietary software, available through ANTIGONE the GAMS modelling platform.[5]

• BARON: BARON is available under the AIMMS, AMPL, and GAMS modeling language and on the NEOS Server.[6] It is a proprietary software [7]
• Couenne: Convex Over and Under ENvelopes for Nonlinear Estimation (Couenne) is an open-source library [8]
• EAGO: Easy-Advanced Global Optimization (EAGO) [9] is an open-source solver in Julia (programming language). It is developed by the University of Connecticut.[10]
• LINDO (Linear, Interactive, and Discrete Optimizer) includes global optimization capabilities.[11]
• MAiNGO: McCormick-based Algorithm for mixed-integer Nonlinear Global Optimization (MAiNGO) [12] is a C++ package with MPI and openMP parallelization and provided open-source [13] under Eclipse Public License - v 2.0.
• Octeract Engine is a proprietary solver with parallelization capabilities. It is developed and licensed by Octeract [14]
• SCIP: SCIP is an open-source suite of optimization solvers which among others solves mixed integer nonlinear programming (MINLP) [15]