Benders decomposition

Assume a problem that occurs in two or more stages, where the decisions for the later stages rely on the results from the earlier ones. An attempt at first-stage decisions can be made without prior knowledge of optimality according to later stage decisions. This first-stage decision is the master problem. Further stages can then be analyzed as separate subproblems. Information from these subproblems is passed back to the master problem. If constraints for a subproblem were violated, they can be added back to the master problem. The master problem is then re-solved.

The master problem represents an initial convex set which is further constrained by information gathered from the subproblems. Because the feasible space only shrinks as information is added, the objective value for the master function provides a lower bound on the objective function of the overall problem.

Benders Decomposition is applicable to problems with a largely block-diagonal structure.

Assume a problem of the following structure:

${\displaystyle {\begin{aligned}&{\text{minimize}}&&\mathbf {c} ^{\mathrm {T} }\mathbf {x} +\mathbf {d} ^{\mathrm {T} }\mathbf {y} \\&{\text{subject to}}&&A\mathbf {x} +B\mathbf {y} \geq \mathbf {b} \\&&&y\in Y\\&&&\mathbf {x} \geq \mathbf {0} \end{aligned}}}$

Where ${\displaystyle A,B}$ represent the constraints shared by both stages of variables and ${\displaystyle Y}$ represents the feasible set for ${\displaystyle \mathbf {y} }$. Notice that for any fixed ${\displaystyle \mathbf {\bar {y}} \in Y}$, the residual problem is

${\displaystyle {\begin{aligned}&{\text{minimize}}&&\mathbf {c} ^{\mathrm {T} }\mathbf {x} +\mathbf {d} ^{\mathrm {T} }\mathbf {\bar {y}} \\&{\text{subject to}}&&A\mathbf {x} \geq \mathbf {b} -B\mathbf {\bar {y}} \\&&&\mathbf {x} \geq \mathbf {0} \end{aligned}}}$

The dual of the residual problem is

${\displaystyle {\begin{aligned}&{\text{maximize}}&&(\mathbf {b} -B\mathbf {\bar {y}} )^{\mathrm {T} }\mathbf {u} +\mathbf {d} ^{\mathrm {T} }\mathbf {\bar {y}} \\&{\text{subject to}}&&A^{\mathrm {T} }\mathbf {u} \leq \mathbf {c} \\&&&\mathbf {u} \geq \mathbf {0} \end{aligned}}}$

Using the dual representation of the residual problem, the original problem can be rewritten as an equivalent minimax problem

${\displaystyle \min _{\mathbf {y} \in Y}\left[\mathbf {d} ^{\mathrm {T} }\mathbf {y} +\max _{\mathbf {u} \geq \mathbf {0} }\left\{(\mathbf {b} -B\mathbf {y} )^{\mathrm {T} }\mathbf {u} \mid A^{\mathrm {T} }\mathbf {u} \leq \mathbf {c} \right\}\right].}$

Benders decomposition relies on an iterative procedure that chooses successive values of ${\displaystyle \mathbf {y} }$ without considering the inner problem except through a set of cut constraints that are created through a pass-back mechanism from the maximization problem. Although the minimax formulation is written in terms of ${\displaystyle (\mathbf {u} ,\mathbf {y} )}$, for an optimal ${\displaystyle \mathbf {\bar {y}} }$ the corresponding ${\displaystyle \mathbf {\bar {x}} }$ can be found by solving the original problem with ${\displaystyle \mathbf {\bar {y}} }$ fixed.

At a high level, the procedure will iteratively consider the master problem and subproblem. Each iteration provides an updated upper and lower bound on the optimal objective value. The result of the subproblem either provides a new constraint to add to the master problem or a certificate that no finite optimal solution exists for the problem. The procedure terminates when it is shown that no finite optimal solution exists or when the gap between the upper and lower bound is sufficiently small. In such a case, the value of ${\displaystyle \mathbf {\bar {x}} }$ is determined by solving the primal residual problem fixing ${\displaystyle \mathbf {\bar {y}} }$.

Formally, the procedure begins with the lower bound set to ${\displaystyle -\inf }$, the upper bound set to ${\displaystyle \inf }$, and the cuts in the master problem empty. An initial solution is produced by selecting any ${\displaystyle \mathbf {\bar {y}} \in Y}$. Then the iterative procedure begins and continues until the gap between the upper and lower bound is at most ${\displaystyle \epsilon }$ or it is shown that no finite optimal solution exists.

The first step of each iteration begins by updating the upper bound by solving the subproblem given the most recent value of ${\displaystyle \mathbf {\bar {y}} }$. There are three possible outcomes from solving the subproblem.

In the first case, the objective value of the subproblem is unbounded above. By duality theory, when a dual problem has unbounded objective the corresponding primal problem is infeasible. This means that the choice of ${\displaystyle \mathbf {\bar {y}} }$ does not satisfy ${\displaystyle A\mathbf {x} +B\mathbf {\bar {y}} \geq \mathbf {b} }$ for any ${\displaystyle \mathbf {x} \geq \mathbf {0} }$. This solution can be removed from the master problem by taking an extreme ray ${\displaystyle \mathbf {\bar {u}} }$ that certifies the subproblem has unbounded objective and adding a constraint to the master asserting that ${\displaystyle (\mathbf {b} -B\mathbf {y} )^{\mathrm {T} }\mathbf {\bar {u}} \leq \mathbf {0} }$.

In the second case, the subproblem is infeasible. Since the dual feasible space to the problem is empty, either the original problem is not feasible or there is a ray in the primal problem that certifies the objective value is unbounded below. In either case, the procedure terminates.

In the third case, the subproblem has a finite optimal solution. By duality theory for linear programs, the optimal value of the subproblem is equal to the optimal value of the original problem constrained on the choice of ${\displaystyle \mathbf {\bar {y}} }$. This allows the upper bound to be updated to the value of the optimal solution of the subproblem, if it is better than the current upper bound. Given an optimal extreme point ${\displaystyle \mathbf {\bar {u}} }$, it also yields a new constraint that requires the master problem to consider the objective value under this particular solution by asserting that ${\displaystyle z\geq (\mathbf {b} -B\mathbf {y} )^{\mathrm {T} }\mathbf {\bar {u}} +\mathbf {d} ^{\mathrm {T} }\mathbf {y} }$. This will strictly increase the value of ${\displaystyle z}$ at the solution ${\displaystyle \mathbf {\bar {y}} }$ in the master problem if the choice of ${\displaystyle \mathbf {\bar {y}} }$ was suboptimal.

Finally, the last part of each iteration is creating a new solution to the master problem by solving the master problem with the new constraint. The new solution ${\displaystyle (\mathbf {\bar {y}} ,z)}$ is used to update the lower bound. If the gap between the best upper and lower bound is less than ${\displaystyle \epsilon }$ then the procedure terminates and the value of ${\displaystyle \mathbf {\bar {x}} }$ is determined by solving the primal residual problem fixing ${\displaystyle \mathbf {\bar {y}} }$. Otherwise, the procedure continues on to the next iteration.

• FortSP solver uses Benders decomposition for solving stochastic programming problems

• Benders, J. F. (Sept. 1962), "Partitioning procedures for solving mixed-variables programming problems", Numerische Mathematik 4(3): 238–252.
• Lasdon, Leon S. (2002), Optimization Theory for Large Systems (reprint of the 1970 Macmillan ed.), Mineola, New York: Dover Publications, pp. xiii+523, MR 1888251.