In this section, we discuss in detail the branch and bound (B&B) method. Without loss of generality, it is assumed that the objective function is to be minimized. That is, the ILP is a minimization problem.
As mentioned earlier, total enumeration is not practical to solve ILPs with a large number of variables. However, the B&B method, which is an implicit enumeration approach, is the most effective and widely used technique for solving large ILPs. The B&B method theoretically allows one to solve any ILP by solving a series of LP relaxation problems (called subproblems).
The B&B method starts with solving the LP relaxation (problem). If the optimal solution to the relaxed LP is integer-valued, the optimal solution to the LP relaxation is also optimal to the ILP, and we are done. However, it is most likely that the optimal solution to the LP relaxation does not satisfy all the integrality restrictions. See, for instance, the optimal solution to the relaxed LP of the capital budgeting problem in Section 3.2.1. In this case, we partition the ILP into a number of subproblems that are generally smaller in size or easier to solve than the original problem. This process of partitioning any given problem into two or more smaller or easier subproblems is commonly called branching, and each subproblem is called a branch. The B&B method is then repeated for each subproblem.
As mentioned earlier, the optimal solution to the LP relaxation is a lower bound on the optimal solution to the corresponding ILP. LetZbe the best known objective function value for the original ILP. For any given subproblem, letZrelaxbe the optimal objective value to its LP relaxation. This means that the optimal integer solution to this subproblem is no better than Zrelax. If Zrelax≥Z, then this subproblem (branch) cannot yield a feasible integer solution better than Z and can be eliminated from further consideration. This process
P3
P9 P10
P14 P17 P4
P6
P18
P13 P5
P11 P12
P8
P16
P20 P15
P19 P7
P1 P2
P0
Zrelax⫽ 80 Zrelax⫽ 93
Zrelax⫽ 92.22
Zrelax ⫽ 101.44
Zrelax⫽ 100
Zrelax ⫽ 99.37
Zrelax⫽ 99.11
Zrelax ⫽ 84 Zrelax ⫽ 96
Zrelax ⫽ 97.6 Zrelax⫽ 89.89
Zrelax⫽ 97 Zrelax ⫽ 97 Zrelax ⫽ 92.6 Zrelax ⫽ 88
Zrelax⫽ 73 y ⫽ 0
y ⫽ 0
y ⫽ 0 y ⫽ 1
y ⫽ 0 y ⫽ 1
y ⫽ 0
y ⫽ 0
y = 0
y ⫽ 0
y = 0
y = 0 y ⫽ 1
y = 1 y ⫽ 1
y = 1
y = 1 y ⫽ 1
y = 1
y ⫽ 1 Zrelax⫽ 98
Zrelax ⫽ 99.74
Zrelax⫽ 99.22
Zrelax ⫽ 97.5
FIGURE 3.2 The complete search tree.
of eliminating a branch from further consideration is usually called fathoming. However, if Zrelax< Z, then a conclusion cannot be reached and further branching from this subproblem is needed. This process continues until all branches are fathomed, and the best integer solution is optimal to the original ILP.
The B&B method can be represented by a search tree that has many levels (Figure 3.2).
The original ILP is the root, the first level consists of all the subproblems branched from the original ILP, the second level consists of all the subproblems branched from all the first level subproblems that have not been fathomed, and so on. We now discuss in more detail each of the three key components of the B&B method: branching, computing bounds, and fathoming.
3.3.1 Branching
For convenience, letP0 denote a given ILP or MILP. The set of all subproblems branched from P0 must represent all of P0 to find an optimal solution to P0. In addition, it is an approximation that any two subproblems are mutually exclusive. For example, let q1, q2, . . ., qr be all the possible values that integer variable yj can assume. Let Pi rep- resent the subproblem obtained by fixingyj atqi, fori= 1,2, . . ., r. IfP0is partitioned into P1, P2, . . ., Pr, then
{P0}={P1} ∪ {P2} ∪. . .∪ {Pr},and {Pi} ∩ {Ps}= ỉ, for alli=s
Note that each subproblemPihas one fewer integer variable sinceyjnow is a fixed constant.
If yj is a binary variable, there are only two obvious branches with yj= 0 and yj= 1.
Among all binary variables, yj should be selected as one that does not equal to 0 or 1 in the optimal solution to the LP relaxation. In the capital budgeting problem given in Section 3.2.1, any of y3,y4, andy5 can be chosen asyj.
If yj is a general integer variable, it may require a lot of effort to find all its possible feasible integer values. A more effective way is to derive two branches:yj≤tandyj≥t+ 1, wheretis a nonnegative integer. Again, one of the variables whose values are not an integer in the optimal solution to the LP relaxation should be selected andt is the largest integer smaller than the corresponding variable value.
As one can see, the branching process is essentially to add additional restrictions to form subproblems. Therefore, the optimal solution to any subproblem is no better than the
branching subproblem. In particular, for minimization, the optimal solution to any subprob- lem is always greater than or equal to the optimal solution to the branching subproblem and thus to the original ILP. As we move further from the root in the search tree, the optimal objective values of subproblems increase or remain the same for minimization ILPs.
3.3.2 Computing Lower and Upper Bounds
Let Zbest be the objective function value of a known integer solution. Then the optimal objective valueZ∗is at least as good asZbest. That is,Zbestis an upper bound onZ∗ for a minimization ILP. If a feasible integer solution with an objective value strictly better than Zbestis found, then replaceZbest by the new solution. In doing so, the upper boundZbest
remains the smallest upper bound and the corresponding feasible integer solution is called the incumbent solution (i.e., the best known integer solution). If no integer solutions have been identified yet,Zbest is set to be∞.
For any subproblem Pj in the B&B search, we attempt to find a lower bound ZL on the optimal objective value of Pj. That is, the optimal solution of Pj cannot be better than ZL. If ZL> Zbest (the objective value of the incumbent integer solution), then the subproblemPj can be discarded from further consideration, and the corresponding branch is then fathomed. However, if ZL≤Zbest, then a conclusion cannot be reached and the subproblemPj needs to be branched further.
In general, it is no easy task to find a lower bound for an optimization problem. Recall, however, that for any ILP, the optimal objective value associated with its LP relaxation provides a lower bound on the optimal objective value of the ILP.
3.3.3 Fathoming
In the B&B search, the ILP (the root) is branched into two or more level-1 subproblems; each level-1 subproblem is either fathomed or branched into two or more level-2 subproblems;
and each level-2 subproblem is either fathomed or further branched into two or more level-3 subproblems, and so on. If all subproblems (branches) are fathomed, the search stops and the incumbent integer solution is optimal to the ILP.
A subproblem may be fathomed in one of the following three ways:
1. An optimal integer solution is found. In this case, further branching from this subproblem is unnecessary and the incumbent solution may be updated.
2. The subproblem is found to be infeasible.
3. The optimal objective valueZLto its LP relaxation is strictly greater thanZbest. 3.3.4 Search Strategies
During the B&B searching process, there are generally many subproblems (at different levels) that remain to be further branched. The question is which remaining subproblem should be selected to branch next? That is, what search strategy should be used?
Width-first and depth-first searches are commonly used search strategies. In width-first search, all subproblems at a level are examined before any subproblem at the next level will be considered, and the search always moves forward from one level to the next. In depth-first search, an arbitrary subproblem at the newest level is examined first, and the search may move backwards. Since depth-first search can yield a feasible solution faster, which can be used to fathom, it is computationally more efficient than width-first search.
However, width-first strategy generally requires less computer memory. Another commonly used strategy is best-bound first, where the sub-problem with the best bound is branched
first. For minimization, the subproblem with the smallest lower bound is branched. In doing so, it is likely to generate a good integer solution early in the searching process and therefore speed up the search process.
In the next section, we present a B&B algorithm using the bound-first search strategy.
3.3.5 A Bound-First B&B Algorithm for Minimization (Maximization)
Step 1: Solve the LP relaxation. If the optimal solution happens to be integer-valued, then stop; and this is the optimal solution to the ILP. Otherwise, set Zbest=∞(=−∞), define the ILP as the candidate problemP, and go to Step 2.
Step 2: Let yj be a variable whose value in the optimal solution to the LP relaxation of P is not integer, and lett be the noninteger value of yj in the solution. BranchP into two subproblems by adding two constraints yj≤int(t) and yj≥int(t) + 1, where int(t) is the integer part oft. Ifyj is binary, letyj= 0 andyj= 1. Solve the LP relaxation problems.
If a subproblem is infeasible, fathom it. Otherwise, proceed to Step 3.
Step 3: If an integer solution is found, fathom the corresponding subproblem, and replace Zbest by the new feasible objective value if it is strictly smaller (greater) thanZbest.
Step 4: If a noninteger solution is found, fathom it if the optimal objective valueZrelax
is strictly greater (smaller) than the current solution, Zbest.
Step 5: If all subproblems are fathomed, then stop; an optimal integer solution has been found with the optimal objective valueZbest.
Step 6: Replace the candidate subproblem P by the remaining subproblem with the smallest (largest) bound Zrelax, and go to Step 2.
3.3.6 An Example
This section presents an example to illustrate the use of the B&B method. For simplicity, a BILP problem is considered. In particular, we use the capital budgeting problem presented in Section 3.2.1. We denote this ILPP0.
Since this is a maximization ILP, setZbest=−∞. The optimal solution to the LP relax- ation of P0 is y1=y2=y5= 1, y3= 0.7796, y4= 0.7627, and y6= 0.0678 with an objective valueZrelax= 101.61. This is a non-integer solution. Choose, for example, y3 as yj. Fixing y3at 0 and 1 respectively yields the following two sub-problems that have only five variables.
P1: maximize Z= 20y1+ 40y2+ 21y4+ 12y5+ 28y6
subject to 5y1+ 9y2+ 7y4+ 3y5+ 8y6≤26 2y1+ 7y2+ 4y4+ 5y5+ 4y6≤22 8y1+ 9y2+ 4y4+ 2y5+ 9y6≤25
y1, y2, y4, y5, y6= 0 or 1 P2: maximize Z= 20y1+ 40y2+ 21y4+ 12y5+ 28y6+ 15
subject to 5y1+ 9y2+ 7y4+ 3y5+ 8y6≤22 2y1+ 7y2+ 4y4+ 5y5+ 4y6≤16 8y1+ 9y2+ 4y4+ 2y5+ 9y6≤22
y1, y2, y4, y5, y6= 0 or 1
The optimal relaxed LP solutions are
P1: y2=y4=y5= 1, y1= 0.8947,andy6= 0.3158 with an objective valueZrelax= 99.74. P2: y1=y2= 1, y4= 0.767, y5= 0.7476,andy6= 0.0485 with an objective valueZrelax=
101.44.
Since P2 has the largest lower bound Zrelax= 101.44, it is selected to branch from the next. Let us choose to sety4= 0 and y4= 1. This will lead to two subproblems with only four decision variablesy1, y2, y5, and y6: P3(y4= 0) and P4(y4= 1). The two relaxed LP solutions are
P3: y2=y6= 1, y1= 0.2778,and y5= 0.8889 withZrelax= 92.22.
P4: y1=y2= 1, y6= 0,andy5= 0.3333 withZrelax= 100.
Of the three remaining subproblems P1, P3, and P4, P4 is selected to branch next by settingy5= 0(P5) andy5= 1(P6).
P5: y2= 1, y1= 0.9474,and y6= 0.1579 withZrelax= 99.37.
P6: y1= 1, y2= 0.6,andy6= 0.2 withZrelax= 97.6.
Among the four remaining subproblemsP1,P3,P5, andP6,P1is branched next by fixing y1= 0(P7) andy1= 1(P8).
P7: y2=y5=y6= 1,andy4= 0.8571 withZrelax= 98.
P8: y2=y4=y5= 1,andy6= 0.2222 withZrelax= 99.22.
OfP3,P5,P6,P7, andP8, the subproblemP5 is the next one to branch from. Choosey1
to be fixed at 0 and 1.
P9: y2= 1, y6= 0.75 withZrelax= 97.
P10: y2= 1, y6= 0.1111 withZrelax= 99.11.
We next branch from subproblemP8 and sety6 equal to 0 and 1.
P11: y2=y4=y5= 1 with Zrelax= 93. This is an integer solution. Since Zrelax>
Zbest=−∞,resetZbest= 93. P11becomes the first incumbent solution. FathomP11. P12: y4=y5= 1,andy2= 0.2222 withZrelax= 89.89.
Since Zrelax< Zbest for subproblems P3 and P12, fathom both P3 and P12. Of the four remaining subproblemsP6,P7,P9, andP10, we branch fromP10 next and fixy6.
P13: y2= 1 with Zrelax= 96.Again, this is an integer solution. SinceZrelax> Zbest= 93, resetZbest= 96,replace the current incumbent solution P11 byP13,and fathom P13. P14: y2= 0 with Zrelax= 84. This solution is all integer-values, and simply fathomP14
sinceZrelax< Zbest= 96.
There are three subproblemsP6,P7, andP9that remain to be examined. We now branch fromP7and fixy4at 0 and 1.
P15: y2=y5=y6= 1 with Zrelax= 80.FathomP15.
P16: y2=y5= 1 and y6= 0.875 with Zrelax= 97.5.FathomP15.
There still are three subproblemsP6,P9, andP16 left. Branch fromP6 by fixingy2 at 0 and 1.
P17: y1= 1 andy6= 0.875 withZrelax= 92.6. P17 is fathomed sinceZrelax< Zbest= 96. P18: y1= 0 andy6= 0 withZrelax= 88. P18 is fathomed since the solution is all integer-
valued withZrelax< Zbest= 96.
There are only two subproblems remaining. Branch fromP16 leads to the following:
P19: y2=y5= 1 withZrelax= 73. FathomP19. P20: y2= 1 andy5= 2/3 withZrelax= 97.
Both the remaining subproblemsP9 andP20 have the sameZrelax= 97. Branching from both leads to four subproblems that will be fathomed since their relaxed LP solutions are all strictly worse thanZbest. Now all subproblems have been fathomed, and the B&B search stops with the current incumbent solution P13(y1=y2=y3=y4= 1 and y5=y6= 0 with Zbest= 96) being the optimal solution to the ILP.
This ILP has six binary variables. Since each variable can assume either 0 or 1, the total enumeration will have to evaluate a total of 26= 64 solutions (many of them may be infeasible). In the above B&B search, a total of only 25 subproblems are evaluated—over 60% reduction of computational effort. As the number of variables increases, computational effort reduction can become more significant. This is especially true when the number of variables that equal 1 in an optimal integer solution is a small portion of all the binary integer variables.
In the B&B search, the number of possible subproblems at least doubles as the level increases by 1. At the final level, for an ILP with k integer variables, there are at least 2k possible sub-problems. For a small k= 20, 2k>1 million, and k= 30, 2k>1 billion.
Too many subproblems! One way to reduce the search effort is to limit the number of subproblems to branch further to some constant (e.g., 50). Such a search method is called a beam search, and the constant is the beam size. Beam search may quickly find an integer solution that may be good, but its optimality is not guaranteed.