The oldest MST algorithm is based on the cut property and uses edge contractions.
Boruvka’s algorithm [28, 148] goes back to 1926 and hence represents one of the oldest graph algorithms. The algorithm operates in phases, and identifies many MST edges in each phase. In a phase, each node identifies the lightest incident edge. These
232 11 Minimum Spanning Trees
edges are added to the MST (here it is assumed that the edge costs are pairwise distinct) and then contracted. Each phase can be implemented to run in time O(m).
Since a phase at least halves the number of remaining nodes, only a single node is left after O(log n)phases, and hence the total running time is O(m log n). Boruvka’s algorithm is not often used, because it is somewhat complicated to implement. It is nevertheless important as a basis for parallel MST algorithms.
There is a randomized linear-time MST algorithm that uses phases of Boruvka’s algorithm to reduce the number of nodes [105, 111]. The second building block of this algorithm reduces the number of edges to about 2n: we sample O(m/2)edges randomly, find an MST Tof the sample, and remove edges e∈E that are the heav- iest edge in a cycle in e∪T. The last step is rather difficult to implement efficiently.
But, at least for rather dense graphs, this approach can yield a practical improve- ment [108]. The linear-time algorithm can also be parallelized [84]. An adaptation to the external-memory model [2] saves a factor ln(n/n)in the asymptotic I/O com- plexity compared with Sibeyn’s algorithm but is impractical for currently interesting values of n owing to its much larger constant factor in the O-notation.
The theoretically best deterministic MST algorithm [35, 155] has the interesting property that it has optimal worst-case complexity, although it is not exactly known what this complexity is. Hence, if you come up with a completely different deter- ministic MST algorithm and prove that your algorithm runs in linear time, then we would know that the old algorithm also runs in linear time.
Minimum spanning trees define a single path between any pair of nodes. Interest- ingly, this path is a bottleneck shortest path [8, Application 13.3], i.e., it minimizes the maximum edge cost for all paths connecting the nodes in the original graph.
Hence, finding an MST amounts to solving the all-pairs bottleneck-shortest-path problem in much less time than that for solving the all-pairs shortest-path problem.
A related and even more frequently used application is clustering based on the MST [8, Application 13.5]: by dropping k−1 edges from the MST, it can be split into k subtrees. The nodes in a subtree T are far away from the other nodes in the sense that all paths to nodes in other subtrees use edges that are at least as heavy as the edges used to cut Tout of the MST.
Many applications lead to MST problems on complete graphs. Frequently, these graphs have a compact description, for example if the nodes represent points in the plane and the edge costs are Euclidean distances (these MSTs are called Euclidean minimum spanning trees). In these situations, it is an important concern whether one can rule out most of the edges as too heavy without actually looking at them. This is the case for Euclidean MSTs. It can be shown that Euclidean MSTs are contained in the Delaunay triangulation [46] of the point set. This triangulation has linear size and can be computed in time O(n log n). This leads to an algorithm of the same time complexity for Euclidean MSTs.
We discussed the application of MSTs to the Steiner tree and the traveling sales- man problem. We refer the reader to the books [8, 13, 117, 115, 200] for more infor- mation about these and related problems.
Generic Approaches to Optimization
A smuggler in the mountainous region of Profitania has n items in his cellar. If he sells an item i across the border, he makes a profit pi. However, the smuggler’s trade union only allows him to carry knapsacks with a maximum weight of M. If item i has weight wi, what items should he pack into the knapsack to maximize the profit from his next trip?
This problem, usually called the knapsack problem, has many other applications.
The books [122, 109] describe many of them. For example, an investment bank might have an amount M of capital to invest and a set of possible investments. Each invest- ment i has an expected profit pi for an investment of cost wi. In this chapter, we use the knapsack problem as an example to illustrate several generic approaches to optimization. These approaches are quite flexible and can be adapted to complicated situations that are ubiquitous in practical applications.
In the previous chapters we have considered very efficient specific solutions for frequently occurring simple problems such as finding shortest paths or minimum spanning trees. Now we look at generic solution methods that work for a much larger range of applications. Of course, the generic methods do not usually achieve the same efficiency as specific solutions. However, they save development time.
Formally, an optimization problem can be described by a setU of potential so- lutions, a setL of feasible solutions, and an objective function f :L →R. In a maximization problem, we are looking for a feasible solution x∗∈L that maximizes the value of the objective function over all feasible solutions. In a minimization prob- lem, we look for a solution that minimizes the value of the objective. In an existence problem, f is arbitrary and the question is whether the set of feasible solutions is nonempty.
For example, in the case of the knapsack problem with n items, a potential so- lution is simply a vector x= (x1, . . . ,xn)with xi∈ {0,1}. Here xi=1 indicates that
“element i is put into the knapsack” and xi=0 indicates that “element i is left out”.
ThusU ={0,1}n. The profits and weights are specified by vectors p= (p1, . . . ,pn) and w= (w1, . . . ,wn). A potential solution x is feasible if its weight does not exceed
234 12 Generic Approaches to Optimization
1 2
2 3
1 2
1 2 3 4
10 20
4 2 30
Instance
5 5 5
3 fractional optimal
greedy Solutions:
p
w M=
Fig. 12.1. The left part shows a knapsack instance with p= (10,20,15,20), w= (1,3,2,4), and M=5. The items are indicated as rectangles whose width and height correspond to weight and profit, respectively. The right part shows three solutions: the one computed by the greedy algorithm from Sect.12.2, an optimal solution computed by the dynamic programming al- gorithm from Sect.12.3, and the solution of the linear relaxation (Sect.12.1.1). The optimal solution has weight 5 and profit 35
the capacity of the knapsack, i.e.,∑1≤i≤nwixi≤M. The dot product wãx is a con- venient shorthand for∑1≤i≤nwixi. We can then say thatL ={x∈U : wãx≤M}is the set of feasible solutions and f(x) =pãx is the objective function.
The distinction between minimization and maximization problems is not essen- tial because setting f :=−f converts a maximization problem into a minimization problem and vice versa. We shall use maximization as our default simply because our example problem is more naturally viewed as a maximization problem.1
We shall present seven generic approaches. We start out with black-box solvers that can be applied to any problem that can be formulated in the problem specification language of the solver. In this case, the only task of the user is to formulate the given problem in the language of the black-box solver. Section12.1introduces this approach using linear programming and integer linear programming as examples.
The greedy approach that we have already seen in Chap. 11 is reviewed in Sect.12.2.
The approach of dynamic programming discussed in Sect.12.3is a more flexible way to construct solutions. We can also systematically explore the entire set of potential solutions, as described in Sect.12.4. Constraint programming, SAT solvers, and ILP solvers are special cases of systematic search. Finally, we discuss two very flexible approaches to exploring only a subset of the solution space. Local search, discussed in Sect.12.5, modifies a single solution until it has the desired quality. Evolutionary algorithms, described in Sect.12.6, simulate a population of candidate solutions.