INTRODUCTION TO ALGORITHMS 3rd phần 9 docx

The class P consists of those problems that are solvable in polynomial time.More specifically, they are problems that can be solved in time O.nk/ for someconstant k, where n is the size o

be either in the interior of the triangle formed by p0, pr, and pi or on a side ofthis triangle (but it is not a vertex of the triangle) Clearly, since pt is within atriangle formed by three other points of Qi, it cannot be a vertex of CH.Qi/.Since ptis not a vertex of CH.Qi/, we have that

Let Pi be the set of points that were popped during iteration i of the for loop.

Since the equality (33.1) applies for all points in Pi, we can apply it repeatedly

to show that CH.Qi Pi/ D CH.Qi/ But Qi Pi D Qj [ fpig, and so weconclude that CH.Qj [ fpig/ D CH.Qi Pi/ D CH.Qi/

We have shown that once we push pi, stack S contains exactly the vertices

of CH.Qi/ in counterclockwise order from bottom to top Incrementing i willthen cause the loop invariant to hold for the next iteration

Termination: When the loop terminates, we have i D m C 1, and so the loop

invariant implies that stack S consists of exactly the vertices of CH.Qm/, which

is CH.Q/, in counterclockwise order from bottom to top This completes theproof

We now show that the running time of GRAHAM-SCAN is O.n lg n/, where

n D jQj Line 1 takes ‚.n/ time Line 2 takes O.n lg n/ time, using merge sort

or heapsort to sort the polar angles and the cross-product method of Section 33.1

to compare angles (We can remove all but the farthest point with the same polarangle in total of O.n/ time over all n points.) Lines 3–6 take O.1/ time Because

m  n  1, the for loop of lines 7–10 executes at most n  3 times Since PUSH

takes O.1/ time, each iteration takes O.1/ time exclusive of the time spent in the

while loop of lines 8–9, and thus overall the for loop takes O.n/ time exclusive of the nested while loop.

We use aggregate analysis to show that the while loop takes O.n/ time overall.

For i D 0; 1; : : : ; m, we push each point pi onto stack S exactly once As in theanalysis of the MULTIPOPprocedure of Section 17.1, we observe that we can pop atmost the number of items that we push At least three points—p0, p1, and pm—arenever popped from the stack, so that in fact at most m  2 POP operations are

performed in total Each iteration of the while loop performs one POP, and so

there are at most m  2 iterations of the while loop altogether Since the test in

line 8 takes O.1/ time, each call of POPtakes O.1/ time, and m  n  1, the total

time taken by the while loop is O.n/ Thus, the running time of GRAHAM-SCAN

is O.n lg n/

p4 p2

p0

p1

right chain left chain

right chain left chain

p3

Figure 33.9 The operation of Jarvis’s march We choose the first vertex as the lowest point p0 The next vertex, p 1 , has the smallest polar angle of any point with respect to p 0 Then, p 2 has the smallest polar angle with respect to p1 The right chain goes as high as the highest point p3 Then,

we construct the left chain by finding smallest polar angles with respect to the negative x-axis.

Jarvis’s march

Jarvis’s march computes the convex hull of a set Q of points by a technique known

as package wrapping (or gift wrapping) The algorithm runs in time O.nh/,

where h is the number of vertices of CH.Q/ When h is o.lg n/, Jarvis’s march isasymptotically faster than Graham’s scan

Intuitively, Jarvis’s march simulates wrapping a taut piece of paper around theset Q We start by taping the end of the paper to the lowest point in the set, that is,

to the same point p0with which we start Graham’s scan We know that this pointmust be a vertex of the convex hull We pull the paper to the right to make it taut,and then we pull it higher until it touches a point This point must also be a vertex

of the convex hull Keeping the paper taut, we continue in this way around the set

of vertices until we come back to our original point p0

More formally, Jarvis’s march builds a sequence H D hp0; p1; : : : ; ph1i of thevertices of CH.Q/ We start with p0 As Figure 33.9 shows, the next vertex p1

in the convex hull has the smallest polar angle with respect to p0 (In case of ties,

we choose the point farthest from p ) Similarly, p has the smallest polar angle

with respect to p1, and so on When we reach the highest vertex, say pk(breakingties by choosing the farthest such vertex), we have constructed, as Figure 33.9

shows, the right chain of CH.Q/ To construct the left chain, we start at pk andchoose pkC1as the point with the smallest polar angle with respect to pk, but from

from the negative x-axis, until we come back to our original vertex p0

We could implement Jarvis’s march in one conceptual sweep around the convexhull, that is, without separately constructing the right and left chains Such imple-mentations typically keep track of the angle of the last convex-hull side chosen andrequire the sequence of angles of hull sides to be strictly increasing (in the range

of 0 to 2 radians) The advantage of constructing separate chains is that we neednot explicitly compute angles; the techniques of Section 33.1 suffice to compareangles

If implemented properly, Jarvis’s march has a running time of O.nh/ For each

of the h vertices of CH.Q/, we find the vertex with the minimum polar angle Eachcomparison between polar angles takes O.1/ time, using the techniques of Sec-tion 33.1 As Section 9.1 shows, we can compute the minimum of n values in O.n/time if each comparison takes O.1/ time Thus, Jarvis’s march takes O.nh/ time

multipli-33.3-3

Given a set of points Q, prove that the pair of points farthest from each other must

be vertices of CH.Q/

33.3-4

For a given polygon P and a point q on its boundary, the shadow of q is the set

of points r such that the segment qr is entirely on the boundary or in the interior

of P As Figure 33.10 illustrates, a polygon P is star-shaped if there exists a

point p in the interior of P that is in the shadow of every point on the boundary

of P The set of all such points p is called the kernel of P Given an n-vertex,

q

q′

Figure 33.10 The definition of a star-shaped polygon, for use in Exercise 33.3-4 (a) A star-shaped

polygon The segment from point p to any point q on the boundary intersects the boundary only at q.

(b) A non-star-shaped polygon The shaded region on the left is the shadow of q, and the shaded

region on the right is the shadow of q0 Since these regions are disjoint, the kernel is empty.

star-shaped polygon P specified by its vertices in counterclockwise order, showhow to compute CH.P / in O.n/ time

33.3-5

In the on-line convex-hull problem, we are given the set Q of n points one point at

a time After receiving each point, we compute the convex hull of the points seen

so far Obviously, we could run Graham’s scan once for each point, with a totalrunning time of O.n2lg n/ Show how to solve the on-line convex-hull problem in

a total of O.n2/ time

33.3-6 ?

Show how to implement the incremental method for computing the convex hull

of n points so that it runs in O.n lg n/ time

We now consider the problem of finding the closest pair of points in a set Q of

n  2 points “Closest” refers to the usual euclidean distance: the distance betweenpoints p1D x1; y1/ and p2 D x2; y2/ isp

.x1 x2/2C y1 y2/2 Two points

in set Q may be coincident, in which case the distance between them is zero Thisproblem has applications in, for example, traffic-control systems A system forcontrolling air or sea traffic might need to identify the two closest vehicles in order

to detect potential collisions

A brute-force closest-pair algorithm simply looks at all then

this problem, whose running time is described by the familiar recurrence T n/ D2T n=2/ C O.n/ Thus, this algorithm uses only O.n lg n/ time.

The divide-and-conquer algorithm

Each recursive invocation of the algorithm takes as input a subset P  Q andarrays X and Y , each of which contains all the points of the input subset P The points in array X are sorted so that their x-coordinates are monotonicallyincreasing Similarly, array Y is sorted by monotonically increasing y-coordinate.Note that in order to attain the O.n lg n/ time bound, we cannot afford to sort

in each recursive call; if we did, the recurrence for the running time would be

T n/ D 2T n=2/ C O.n lg n/, whose solution is T n/ D O.n lg2n/ (Use theversion of the master method given in Exercise 4.6-2.) We shall see a little laterhow to use “presorting” to maintain this sorted property without actually sorting ineach recursive call

A given recursive invocation with inputs P , X , and Y first checks whether

jP j  3 If so, the invocation simply performs the brute-force method describedabove: try all jP j

2

pairs of points and return the closest pair If jP j > 3, therecursive invocation carries out the divide-and-conquer paradigm as follows

Divide: Find a vertical line l that bisects the point set P into two sets PLand PR

such that jPLj D djP j =2e, jPRj D bjP j =2c, all points in PLare on or to theleft of line l, and all points in PRare on or to the right of l Divide the array Xinto arrays XLand XR, which contain the points of PL and PR respectively,sorted by monotonically increasing x-coordinate Similarly, divide the array Yinto arrays YL and YR, which contain the points of PL and PR respectively,sorted by monotonically increasing y-coordinate

Conquer: Having divided P into PLand PR, make two recursive calls, one to findthe closest pair of points in PLand the other to find the closest pair of points

in PR The inputs to the first call are the subset PLand arrays XLand YL; thesecond call receives the inputs PR, XR, and YR Let the closest-pair distancesreturned for PLand PRbe ıLand ıR, respectively, and let ı D min.ıL; ıR/

Combine: The closest pair is either the pair with distance ı found by one of the

recursive calls, or it is a pair of points with one point in PLand the other in PR.The algorithm determines whether there is a pair with one point in PLand theother point in PR and whose distance is less than ı Observe that if a pair ofpoints has distance less than ı, both points of the pair must be within ı units

of line l Thus, as Figure 33.11(a) shows, they both must reside in the 2ı-widevertical strip centered at line l To find such a pair, if one exists, we do thefollowing:

1 Create an array Y0, which is the array Y with all points not in the 2ı-widevertical strip removed The array Y0is sorted by y-coordinate, just as Y is.

2 For each point p in the array Y0, try to find points in Y0 that are within ıunits of p As we shall see shortly, only the 7 points in Y0that follow p need

be considered Compute the distance from p to each of these 7 points, andkeep track of the closest-pair distance ı0found over all pairs of points in Y0

3 If ı0 < ı, then the vertical strip does indeed contain a closer pair than therecursive calls found Return this pair and its distance ı0 Otherwise, returnthe closest pair and its distance ı found by the recursive calls

The above description omits some implementation details that are necessary toachieve the O.n lg n/ running time After proving the correctness of the algorithm,

we shall show how to implement the algorithm to achieve the desired time bound

Correctness

The correctness of this closest-pair algorithm is obvious, except for two aspects.First, by bottoming out the recursion whenjP j  3, we ensure that we never try tosolve a subproblem consisting of only one point The second aspect is that we needonly check the 7 points following each point p in array Y0; we shall now prove thisproperty

Suppose that at some level of the recursion, the closest pair of points is pL2 PL

and pR 2 PR Thus, the distance ı0 between pL and pR is strictly less than ı.Point pLmust be on or to the left of line l and less than ı units away Similarly, pR

is on or to the right of l and less than ı units away Moreover, pL and pR arewithin ı units of each other vertically Thus, as Figure 33.11(a) shows, pLand pR

are within a ı 2ı rectangle centered at line l (There may be other points withinthis rectangle as well.)

We next show that at most 8 points of P can reside within this ı 2ı rectangle.Consider the ı ı square forming the left half of this rectangle Since all pointswithin PLare at least ı units apart, at most 4 points can reside within this square;Figure 33.11(b) shows how Similarly, at most 4 points in PR can reside withinthe ı ı square forming the right half of the rectangle Thus, at most 8 points of Pcan reside within the ı 2ı rectangle (Note that since points on line l may be ineither PLor PR, there may be up to 4 points on l This limit is achieved if there aretwo pairs of coincident points such that each pair consists of one point from PLandone point from PR, one pair is at the intersection of l and the top of the rectangle,and the other pair is where l intersects the bottom of the rectangle.)

Having shown that at most 8 points of P can reside within the rectangle, wecan easily see why we need to check only the 7 points following each point in thearray Y0 Still assuming that the closest pair is p and p , let us assume without

Figure 33.11 Key concepts in the proof that the closest-pair algorithm needs to check only 7 points following each point in the array Y0 (a) If pL2 P L and pR2 P R are less than ı units apart, they

must reside within a ı 2ı rectangle centered at line l (b) How 4 points that are pairwise at least ı

units apart can all reside within a ı ı square On the left are 4 points in PL, and on the right are 4 points in PR The ı 2ı rectangle can contain 8 points if the points shown on line l are actually pairs of coincident points with one point in PLand one in PR.

loss of generality that pLprecedes pRin array Y0 Then, even if pLoccurs as early

as possible in Y0and pRoccurs as late as possible, pRis in one of the 7 positionsfollowing pL Thus, we have shown the correctness of the closest-pair algorithm

Implementation and running time

As we have noted, our goal is to have the recurrence for the running time be T n/ D2T n=2/ C O.n/, where T n/ is the running time for a set of n points The maindifficulty comes from ensuring that the arrays XL, XR, YL, and YR, which arepassed to recursive calls, are sorted by the proper coordinate and also that thearray Y0 is sorted by y-coordinate (Note that if the array X that is received by arecursive call is already sorted, then we can easily divide set P into PLand PRinlinear time.)

The key observation is that in each call, we wish to form a sorted subset of asorted array For example, a particular invocation receives the subset P and thearray Y , sorted by y-coordinate Having partitioned P into PLand PR, it needs toform the arrays YL and YR, which are sorted by y-coordinate, in linear time Wecan view the method as the opposite of the MERGEprocedure from merge sort in

Section 2.3.1: we are splitting a sorted array into two sorted arrays The followingpseudocode gives the idea.

1 let YLŒ1 : : Y:length and YRŒ1 : : Y:length be new arrays

2 YL:length D YR:length D 0

3 fori D 1 to Y:length

4 ifY Œi  2 PL

5 YL:length D YL:length C 1

6 YLŒYL:length D Y Œi 

7 elseYR:length D YR:length C 1

8 YRŒYR:length D Y Œi 

We simply examine the points in array Y in order If a point Y Œi  is in PL, weappend it to the end of array YL; otherwise, we append it to the end of array YR.Similar pseudocode works for forming arrays XL, XR, and Y0

The only remaining question is how to get the points sorted in the first place We

presort them; that is, we sort them once and for all before the first recursive call.

We pass these sorted arrays into the first recursive call, and from there we whittlethem down through the recursive calls as necessary Presorting adds an additionalO.n lg n/ term to the running time, but now each step of the recursion takes lineartime exclusive of the recursive calls Thus, if we let T n/ be the running time ofeach recursive step and T0.n/ be the running time of the entire algorithm, we get

Professor Williams comes up with a scheme that allows the closest-pair algorithm

to check only 5 points following each point in array Y0 The idea is always to placepoints on line l into set PL Then, there cannot be pairs of coincident points online l with one point in PLand one in PR Thus, at most 6 points can reside inthe ı 2ı rectangle What is the flaw in the professor’s scheme?

33.4-2

Show that it actually suffices to check only the points in the 5 array positions lowing each point in the array Y0

We can define the distance between two points in ways other than euclidean Inthe plane, the Lm-distance between points p1 and p2 is given by the expres-sion jx1 x2jmC jy1 y2jm/1=m Euclidean distance, therefore, is L2-distance.Modify the closest-pair algorithm to use the L1-distance, which is also known as

the Manhattan distance.

33.4-6

Suggest a change to the closest-pair algorithm that avoids presorting the Y array

but leaves the running time as O.n lg n/ (Hint: Merge sorted arrays YLand YRtoform the sorted array Y )

Problems

33-1 Convex layers

Given a set Q of points in the plane, we define the convex layers of Q inductively.

The first convex layer of Q consists of those points in Q that are vertices of CH.Q/.For i > 1, define Qito consist of the points of Q with all points in convex layers1; 2; : : : ; i  1 removed Then, the i th convex layer of Q is CH.Qi/ if Qi ¤ ; and

is undefined otherwise

a Give an O.n2/-time algorithm to find the convex layers of a set of n points

b Prove that .n lg n/ time is required to compute the convex layers of a set of n

points with any model of computation that requires .n lg n/ time to sort n realnumbers

33-2 Maximal layers

Let Q be a set of n points in the plane We say that point x; y/ dominates

point x0; y0/ if x  x0and y  y0 A point in Q that is dominated by no other

points in Q is said to be maximal Note that Q may contain many maximal points, which can be organized into maximal layers as follows The first maximal layer L1

is the set of maximal points of Q For i > 1, the i th maximal layer Liis the set ofmaximal points in Q Si 1

j D1Lj.Suppose that Q has k nonempty maximal layers, and let yibe the y-coordinate

of the leftmost point in Lifor i D 1; 2; : : : ; k For now, assume that no two points

in Q have the same x- or y-coordinate

a Show that y1 > y2>    > yk

Consider a point x; y/ that is to the left of any point in Q and for which y isdistinct from the y-coordinate of any point in Q Let Q0D Q [ f.x; y/g

b Let j be the minimum index such that yj < y, unless y < yk, in which case

we let j D k C 1 Show that the maximal layers of Q0are as follows:

 If j  k, then the maximal layers of Q0 are the same as the maximal layers

of Q, except that Lj also includes x; y/ as its new leftmost point

 If j D k C 1, then the first k maximal layers of Q0are the same as for Q, but

in addition, Q0has a nonempty k C 1/st maximal layer: LkC1D f.x; y/g

c Describe an O.n lg n/-time algorithm to compute the maximal layers of a set Q

of n points (Hint: Move a sweep line from right to left.)

d Do any difficulties arise if we now allow input points to have the same x- or

y-coordinate? Suggest a way to resolve such problems

33-3 Ghostbusters and ghosts

A group of n Ghostbusters is battling n ghosts Each Ghostbuster carries a protonpack, which shoots a stream at a ghost, eradicating it A stream goes in a straightline and terminates when it hits the ghost The Ghostbusters decide upon the fol-lowing strategy They will pair off with the ghosts, forming n Ghostbuster-ghostpairs, and then simultaneously each Ghostbuster will shoot a stream at his cho-

sen ghost As we all know, it is very dangerous to let streams cross, and so the

Ghostbusters must choose pairings for which no streams will cross

Assume that the position of each Ghostbuster and each ghost is a fixed point inthe plane and that no three positions are colinear

a Argue that there exists a line passing through one Ghostbuster and one ghost

such that the number of Ghostbusters on one side of the line equals the number

of ghosts on the same side Describe how to find such a line in O.n lg n/ time

b Give an O.n2lg n/-time algorithm to pair Ghostbusters with ghosts in such away that no streams cross.

33-4 Picking up sticks

Professor Charon has a set of n sticks, which are piled up in some configuration.Each stick is specified by its endpoints, and each endpoint is an ordered triplegiving its x; y; ´/ coordinates No stick is vertical He wishes to pick up all thesticks, one at a time, subject to the condition that he may pick up a stick only ifthere is no other stick on top of it

a Give a procedure that takes two sticks a and b and reports whether a is above,

below, or unrelated to b

b Describe an efficient algorithm that determines whether it is possible to pick up

all the sticks, and if so, provides a legal order in which to pick them up

33-5 Sparse-hulled distributions

Consider the problem of computing the convex hull of a set of points in the planethat have been drawn according to some known random distribution Sometimes,the number of points, or size, of the convex hull of n points drawn from such adistribution has expectation O.n1/ for some constant  > 0 We call such a

distribution sparse-hulled Sparse-hulled distributions include the following:

 Points drawn uniformly from a unit-radius disk The convex hull has expectedsize ‚.n1=3/

 Points drawn uniformly from the interior of a convex polygon with k sides, forany constant k The convex hull has expected size ‚.lg n/

 Points drawn according to a two-dimensional normal distribution The convexhull has expected size ‚.p

lg n/

a Given two convex polygons with n1and n2vertices respectively, show how tocompute the convex hull of all n1Cn2points in O.n1Cn2/ time (The polygonsmay overlap.)

b Show how to compute the convex hull of a set of n points drawn independently

according to a sparse-hulled distribution in O.n/ average-case time (Hint:

Recursively find the convex hulls of the first n=2 points and the second n=2points, and then combine the results.)

Chapter notes

This chapter barely scratches the surface of computational-geometry algorithmsand techniques Books on computational geometry include those by Preparata andShamos [282], Edelsbrunner [99], and O’Rourke [269]

Although geometry has been studied since antiquity, the development of rithms for geometric problems is relatively new Preparata and Shamos note thatthe earliest notion of the complexity of a problem was given by E Lemoine in 1902

algo-He was studying euclidean constructions—those using a compass and a ruler—anddevised a set of five primitives: placing one leg of the compass on a given point,placing one leg of the compass on a given line, drawing a circle, passing the ruler’sedge through a given point, and drawing a line Lemoine was interested in thenumber of primitives needed to effect a given construction; he called this amountthe “simplicity” of the construction

The algorithm of Section 33.2, which determines whether any segments sect, is due to Shamos and Hoey [313]

inter-The original version of Graham’s scan is given by Graham [150] inter-The wrapping algorithm is due to Jarvis [189] Using a decision-tree model of com-putation, Yao [359] proved a worst-case lower bound of .n lg n/ for the runningtime of any convex-hull algorithm When the number of vertices h of the con-vex hull is taken into account, the prune-and-search algorithm of Kirkpatrick andSeidel [206], which takes O.n lg h/ time, is asymptotically optimal

package-The O.n lg n/-time divide-and-conquer algorithm for finding the closest pair ofpoints is by Shamos and appears in Preparata and Shamos [282] Preparata andShamos also show that the algorithm is asymptotically optimal in a decision-treemodel

Almost all the algorithms we have studied thus far have been polynomitime

al-gorithms: on inputs of size n, their worst-case running time is O.nk/ for some

con-stant k You might wonder whether all problems can be solved in polynomial time.

The answer is no For example, there are problems, such as Turing’s famous ing Problem,” that cannot be solved by any computer, no matter how much time weallow There are also problems that can be solved, but not in time O.nk/ for anyconstant k Generally, we think of problems that are solvable by polynomial-timealgorithms as being tractable, or easy, and problems that require superpolynomialtime as being intractable, or hard

“Halt-The subject of this chapter, however, is an interesting class of problems, calledthe “NP-complete” problems, whose status is unknown No polynomial-time al-gorithm has yet been discovered for an NP-complete problem, nor has anyone yetbeen able to prove that no polynomial-time algorithm can exist for any one of them.This so-called P ¤ NP question has been one of the deepest, most perplexing openresearch problems in theoretical computer science since it was first posed in 1971.Several NP-complete problems are particularly tantalizing because they seem

on the surface to be similar to problems that we know how to solve in polynomialtime In each of the following pairs of problems, one is solvable in polynomialtime and the other is NP-complete, but the difference between problems appears to

be slight:

Shortest vs longest simple paths: In Chapter 24, we saw that even with negative

edge weights, we can find shortest paths from a single source in a directed graph G D V; E/ in O.VE/ time Finding a longest simple path between two

vertices is difficult, however Merely determining whether a graph contains asimple path with at least a given number of edges is NP-complete

Euler tour vs hamiltonian cycle: An Euler tour of a connected, directed graph

G D V; E/ is a cycle that traverses each edge of G exactly once, although

it is allowed to visit each vertex more than once By Problem 22-3, we candetermine whether a graph has an Euler tour in only O.E/ time and, in fact,

we can find the edges of the Euler tour in O.E/ time A hamiltonian cycle of

a directed graph G D V; E/ is a simple cycle that contains each vertex in V

Determining whether a directed graph has a hamiltonian cycle is NP-complete

(Later in this chapter, we shall prove that determining whether an undirected

graph has a hamiltonian cycle is NP-complete.)

2-CNF satisfiability vs 3-CNF satisfiability: A boolean formula contains

vari-ables whose values are 0 or 1; boolean connectives such as ^ (AND), _ (OR),

and : (NOT); and parentheses A boolean formula is satisfiable if there exists

some assignment of the values 0 and 1 to its variables that causes it to evaluate

to 1 We shall define terms more formally later in this chapter, but informally, aboolean formula is ink-conjunctive normal form, or k-CNF, if it is the AND

of clauses of ORs of exactly k variables or their negations For example, theboolean formula x1_ :x2/ ^ :x1_ x3/ ^ :x2_ :x3/ is in 2-CNF (It hasthe satisfying assignment x1 D 1; x2 D 0; x3 D 1.) Although we can deter-mine in polynomial time whether a 2-CNF formula is satisfiable, we shall seelater in this chapter that determining whether a 3-CNF formula is satisfiable isNP-complete

NP-completeness and the classes P and NP

Throughout this chapter, we shall refer to three classes of problems: P, NP, andNPC, the latter class being the NP-complete problems We describe them infor-mally here, and we shall define them more formally later on

The class P consists of those problems that are solvable in polynomial time.More specifically, they are problems that can be solved in time O.nk/ for someconstant k, where n is the size of the input to the problem Most of the problemsexamined in previous chapters are in P

The class NP consists of those problems that are “verifiable” in polynomial time.What do we mean by a problem being verifiable? If we were somehow given a

“certificate” of a solution, then we could verify that the certificate is correct in timepolynomial in the size of the input to the problem For example, in the hamiltonian-cycle problem, given a directed graph G D V; E/, a certificate would be a se-quence h1; 2; 3; : : : ; jV ji of jV j vertices We could easily check in polynomialtime that i; i C1/ 2 E for i D 1; 2; 3; : : : ; jV j  1 and that jV j; 1/ 2 E as well

As another example, for 3-CNF satisfiability, a certificate would be an assignment

of values to variables We could check in polynomial time that this assignmentsatisfies the boolean formula

Any problem in P is also in NP, since if a problem is in P then we can solve it

in polynomial time without even being supplied a certificate We shall formalizethis notion later in this chapter, but for now we can believe that P  NP The openquestion is whether or not P is a proper subset of NP

Informally, a problem is in the class NPC—and we refer to it as being

NP-complete—if it is in NP and is as “hard” as any problem in NP We shall formally

define what it means to be as hard as any problem in NP later in this chapter

In the meantime, we will state without proof that if any NP-complete problem can be solved in polynomial time, then every problem in NP has a polynomial-

time algorithm Most theoretical computer scientists believe that the NP-completeproblems are intractable, since given the wide range of NP-complete problemsthat have been studied to date—without anyone having discovered a polynomial-time solution to any of them—it would be truly astounding if all of them could

be solved in polynomial time Yet, given the effort devoted thus far to provingthat NP-complete problems are intractable—without a conclusive outcome—wecannot rule out the possibility that the NP-complete problems are in fact solvable

in polynomial time

To become a good algorithm designer, you must understand the rudiments of thetheory of NP-completeness If you can establish a problem as NP-complete, youprovide good evidence for its intractability As an engineer, you would then dobetter to spend your time developing an approximation algorithm (see Chapter 35)

or solving a tractable special case, rather than searching for a fast algorithm thatsolves the problem exactly Moreover, many natural and interesting problems that

on the surface seem no harder than sorting, graph searching, or network flow are

in fact NP-complete Therefore, you should become familiar with this remarkableclass of problems

Overview of showing problems to be NP-complete

The techniques we use to show that a particular problem is NP-complete differfundamentally from the techniques used throughout most of this book to designand analyze algorithms When we demonstrate that a problem is NP-complete,

we are making a statement about how hard it is (or at least how hard we think itis), rather than about how easy it is We are not trying to prove the existence of

an efficient algorithm, but instead that no efficient algorithm is likely to exist Inthis way, NP-completeness proofs bear some similarity to the proof in Section 8.1

of an .n lg n/-time lower bound for any comparison sort algorithm; the specifictechniques used for showing NP-completeness differ from the decision-tree methodused in Section 8.1, however

We rely on three key concepts in showing a problem to be NP-complete:

Decision problems vs optimization problems

Many problems of interest are optimization problems, in which each feasible (i.e.,

“legal”) solution has an associated value, and we wish to find a feasible solutionwith the best value For example, in a problem that we call SHORTEST-PATH,

we are given an undirected graph G and vertices u and , and we wish to find apath from u to  that uses the fewest edges In other words, SHORTEST-PATH

is the single-pair shortest-path problem in an unweighted, undirected graph

NP-completeness applies directly not to optimization problems, however, but to

deci-sion problems, in which the answer is simply “yes” or “no” (or, more formally, “1”

or “0”)

Although NP-complete problems are confined to the realm of decision problems,

we can take advantage of a convenient relationship between optimization problemsand decision problems We usually can cast a given optimization problem as a re-lated decision problem by imposing a bound on the value to be optimized Forexample, a decision problem related to SHORTEST-PATH is PATH: given a di-rected graph G, vertices u and , and an integer k, does a path exist from u to consisting of at most k edges?

The relationship between an optimization problem and its related decision lem works in our favor when we try to show that the optimization problem is

prob-“hard.” That is because the decision problem is in a sense “easier,” or at least “noharder.” As a specific example, we can solve PATH by solving SHORTEST-PATHand then comparing the number of edges in the shortest path found to the value

of the decision-problem parameter k In other words, if an optimization lem is easy, its related decision problem is easy as well Stated in a way that hasmore relevance to NP-completeness, if we can provide evidence that a decisionproblem is hard, we also provide evidence that its related optimization problem ishard Thus, even though it restricts attention to decision problems, the theory ofNP-completeness often has implications for optimization problems as well

input to a particular problem an instance of that problem; for example, in PATH,

an instance would be a particular graph G, particular vertices u and  of G, and aparticular integer k Now suppose that we already know how to solve a differentdecision problem B in polynomial time Finally, suppose that we have a procedurethat transforms any instance ˛ of A into some instance ˇ of B with the followingcharacteristics:

 The transformation takes polynomial time

 The answers are the same That is, the answer for ˛ is “yes” if and only if theanswer for ˇ is also “yes.”

polynomial-time reduction algorithm

we use the answer for ˇ as the answer for ˛.

We call such a procedure a polynomial-time reduction algorithm and, as

Fig-ure 34.1 shows, it provides us a way to solve problem A in polynomial time:

1 Given an instance ˛ of problem A, use a polynomial-time reduction algorithm

to transform it to an instance ˇ of problem B

2 Run the polynomial-time decision algorithm for B on the instance ˇ

3 Use the answer for ˇ as the answer for ˛

As long as each of these steps takes polynomial time, all three together do also, and

so we have a way to decide on ˛ in polynomial time In other words, by “reducing”solving problem A to solving problem B, we use the “easiness” of B to prove the

“easiness” of A

Recalling that NP-completeness is about showing how hard a problem is ratherthan how easy it is, we use polynomial-time reductions in the opposite way to showthat a problem is NP-complete Let us take the idea a step further, and show how wecould use polynomial-time reductions to show that no polynomial-time algorithmcan exist for a particular problem B Suppose we have a decision problem A forwhich we already know that no polynomial-time algorithm can exist (Let us notconcern ourselves for now with how to find such a problem A.) Suppose furtherthat we have a polynomial-time reduction transforming instances of A to instances

of B Now we can use a simple proof by contradiction to show that no time algorithm can exist for B Suppose otherwise; i.e., suppose that B has apolynomial-time algorithm Then, using the method shown in Figure 34.1, wewould have a way to solve problem A in polynomial time, which contradicts ourassumption that there is no polynomial-time algorithm for A

For NP-completeness, we cannot assume that there is absolutely no time algorithm for problem A The proof methodology is similar, however, in that

polynomial-we prove that problem B is NP-complete on the assumption that problem A is alsoNP-complete

A first NP-complete problem

Because the technique of reduction relies on having a problem already known to

be NP-complete in order to prove a different problem NP-complete, we need a

“first” NP-complete problem The problem we shall use is the circuit-satisfiabilityproblem, in which we are given a boolean combinational circuit composed of AND,

OR, and NOT gates, and we wish to know whether there exists some set of booleaninputs to this circuit that causes its output to be 1 We shall prove that this firstproblem is NP-complete in Section 34.3

Chapter outline

This chapter studies the aspects of NP-completeness that bear most directly on theanalysis of algorithms In Section 34.1, we formalize our notion of “problem” anddefine the complexity class P of polynomial-time solvable decision problems Wealso see how these notions fit into the framework of formal-language theory Sec-tion 34.2 defines the class NP of decision problems whose solutions are verifiable

in polynomial time It also formally poses the P ¤ NP question

Section 34.3 shows we can relate problems via polynomial-time “reductions.”

It defines NP-completeness and sketches a proof that one problem, called “circuitsatisfiability,” is NP-complete Having found one NP-complete problem, we show

in Section 34.4 how to prove other problems to be NP-complete much more simply

by the methodology of reductions We illustrate this methodology by showing thattwo formula-satisfiability problems are NP-complete With additional reductions,

we show in Section 34.5 a variety of other problems to be NP-complete

We begin our study of NP-completeness by formalizing our notion of time solvable problems We generally regard these problems as tractable, but forphilosophical, not mathematical, reasons We can offer three supporting argu-ments

polynomial-First, although we may reasonably regard a problem that requires time ‚.n100/

to be intractable, very few practical problems require time on the order of such ahigh-degree polynomial The polynomial-time computable problems encountered

in practice typically require much less time Experience has shown that once thefirst polynomial-time algorithm for a problem has been discovered, more efficientalgorithms often follow Even if the current best algorithm for a problem has arunning time of ‚.n100/, an algorithm with a much better running time will likelysoon be discovered

Second, for many reasonable models of computation, a problem that can besolved in polynomial time in one model can be solved in polynomial time in an-other For example, the class of problems solvable in polynomial time by the serialrandom-access machine used throughout most of this book is the same as the class

of problems solvable in polynomial time on abstract Turing machines.1 It is alsothe same as the class of problems solvable in polynomial time on a parallel com-puter when the number of processors grows polynomially with the input size.Third, the class of polynomial-time solvable problems has nice closure proper-ties, since polynomials are closed under addition, multiplication, and composition.For example, if the output of one polynomial-time algorithm is fed into the input ofanother, the composite algorithm is polynomial Exercise 34.1-5 asks you to showthat if an algorithm makes a constant number of calls to polynomial-time subrou-tines and performs an additional amount of work that also takes polynomial time,then the running time of the composite algorithm is polynomial

Abstract problems

To understand the class of polynomial-time solvable problems, we must first have

a formal notion of what a “problem” is We define an abstract problem Q to be a binary relation on a set I of problem instances and a set S of problem solutions.

For example, an instance for SHORTEST-PATH is a triple consisting of a graphand two vertices A solution is a sequence of vertices in the graph, with perhapsthe empty sequence denoting that no path exists The problem SHORTEST-PATHitself is the relation that associates each instance of a graph and two vertices with

a shortest path in the graph that connects the two vertices Since shortest paths arenot necessarily unique, a given problem instance may have more than one solution.This formulation of an abstract problem is more general than we need for ourpurposes As we saw above, the theory of NP-completeness restricts attention to

decision problems: those having a yes/no solution In this case, we can view an

abstract decision problem as a function that maps the instance set I to the solutionset f0; 1g For example, a decision problem related to SHORTEST-PATH is theproblem PATH that we saw earlier If i D hG; u; ; ki is an instance of the decisionproblem PATH, then PATH.i / D 1 (yes) if a shortest path from u to  has atmost k edges, and PATH.i / D 0 (no) otherwise Many abstract problems are not

decision problems, but rather optimization problems, which require some value to

be minimized or maximized As we saw above, however, we can usually recast anoptimization problem as a decision problem that is no harder

1 See Hopcroft and Ullman [180] or Lewis and Papadimitriou [236] for a thorough treatment of the Turing-machine model.

In order for a computer program to solve an abstract problem, we must represent

problem instances in a way that the program understands An encoding of a set S

of abstract objects is a mapping e from S to the set of binary strings.2 For example,

we are all familiar with encoding the natural numbersN D f0; 1; 2; 3; 4; : : :g asthe strings f0; 1; 10; 11; 100; : : :g Using this encoding, e.17/ D 10001 If youhave looked at computer representations of keyboard characters, you probably haveseen the ASCII code, where, for example, the encoding of A is 1000001 We canencode a compound object as a binary string by combining the representations ofits constituent parts Polygons, graphs, functions, ordered pairs, programs—all can

be encoded as binary strings

Thus, a computer algorithm that “solves” some abstract decision problem ally takes an encoding of a problem instance as input We call a problem whose

actu-instance set is the set of binary strings a concrete problem We say that an rithm solves a concrete problem in time O.T n// if, when it is provided a problem

algo-instance i of length n D ji j, the algorithm can produce the solution in O.T n//time.3 A concrete problem is polynomial-time solvable, therefore, if there exists

an algorithm to solve it in time O.nk/ for some constant k

We can now formally define the complexity class P as the set of concrete

deci-sion problems that are polynomial-time solvable

We can use encodings to map abstract problems to concrete problems Given

an abstract decision problem Q mapping an instance set I tof0; 1g, an encoding

e W I ! f0; 1gcan induce a related concrete decision problem, which we denote

by e.Q/.4 If the solution to an abstract-problem instance i 2 I is Q.i / 2 f0; 1g,then the solution to the concrete-problem instance e.i / 2f0; 1gis also Q.i / As atechnicality, some binary strings might represent no meaningful abstract-probleminstance For convenience, we shall assume that any such string maps arbitrarily

to 0 Thus, the concrete problem produces the same solutions as the abstract lem on binary-string instances that represent the encodings of abstract-probleminstances

prob-We would like to extend the definition of polynomial-time solvability from crete problems to abstract problems by using encodings as the bridge, but we would

con-2The codomain of e need not be binary strings; any set of strings over a finite alphabet having at

least 2 symbols will do.

3 We assume that the algorithm’s output is separate from its input Because it takes at least one time step to produce each bit of the output and the algorithm takes O.T n// time steps, the size of the output is O.T n//.

4 We denote by f0; 1gthe set of all strings composed of symbols from the set f0; 1g.

like the definition to be independent of any particular encoding That is, the ficiency of solving a problem should not depend on how the problem is encoded.Unfortunately, it depends quite heavily on the encoding For example, suppose that

ef-an integer k is to be provided as the sole input to ef-an algorithm, ef-and suppose that

the running time of the algorithm is ‚.k/ If the integer k is provided in unary—a

string of k 1s—then the running time of the algorithm is O.n/ on length-n inputs,which is polynomial time If we use the more natural binary representation of theinteger k, however, then the input length is n Dblg kc C 1 In this case, the run-ning time of the algorithm is ‚.k/ D ‚.2n/, which is exponential in the size of theinput Thus, depending on the encoding, the algorithm runs in either polynomial

or superpolynomial time

How we encode an abstract problem matters quite a bit to how we understandpolynomial time We cannot really talk about solving an abstract problem withoutfirst specifying an encoding Nevertheless, in practice, if we rule out “expensive”encodings such as unary ones, the actual encoding of a problem makes little dif-ference to whether the problem can be solved in polynomial time For example,representing integers in base 3 instead of binary has no effect on whether a prob-lem is solvable in polynomial time, since we can convert an integer represented inbase 3 to an integer represented in base 2 in polynomial time

We say that a function f W f0; 1g ! f0; 1gis polynomial-time computable

if there exists a polynomial-time algorithm A that, given any input x 2 f0; 1g,produces as output f x/ For some set I of problem instances, we say that two en-codings e1and e2are polynomially related if there exist two polynomial-time com-

putable functions f12and f21such that for any i 2 I , we have f12.e1.i // D e2.i /and f21.e2.i // D e1.i /.5That is, a polynomial-time algorithm can compute the en-coding e2.i / from the encoding e1.i /, and vice versa If two encodings e1and e2of

an abstract problem are polynomially related, whether the problem is time solvable or not is independent of which encoding we use, as the followinglemma shows

polynomial-Lemma 34.1

Let Q be an abstract decision problem on an instance set I , and let e1 and e2 bepolynomially related encodings on I Then, e1.Q/ 2 P if and only if e2.Q/ 2 P

5 Technically, we also require the functions f 12 and f 21 to “map noninstances to noninstances.”

A noninstance of an encoding e is a string x 2f0; 1gsuch that there is no instance i for which e.i / D x We require that f 12 x/ D y for every noninstance x of encoding e 1 , where y is some non- instance of e2, and that f21.x0/ D y0for every noninstance x0of e2, where y0is some noninstance

of e1.

Proof We need only prove the forward direction, since the backward direction issymmetric Suppose, therefore, that e1.Q/ can be solved in time O.nk/ for someconstant k Further, suppose that for any problem instance i , the encoding e1.i /can be computed from the encoding e2.i / in time O.nc/ for some constant c, where

n D je2.i /j To solve problem e2.Q/, on input e2.i /, we first compute e1.i / andthen run the algorithm for e1.Q/ on e1.i / How long does this take? Convertingencodings takes time O.nc/, and therefore je1.i /j D O.nc/, since the output of

a serial computer cannot be longer than its running time Solving the problem

on e1.i / takes time O.je1.i /jk/ D O.nck/, which is polynomial since both c and kare constants

Thus, whether an abstract problem has its instances encoded in binary or base 3does not affect its “complexity,” that is, whether it is polynomial-time solvable ornot; but if instances are encoded in unary, its complexity may change In order to

be able to converse in an encoding-independent fashion, we shall generally assumethat problem instances are encoded in any reasonable, concise fashion, unless wespecifically say otherwise To be precise, we shall assume that the encoding of aninteger is polynomially related to its binary representation, and that the encoding of

a finite set is polynomially related to its encoding as a list of its elements, enclosed

in braces and separated by commas (ASCII is one such encoding scheme.) Withsuch a “standard” encoding in hand, we can derive reasonable encodings of othermathematical objects, such as tuples, graphs, and formulas To denote the standardencoding of an object, we shall enclose the object in angle braces Thus, hGidenotes the standard encoding of a graph G

As long as we implicitly use an encoding that is polynomially related to thisstandard encoding, we can talk directly about abstract problems without reference

to any particular encoding, knowing that the choice of encoding has no effect onwhether the abstract problem is polynomial-time solvable Henceforth, we shallgenerally assume that all problem instances are binary strings encoded using thestandard encoding, unless we explicitly specify the contrary We shall also typicallyneglect the distinction between abstract and concrete problems You should watchout for problems that arise in practice, however, in which a standard encoding isnot obvious and the encoding does make a difference

A formal-language framework

By focusing on decision problems, we can take advantage of the machinery offormal-language theory Let’s review some definitions from that theory An

alphabet † is a finite set of symbols A language L over † is any set of

strings made up of symbols from † For example, if † D f0; 1g, the set

L D f10; 11; 101; 111; 1011; 1101; 10001; : : :g is the language of binary

represen-tations of prime numbers We denote the empty string by ", the empty language

by ;, and the language of all strings over † by † For example, if † D f0; 1g,then † D f"; 0; 1; 00; 01; 10; 11; 000; : : :g is the set of all binary strings Everylanguage L over † is a subset of †

We can perform a variety of operations on languages Set-theoretic operations,

such as union and intersection, follow directly from the set-theoretic definitions.

We define the complement of L by L D † L The concatenation L1L2of twolanguages L1and L2is the language

L D fx1x2 W x12 L1and x2 2 L2g :

The closure or Kleene star of a language L is the language

LD f"g [ L [ L2[ L3[    ;

where Lk is the language obtained by concatenating L to itself k times

From the point of view of language theory, the set of instances for any decisionproblem Q is simply the set †, where † Df0; 1g Since Q is entirely character-ized by those problem instances that produce a 1 (yes) answer, we can view Q as

a language L over † Df0; 1g, where

L D fx 2 †W Q.x/ D 1g :

For example, the decision problem PATH has the corresponding language

PATH D fhG; u; ; ki W G D V; E/ is an undirected graph,

u;  2 V;

k  0 is an integer, andthere exists a path from u to  in Gconsisting of at most k edgesg :(Where convenient, we shall sometimes use the same name—PATH in this case—

to refer to both a decision problem and its corresponding language.)

The formal-language framework allows us to express concisely the relation tween decision problems and algorithms that solve them We say that an al-

be-gorithm A accepts a string x 2 f0; 1g if, given input x, the algorithm’s

out-put A.x/ is 1 The language accepted by an algorithm A is the set of strings

L D fx 2 f0; 1gW A.x/ D 1g, that is, the set of strings that the algorithm accepts

An algorithm A rejects a string x if A.x/ D 0.

Even if language L is accepted by an algorithm A, the algorithm will not sarily reject a string x 62 L provided as input to it For example, the algorithm may

neces-loop forever A language L is decided by an algorithm A if every binary string

in L is accepted by A and every binary string not in L is rejected by A A

lan-guage L is accepted in polynomial time by an algorithm A if it is accepted by A

and if in addition there exists a constant k such that for any length-n string x 2 L,

algorithm A accepts x in time O.nk/ A language L is decided in polynomial

time by an algorithm A if there exists a constant k such that for any length-n string

x 2 f0; 1g, the algorithm correctly decides whether x 2 L in time O.nk/ Thus,

to accept a language, an algorithm need only produce an answer when provided astring in L, but to decide a language, it must correctly accept or reject every string

inf0; 1g

As an example, the language PATH can be accepted in polynomial time Onepolynomial-time accepting algorithm verifies that G encodes an undirected graph,verifies that u and  are vertices in G, uses breadth-first search to compute a short-est path from u to  in G, and then compares the number of edges on the shortestpath obtained with k If G encodes an undirected graph and the path found from u

to  has at most k edges, the algorithm outputs 1 and halts Otherwise, the rithm runs forever This algorithm does not decide PATH, however, since it doesnot explicitly output 0 for instances in which a shortest path has more than k edges

algo-A decision algorithm for Palgo-ATH must explicitly reject binary strings that do not long to PATH For a decision problem such as PATH, such a decision algorithm iseasy to design: instead of running forever when there is not a path from u to  with

be-at most k edges, it outputs 0 and halts (It must also output 0 and halt if the inputencoding is faulty.) For other problems, such as Turing’s Halting Problem, thereexists an accepting algorithm, but no decision algorithm exists

We can informally define a complexity class as a set of languages, membership

in which is determined by a complexity measure, such as running time, of an

algorithm that determines whether a given string x belongs to language L Theactual definition of a complexity class is somewhat more technical.6

Using this language-theoretic framework, we can provide an alternative tion of the complexity class P:

defini-P D fL f0; 1gW there exists an algorithm A that decides L

in polynomial timeg :

In fact, P is also the class of languages that can be accepted in polynomial time

Theorem 34.2

P DfL W L is accepted by a polynomial-time algorithmg :

Proof Because the class of languages decided by polynomial-time algorithms is

a subset of the class of languages accepted by polynomial-time algorithms, weneed only show that if L is accepted by a polynomial-time algorithm, it is de-cided by a polynomial-time algorithm Let L be the language accepted by some

6 For more on complexity classes, see the seminal paper by Hartmanis and Stearns [162].

polynomial-time algorithm A We shall use a classic “simulation” argument toconstruct another polynomial-time algorithm A0 that decides L Because A ac-cepts L in time O.nk/ for some constant k, there also exists a constant c suchthat A accepts L in at most cnk steps For any input string x, the algorithm A0simulates cnksteps of A After simulating cnksteps, algorithm A0inspects the be-havior of A If A has accepted x, then A0accepts x by outputting a 1 If A has notaccepted x, then A0 rejects x by outputting a 0 The overhead of A0simulating Adoes not increase the running time by more than a polynomial factor, and thus A0

is a polynomial-time algorithm that decides L

Note that the proof of Theorem 34.2 is nonconstructive For a given language

L 2 P, we may not actually know a bound on the running time for the algorithm Athat accepts L Nevertheless, we know that such a bound exists, and therefore, that

an algorithm A0 exists that can check the bound, even though we may not be able

to find the algorithm A0easily

Exercises

34.1-1

Define the optimization problem LONGEST-PATH-LENGTH as the relation thatassociates each instance of an undirected graph and two vertices with the num-ber of edges in a longest simple path between the two vertices Define the de-cision problem LONGEST-PATH D fhG; u; ; ki W G D V; E/ is an undi-rected graph, u;  2 V , k  0 is an integer, and there exists a simple pathfrom u to  in G consisting of at least k edgesg Show that the optimization prob-lem LONGEST-PATH-LENGTH can be solved in polynomial time if and only ifLONGEST-PATH 2 P

34.1-2

Give a formal definition for the problem of finding the longest simple cycle in anundirected graph Give a related decision problem Give the language correspond-ing to the decision problem

34.1-3

Give a formal encoding of directed graphs as binary strings using an matrix representation Do the same using an adjacency-list representation Arguethat the two representations are polynomially related

adjacency-34.1-4

Is the dynamic-programming algorithm for the 0-1 knapsack problem that is askedfor in Exercise 16.2-2 a polynomial-time algorithm? Explain your answer

Show that if an algorithm makes at most a constant number of calls to time subroutines and performs an additional amount of work that also takes polyno-mial time, then it runs in polynomial time Also show that a polynomial number ofcalls to polynomial-time subroutines may result in an exponential-time algorithm

we can solve PATH in linear time—and so verifying membership from a givencertificate takes as long as solving the problem from scratch We shall now examine

a problem for which we know of no polynomial-time decision algorithm and yet,given a certificate, verification is easy

Hamiltonian cycles

The problem of finding a hamiltonian cycle in an undirected graph has been

stud-ied for over a hundred years Formally, a hamiltonian cycle of an undirected graph

G D V; E/ is a simple cycle that contains each vertex in V A graph that

con-tains a hamiltonian cycle is said to be hamiltonian; otherwise, it is

nonhamilto-nian The name honors W R Hamilton, who described a mathematical game on

the dodecahedron (Figure 34.2(a)) in which one player sticks five pins in any fiveconsecutive vertices and the other player must complete the path to form a cycle

(a) (b)

Figure 34.2 (a) A graph representing the vertices, edges, and faces of a dodecahedron, with a

hamiltonian cycle shown by shaded edges (b) A bipartite graph with an odd number of vertices.

Any such graph is nonhamiltonian.

containing all the vertices.7 The dodecahedron is hamiltonian, and Figure 34.2(a)shows one hamiltonian cycle Not all graphs are hamiltonian, however For ex-ample, Figure 34.2(b) shows a bipartite graph with an odd number of vertices.Exercise 34.2-2 asks you to show that all such graphs are nonhamiltonian

We can define the hamiltonian-cycle problem, “Does a graph G have a

hamil-tonian cycle?” as a formal language:

HAM-CYCLE DfhGi W G is a hamiltonian graphg :

How might an algorithm decide the language HAM-CYCLE? Given a probleminstance hGi, one possible decision algorithm lists all permutations of the vertices

of G and then checks each permutation to see if it is a hamiltonian path What isthe running time of this algorithm? If we use the “reasonable” encoding of a graph

as its adjacency matrix, the number m of vertices in the graph is .pn/, where

n D jhGij is the length of the encoding of G There are mŠ possible permutations

7 In a letter dated 17 October 1856 to his friend John T Graves, Hamilton [157, p 624] wrote, “I have found that some young persons have been much amused by trying a new mathematical game which the Icosion furnishes, one person sticking five pins in any five consecutive points and the other player then aiming to insert, which by the theory in this letter can always be done, fifteen other pins, in cyclical succession, so as to cover all the other points, and to end in immediate proximity to the pin wherewith his antagonist had begun.”

of the vertices, and therefore the running time is .mŠ/ D .pn Š/ D .2pn/,which is not O.nk/ for any constant k Thus, this naive algorithm does not run

in polynomial time In fact, the hamiltonian-cycle problem is NP-complete, as weshall prove in Section 34.5

Verification algorithms

Consider a slightly easier problem Suppose that a friend tells you that a givengraph G is hamiltonian, and then offers to prove it by giving you the vertices inorder along the hamiltonian cycle It would certainly be easy enough to verify theproof: simply verify that the provided cycle is hamiltonian by checking whether

it is a permutation of the vertices of V and whether each of the consecutive edgesalong the cycle actually exists in the graph You could certainly implement thisverification algorithm to run in O.n2/ time, where n is the length of the encoding

of G Thus, a proof that a hamiltonian cycle exists in a graph can be verified inpolynomial time

We define a verification algorithm as being a two-argument algorithm A, where

one argument is an ordinary input string x and the other is a binary string y called

a certificate A two-argument algorithm A verifies an input string x if there exists

a certificate y such that A.x; y/ D 1 The language verified by a verification

algorithm A is

L D fx 2 f0; 1gW there exists y 2 f0; 1gsuch that A.x; y/ D 1g :

Intuitively, an algorithm A verifies a language L if for any string x 2 L, thereexists a certificate y that A can use to prove that x 2 L Moreover, for any string

x 62 L, there must be no certificate proving that x 2 L For example, in thehamiltonian-cycle problem, the certificate is the list of vertices in some hamilto-nian cycle If a graph is hamiltonian, the hamiltonian cycle itself offers enoughinformation to verify this fact Conversely, if a graph is not hamiltonian, therecan be no list of vertices that fools the verification algorithm into believing that thegraph is hamiltonian, since the verification algorithm carefully checks the proposed

“cycle” to be sure

The complexity class NP

The complexity class NP is the class of languages that can be verified by a

poly-nomial-time algorithm.8 More precisely, a language L belongs to NP if and only ifthere exist a two-input polynomial-time algorithm A and a constant c such that

L D fx 2 f0; 1gW there exists a certificate y with jyj D O.jxjc/

such that A.x; y/ D 1g :

We say that algorithm A verifies language L in polynomial time.

From our earlier discussion on the hamiltonian-cycle problem, we now see thatHAM-CYCLE 2 NP (It is always nice to know that an important set is nonempty.)Moreover, if L 2 P, then L 2 NP, since if there is a polynomial-time algorithm

to decide L, the algorithm can be easily converted to a two-argument verificationalgorithm that simply ignores any certificate and accepts exactly those input strings

it determines to be in L Thus, P  NP

It is unknown whether P D NP, but most researchers believe that P and NP arenot the same class Intuitively, the class P consists of problems that can be solvedquickly The class NP consists of problems for which a solution can be verifiedquickly You may have learned from experience that it is often more difficult tosolve a problem from scratch than to verify a clearly presented solution, especiallywhen working under time constraints Theoretical computer scientists generallybelieve that this analogy extends to the classes P and NP, and thus that NP includeslanguages that are not in P

There is more compelling, though not conclusive, evidence that P ¤ NP—theexistence of languages that are “NP-complete.” We shall study this class in Sec-tion 34.3

Many other fundamental questions beyond the P ¤ NP question remain solved Figure 34.3 shows some possible scenarios Despite much work by manyresearchers, no one even knows whether the class NP is closed under comple-

unre-ment That is, does L 2 NP imply L 2 NP? We can define the complexity class

co-NP as the set of languages L such that L 2 NP We can restate the question

of whether NP is closed under complement as whether NP D co-NP Since P isclosed under complement (Exercise 34.1-6), it follows from Exercise 34.2-9 that

P  NP \ co-NP Once again, however, no one knows whether P D NP \ co-NP

or whether there is some language in NP \ co-NP  P

8 The name “NP” stands for “nondeterministic polynomial time.” The class NP was originally studied

in the context of nondeterminism, but this book uses the somewhat simpler yet equivalent notion of verification Hopcroft and Ullman [180] give a good presentation of NP-completeness in terms of nondeterministic models of computation.

co-NP NP

(c)

(d) P

P = NP = co-NP

(a)

NP = co-NP

(b) P

NP ∩ co-NP

Figure 34.3 Four possibilities for relationships among complexity classes In each diagram, one

region enclosing another indicates a proper-subset relation (a) P D NP D co-NP Most researchers regard this possibility as the most unlikely (b) If NP is closed under complement, then NP D co-NP, but it need not be the case that P D NP (c) P D NP\co-NP, but NP is not closed under complement.

(d) NP ¤ co-NP and P ¤ NP \ co-NP Most researchers regard this possibility as the most likely.

Thus, our understanding of the precise relationship between P and NP is fully incomplete Nevertheless, even though we might not be able to prove that aparticular problem is intractable, if we can prove that it is NP-complete, then wehave gained valuable information about it

woe-Exercises

34.2-1

Consider the language GRAPH-ISOMORPHISM D fhG1; G2i W G1and G2 areisomorphic graphsg Prove that GRAPH-ISOMORPHISM 2 NP by describing apolynomial-time algorithm to verify the language

A hamiltonian path in a graph is a simple path that visits every vertex exactly

once Show that the language HAM-PATH D fhG; u; i W there is a hamiltonianpath from u to  in graph Gg belongs to NP

34.2-7

Show that the hamiltonian-path problem from Exercise 34.2-6 can be solved inpolynomial time on directed acyclic graphs Give an efficient algorithm for theproblem

34.2-8

Let  be a boolean formula constructed from the boolean input variables x1; x2;: : : ; xk, negations (:), ANDs (^), ORs (_), and parentheses The formula  is a

tautology if it evaluates to 1 for every assignment of 1 and 0 to the input variables.

Define TAUTOLOGY as the language of boolean formulas that are tautologies.Show that TAUTOLOGY 2 co-NP

of length at most 3 Prove that G3is hamiltonian (Hint: Construct a spanning tree

for G, and use an inductive argument.)

34.3 NP-completeness and reducibility

Perhaps the most compelling reason why theoretical computer scientists believethat P ¤ NP comes from the existence of the class of “NP-complete” problems

This class has the intriguing property that if any NP-complete problem can be solved in polynomial time, then every problem in NP has a polynomial-time solu-

tion, that is, P D NP Despite years of study, though, no polynomial-time algorithmhas ever been discovered for any NP-complete problem

The language HAM-CYCLE is one NP-complete problem If we could decideHAM-CYCLE in polynomial time, then we could solve every problem in NP inpolynomial time In fact, if NP  P should turn out to be nonempty, we could saywith certainty that HAM-CYCLE 2 NP  P

The NP-complete languages are, in a sense, the “hardest” languages in NP Inthis section, we shall show how to compare the relative “hardness” of languagesusing a precise notion called “polynomial-time reducibility.” Then we formallydefine the NP-complete languages, and we finish by sketching a proof that onesuch language, called CIRCUIT-SAT, is NP-complete In Sections 34.4 and 34.5,

we shall use the notion of reducibility to show that many other problems are complete

NP-Reducibility

Intuitively, a problem Q can be reduced to another problem Q0if any instance of Qcan be “easily rephrased” as an instance of Q0, the solution to which provides asolution to the instance of Q For example, the problem of solving linear equations

in an indeterminate x reduces to the problem of solving quadratic equations Given

an instance ax C b D 0, we transform it to 0x2 C ax C b D 0, whose solutionprovides a solution to ax C b D 0 Thus, if a problem Q reduces to anotherproblem Q0, then Q is, in a sense, “no harder to solve” than Q0

Returning to our formal-language framework for decision problems, we say that

a language L1 is polynomial-time reducible to a language L2, written L1 P L2,

if there exists a polynomial-time computable function f Wf0; 1g ! f0; 1gsuchthat for all x 2f0; 1g,

then f x/ 2 L2 Moreover, if x 62 L1, then f x/ 62 L2 Thus, the reduction tion maps any instance x of the decision problem represented by the language L1

func-to an instance f x/ of the problem represented by L2 Providing an answer towhether f x/ 2 L2directly provides the answer to whether x 2 L1

Polynomial-time reductions give us a powerful tool for proving that various guages belong to P

Figure 34.5 illustrates how we construct A1 For a given input x 2 f0; 1g,algorithm A1uses F to transform x into f x/, and then it uses A2to test whether

f x/ 2 L2 Algorithm A1 takes the output from algorithm A2 and produces thatanswer as its own output

The correctness of A1follows from condition (34.1) The algorithm runs in nomial time, since both F and A2run in polynomial time (see Exercise 34.1-5)

why the “less than or equal to” notation for reduction is mnemonic We can nowdefine the set of NP-complete languages, which are the hardest problems in NP.

A language L f0; 1gis NP-complete if

1 L 2 NP, and

2 L0 PL for every L02 NP

If a language L satisfies property 2, but not necessarily property 1, we say that L

is NP-hard We also define NPC to be the class of NP-complete languages.

As the following theorem shows, NP-completeness is at the crux of decidingwhether P is in fact equal to NP

Theorem 34.4

If any NP-complete problem is polynomial-time solvable, then P D NP lently, if any problem in NP is not polynomial-time solvable, then no NP-completeproblem is polynomial-time solvable

Equiva-Proof Suppose that L 2 P and also that L 2 NPC For any L0 2 NP, wehave L0 P L by property 2 of the definition of NP-completeness Thus, byLemma 34.3, we also have that L0 2 P, which proves the first statement of thetheorem

To prove the second statement, note that it is the contrapositive of the first ment

state-It is for this reason that research into the P ¤ NP question centers around theNP-complete problems Most theoretical computer scientists believe that P ¤ NP,which leads to the relationships among P, NP, and NPC shown in Figure 34.6.But, for all we know, someone may yet come up with a polynomial-time algo-rithm for an NP-complete problem, thus proving that P D NP Nevertheless, since

no polynomial-time algorithm for any NP-complete problem has yet been

discov-NPC P

Unfortunately, the formal proof that the circuit-satisfiability problem is complete requires technical detail beyond the scope of this text Instead, we shallinformally describe a proof that relies on a basic understanding of boolean combi-national circuits

NP-Boolean combinational circuits are built from boolean combinational elements

that are interconnected by wires A boolean combinational element is any circuit

element that has a constant number of boolean inputs and outputs and that performs

a well-defined function Boolean values are drawn from the set f0; 1g, where 0representsFALSEand 1 representsTRUE

The boolean combinational elements that we use in the circuit-satisfiability

prob-lem compute simple boolean functions, and they are known as logic gates

Fig-ure 34.7 shows the three basic logic gates that we use in the circuit-satisfiability

problem: the NOT gate (or inverter), the AND gate, and the OR gate The NOT gate takes a single binary input x, whose value is either 0 or 1, and produces a binary output ´ whose value is opposite that of the input value Each of the other

two gates takes two binary inputs x and y and produces a single binary output ´

We can describe the operation of each gate, and of any boolean combinational

element, by a truth table, shown under each gate in Figure 34.7 A truth table gives

the outputs of the combinational element for each possible setting of the inputs For

Figure 34.7 Three basic logic gates, with binary inputs and outputs Under each gate is the truth

table that describes the gate’s operation (a) The NOT gate (b) The AND gate (c) The OR gate.

example, the truth table for the OR gate tells us that when the inputs are x D 0and y D 1, the output value is ´ D 1 We use the symbols : to denote the NOTfunction, ^ to denote the AND function, and _ to denote the OR function Thus,for example, 0 _ 1 D 1

We can generalize AND and OR gates to take more than two inputs An ANDgate’s output is 1 if all of its inputs are 1, and its output is 0 otherwise An OR gate’soutput is 1 if any of its inputs are 1, and its output is 0 otherwise

A boolean combinational circuit consists of one or more boolean combinational elements interconnected by wires A wire can connect the output of one element

to the input of another, thereby providing the output value of the first element as aninput value of the second Figure 34.8 shows two similar boolean combinationalcircuits, differing in only one gate Part (a) of the figure also shows the values onthe individual wires, given the input hx1 D 1; x2D 1; x3 D 0i Although a singlewire may have no more than one combinational-element output connected to it, itcan feed several element inputs The number of element inputs fed by a wire is

called the fan-out of the wire If no element output is connected to a wire, the wire

is a circuit input, accepting input values from an external source If no element input is connected to a wire, the wire is a circuit output, providing the results of

the circuit’s computation to the outside world (An internal wire can also fan out

to a circuit output.) For the purpose of defining the circuit-satisfiability problem,

we limit the number of circuit outputs to 1, though in actual hardware design, aboolean combinational circuit may have multiple outputs

Boolean combinational circuits contain no cycles In other words, suppose wecreate a directed graph G D V; E/ with one vertex for each combinational elementand with k directed edges for each wire whose fan-out is k; the graph contains

a directed edge u; / if a wire connects the output of element u to an input ofelement  Then G must be acyclic

1 1

1

1 0

Figure 34.8 Two instances of the circuit-satisfiability problem (a) The assignment hx1 D 1;

x 2 D 1; x 3 D 0i to the inputs of this circuit causes the output of the circuit to be 1 The circuit

is therefore satisfiable (b) No assignment to the inputs of this circuit can cause the output of the

circuit to be 1 The circuit is therefore unsatisfiable.

A truth assignment for a boolean combinational circuit is a set of boolean input values We say that a one-output boolean combinational circuit is satisfiable if it has a satisfying assignment: a truth assignment that causes the output of the circuit

to be 1 For example, the circuit in Figure 34.8(a) has the satisfying assignment

hx1 D 1; x2 D 1; x3D 0i, and so it is satisfiable As Exercise 34.3-1 asks you toshow, no assignment of values to x1, x2, and x3causes the circuit in Figure 34.8(b)

to produce a 1 output; it always produces 0, and so it is unsatisfiable

The circuit-satisfiability problem is, “Given a boolean combinational circuit

composed of AND, OR, and NOT gates, is it satisfiable?” In order to pose thisquestion formally, however, we must agree on a standard encoding for circuits

The size of a boolean combinational circuit is the number of boolean

combina-tional elements plus the number of wires in the circuit We could devise a graphlikeencoding that maps any given circuit C into a binary string hC i whose length ispolynomial in the size of the circuit itself As a formal language, we can thereforedefine

CIRCUIT-SAT DfhC i W C is a satisfiable boolean combinational circuitg :The circuit-satisfiability problem arises in the area of computer-aided hardwareoptimization If a subcircuit always produces 0, that subcircuit is unnecessary;the designer can replace it by a simpler subcircuit that omits all logic gates andprovides the constant 0 value as its output You can see why we would like to have

a polynomial-time algorithm for this problem

Given a circuit C , we might attempt to determine whether it is satisfiable bysimply checking all possible assignments to the inputs Unfortunately, if the circuithas k inputs, then we would have to check up to 2k possible assignments When

the size of C is polynomial in k, checking each one takes .2k/ time, which issuperpolynomial in the size of the circuit.9 In fact, as we have claimed, there isstrong evidence that no polynomial-time algorithm exists that solves the circuit-satisfiability problem because circuit satisfiability is NP-complete We break theproof of this fact into two parts, based on the two parts of the definition of NP-completeness.

Lemma 34.5

The circuit-satisfiability problem belongs to the class NP

Proof We shall provide a two-input, polynomial-time algorithm A that can verifyCIRCUIT-SAT One of the inputs to A is (a standard encoding of) a boolean com-binational circuit C The other input is a certificate corresponding to an assignment

of boolean values to the wires in C (See Exercise 34.3-4 for a smaller certificate.)

We construct the algorithm A as follows For each logic gate in the circuit, itchecks that the value provided by the certificate on the output wire is correctlycomputed as a function of the values on the input wires Then, if the output of theentire circuit is 1, the algorithm outputs 1, since the values assigned to the inputs

of C provide a satisfying assignment Otherwise, A outputs 0

Whenever a satisfiable circuit C is input to algorithm A, there exists a certificatewhose length is polynomial in the size of C and that causes A to output a 1 When-ever an unsatisfiable circuit is input, no certificate can fool A into believing thatthe circuit is satisfiable Algorithm A runs in polynomial time: with a good imple-mentation, linear time suffices Thus, we can verify CIRCUIT-SAT in polynomialtime, and CIRCUIT-SAT 2 NP

The second part of proving that CIRCUIT-SAT is NP-complete is to show thatthe language is NP-hard That is, we must show that every language in NP ispolynomial-time reducible to CIRCUIT-SAT The actual proof of this fact is full

of technical intricacies, and so we shall settle for a sketch of the proof based onsome understanding of the workings of computer hardware

A computer program is stored in the computer memory as a sequence of structions A typical instruction encodes an operation to be performed, addresses

in-of operands in memory, and an address where the result is to be stored A

spe-cial memory location, called the program counter, keeps track of which

instruc-9 On the other hand, if the size of the circuit C is ‚.2k/, then an algorithm whose running time

is O.2k/ has a running time that is polynomial in the circuit size Even if P ¤ NP, this tion would not contradict the NP-completeness of the problem; the existence of a polynomial-time algorithm for a special case does not imply that there is a polynomial-time algorithm for all cases.

situa-tion is to be executed next The program counter automatically increments uponfetching each instruction, thereby causing the computer to execute instructions se-quentially The execution of an instruction can cause a value to be written to theprogram counter, however, which alters the normal sequential execution and allowsthe computer to loop and perform conditional branches.

At any point during the execution of a program, the computer’s memory holdsthe entire state of the computation (We take the memory to include the programitself, the program counter, working storage, and any of the various bits of statethat a computer maintains for bookkeeping.) We call any particular state of com-

puter memory a configuration We can view the execution of an instruction as

mapping one configuration to another The computer hardware that accomplishesthis mapping can be implemented as a boolean combinational circuit, which wedenote by M in the proof of the following lemma

Lemma 34.6

The circuit-satisfiability problem is NP-hard

Proof Let L be any language in NP We shall describe a polynomial-time rithm F computing a reduction function f that maps every binary string x to acircuit C D f x/ such that x 2 L if and only if C 2 CIRCUIT-SAT

algo-Since L 2 NP, there must exist an algorithm A that verifies L in polynomialtime The algorithm F that we shall construct uses the two-input algorithm A tocompute the reduction function f

Let T n/ denote the worst-case running time of algorithm A on length-n inputstrings, and let k  1 be a constant such that T n/ D O.nk/ and the length of thecertificate is O.nk/ (The running time of A is actually a polynomial in the totalinput size, which includes both an input string and a certificate, but since the length

of the certificate is polynomial in the length n of the input string, the running time

is polynomial in n.)

The basic idea of the proof is to represent the computation of A as a sequence

of configurations As Figure 34.9 illustrates, we can break each configuration intoparts consisting of the program for A, the program counter and auxiliary machinestate, the input x, the certificate y, and working storage The combinational cir-cuit M , which implements the computer hardware, maps each configuration ci tothe next configuration ci C1, starting from the initial configuration c0 Algorithm Awrites its output—0 or 1—to some designated location by the time it finishes ex-ecuting, and if we assume that thereafter A halts, the value never changes Thus,

if the algorithm runs for at most T n/ steps, the output appears as one of the bits

in cT n/

The reduction algorithm F constructs a single combinational circuit that putes all configurations produced by a given initial configuration The idea is to

com-M

tautology if it evaluates to for every assignment of and to the input variables.

Define TAUTOLOGY as the language of boolean formulas that are tautologies.Show that TAUTOLOGY... vertices in some hamilto-nian cycle If a graph is hamiltonian, the hamiltonian cycle itself offers enoughinformation to verify this fact Conversely, if a graph is not hamiltonian, therecan be no... that

con-tains a hamiltonian cycle is said to be hamiltonian; otherwise, it is

nonhamilto-nian The name honors W R Hamilton, who described a mathematical game