A branch-and-cut algorithm for nonconvex quadratic programs with box constraints

Math. Program., Ser. A 102: 559–575 (2005) Digital Object Identifier (DOI) 10.1007/s10107-004-0550-7

Dieter Vandenbussche · George L. Nemhauser

A branch-and-cut algorithm for nonconvex quadratic programs with box constraints Received: June 27, 2003 / Accepted: July 22, 2004 Published online: 1 October 2004 – © Springer-Verlag 2004 Abstract. We present the implementation of a branch-and-cut algorithm for bound constrained nonconvex quadratic programs. We use a class of inequalities developed in [12] as cutting planes. We present various branching strategies and compare the algorithm to several other methods to demonstrate its effectiveness.

1. Introduction In [12], we studied Quadratic Programming with Bounds (QPB) defined by 1 T x Qx + cT x 2 subject to 0 ≤ xj ≤ 1 ∀j ∈ N, max f (x) =

where Q = [qij ] is an n × n symmetric matrix, c is a column vector of dimension n, and N = {1, . . . , n}. We presented a reformulation of QPB 1
1
ci x i + yi 2 2 i∈N i∈N
subject to yi − qij xj − zi = ci max

j ∈N

yi (1 − xi ) = 0

∀i ∈ N

zi xi = 0 ∀i ∈ N x≤1 x, y, z ≥ 0.

(1) ∀i ∈ N

(2) (3) (4) (5) (6)

based on the work on general quadratic programs by Balas [1]. In this paper, we give a branch-and-cut algorithm for solving this reformulation. Let   S = (x, y, z) ∈ R3n : (x, y, z) satisfies (2) – (6) , D. Vandenbussche: Mechanical and Industrial Engineering, University of Illinois 1206 West Green Str., Urbana, IL 61801, USA. e-mail: [email protected] G.L. Nemhauser: Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0205, USA. e-mail: [email protected] Mathematics Subject Classification (2000): 90C26, 90C27, 90C20

560

D. Vandenbussche, G.L. Nemhauser

P S = conv(S), and   LP S = (x, y, z) ∈ R3n : (x, y, z) satisfies (2), (5), and (6) be the LP relaxation of S obtained by dropping the complementarity constraints (3) and (4). In [12], we characterized P Si , the convex hull of 
Si = (yi , zi , x) ∈ Rn+2 : yi − qij xj − zi = ci j ∈N

yi (1 − xi ) = 0, zi xi = 0

 xj ≤ 1 ∀j ∈ N, yi , zi , x ≥ 0 ,

which we referred to as a one row relaxation. We will use the inequalities that define P Si as cutting planes to improve the upper bounds provided by the LP’s. In Section 2, we present necessary optimality conditions, first reported by Hansen et al. [5], that guide the branching. In addition, we present various options in selecting branching candidates. Section 3 discusses the implementation details of the cutting planes. In Section 4, we present computational experiments that demonstrate the effectiveness of this branch-and-cut algorithm. Lastly, Section 5 presents conclusions and some potential improvements and generalizations of our approach.

2. Branching After a linear program is solved in the branch-and-bound tree, the algorithm checks if this solution satisfies the complementarity conditions (3) and (4). If this is not the case, then we have the choice of branching or cutting. We begin by studying exactly how the branching decision is carried out. Hansen et al. [5] presented a branch-and-bound algorithm for QPB. They showed that we can use the sign of qii to determine how one partitions the feasible set, specifically they showed that if we denote the i th component of the gradient of f (x) by ∇fi (x), then Proposition 1 ([5]). Suppose x ∗ is a local maximizer of QPB, then if qii > 0, then either ∇fi (x ∗ ) < 0 and xi∗ = 0, or ∇fi (x ∗ ) > 0 and xi∗ = 1; if qii < 0, then either ∇fi (x ∗ ) < 0 and xi∗ = 0, or ∇fi (x ∗ ) = 0 and 0 ≤ xi∗ ≤ 1, or ∇fi (x ∗ ) > 0 and xi∗ = 1. Proposition 2 ([5]). If qii = 0 for some i ∈ N, then there exists a global maximizer x ∗ such that either ∇fi (x ∗ ) ≤ 0 and xi∗ = 0 or ∇fi (x ∗ ) > 0 and xi∗ = 1.

Branch-and-cut for quadratic programming

561

The authors use these two propositions as guidelines for a branching scheme in their algorithm. Each subproblem is of the form max f (x) subject to ∇fi (x) < 0

∀i ∈ I −

∇fi (x) = 0

∀i ∈ I 0

∇fi (x) > 0

∀i ∈ I +

∇fi (x) ≤ 0

∀i ∈ I 0−

0 ≤ xj ≤ 1

∀j ∈ N,

where I − , I 0 , I + , I 0− ⊂ N. Using the constraints, the algorithm attempts to shrink the domain of each subproblem, after which interval arithmetic is used to obtain an upper bound on the objective value. It is important to note that they do not (and cannot) solve each subproblem as a linear program. Their algorithm is related to our approach, however. This becomes clear if we rewrite (2) as ∇fi (x) = yi − zi . Since (3) and (4) imply that yi zi = 0, we see that restricting ∇fi (x) < 0 implies zi > 0, and similarly, ∇fi (x) > 0, implies yi > 0. Since we want to use LP relaxations, we cannot use zi > 0 or yi > 0 as constraints. Instead, we replace zi > 0 by its implications, xi = 0 and yi = 0. Similarly, we can replace yi > 0 with xi = 1 and zi = 0. Hence, if qii < 0, then according to Proposition 1, the suggested partition is   (xi = 0, yi = 0) (xi = 1, zi = 0) (yi = 0, zi = 0). However, some initial experiments indicated that creating three children is not the most effective approach. Instead, if yi∗ (1 − xi∗ ) > 0 for the current LP solution, we partition the feasible set as  (7) (yi = 0) (xi = 1, zi = 0) or if zi∗ xi∗ > 0, we use (zi = 0)

 (xi = 0, yi = 0).

(8)

This eliminates explicitly creating the branch (yi = 0, zi = 0) which is, in general, unlikely to produce an optimal solution. If, on the other hand, qii ≥ 0, then Propositions 1 and 2 suggest using  (9) (xi = 0, yi = 0) (xi = 1, zi = 0) as the branching scheme. Next we need to determine the index to branch on. We implemented a method analogous to the “closest to 21 ” rule in integer programming. We compute   ∗  y (1 − xi∗ ) zi∗ xi∗ , , i ∗ ∈ arg max max i i∈N y¯i z¯ i

562

D. Vandenbussche, G.L. Nemhauser

where y¯i and z¯ i are upper bounds for yi and zi defined in Section 3. If qi ∗ i ∗ ≥ 0, then we can branch on this index using (9). If qi ∗ i ∗ < 0, then we choose the branching (7) y ∗ (1−x ∗ ) z∗ x ∗ when i y¯i i > iz¯ i i , otherwise we use (8). We refer to this rule as the most violated rule. A more computationally intensive way of selecting the branching index is the strong branching rule, which has been incorporated in CPLEX ([6]). This rule often reduces the number of evaluated nodes enough to warrant its computational expense. For each potential partition, we add the bounds implied by the first child to the current LP and 1 , then repeat the process for the second child resolve to obtain its objective value zLP 2 to get the objective value zLP . Now choose the partition with the smallest value for 1 + z2 . In effect, this attempts to choose the partition that will reduce the upper zLP LP bound by the greatest amount. A way to reduce the computational expense of strong branching is to limit the number of dual simplex pivots one uses to solve the required LP’s. Of course, this reveals the natural tradeoff between the accuracy of the estimate and the time expended to obtain it. We will perform some computational experiments to compare the effectiveness of the strong branching rule versus the most violated branching rule.

3. Cutting planes Since the polyhedron defined by LP S is unbounded, whereas P S is bounded, we add some initial inequalities to bound LP S. To do this, we recall some definitions from [12]. Let  y¯i = ci + qii + j ∈N + qij , and i  z¯ i = −ci − j ∈N − qij , i

where Ni+

= {j ∈ N \ i : qij ≥ 0} and Ni− = N \ (i ∪ Ni+ ). Define (a)+ = max{0, a}. From Theorem 3 in [12], we find Proposition 3. The inequalities yi ≤ (y¯i )+ xi

(10)

zi + (¯zi )+ xi ≤ (¯zi )+

(11)

and

are valid inequalities for P Si . If y¯i > 0 and z¯ i > 0, then they define facets of P Si . We add (10) and (11) for each i ∈ N to LP S to make it bounded. We now explain how we obtain valid inequalities from the one row relaxations. In [12], we showed that if z¯ i > 0 and y¯i > 0, then any facet defining inequality of P Si not induced by the bounds is either of the form

αj xj ≤ αj (12) yi + j ∈N

j ∈Ni−

Branch-and-cut for quadratic programming

or zi +

563







αj xj ≤

αj .

(13)

j ∈Ni+ ∪i

j ∈N

We refer to such inequalities as nontrivial facets. If (12) is a nontrivial facet, then α is a vertex of   j ∈Ni− αj − j ∈Ni+ αj − αi = y¯i y −qij ≤ αj ≤ 0 ∀j ∈ Ni+ (SEPi ) 0 ≤ αj ≤ −qij ∀j ∈ Ni− αi ≤ 0 and similarly, if (13) is a nontrivial facet, then α is a vertex of   αi + j ∈N + αj − j ∈N − αj = z¯ i i i 0 ≤ αj ≤ qij ∀j ∈ Ni+ qij ≤ αj ≤ 0 ∀j ∈ Ni− αi ≥ 0.

(SEPiz )

y

In [12], we showed that “most” vertices of SEPi and SEPiz yield nontrivial facet definy ing inequalities of P Si . Due to the simplicity of SEPi and SEPiz , they are the natural choice for use in separation. For instance, to check if an LP solution (y ∗ , z∗ , x ∗ ) violates any nontrivial facet (13), we solve

max zi∗ + αj xj∗ − αj j ∈N

subject to

α ∈ SEPiz .

j ∈Ni+ ∪i

To solve this problem, we rewrite it as a continuous knapsack problem. By substituting out αi and setting αj = qij αj ∀j ∈ N \ i, we obtain the linear program

max qij (xj∗ − xi∗ )αj + qij (xj∗ + xi∗ − 1)αj j ∈Ni+

subject to




j ∈N\i

j ∈Ni−

|qij |αj ≤ z¯ i

0 ≤ αj ≤ 1

∀j ∈ N \ i.

Balas and Zemel [2] showed that this problem can be solved in O(n) time. However, for ease of implementation, we used the well-known greedy algorithm which requires sorting the variables and hence runs in O(n log n) time. A similar approach is used to separate inequalities (12). Whenever the algorithm decides to add cuts, it checks each index i to see if yi∗ (1 − y xi∗ ) > 0. If it is, then we solve the separation problem associated with SEPi . Similarly, z if zi∗ xi∗ > 0, then we solve the separation problem associated with SEPi . If any violated inequalities are found, they are added to the formulation and the algorithm resolves the LP.

564

D. Vandenbussche, G.L. Nemhauser

3.1. Global vs. local cuts An important question to answer is whether or not separation should take into account the variables that have been fixed by branching. At a node of the branch-and-bound tree, we define the sets F1 ⊂ N and F0 ⊂ N as the indices of the x variables that have been fixed to 1 or 0, respectively. Let F = F1 ∪ F0 , Fl+ = Fl ∩ Ni+ , and Fl− = Fl ∩ Ni− for l = 0, 1. Note that if in the current LP solution, xi∗ ∈ {0, 1} for some i, then the inequalities yi ≤ (y¯i )+ xi and zi + (¯zi )+ xi ≤ (¯zi )+ enforce complementarity. Hence separation is not needed for any i ∈ F . Suppose we have i ∈ N \ F for which zi∗ xi∗ > 0. We define 

S˜i = (yi , zi , x) : yi − qij xj − zi = ci + qik j ∈N\F

k∈F1

yi (1 − xi ) = 0,

zi xi = 0

xj ≤ 1 ∀j ∈ N \ F, yi , zi , x ≥ 0



as the one row with respect to F . We recompute  the upper bound  relaxation  on zi as z˜ i = z¯ i + k∈F − qik − k∈F + qik . Similarly, y˜i = y¯i − k∈F + qik + k∈F − qik . 0 1 0 1 Assume z ˜ > 0 and y ˜ > 0. In this case, we know any nontrivial facet of conv( S˜i ), i i   + zi + j ∈N\F α˜ j xj ≤ j ∈N\F α˜ j , must satisfy   α˜ i + j ∈N + \F α˜ j − j ∈N − \F α˜ j = z˜ i i i 0 ≤ α˜ j ≤ qij ∀j ∈ Ni+ \ F qij ≤ α˜ j ≤ 0 ∀j ∈ Ni− \ F α˜ i ≥ 0.

z


i) (SEP

z


i . Now let αk = qik , ∀k ∈ F + ∪ F − , αk = 0 ∀k ∈ F + ∪ F − , where α˜ is a vertex of SEP 1 0 0 1  and let αj = α˜ j , ∀j ∈ N \ F . Hence j ∈N |αj | = z¯ i , which implies we now have a vertex of SEPiz . In addition, the equality
j ∈(Ni+ ∪i)\F

α˜ j − zi∗ −


j ∈N\F

α˜ j xj∗ =


j ∈Ni+ ∪i

αj − zi∗ −


j ∈N\F

αj xj∗ −




αk .

k∈F1+

shows that the violation of the inequalities is the same. Hence, we can extend any an inequality for  Si with the same violation. The difference is that inequality for S˜i to the inequality zi + j ∈N\F α˜ j xj ≤ j ∈(N + ∪i)\F α˜ j is valid only for this node and its i   descendants (i.e. it is a local cut), whereas zi + j ∈N αj xj ≤ j ∈N + ∪i αj is valid at i every node (a global cut). We have shown for any such local cut, there is a global cut with the same violation. This turns out to be a major computational advantage, since local cuts require more bookkeeping and global cuts generated at one node can prove to be useful at nodes other than its descendants. Note that the above argument does not hold if z˜ i ≤ 0 or y˜i ≤ 0. In that case, S˜i falls under one of the exceptions discussed

Branch-and-cut for quadratic programming

565

in [12] and nontrivial facets (13) need not correspond to vertices of SEPiz . Regardless, due to the disadvantages of local cuts, and since computational experiments indicated complementarity was rarely violated when z˜ i ≤ 0 or y˜i ≤ 0, we still use SEPiz to obtain cuts in this case. A similar analysis of inequalities (12) shows that we need only consider y globally valid cuts derived from SEPi . 3.2. Parameters Using the separation routine, many inequalities are generated and added to the LP formulation. We introduce a parameter CUTFREQ to control the frequency of cut addition. Cuts are only added at every CUTFREQ nodes. However, as more rows are added, the time required to solve each LP increases. Hence, it is advisable to delete cuts that are no longer active. On the other hand, it may not be wise to discard cuts too quickly as they may again become active at the next node that is evaluated. In order to manage this tradeoff, the algorithm uses two parameters: DELBND and DEACTFREQ. For every DEACTFREQ linear programs that are solved, the dual variable for each cut is checked. If the dual is zero, a counter for that cut is incremented; otherwise the counter is set to zero. If the counter for some cut is greater than DELBND, then the cut is removed from the formulation. Another way the algorithm hedges against deleting a cut that may prove useful again is by way of a cut pool. When the algorithm removes a cut, it is not immediately discarded, but instead it is added to a cut pool. The cut pool has a limited size (indicated by the parameter POOLSZ) and is managed as a first-in, first-out queue. After solving a linear program, the algorithm always checks if any cuts in the pool are violated, adds them and resolves, if necessary. If a cut is pushed out of the pool due to the size limitation, then it is permanently discarded. The algorithm forces branching at a node (that is, terminates cut generation, both from the separation routine and from the cut pool) if the improvement in the LP objective over the last few rounds of cuts is less than 0.5 % of the current LP value. 4. Computational experiments The branch-and-cut algorithm is implemented with MINTO 3.0 [9] which uses CPLEX 7.5 [6] to solve the LP relaxations. At the root node, we find a good feasible solution using the large-scale optimization algorithm in the MATLAB Optimization Toolbox, version 2.0 [8]. This algorithm is based on a Newton method that finds good local solutions to QP’s (see Coleman and Li [3]). For many instances, this method found the optimal solution, which prompted our decision never to search for better primal solutions at nodes other than the root. We do not report the time required to find the primal solution as it is always a negligible fraction of the total solution time. Initial experiments also showed that finding such good solutions at the root had little bearing on the overall solution time, again indicating there is no need to search for better solutions during the branch-andbound process. We used the standard best-bound rule when selecting the next node to evaluate. The tests are run on a SUN Ultra-80 with 2x450-MHz UltraSPARC-II processors and 1-GB Memory. We follow the lead of Hansen et al. and randomly generate instances.

566

D. Vandenbussche, G.L. Nemhauser

Since the density of the matrix Q greatly affects the running time of the algorithm, we generate instances of various density and size. Entries of Q and c are randomly generated integers between -50 and 50. We will denote the various instances we generate as N-D-S, where Q is an N × N matrix, D is the density expressed as a percentage and S is the seed used for the random number generator. For example, 30-60-1 is a 30 variable instance with density 60%, generated with seed 1. 4.1. Tuning parameters We begin with some preliminary experiments to determine good settings for the parameters CUTFREQ, DELBND, DEACTFREQ, and POOLSZ. We test various combinations of the parameters on several nontrivial instances of different sizes and densities. We use the most violated rule to select the index to branch on. The results are shown in Table 1. Each row presents results for a particular setting for the parameters (CUTFREQ, DELBND, DEACTFREQ, POOLSZ). Except for instance 40-40-1, where having POOLSZ = 250 has a negligible edge over having POOLSZ = 100, it appears that the best settings are to have CUTFREQ = 1, DELBND = 2, DEACTFREQ = 1, and POOLSZ = 100. In essence, the table indicates that the best strategy is to add cuts as aggressively as possible and to delete them very aggressively as well. This result can be justified intuitively. Since our computational experiments seem to indicate that the primal algorithm finds an optimal solution at the root, it makes sense to try to reduce the upper bounds from the LP’s aggressively. However, in order to minimize the solution time for these LP’s, it also makes sense to delete rows soon after they become inactive. Since generating inequalities can be done quickly, we do not pay too high a price in reproducing deleted inequalities and we do not need to keep a large cut pool. 4.2. Branch-and-bound In order to test the effectiveness of the cuts, we compare the performance of the branchand-bound algorithm with and without cuts. We report the results in Table 2. The first set of columns gives results for the branch-and-cut approach. The tables present the total time, number of nodes, number of LP’s solved, depth of the tree, initial value of the root LP relaxation, the value of the root LP after cuts are added, the optimal value, the value of the feasible solution found at the root, the maximum number of rows in any LP relaxation, and the total number of cuts generated. For branch-and-bound, i.e. when no cuts are added, we report the total time, number of nodes, and the depth of the tree. Note that a  indicates that the computation was terminated after 4000 seconds, in which case we list the relative gap between the current best upper bound and the best lower bound. Out of the 51 instances listed in Table 2, branch-and-cut was unable to complete one of them whereas branch-and-bound could not finish 17 of them within the 4000 second time limit. In the Totals row, we report the ratio between the total branch-and-bound time and the total branch-and-cut time, and similarly for the number of nodes column. In these totals, we do not include instances that did not terminate with the branch-and-bound approach.

(1, 2, 1, 100) (1, 2, 5, 100) (1, 5, 1, 100) (1, 5, 5, 100) (5, 2, 1, 100) (5, 2, 5, 100) (5, 5, 1, 100) (5, 5, 5, 100) (1, 2, 1, 250) (1, 2, 5, 250) (1, 5, 1, 250) (1, 5, 5, 250) (5, 2, 1, 250) (5, 2, 5, 250) (5, 5, 1, 250) (5, 5, 5, 250) (1, 2, 1, 50) (1, 2, 5, 50) (1, 5, 1, 50) (1, 5, 5, 50) (5, 2, 1, 50) (5, 2, 5, 50) (5, 5, 1, 50) (5, 5, 5, 50)

30-60-1 22.53 59.03 30.50 91.84 28.48 40.34 37.91 59.42 42.56 50.78 33.34 81.76 40.22 51.76 31.11 65.19 32.01 64.94 28.60 105.68 45.51 48.41 37.24 64.69

Time (s) 40-40-1 146.41 268.38 157.02 413.02 195.49 223.99 225.32 319.97 141.00 232.17 141.52 373.62 259.42 215.73 212.02 311.71 174.18 284.71 166.04 511.94 194.97 212.04 252.15 327.80

50-40-1 468.50 946.27 517.15 1557.27 1149.84 1013.82 957.79 1326.03 545.93 829.23 648.04 1295.93 1142.96 1275.32 1492.34 1413.55 697.17 1097.01 648.28 1976.02 994.30 974.97 924.82 1386.44

30-60-1 397 511 465 483 1711 1501 1933 1225 725 481 479 501 1665 1181 973 945 529 491 395 469 3303 1983 2495 1243

Nodes 40-40-1 1597 1511 1445 1373 7735 5991 7797 5035 1563 1485 1273 1593 7507 4023 5145 3375 1753 1363 1355 1381 8325 5599 9119 4821 50-40-1 2756 2369 2349 2205 19977 13725 16339 9715 3239 2663 2945 2357 17469 13465 18581 9303 2833 2281 2231 2247 19043 12703 15651 10091

Table 1. Parameter Tuning: (CUTFREQ, DELBND, DEACTFREQ, POOLSZ) 30-60-1 1772 1907 1744 1796 3706 2866 3954 2403 3127 1837 1944 1946 3985 2699 2224 2187 2288 1829 1460 1683 6289 3442 4447 2099

LP’s solved 40-40-1 6833 5287 5606 4696 15806 11214 15096 9311 6518 5469 5138 5708 17185 8573 11425 7221 6815 4606 4858 4560 15606 9580 16113 8109 50-40-1 11356 8427 9046 7625 41968 25916 32361 18308 13551 9793 11844 8690 40561 28901 41014 19923 10535 7875 7692 7556 35614 21447 27556 16682

Branch-and-cut for quadratic programming 567

20-100-1 20-100-2 20-100-3 30-60-1 30-60-2 30-60-3 30-70-1 30-70-2 30-70-3 30-80-1 30-80-2 30-80-3 30-90-1 30-90-2 30-90-3 30-100-1 30-100-2 30-100-3 40-30-1 40-30-2 40-30-3 40-40-1 40-40-2 40-40-3 40-50-1 40-50-2 40-50-3

time 0.78 1.58 0.82 22.86 1.07 15.39 109.41 4.97 5.33 85.1 2.01 2.32 41.31 19.12 14.83 77.3 80.18 22.92 3.3 2.8 2.63 143.61 13.55 140.51 131.6 70.22 55.31

nodes 27 61 31 397 17 275 1451 83 97 1211 29 33 523 273 181 979 877 291 59 53 37 1597 177 1411 1209 727 559

LPs solved 69 241 81 1772 55 1185 6676 315 367 5151 113 121 2327 1156 804 4373 4259 1278 186 201 139 6833 668 5861 5414 3078 2254

depth 7 10 9 16 5 16 17 11 14 17 6 8 13 16 13 18 16 17 8 9 6 20 14 17 17 15 16

Branch-and-Cut (P S) zinit zroot zopt 1054.83 1010.75 706.50 1320.01 1227.75 856.50 1142.49 1115.50 772.00 1420.70 1405.25 706.00 1769.60 1637.00 1377.17 2066.69 1966.00 1293.50 1552.31 1525.50 654.00 1929.40 1836.25 1313.00 2378.74 2199.50 1657.40 2038.93 2036.50 952.73 2284.02 2138.00 1597.00 2439.91 2349.00 1809.78 2356.31 2346.75 1296.50 2593.57 2578.50 1466.84 2487.04 2460.50 1494.00 2500.65 2481.00 1227.13 2677.71 2668.50 1260.50 2662.59 2655.75 1511.05 1161.97 1046.36 839.50 1859.88 1600.50 1429.00 1468.53 1291.00 1086.00 1633.45 1545.61 837.00 2033.05 1870.75 1428.00 2038.59 1994.75 1173.50 2162.80 2095.50 1154.50 2442.95 2312.50 1430.98 2670.25 2590.00 1653.63 zprimal 706.50 848.50 772.00 694.50 1377.17 1271.38 631.24 1313.00 1651.65 952.73 1597.00 1809.78 1296.50 1466.84 1494.00 1227.13 1218.70 1504.47 839.50 1420.16 1086.00 837.00 1428.00 1173.50 1150.05 1430.98 1653.63

Max rows 237 203 231 324 319 290 349 326 300 324 332 375 339 323 340 332 352 338 357 316 377 395 397 440 416 436 439

totalcuts 658 1562 677 18650 869 12684 75581 3805 3701 60950 1419 1522 30088 14618 11232 52089 58290 16408 1856 1526 1569 84850 8125 82683 77939 41364 33985

Table 2. CUTFREQ=1,DELBND=2,DEACTFREQ=1,CUTPOOLSZ=100 results Branch-and-Bound time nodes depth 0.61 313 16 1.13 695 16 0.85 423 16 172.98 31957 25 2.05 783 24 46.63 11997 26 551.17 61383 29 14.2 4055 23 14.54 4581 23 220.5 37379 31 7.09 2295 23 3.36 977 25 59.81 12827 29 73.65 14949 30 25.48 6083 28 183.93 31319 29 242.7 36043 28 34.03 7619 32 23.47 7067 26 32.02 8185 26 15.71 4677 28 2.86% 160684 33 65.48 13419 32 1.38% 158939 34 1935.01 123935 35 1021.75 87153 34 828.58 76041 39

568 D. Vandenbussche, G.L. Nemhauser

40-60-1 40-60-2 40-60-3 40-70-1 40-70-2 40-70-3 40-80-1 40-80-2 40-80-3 40-90-1 40-90-2 40-90-3 40-100-1 40-100-2 40-100-3 50-30-1 50-30-2 50-30-3 50-40-1 50-40-2 50-40-3 60-20-1 60-20-2 60-20-3 Totals

time 983.32 14.42 10.09 229.41 138 26.86 359.03 515.81 117.31 199.48 241 73.32 263.61 2169.8 58.84% 13.28 127.07 87.91 464.57 455.61 263.06 101.89 18.04 141.46

nodes 4575 133 89 1681 1019 267 2169 2821 845 1375 1595 557 1581 6103 4330 129 1039 751 2765 2529 1627 773 145 1045

LPs solved 20590 568 350 7622 4490 1081 9729 12690 3687 6246 7202 2589 7190 27928 36207 434 4285 2827 11356 10561 6464 2781 490 3876

depth 24 15 11 22 17 24 20 21 24 23 25 20 24 23 21 13 18 17 23 21 17 21 13 20

Branch-and-Cut (P S) zinit zroot 2772.42 2763.75 2963.96 2827.75 3466.70 3338.25 3019.23 2996.25 3363.89 3276.75 3825.06 3688.75 3731.54 3730.75 3798.02 3717.75 4284.44 4260.75 4306.50 4274.25 4273.61 4251.00 4407.36 4390.00 4854.50 4854.50 4807.75 4807.75 4958.75 4958.75 1931.95 1817.25 2537.25 2315.25 2202.08 2080.25 2576.17 2529.25 2945.56 2860.50 3307.72 3198.50 1844.09 1733.75 2442.53 2221.75 2177.74 2063.25 zopt 1322.67 2004.23 2454.50 1605.00 1867.50 2436.50 1838.50 1952.50 2545.50 2135.50 2113.00 2535.00 2476.38 2102.50 – 1324.50 1668.00 1453.61 1411.00 1745.76 2094.50 1212.00 1925.50 1483.00

zprimal 1303.50 2004.23 2454.50 1605.00 1867.50 2436.50 1838.50 1952.50 2545.50 2132.50 2113.00 2535.00 2476.38 2102.50 1847.06 1324.50 1649.00 1453.61 1411.00 1745.76 2094.50 1212.00 1925.50 1483.00

Max rows 441 415 471 439 441 429 469 463 471 460 468 443 482 499 494 507 509 499 550 543 553 603 572 571

total cuts 317378 8510 6039 109000 74903 16501 165197 210223 61630 101621 120944 41132 122704 493923 659599 7116 63591 45628 190041 190653 116742 48850 8758 66387

Table 2. (cont.) CUTFREQ=1,DELBND=2,DEACTFREQ=1,CUTPOOLSZ=100 results Branch-and-Bound time nodes depth 21.51% 110070 30 443.4 50905 32 63.86 10105 30 2682.37 143807 37 3679.27 167565 36 1419.91 98933 33 12.37% 82072 32 3.20% 144271 35 1727.67 107183 37 9.10% 123764 31 8.80% 129598 33 1603.15 104291 38 3718.04 159891 40 20.78% 110071 31 54.16% 103321 26 459.54 52825 36 10.19% 112540 28 3.59% 133611 33 14.44% 111166 35 16.5% 106206 32 10.16% 104299 30 9.52% 115302 33 0.9% 161597 38 12.98% 108746 30 12.78 92.85

Branch-and-cut for quadratic programming 569

570

D. Vandenbussche, G.L. Nemhauser

Table 2 indicates that adding cutting planes aggressively can significantly improve upon the computation time required to solve the instances with plain branch-and-bound. In fact, the tables suggest that, on average, adding cuts allows one to solve these instances at least 12 times faster than without cuts. This speedup factor would be even more dramatic had we solved all instances to optimality. The effectiveness of branch-and-cut over branch-and-bound is in part due to the fact that the maximum number of rows is never more than 603. Observe also that the cuts at the root do not close the gap significantly, which indicates the need to add cuts throughout the tree. The instances we have reported show the approximate envelope of size and density that the branchand-cut approach can complete within the self-imposed time limit of 4000 seconds. Clearly, this envelope would increase if we were willing to terminate when the algorithm has found a solution with a value within an acceptable percentage of the optimal.

4.3. Strong branching In order to gauge the effectiveness of strong branching, we have also tested it on a subset of the generated instances. Initial experiments showed that instances that were solved quickly using the most violated rule, could not be solved faster using strong branching, regardless of the iteration limit on the dual simplex pivots. Consequently, we have decided to take all instances from Table 2 that required more than 100 seconds with branch-and-cut and that terminated within the time limit. Table 3 shows the total solution time, the amount of time spent on branching, and the number of nodes for various iteration limits. Strong branching greatly reduces the number of nodes evaluated. However, this is at the cost of a great deal of overhead. Hence Table 3 also contains columns called Branch that report the time spent selecting the branching candidate. In the row labeled Totals, we take the total of each branch column and divide that by the total solution time to see the percentage of the solution time spent selecting branching candidates. In addition, for each time and nodes column, we report the total of that column divided by the total of the corresponding column for the most violated method. We see that, overall, strong branching requires about 30% of the number of nodes that the most violated method requires. In fact, for most instances, the addional time required to select a branching index is offset by the significant reduction in the number of nodes evaluated. Hence we find that, on average, we can reduce the solution time for “difficult” instances by approximately 30 % with the use of strong branching with an iteration limit of 40.

4.4. MIP formulation Problem P S can also be reformulated as a mixed integer program. The MIP formulation requires a number of auxiliary variables and some additional constraints. Define K + = {i ∈ N : qii ≥ 0} and K − = N \ K + . The MIP formulation is

30-70-1 40-40-1 40-40-3 40-50-1 40-60-1 40-70-1 40-70-2 40-80-1 40-80-2 40-80-3 40-90-1 40-90-2 40-100-1 40-100-2 50-30-2 50-40-1 50-40-2 50-40-3 60-20-1 60-20-3 Totals

Most Violated Time Nodes 109.41 1451 143.61 1597 140.51 1411 131.6 1209 983.32 4575 229.41 1681 138 1019 359.03 2169 515.81 2821 117.31 845 199.48 1375 241.00 1595 263.61 1581 2169.80 6103 127.07 1039 464.57 2765 455.61 2529 263.06 1627 101.89 773 141.46 1045 1 1

Time 96.82 123.30 109.29 100.29 804.49 202.07 97.77 524.83 133.31 90.29 323.84 170.68 327.38 1066.20 80.89 810.76 420.59 225.40 122.48 180.38 0.82

20 Branch 60.30 87.24 77.62 69.40 487.02 142.59 66.44 341.80 88.44 62.17 211.20 112.50 217.11 609.31 60.14 563.59 298.16 167.04 94.01 135.55 0.66 Nodes 659 545 437 453 2463 647 361 1411 429 279 937 651 799 2665 237 1805 1097 521 245 381 0.43

Time 105.80 111.55 139.98 105.70 544.86 187.30 84.84 566.25 170.03 93.44 213.31 173.18 246.14 748.35 85.86 520.24 346.82 239.34 183.67 196.72 0.69

40 Branch 75.02 85.90 106.24 80.33 391.74 144.05 65.95 405.15 128.38 75.18 158.63 124.61 184.27 518.95 68.25 412.81 270.45 193.92 152.40 161.88 0.75 Nodes 557 419 477 355 1531 517 221 1249 441 173 521 607 427 1601 213 971 721 405 251 291 0.30

Time 102.10 151.89 125.02 109.24 581.90 163.71 100.73 602.99 151.12 102.64 244.42 161.23 287.85 719.40 109.50 695.33 374.55 220.76 174.92 222.64 0.74

Iteration Limit

Table 3. Strong Branching results 90 Branch 77.19 124.70 101.83 88.21 450.59 132.54 84.49 473.19 120.75 86.64 195.72 124.66 234.84 546.98 91.88 588.10 313.23 188.48 151.53 193.52 0.81 Nodes 529 467 343 311 1387 357 201 1111 325 167 479 473 395 1269 231 989 573 279 223 239 0.26

Time 109.02 150.07 172.11 111.03 609.64 162.98 101.10 530.67 141.92 99.68 256.21 152.26 238.87 922.79 135.01 688.57 425.41 224.64 208.68 294.31 0.79

∞ Branch 82.85 122.49 141.91 90.40 471.72 132.04 84.33 423.73 114.59 86.21 205.17 116.42 185.89 698.83 117.53 586.99 357.79 190.32 183.22 261.17 0.81 Nodes 527 443 473 305 1525 349 211 917 277 147 489 471 381 1565 213 925 649 329 253 287 0.27

Branch-and-cut for quadratic programming 571

572

D. Vandenbussche, G.L. Nemhauser

max

1
(ci xi + yi ) 2 i∈N
yi − qij xj − zi = ci j ∈N u yi ≤ y¯i yi , zi ≤ z¯ i ziu xi ≤ xiu , 1 − xi ≤ xil xiu + ziu ≤ 1, xil + yiu

yi ≤ y¯i xi , xi ∈ {0, 1}

∀i ∈ K − ∀i ∈ K − ≤1

zi + z¯ i xi ≤ z¯ i

xiu , xil , yiu , ziu ∈ {0, 1}

∀i ∈ N

∀i ∈ K − ∀i ∈ K + ∀i ∈ K − ∀i ∈ K +

x, y, z ≥ 0. In this formulation, we add binary variables for each i ∈ K − . For instance, xiu is 1 if xi > 0, and we use ziu to indicate if zi > 0. We then enforce the appropriate complementarity constraint by ensuring that xiu and ziu are not simultaneously 1. Note that we do not need to add auxiliary variables for i ∈ K + since, by Proposition 1, if qii ≥ 0, we may assume xi ∈ {0, 1}. In an attempt to obtain a fair comparison, we solved these MIP’s using default MINTO settings. The results are shown in Table 4, together with the results of our branch-and-cut approach using the most violated rule. Columns labeled P S denote our branch-and-cut approach. In the Totals row we report the ratio of the total MIP time to the total time spent with our branch-and-cut approach. Similar ratios are reported for the number of nodes and number of LP’s solved. These totals do not include instances that did not terminate within the time limit with the MIP approach. The MIP approach was unable to complete 4 of the 51 instances within the time limit. With solution times on average 5 times smaller than with the MIP approach, the results indicate the computational advantage of handling the complementarity conditions directly with branch-and-cut, as opposed to using auxiliary binary variables to enforce them. In addition, we generated some instances where each diagonal is a random integer in the interval [−50,−1] and all other entries are generated as before. In this case, the MIP formulation contains auxiliary binary variables for each i ∈ N. This will make the problem more difficult for our approach and the MIP approach. However, as Table 5 indicates, these instances accentuate the difficulties for the MIP formulation in comparison with our formulation. In Table 4, we saw that on average the MIP approach took at least 5.71 times longer, whereas with the negative diagonal instances, the MIP took almost 20 times longer. These instances allow us to accentuate the advantage of using P S as a formulation as opposed to the much larger MIP formulation. 4.5. BARON We also solved each instance using the BARON global optimization algorithm, see [10], which was recently incorporated into the suite of solvers of GAMS [4]. Initial

20-100-1 20-100-2 20-100-3 30-60-1 30-60-2 30-60-3 30-70-1 30-70-2 30-70-3 30-80-1 30-80-2 30-80-3 30-90-1 30-90-2 30-90-3 30-100-1 30-100-2 30-100-3 40-30-1 40-30-2 40-30-3 40-40-1 40-40-2 40-40-3 40-50-1 40-50-2 40-50-3

Time (s) MIP PS 2.80 0.78 7.62 1.58 1.86 0.82 205.85 22.86 5.22 1.07 67.66 15.39 246.31 109.41 53.54 4.97 56.77 5.33 694.43 85.10 15.59 2.01 6.23 2.32 141.30 41.31 228.58 19.12 72.34 14.83 448.30 77.30 325.58 80.18 286.81 22.92 16.10 3.30 20.86 2.80 16.22 2.63 533.43 143.61 71.58 13.55 499.50 140.51 600.95 131.60 289.77 70.22 316.14 55.31

Nodes MIP PS 425 27 971 61 465 31 18601 397 577 17 4799 275 20501 1451 5279 83 3438 97 70109 1211 1873 29 767 33 16275 523 21539 273 7401 181 36389 979 31181 877 18555 291 1869 59 1267 53 1421 37 22019 1597 5005 177 33071 1411 40057 1209 21015 727 22759 559 40-60-1 40-60-2 40-60-3 40-70-1 40-70-2 40-70-3 40-80-1 40-80-2 40-80-3 40-90-1 40-90-2 40-90-3 40-100-1 40-100-2 40-100-3 50-30-1 50-30-2 50-30-3 50-40-1 50-40-2 50-40-3 60-20-1 60-20-2 60-20-3 Totals

Time (s) MIP PS  102.32% 983.32 182.55 14.42 224.12 10.09 1584.54 229.41 401.94 138.00 277.92 26.86 2310.23 359.03 755.06 515.81 371.19 117.31 1186.46 199.48 1033.39 241.00 722.22 73.32 556.66 263.61  128.67% 2169.80  168.47%  58.84% 538.49 13.28 528.87 127.07 612.58 87.91  82.58% 464.57 2872.36 455.61 1706.96 263.06 2435.19 101.89 131.73 18.04 1836.86 141.46 5.71 –

Table 4. MIP Formulation results LPs solved MIP PS 499 69 1084 241 517 81 21775 1772 666 55 5612 1185 24613 6676 5975 315 3998 367 81805 5151 2073 113 839 121 18490 2327 25192 1156 8603 804 41548 4373 35842 4259 21931 1278 2054 186 1406 201 1567 139 26960 6833 5810 668 40858 5861 48693 5414 24938 3078 27378 2254 Nodes MIP PS 178672 4575 8283 133 10911 89 63379 1681 18759 1019 13103 267 123447 2169 35803 2821 15453 845 75957 1375 42861 1595 38565 557 27571 1581 262407 6103 333438 4330 24971 129 17575 1039 32001 751 188000 2765 102691 2529 70701 1627 73629 773 4029 145 52379 1045 36.16 –

LPs solved MIP PS 223017 20590 10204 568 13295 350 77143 7622 22435 4490 15806 1081 148247 9729 43030 12690 18872 3687 90882 6246 51843 7202 46314 2589 32691 7190 308164 27928 374295 36207 31338 434 21673 4285 39285 2827 235286 11356 129650 10561 88273 6464 89907 2781 4812 490 64360 3876 10.13 –

Branch-and-cut for quadratic programming 573

574

D. Vandenbussche, G.L. Nemhauser

Table 5. Negative Diagonal MIP results

30-60-1 30-60-2 30-60-3 30-70-1 30-70-2 30-70-3 40-40-1 40-40-2 40-40-3 Totals

Time MIP PS 1378.19 136.21 99.06 3.77 260.99 13.95 1049.79 103.22 446.75 20.07 229.11 5.44 2388.42 57.06 3947.98 159.53  78.40% 172.22 19.63 –

Nodes MIP PS 58707 1789 3959 69 15111 277 41811 1413 20915 301 8521 117 43567 681 56947 1699 70188 1565 39.32 –

LPs solved MIP PS 74603 8248 4784 258 17754 1159 52874 6235 25443 1315 10134 425 56547 2916 72440 6945 88796 7027 11.44 –

computations indicated that BARON performed better when applied to the original problem QPB, instead of the reformulation with complementarities. Due to platform incompatibilities, we could only run BARON on a Linux platform. The tests were performed on a Pentium-4 1.8 GHz processor with 512 MB of main memory. We solved a sample of linear programs on both platforms to obtain a sense of the relative speeds of each machine. On average, we found the Linux platform to be three times faster than the SUN platform. The solution times reported in Table 6 have not been adjusted to account for this difference in speed and yet we find that the branch-and-cut approach was significantly faster, despite the slower machine. In fact, as the Totals row indicates, the unadjusted branch-and-cut times were smaller by a factor of approximately 22 and that number only accounts for the instances that BARON could solve within the 4000 second time limit. BARON did not finish 21 of the 51 instances within the time limit on a faster machine. Of course, this should be moderated by the fact that BARON is a general purpose global optimization solver whereas our approach only solves QPB instances.

5. Conclusions and future work The computational results in this paper demonstrate the clear advantage of using a formulation such as PS to solve QPB. The combination of a compact formulation with strong cuts which effectively reduce the upper bounds allows this approach to dominate the other methods. In addition, we showed that strong branching is able to make decisions which significantly decrease the number of nodes to be examined. This suggests the possibility of using estimation methods such as pseudocosts to make the branching decisions (see [7] for a survey). Such methods may reduce the computation required to make the branching decisions without a significant increase in the size of the tree. With the very encouraging results obtained for QPB, we will next apply this approach to general linearly constrained nonconvex quadratic programs. Acknowledgements. This research was supported by NSF grants DMI-0100020 and DMI-0121495 and formed the basis of Vandenbussche’s Ph.D. dissertation at Georgia Tech [11]. We would like to thank an anonymous referee for a thorough review and many useful comments.

Branch-and-cut for quadratic programming

575

Table 6. BARON results

20-100-1 20-100-2 20-100-3 30-60-1 30-60-2 30-60-3 30-70-1 30-70-2 30-70-3 30-80-1 30-80-2 30-80-3 30-90-1 30-90-2 30-90-3 30-100-1 30-100-2 30-100-3 40-30-1 40-30-2 40-30-3 40-40-1 40-40-2 40-40-3 40-50-1 40-50-2 40-50-3

Time (s) BARON PS 1.65 0.78 4.80 1.58 3.47 0.82 1282.43 22.86 1.25 1.07 193.58 15.39  9.92% 109.41 35.46 4.97 35.09 5.33  10.84% 85.10 9.60 2.01 14.03 2.32 2372.42 41.31 2233.03 19.12 824.37 14.83  11.03% 77.30  13.48% 80.18 2608.60 22.92 1.99 3.30 1.20 2.80 1.08 2.63 2879.36 143.61 15.12 13.55 3747.10 140.51  7.43% 131.60 2051.12 70.22 2781.58 55.31

40-60-1 40-60-2 40-60-3 40-70-1 40-70-2 40-70-3 40-80-1 40-80-2 40-80-3 40-90-1 40-90-2 40-90-3 40-100-1 40-100-2 40-100-3 50-30-1 50-30-2 50-30-3 50-40-1 50-40-2 50-40-3 60-20-1 60-20-2 60-20-3 Totals

Time (s) BARON PS  24.92% 983.32 593.48 14.42 305.33 10.09  15.60% 229.41  11.84% 138.00  4.88% 26.86  25.63% 359.03  19.20% 515.81  14.52% 117.31  26.60% 199.48  25.3% 241.00  16.24% 73.32  25.95% 263.61  34.21% 2169.80  45.42% 58.84% 56.57 13.28 1291.93 127.07 478.81 87.91  16.26% 464.57  11.19% 455.61  7.19% 263.06 502.34 101.89 5.52 18.04 739.32 141.46 22.76 –

References 1. Balas, E.: Nonconvex quadratic programming via generalized polars. SIAM J. Appl. Math. 28, 335–349 (1975) 2. Balas, E., Zemel, E.: An algorithm for large zero-one knapsack problems. Oper. Res. 28, 1130–1154 (1980) 3. Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996) 4. GAMS Development Corporation: General Algebraic Modeling System, Version 2.50, 1998 5. Hansen, P., Jaumard, B., Ruiz, M., Xiong, J.: Global minimization of indefinite quadratic functions subject to box constraints. Nav. Res. Logist. 40, 373–392 (1993) 6. ILOG, Inc.: ILOG CPLEX 7.5, User Manual, 2001 7. Linderoth, J.T., Savelsbergh, M.W.P.: A computational study of search strategies for mixed integer programming. Informs J. Comput. 11, 173–187 (1999) 8. Mathworks, Inc.: MATLAB Optimization Toolbox 2.0, 1998 9. Nemhauser, G.L., Savelsbergh, M.W.P., Sigismondi, G.S.: MINTO, a Mixed INTeger Optimizer. Oper. Res. Lett. 15, 47–58 (1994) 10. Sahinidis, N.V.: BARON: a general purpose global optimization software package. J. Glob. Optim. 8, 201–205 (1996) 11. Vandenbussche, D.: Polyhedral Approaches to Solving Nonconvex Quadratic Programs. PhD thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology, 2003 12. Vandenbussche, D., Nemhauser, G.L.: A polyhedral study of nonconvex quadratic programs with box constraints. Technical Report TLI-03-04, Georgia Institute of Technology. Math. Prog. 102, 531–557 (2005)