03-simplex-week2

The simplex algorithm — Discussions in class
Michel Bierlaire
1
Summary
The simplex algorithm solves a linear optimization problem by visiting the
vertices of the constraint polyhedron. It exploits the representation of vertices by feasible basic solutions, and follows the basic directions to move from
one vertex to the next.
1.1
Main concepts
• Basic directions,
• reduced costs,
• solution on a vertex,
• from vertex to vertex,
• the simplex tableau,
B−1 A
B−1 b
cT − cTB B−1 A −cTB B−1 b
• the initial tableau: the simple case,
• the initial tableau: the general case.
For the multiple choice questions below, the correct responses are in bold
font.
1
2
Reduced costs
Consider the condition “the reduced costs are all non negative”. Is it a
sufficient/necessary optimality condition?
• sufficient and necessary,
• sufficient and not necessary,
• not sufficient but necessary,
• neither sufficient, neither necessary.
3
Degeneracy: example
We propose an example where the optimal solution is degenerate, and the
reduced costs are non negative.
min x1 + x2
subject to
x1 ≥ x2 ,
x1 ≥ 0,
x2 ≥ 0.
Standard form:
min x1 + x2
subject to
−x1 + x2 + x3
x1
x2
x3
= 0,
≥ 0,
≥ 0,
≥ 0.


1
A = (−1 1 1), b = 0, c =  1  .
0
Consider the feasible basic direction where x2 is in the basis, and calculate
the reduced costs.
1. Is it an optimal vertex? Yes. If x2 is in the basis, we have B = B−1 = 1,
and xB = B−1 b = 1 · 0 = 0. Therefore, all the variables are equal to
zero. The value of the objective function at (0, 0) is zero. It cannot
reach a lower value without violating the constraints.
2
2. Why is there a negative reduced cost?
(a) Because the slack variable is out of the basis.
(b) Because the basis is degenerate. It is false, because a basis can
be degenerate, and have all the reduced costs non negative, as
illustrated by the last example below.
(c) Because it is a minimization problem.
(d) Because the corresponding basic direction is infeasible.
(e) Because there is only one basic variable.
If x2 is in the basis, we have B = B−1 = 1. The basic entry of the basic
direction that tries to put x3 in the basis is
(d3 )B = −B−1 A3 = −(1)(1) = −1.
Therefore, the direction is

 
0
0
d3 =  (d3 )B  =  −1 
1
1

and the reduced cost is
c̄3 = c3 − cTB B−1 A3 = 0 − (1)(1)(1) = −1.
Consequently, the objective function decreases along this direction.
The basic entry of the basic direction that tries to put x1 in the basis
is
(d1 )B = −B−1 A1 = −(1)(−1) = 1.
Therefore, the direction is
  
1
1



1 
(d1 )B
=
d1 =
0
0

and the reduced cost is
c̄1 = c1 − cTB B−1 A1 = 1 − (1)(1)(−1) = 2.
Consequently, the objective function increases along this direction.
3
d1
d3
3. If, instead of minimizing the objective function, we maximize it on the
same set of constraints, what happens?
• As the polyhedron has only one vertex, it corresponds to the optimal solution of the maximization problem as well.
• There is no optimal solution.
• The optimal solution is not on a vertex.
4
Solution on a vertex
1. Consider a linear optimization problem, with at least one optimal solution. Which one of the following statements is correct?
• There exists an optimal vertex of the constraints polyhedron. Indeed, this is Theorem 16.2, p.365.
• An optimal solution is always at a vertex of the constraints polyhedron. No. It may happen that all solutions in a face of the
polyhedron are optimal. In that case, solutions that are not vertices are also optimal.
• There exists a vertex of the constraints polyhedron which is optimal only if the problem is written in standard form. No. The
property is true irrespectively of how the polyhedron is written.
Indeed, any polyhedron can be written in standard form, and the
proof of Theorem 16.2 applies.
2. One of the following optimization problems has an optimal solution
which is not a vertex. Which one?
4
(a) min x s.t. 0 ≤ x ≤ 1. The unique solution is x∗ = 0, which is on
a vertex.
(b) min x1 + x2 , s.t. x1 , x2 ≥ 0. The unique solution is x1 = x2 = 0,
which is on a vertex.
(c) min x1 + 2x2 s.t. x1 + 2x2 ≤ 4, x1 , x2 ≥ 0. The unique solution is
x1 = x2 = 0, which is on a vertex.
(d) max x1 + 2x2 s.t. x1 + 2x2 ≤ 4, x1 , x2 ≥ 0. Note that it is
a maximization problem. All solutions such that x1 + 2x2 = 4
and x1 , x2 ≥ 0 are optimal. In particular, the solutions x1 = 0,
x2 = 2 and x1 = 4, x2 = 0 are optimal, and are on a vertex of the
polyhedron. But the solution x1 = 2, x2 = 1 is also optimal, and
is not on a vertex.
(e) max x s.t. 0 ≤ x ≤ 1. The unique solution is x = 1, which is on a
vertex.
(f) max x1 + x2 s.t. x1 , x2 ≥ 0. This problem is not bounded, and has
no optimal solution.
5
Step along the direction
1. The length of the step α to perform along the basic direction dp is
determined using exactly one of the following conditions.
• The gradient at x + αdp is zero. No, the gradient is constant and
always equal to c. Therefore, except for a trivial problem where
c = 0, the gradient is never zero.
• α = 1. No, the value one is arbitrary.
• α is equal to the reduced cost associated with dp . No, the reduced cost is the slope, the directional derivative, of the objective
function in the direction dp .
• α corresponds to the first constraint that is activated. Yes,
we follow the descent direction dp as far as we can, which is when
the first constraint is met. Beyond, the iterate would become
infeasible.
• α corresponds to the last constraint that is activated. No, it must
be the first, as discussed above.
• There are m + 1 variables in the basis at x + αdp . No, there are
always m variables in the basis.
5
• There are m − 1 variables in the basis at x + αd. No, there are
always m variables in the basis.
2. Consider an optimal feasible basic solution, such that the reduced cost
associated with the non basic variable xj is c̄j < 0. What is the length
α of the step to perform along the basic direction dj ?
(a) 0,
(b) 1,
(c) +∞,
(d) this situation is not possible.
The situation is possible, when the optimal basis is degenerate. As the
solution is optimal, any descent direction must be infeasible. And dj
is a descent direction, as the reduced cost is negative. Therefore, the
step is zero.
3. Consider the problem
max x1
x∈R2
subject to
x1 + x2 = 0,
x1 , x2 ≥ 0.
Write the problem in standard form, and consider the feasible basic
solution where x2 is in the basis. Which one of the following statements
is correct?
• The reduced cost is negative, and α = 0.
• The reduced cost is negative and α = 1.
• The reduced cost is negative and α = +∞.
• The reduced cost is positive, and there is no need to calculate α.
• The reduced cost is zero, and there is no need to calculate α.
The problem in standard form is
min −x1
x∈R2
6
subject to
x1 + x2 = 0,
x1 , x2 ≥ 0.
The data of the problem is
A = (1 1), b = 0, c =
−1
0
.
Note that the only feasible solution is x1 = 0, x2 = 0, as illustrated in
the following figure.
1 x2
d2
0.5
x1
−1
−0.5
0.5
−0.5
1
d1
−1
If x2 is in the basis, we have x1 = 0. As B = 1, x2 = xB = B−1 b = 1·0 =
0. The basic direction d1 is such that (d1 )B = −B−1 A1 = −1 · 1 = −1,
so that
1
.
d1 =
−1
The reduced cost is
c̄1 = c1 − cTB B−1 A1 = −1.
Note that, if we decide to put x1 in the basis, we have x2 = 0. As
B = 1, x1 = xB = B−1 b = 1 · 0 = 0. The basic direction d2 is such that
(d2 )B = −B−1 A1 = −1 · 1 = −1, so that
−1
.
d2 =
1
The reduced cost is
c̄2 = c2 − cTB B−1 A2 = 0 − (−1)(1)(1) = 1.
7
6
Tableau
Consider the following tableau.
x1
x2
x3
x4
x5
x6
1
-1
0
-1
-1
2
1
0
-1
0
-1
1
1
0
0
0
1
0
2
6
-8
Only one of the following statements is incorrect.
1. x3 is non basic. Indeed, if it were basic, the corresponding column
would contain only zero’s, except for one “one”.
2. x5 = 2. Indeed. x5 is in the basis, and the position of the “one” refers
to the corresponding row. Therefore, we read its value at row one in
the last column.
3. x2 = 0. Indeed, x2 is out of the basis.
4. The solution is not optimal. Indeed, the reduced cost of x3 is negative,
so that the tableau is not optimal. It may happen that a tableau which
is not optimal corresponds to an optimal solution. However, it is not
the case here as the feasible basic solution is not degenerate.
5. The value of the objective function is −8. No. The value of the
objective function is the value in the last row and the last column, with
a different sign. The value of the objective function is 8.
6. x6 is in the basis. Indeed, the corresponding column contains only
zero’s, and one “one”.
7. If x3 enters the basis and x5 exits the basis, the length of the step along
the basic direction is α = 2. Indeed, the value of α5 is 2/1 = 2, and
the value of α6 is +∞. The value of α is the smallest of the two, that
is 2.
8. If x3 enters the basis, the basic direction is


0
 0 


 1 

d3 = 
 0 .


 −1 
0
8
Indeed, the entries of the upper-left part of the tableau contain −dB .
Therefore,
−1
.
(d3 )B =
0
As the variables in the basis are x5 and x6 , we have (d3 )5 = −1 and
(d3 )6 = 0. As it is the third basic direction, we have (d3 )3 = 1. All the
other entries are zero, so that


0
 0 


 1 
.

d3 = 

0


 −1 
0
7
The initial tableau
At the end of phase I, we obtain the following tableau:
x1
x2
x3
x4
xa1
xa2
xa3
xa4
1
0
0
0
0
0
1
0
0
0
0
0
0
1
0
1/2
−3/4
0
1/3
0
1/2
1/4
−1
0
2
−1/2
1/4
−1
0
2
0
0
1
0
0
1/2
−3/4
0
1/3
1
1
1/2
0
1/3
0
x1
x2
xa3
x3
What should be done next.
1. Pivot the tableau to take xa3 out of the basis. xa3 should indeed not
stay in the basis, but there is no valid pivot. Indeed, the only original
variable candidate to enter the basis is x4 , and the corresponding pivot
is zero.
2. Stop the algorithm as the original problem is not feasible. No. As the
objective function of the auxiliary problem is zero, we have found a
feasible solution to the original problem: x1 = 1, x2 = 1/2, x3 = 1/3,
x4 = 0.
3. Calculate the last row of the tableau for the original problem. No. The
tableau must be “cleaned” first, in the sense that all auxiliary variables
must be removed.
9
4. Remove the third row of the tableau. Indeed, as discussed above,
there is no way to pivot xa3 out of the basis. It means that the corresponding constraint is redundant, and the row can be removed.
5. Remove the columns of the tableau corresponding to the auxiliary variables. No. This cannot be done while an auxiliary variable is in the
basis.
If time allows...
A
Reduced costs: example
Consider the lame linear optimization problem as last week:
max x1
subject to
x1 ≤ 1,
x2 ≤ 2,
x1 , x2 ≥ 0.
In standard form: Standard form:
min −x1
subject to
x1 + x3 = 1
x2 + x4 = 2
x1 , x2 , x3 , x4 ≥ 0

−1
 0 
1
1 0 1 0

, c=
, b=
A=
 0 .
2
0 1 0 1
0

The feasible set is represented in the following picture:
•
•
•
•
10
Consider x1 and x4 in the basis: x1 = 1, x2 = 0, x3 = 0, x4 = 2. The
reduced cost associated with x2 is
• positive,
• negative,
• zero.
The formula for the reduced costs is
c̄j = cj − cTB B−1 Aj .
where j is the index of a non basic variable. As
0
−1
,
, and A2 =
B = I, cB =
1
0
we have
c̄2 = 0.
From a geometrical point of view, the direction starts from the point (1, 0)
and increases the value of x2 along the face. In this direction, the value of
x1 does not change. Therefore, the value of the objective function does not
change. And the directional derivative, that is the reduced cost, is zero.
•
(1, 0)
A.1
Degeneracy: another example
Consider the problem
min x1 + x2
subject to
x2 − x3
x1
x2
x3
11
= 0,
≥ 0,
≥ 0,
≥ 0.
The data of the problem are

1
A = (0 1 − 1), b = 0, c =  1  .
0

We have three possible bases.
1. x1 in the basis: impossible, as B = 0 would not be invertible.
2. x2 in the basis: B = 1, cB = 1. x2 = xB = B−1 b = 1 · 0 = 0. All
variables are equal to zero. The reduced costs are
c̄1 = c1 − cTB B−1 A1 = 1 − (1)(1)(0) = 1,
c̄3 = c3 − cTB B−1 A3 = 0 − (1)(1)(−1) = 1.
They are both non negative. The sufficient condition for optimality is
verified.
3. x3 in the basis: B = −1, cB = 0. x3 = xB = B−1 b = −1 · 0 = 0. All
variables are equal to zero. This is the same vertex as the previous
basis. The reduced costs are
c̄1 = c1 − cTB B−1 A1 = c1 = 1,
and
c̄2 = c2 − cTB B−1 A2 = c2 = 1.
They are both non negative. The sufficient condition for optimality is
also verified.
Note that the constraints x2 − x3 = 0 and x3 ≥ 0 are equivalent to the
constraint x2 ≥ 0, and can be removed. Therefore, the problem can be
written
min x1 + x2
subject to
x1 ≥ 0,
x2 ≥ 0.
In this case, the optimal solution is the same, but the feasible basic solution
is not degenerate.
12