Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) Solution of question 1: 1. The feasible region is shown in gray in Figure 1. x2 F 5 4 C 3 2 B 1 E -3 A D 0 -2 -1 0 1 2 3 4 5 x1 Figure 1: Feasible domain 2. Constraints are active when they are verified as equality. Therefore, we search for the solutions to this system of equations: x1 + x2 = 5, −x1 + 2x2 = 3. We solve the system of equations by substitution or by elimination and we obtain 8 x2 = , 3 7 x1 = . 3 If constraints (1) and (2) are active, it corresponds to the vertex ( 73 , 83 ), that is point C in Figure 1. 3. The basic solutions correspond to the intersections of all pairs of constraints: A = (0, 0), B = (0, 23 ), C = ( 37 , 83 ), D = (5, 0), E = (−3, 0), F = (0, 5). 1 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) 4. The feasible basic solutions are the following 4 vertices: A = (0, 0), B = (0, 32 ), C = ( 73 , 83 ), D = (5, 0). Indeed, they satisfy all constraints (1-4). E = (−3, 0) does not satisfy constraint (3), while F = (0, 5) does not satisfy constraint (2). 5. We have to add the level lines corresponding to different values of the objective function. We start by drawing an arbitrary level line intersecting the feasible domain, say f = −2x1 + 2x2 = 0. The gradient corresponds to the steepest ascent direction. As we minimize, we need to identify the steepest descent direction, that is 2 −∇f (x) = −c = . −2 We move this level line parallel to itself as far as possible in the direction of −c as long as it intersects the feasible domain. The minimum value that the objective function can take is f = −10. The optimal solution corresponds to vertex D (5, 0), as shown in Figure 2. x2 F 5 4 C 3 2 B 1 E -3 A D 0 -2 -1 0 1 2 3 4 5 f = −10 f =0 Figure 2: Graphical method 2 x1 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) Solution of question 2: 1. There are two basic variables. Recall that the columns corresponding to the basic variables are composed of zeros, except one "one". In this case, the basic variables are x1 and x4 . The position of the "one" in the column of a basic variable identifies the corresponding row. Therefore, the basic variable x1 corresponds to row 1, and the basic variable x4 corresponds to row 2. 2. In order to identify the vertex xT = (x1 , x2 , x3 , x4 ), we need to find the values of x1 , x2 , x3 and x4 . We know that the non basic variables are equal to zero: x2 = x3 = 0. Also, the value of each basic variable is found in the rightmost column of the row that identifies the basic variable: x1 = 2 and x4 = 4. Therefore, xT = (2, 0, 0, 4). 3. The value in the lower-right part of the tableau is equal to the opposite of the objective function value. Therefore, the objective function value is −4. Alternatively, the value of the objective function can be calculated as the scalar product cT x: 2 0 cT x = −2 −1 0 0 0 = −4 + 0 + 0 + 0 = −4. 4 4. The reduced costs can be found in the lower-left part of the tableau. The reduced cost for x2 is −3. It is negative, meaning that the objective function decreases along the corresponding basic direction. The reduced cost for x3 is 2. It is positive, meaning that the objective function increases along the corresponding basic direction. 5. We consider the basic solution where x1 and x4 are in the basis: 2 −1 xB = B b = . 4 The basic entries of the basic directions are found in the top-left part 3 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) of the tableau, with a different sign. Therefore, 1 1 1 (d2 )B = , d2 = 0 , −2 −2 and −1 −1 0 (d3 )B = , d3 = 1 . 1 1 Moving along d2 , we obtain 2 1 2+α 0 1 α x+ = 0 + α 0 = 0 , 4 −2 4 − 2α which is feasible for 0 ≤ α ≤ 2. Therefore, the basic direction d2 is feasible. Moving along d3 , we obtain 2 −1 2−α 0 0 0 x+ = 0 + α 1 = α , 4 1 4+α which is feasible for 0 ≤ α ≤ 2. Therefore, the basic direction d3 is feasible. The feasible region and the two directions are represented on the following graph. 4 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) x2 C 5 4 3 D 2 d2 1 d3 B E 0 -3 -2 -1 0 1 2A 3 4 5 x1 F Figure 3: Feasible region and directions Solution of question 3: 1. Which of the following statements about a feasible basic solution is true for a minimization problem? (a) If c̄ < 0, that is, all reduced costs are negative, the solution is optimal. (b) If the values of the basic variables are negative, the solution is optimal. (c) If c̄N > 0, that is, the reduced costs of the non basic variables are positive, the solution is optimal. Answer: (c). Answer (a) is false. Theorem 6.30 (necessary optimality conditions) says that if the solution is optimal, then the reduced costs are non negative. The contrapositive is that, if the reduced costs are negative, the solution is not optimal. Answer (b) is false because if the values of the basic variables are negative, then the solution is infeasible, since 5 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) the non negativity constraints are not respected. It cannot therefore be optimal. Answer (c) is correct. Indeed, the reduced costs of the basic variables are zero, so that all reduced costs are non negative (sufficient optimality conditions). 2. Consider a linear minimization problem with four variables (x = (x1 , x2 , x3 , x4 )T ). In the current iteration of the simplex algorithm, we obtain the reduced costs vector c̄ = (−1, 0, 0, 0)T and the upper-right column of the simplex tableau is given by B −1 b = (2, 4, 3)T . Which of the following propositions is false? (a) x1 enters the basis in the following iteration of the simplex. (b) x3 is a basic variable. (c) The current solution is not feasible. Answer: (c). B −1 b has size 3, so there must be 3 basic variables. The size of x is 4, so we know that there is one non basic variable. The non basic variable is x1 , since its associated reduced cost is different from 0. As a consequence, x2 , x3 and x4 are all basic variables and answer (b) is true. Answer (a) is also true, since the reduced cost associated with x1 is the only negative reduced cost in the current iteration. Answer (c) is false because the current solution is feasible, since all basic variables are non negative. 3. Which is the feasible basic solution associated with the following simplex tableau? x1 1 0 0 x2 4 1 2 x3 1 3 3 (a) x1 = 5, x2 = 6, x3 = s1 = s2 = 0 (b) x3 = 5, s1 = 6, x1 = x2 = s2 = 0 (c) x3 = 5, x2 = 6, x1 = s1 = s2 = 0 6 s1 0 1 0 s2 5 2 5 5 6 -11 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) (d) x1 = 5, s1 = 6, x2 = x3 = s2 = 0 Answer: (d). The variables x1 and s1 are the basic variables. The corresponding columns in the tableau contain only zeros, except for one "one". All reduced costs are non negative, therefore the tableau is optimal. The value of the objective function is 11, that is the value in the lowerright part of the tableau, with an opposite sign. 4. The following tableau represents a basic solution for phase I of the simplex algorithm. a1 and a2 are the auxiliary variables associated with the two constraints. x1 x2 2/3 1 -1/3 0 0 0 x3 0 1 0 x4 1/3 1/3 0 a1 0 -1 1 a2 1/3 1/3 1 6 2 0 Which of the following propositions is true? (a) There is a feasible solution for the original problem and there are no auxiliary variables in the basis. (b) There is a feasible solution for the original problem, but there are still auxiliary variables in the basis. (c) The original problem does not admit any feasible solution. Answer: (a). All the reduced costs are non negative, so this is an optimal tableau for phase I of the simplex algorithm. The value of objective function is zero, so there exists a feasible solution for the original problem. Answer (c) is therefore incorrect. The variables x2 and x3 are the two basic variables, since the corresponding columns in the tableau contain only zeros, except for one "one". There are no auxiliary variables in the basis, therefore answer (b) is incorrect. Answer (a) is the only correct proposition. 7 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) Solution of question 4: 1. The feasible region is shown in gray in Figure 4. To solve the problem using the graphical method, we add the level lines corresponding to different values of the objective function. We start by drawing an arbitrary level line intersecting the feasible domain, say f = −3x1 + 4x2 = −12. The steepest descent direction is 3 −∇f (x) = −c = . −4 The minimum value that the objective function can take is f = −27. The optimal solution corresponds to vertex D = (9, 0). x2 7 6 B 5 4 A 3 2 1 C 0 0 1 2 3 4 D 5 f = −12 6 7 8 9 10 x1 f = −27 Figure 4: Feasible region and graphical method 2. To solve the problem using the simplex method, we start by writing the problem in standard form. To do so, we introduce the slack variables e1 and e2 : min −3x1 + 4x2 8 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) subject to: x1 + x2 − e1 = 4 2x1 + 3x2 + e2 = 18 x1 , x2 , e1 , e2 ≥ 0 We can now identify the matrix A, the vector b and the vector c: −3 1 1 −1 0 4 4 A= , b= , c= 0 . 2 3 0 1 18 0 In this case, we cannot start the simplex algorithm by simply choosing the slack variables as basic variables, since x1 = x2 = 0 is not a feasible basic solution. Therefore, we first solve the auxiliary problem. Phase I: the auxiliary problem The goal of phase I is to find an initial tableau for the simplex algorithm, which corresponds to a feasible solution to the original problem. We introduce the auxiliary variables a1 and a2 . The auxiliary problem is defined as follows: min a1 + a2 subject to: x1 + x2 − e1 + a1 = 4 2x1 + 3x2 + e2 + a2 = 18 x1 , x2 , e1 , e2 , a1 , a2 ≥ 0. Iteration 1 We can choose as basic variables a1 and a2 . The initial tableau has 7 columns: 4 for the variables of the original problem, 2 for the auxiliary variables and 1 for the right-side column. The matrix A contains the columns corresponding to the variables of the original problem. Then 9 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) we write the identity matrix in the columns corresponding to the auxiliary variables that are in the base. For these latter columns, the value in the last row is 0, as the auxiliary variables are in the basis. Then, we write the vector b in the rightmost column. Then, we calculate the values of the last row for the columns corresponding to non basic variables, exploiting the structure of phase I of the simplex tableau, i.e. we sum the values of the rows above and we change the sign of the result. Lastly, the value in the lower-right part of the table is equal to the opposite of the sum of the elements of vector b, and is equal to the opposite of the objective function value. x1 x2 a1 e1 e2 a1 A a2 − P j a2 I2 Ai,j 0 b − 0 P j bj We can now fill the tableau: a1 a2 x1 1 2 -3 x2 1 3 -4 e1 -1 0 1 e2 0 1 -1 a1 1 0 0 a2 0 1 0 4 18 -22 The first six values of the last row correspond to the reduced costs. You can see that the opposite of the sum of the elements of the vector b is equal to the opposite of the objective function value. We identify the variable that enters the base. We apply Bland’s rule and select the leftmost variables associated with a negative reduced cost, i.e. x1 . Now we calculate the distances to the constraints: 4 θa1 = = 4, 1 18 θa2 = = 9. 2 10 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) Therefore, a1 exits the base. We can calculate the decrease in the objective function, which is equal to θa1 c̄x1 = −12. Finally, we prepare the tableau for the next iteration. First, we need to ensure that the pivot value is equal to 1. If that is not the case, we can simply divide the whole row of the pivot by the pivot value itself. We see that this step is not necessary in this case. Then, we need to make all other values in the column of the entering variable equal to 0. For the second row, we subtract twice the first row. For the last row, we add three times the first row. Iteration 2 The tableau for the second iteration is the following: x1 a2 x1 1 0 0 x2 1 1 -1 e1 -1 2 -2 e2 0 1 -1 a1 1 -2 3 a2 0 1 0 4 10 -10 We identify the variable that enters the base. We apply Bland’s rule and select the leftmost variables associated with a negative reduced cost, i.e. x2 . Now we calculate the distances to the constraints: 4 = 4, 1 10 = = 10. 1 θx1 = θa2 Therefore, x1 exits the base. We can calculate the decrease in the objective function, which is equal to θx1 c̄x2 = −4. The pivot value is equal to 1. Finally, we need to make all other values in the column of the entering variable equal to 0. For the second row, we subtract the first row. For the last row, we add the first row. Iteration 3 11 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) The tableau for the third iteration is the following: x2 a2 x1 1 -1 1 x2 1 0 0 e1 -1 3 -3 e2 0 1 -1 a1 1 -3 4 a2 0 1 0 4 6 -6 We identify the variable that enters the base. We apply Bland’s rule and select the leftmost variables associated with a negative reduced cost, i.e. e1 . Now we calculate the distances to the constraints for the positive entries of the column of the pivot: θa2 = 6 = 2. 3 Recall that the distance from the selected constraint cannot be negative. Therefore, a2 exits the base. We can calculate the decrease in the objective function, which is equal to θa2 c̄e1 = −6. The pivot value is equal to 3, so we divide the second row by 3 in order to have a pivot equal to 1. x2 a2 x1 x2 1 1 -1/3 0 1 0 e1 -1 1 -3 e 2 a1 0 1 1/3 -1 -1 4 a2 0 1/3 0 4 2 -6 Finally, we need to make all other values in the column of the entering variable equal to 0. For the first row, we add the second row. For the last row, we add three times the second row. Iteration 4 The tableau for the fourth iteration is the following: 12 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) x2 e1 x 1 x2 2/3 1 -1/3 0 0 0 e1 0 1 0 e2 a1 1/3 0 1/3 -1 0 1 a2 1/3 1/3 1 6 2 0 Since all reduced costs are non negative, we have found the optimal solution of the auxiliary problem. The solution is the following: x1 , e2 , a1 and a2 are non basic variables and are, therefore, equal to 0. The basic variables are x2 = 6 and e1 = 2. You can verify that this solution satisfies all the constraints of the original problem written in standard form. Now, we can move to solving the optimization problem using the simplex algorithm and this solution as initial feasible solution. Phase II: the initial problem Constructing the tableau for the first iteration is simple, as we only have to remove the columns of the auxiliary variables. Indeed, none of them is in the basis. Iteration 1 The tableau for the first iteration is the following: x2 e1 x1 x2 2/3 1 -1/3 0 c̄x1 0 e1 0 1 0 e2 1/3 1/3 c̄e2 6 2 −cT x We already know the reduced costs for the basic variables, but we need to compute the reduced costs of the non basic variables: 2/3 T c̄x1 = c1 − cB A1 = −3 − 4 0 = −17/3 −1/3 1/3 T −1 c̄e2 = c4 − cB B A4 = 0 − 4 0 = −4/3 1/3 13 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) We can also calculate the value of the lower-right cell, which is the value of the objective function with the opposite sign: 0 6 −cT x = − −3 4 0 0 2 = −24 0 We obtain the full tableau: x2 e1 x1 x2 2/3 1 -1/3 0 -17/3 0 e1 0 1 0 e2 1/3 1/3 -4/3 6 2 -24 We identify the variable that enters the basis. We apply Bland’s rule and select the leftmost variable with a negative reduced cost, i.e. x1 . Now we calculate the distances to the constraints for the positive entries of the column of the pivot: θx2 = 6 = 9. 2/3 Therefore, x2 exits the basis. We can calculate the decrease in the objective function, which is equal to θx2 c̄x1 = −51. The pivot value is equal to 2/3, so we multiply the first row by 3/2. x2 e1 x1 x2 1 3/2 -1/3 0 -17/3 0 e1 0 1 0 e2 1/2 1/3 -4/3 9 2 -24 Finally, we need to make all other values in the column of the entering variable equal to 0. For the second row, we add the first row multiplied by 1/3. For the last row, we add the first row multiplied by 17/3. 14 Introduction to optimization and operations research (2019–2020) Lecturer: Michel Bierlaire, Teaching assistants: Nourelhouda Dougui, Stefano Bortolomiol Simplex – correction (18 October 2019) Iteration 2 The tableau for the second iteration is the following: x1 e1 x1 1 0 0 x2 3/2 1/2 17/2 e1 0 1 0 e2 1/2 1/2 3/2 9 5 27 All reduced costs are non negative, therefore we have found the optimal solution of the initial problem. The solution is x = (9, 0, 5, 0)T , with cT x = −27. The optimal solution corresponds to point D in Figure 4, consistently with what was found by using the graphical method. 15
