Dual linear programming problem and its applications

Duality theorems, in particular, the Complementary slackness orem for applications for test optimization and the relationship betweenthe optimal solution of the original problem and the

MINISTRY OF EDUCATION AND TRAININGHANOI PEDAGOGICAL UNIVERSITY 2

——————–o0o———————

NGUYEN THI THUY DUONG

DUAL LINEAR PROGRAMMING PROBLEM

AND ITS APPLICATIONS

BACHELOR THESIS

Hanoi - 2019

MINISTRY OF EDUCATION AND TRAININGHANOI PEDAGOGICAL UNIVERSITY 2

——————–o0o———————

NGUYEN THI THUY DUONG

DUAL LINEAR PROGRAMMING PROBLEM

AND ITS APPLICATIONS

Major: Applied Mathematics

BACHELOR THESIS

Advisor:

Dr Nguyen Trung Dung

Hanoi - 2019

Thesis Assurance

I assure that the data and the results of this thesis are not true andnot identical to other topics I also assure that all the help for this thesishas been acknowledged and that the results presented in the thesis hasbeen identified clearly

Student

Nguyen Thi Thuy Duong

Thesis Acknowledgement

First and foremost, my heartfele goes to my Advisor Dr NguyenTrung Dung (Hanoi Pedagogical University No2), for his continuous sup-ports when I met obstacles during the journey The completion of thisstudy would not have been possible without his expert advice, close at-tention and unswerving guidance

Secondly, I am keen to express my deep gratitude for my familyfor encourage me to continute this thesis I owe my special thanks to

my parents for their emotional and material sacrifices as well as theirunderstanding and unconditional supports

Finally, I own my thanks to many people who helped me and courage me during my work I am specifically thankful to all my bestfriends at university for endless encouragement

en-Student

Nguyen Thi Thuy Duong

Thesis Assurance 1

Thesis Acknowledgement 2

Notations 4

Introduction 6

1 DUAL LINEAR PROGRAMING PROBLEM 8

1.1 The Linear Programming Problem 8

1.1.1 The general form of LPP 8

1.1.2 The standard form of LPP 10

1.1.3 The canonical form of LPP 11

1.2 The Dual Problem 12

1.2.1 Lagrange Function and Saddle Point 12

1.2.2 Standard Form of Duality 13

1.2.3 Canonical Form of Duality 15

1.2.4 General Form of Duality 16

1.3 Primal-Dual Relationships 18

2 DUAL SIMPLEX ALGORITHM 22

2.1 The basic of the dual simplex algorithm 22

2.2 The dual simplex algorithm 25

2.3 Using Matlab to solve LPP 29

2.4 Applications of the dual linear programming 32

2.4.1 Checking a feasible solution whether or not an opti-mal solution 32

2.4.2 Find the optimal solution set of dual problem 34CONCLUSION 36REFERENCE 37

R+ Set of non-negative real numbers

Rn n-dimensional Euclidean vector space

A> Transpose matrix of matrix A

1 Motivation

Duality theory is an important part of optimization theory and hasmany applications in reality For an optimal problem, mathematiciansresearched a problem closely related to it that is called duality problem

If the Origin problem is a minimum problem, the Dual problem will bethe maximum problem, it is expected that the dual problem is easier

to handle than the origin problem From that desire, new methods andsoftware are created to help solve those problems more easily On theother hand, mathematicians also researched the practical applications ofduality problem through the theory system

Duality theorems, in particular, the Complementary slackness orem for applications for test optimization and the relationship betweenthe optimal solution of the original problem and the corresponding dualproblem The theory of duality and application has been researched bymany domestic and foreign authors and obtained many impressive re-sults Due to the enthusiastic help of Dr Nguyen Trung Dung, alongwith my desire to research more deeply about the Dual problem, I wouldlike to choose the topic ”Dual linear programming problem and itsapplications” as my research topic.The thesis focuses on the issues ofdual theory and applications including statements and how to construct adual problem from the original problem; Understanding the relationshipbetween the original problem and the dual problem; How to use DualSimplex Algorithm solving primal linear programming problem; UsingMatlab to find solutions of linear programming problems; Application ofduality problem through weak theorem of offset deviation

the-2 Thesis Objectives

The main purpose of the thesis is to understand the duality problemand its applications More specifically, the thesis focuses on the followingtwo main topics:

1 Dual linear programing problem and its applications

2 Dual Simplex algorithm

3 Research methods

This thesis uses methods to collect, synthesize, analyze and research

on documents

4 Thesis organization

This thesis is organised as follows:

• Chapter 1 Dual linear programing problem

In this chapter, Some basic concepts involving linear programmingproblems; how to construct a Dual linear programming problem fromthe forms of an original Linear programing problem; Relationshipsbetween a pair of primal-dual linear problem is presented, respec-tively

• Chapter 2 Dual Simplex Algorithm

This chapter addresses the ways to solve a Primal linear ming problem by using Dual Simplex Algorithm and using Matlab

program-to solve the Linear programming problems

Chapter 1

DUAL LINEAR PROGRAMING PROBLEM

In this chapter, we recall some concepts and results of dual linearprogramming problems that using to next chapter

1.1 The Linear Programming Problem

In this section, we introduce some forms of linear programmingproblems such as general form, standard form, and canonical form.1.1.1 The general form of LPP

Find a vector x = (x1, x2, , xn)T ∈ Rn such that

Common terminology for the LPP can be summarized as follows:

• The variables x1, x2, , xn are called the decision variables

• The constants cj, bi, aij are the input constants of LPP.

• The function being minimized or maximized f (x) is called the jective function or cost function

ob-• The restrictions are referred to as constraints

• The first m constraints (those a function of all variables

n

X

j=1

aijxj onthe left-hand side) are called functional constraints

• The variables xj ≥ 0, xj ≤ 0 are called the sign constraints

For the LPP (1.1), we have some simple properties as follows:

1 Because maximizing a quantity is eqivalent to minimizing its tive, so each maximization problem can be transformed to a mini-mization problem:

nega-f (x∗) = min{f (x), x ∈ D} ⇐⇒ −f (x∗) = max{−f (x), x ∈ D}

2 Each inequality can be transformed to an equality by introducing

an extra nonnegative variable:

5 Each unrestricted variable xj can be eliminated by the followingvariable substitution:

xj = x0j − x00j, x0j ≥ 0, x00j ≥ 0

From the above properties, we can convert any LP problems into one of

two forms: standard form or canonical form that will introduce in the

next section

1.1.2 The standard form of LPP

We say that a linear programming problem is in standard form if ithas the following form:

Find a vector x = (x1, x2, , xn)T ∈ Rn such that

the standard form of LPP (1.2) can be put into compact form below

(so-called matrix form of LPP):

1.1.3 The canonical form of LPP

We say that a linear programming problem is in canonical form if

it has the following form:

Find a vector x = (x1, x2, , xn)T ∈ Rn such that

Using the same notations as the standard form, the canonical form

of LPP (1.4) can be put into compact form below (so-called matrix form

So, the standard form of LPP (1.6) as the follows:

−2x1 + 3x2 − 4x3 + 5x4 → minsubject to

x1, x2, x3, x4, x5, x6 ≥ 0

1.2 The Dual Problem

In this section, we recall Lagrange Function and Saddle Point Weestablish Standard form of duality, Canonical form of duality and Gen-eral form of duality

1.2.1 Lagrange Function and Saddle Point

Consider a linear programming problem in standard form as follows:

f (x) = c>x → minsubject to

Definition 1.2.1 (Lagrange Function) A function L(x, y) is called theLagrange function of problem (P) if it has the following form:

L(x, y) = c>x + (b − Ax)>y, x ∈ Rn+, y ∈ Rm+Definition 1.2.2 (Saddle Point) A point (x∗, y∗) ∈ Rn+ ×Rm

+ is calledthe saddle point if the following inequality holds:

L(x∗, y) ≤ L(x∗, y∗) ≤ L(x, y∗), ∀(x, y) ∈ Rn+×Rm+

Figure 1.1: The saddle point (x∗, y∗).

1.2.2 Standard Form of Duality

Obiviously, for each x ∈ Rn+ satisfying Ax ≥ b implies that b − Ax ≤

Therefore, the linear programming (P) is equivalent to the followingproblem:

The problem (1.7) is called the primal problem, the problem (1.8) iscalled the dual problem.

Next, we will find a linear programming problem so that it is alent to the problem (1.8) Letting

g(y) = 8y1 − 4y2 + 6y3 → max

−3y1 +y2 −5y3 ≤ −3

yi ≥ 0, i = 1, 3

1.2.3 Canonical Form of Duality

Consider a linear programming problem in canonical form as follows:

f (x) = c>x → minsubject to

g(y) = 12y1 + 4y2 + 6y3 → max

−3y1 +y2 −5y3 ≤ 4

y1, y2, y3 free

1.2.4 General Form of Duality

Correspondence between primal and dual problems are summarized

to the following table:

Primal Problem Index Dual Problem

Table 1.1: Relationships Between Primal and Dual Problems

Example 1.2.3 Find the dual problem of the following primal problem:

f (x) = 3x1 + 2x2 + 4x3 − x4 → minsubject to

x1 ≥ 0, x2 ≤ 0, x3 ≥ 0, x4 free

Applying the rules in Table 1.1, the dual problem is

g(y) = 12y1 + 4y2 + 6y3 → max

−3y1 +5y2 −5y3 ≤ 4

y1 −2y2 −y3 = −1

y1 ≥ 0, y2 free, y3 ≤ 0

Proof By Theorem 1.3.1, we have f (x∗) = g(y∗) ≤ f (x) for all x ∈ DP.Therefore, x∗ is an optimal solution of the primal problem Analogously,

y∗ is an optimal solution of the primal problem

From Theorem 1.3.1, we have the following corollary.

Corollary 1.3.3 For the pair of the primal and dual problem ((P), (Q)),

we have the following statements:

i) If the primal problem is unbounded below, then the dual problem has

no feasible solution

ii) If the dual problem is unbounded upper, then the primal problem has

no feasible solution

Theorem 1.3.4 For the pair of the primal and dual problem ((P), (Q)),

we have the following statements:

i) If x∗ is an optimal solution of the primal problem, then the dualproblem has an optimal solution y∗ Moreover, we have f (x∗) =g(y∗)

ii) If y∗ is an optimal solution of the dual problem, then the primalproblem has an optimal solution x∗ Moreover, we have f (x∗) =g(y∗)

Proof First, we prove the statement i) Without lost of generality, weassume x∗ is an optimal basic feasible solution of the primal problemwith the basis J Since x∗j = 0, ∀j ∈ J , so we have

AJx∗ = b or x∗J = A−1J b (1.11)

On the other hand, x∗ is the optimal basic feasible solution, i.e., ∆k ≤ 0for all k = 1, 2, , n This implies

c>Jzk − ck ≤ 0, k = 1, 2, , n, (1.12)where zk is the coordinates of the column vector Ak with respect to thebasis J , i.e, Ak = AJzk Substituting zk = A−1J Ak into (1.12), we have

c>Jzk − ck ≤ 0, k = 1, 2, , n, (1.13)or

c>JA−1J Ak − ck ≤ 0, k = 1, 2, , n (1.14)

This yields

c>JA−1J A − c ≤ 0 (1.15)Letting y∗ = (A−1J )>cJ From (1.15), we have A>(A−1J )>cJ − c ≤ 0this implies A>y∗ ≤ c, i.e., y∗ ∈ DQ Moreover, we have

g(y∗) = b>y∗ = b>(A−1J )>cJ

= c>JA−1J b = c>Jx∗J = f (x∗)

By Theorem 1.3.2, y∗ is the optimal solution of the dual problem

Using the fact that the dual of the dual problem is the primal lem, so the statement ii) also holds

prob-From Theorem 1.3.2 and 1.3.4 we have the following corollary.Corollary 1.3.5 Assume that x∗ ∈ DP and y∗ ∈ DQ Then x∗ is anoptimal of (P) and y∗ is an optimal of (Q) if and only if f (x∗) = g(y∗).Next, we consider a pair of primal and dual problems in standardform as follows:

f (x) = c>x → minsubject to

This condition is eqivalent to α + β = 0.

On the other hand, we have

Chapter 2

DUAL SIMPLEX ALGORITHM

In the last chapter, we know that for any linear programming lem, there is a corresponding dual linear programming problem Thedual linear programming problem which constructed from the cost andconstraints of the original linear programming problem The originallinear programming problem is called primal linear programming prob-lem Moreover, a linear programming problem is solved by the simplexmethod Therefore, a dual linear programming problem can also besolved using the simplex method because every dual linear programmingproblem is a linear programming problem Duality is used to improvethe performance of the simplex algorithm (leading to dual algorithm)

prob-In this chapter we present the dual simplex algorithm to solve LPP

2.1 The basic of the dual simplex algorithm

The dual simplex algorithm is an attractive alternative for solvinglinear programming problems (LPs) The dual simplex algorithm is veryefficient on many types of problems and is especially useful in integerlinear programming

Consider a pair of primal and dual problems in standard form asfollows:

f (x) = c>x → minsubject to

g(y) = b>y → maxsubject to

Assume that rank(A) = m and the vectors {Aj, j ∈ J } consisting

m column vectors of the matrix A are linearly independent Then thecolumn vectors {Aj, j ∈ J } is called a basis of the matrix A and denoted

by AJ Denote the set K = {1, 2, , n} \ J The basic solution (xJ, xK)

of the problem (P) corresponding to the basis J is defined by

cor-J is an optimal solution

Definition 2.1.2 A vector y is called a dual basic solution ing to the basis J if y = (A>J)−1cJ ∈ DQ A basis J is called a dualfeasible basis if the corresponding dual basic solution is a feasible solu-tion of dual problem (Q)

correspond-Example 2.1.1 Consider the following linear programming problem

f (x) = x1 + x2 + x3 → minsubject to

We have that det(AJ) = −3 6= 0 where AJ = 

On the other hand, we also have A−1J = −1

A>y ≤ c or A>(A>J)−1cJ ≤ c

Propo-is the optimal condition for the dual simplex algorithm.

Proposition 2.1.2 Consider the primal-dual problem ((P), (Q)) sume that J is a dual feasible basis then if there exists an index j ∈

As-J, xj < 0 such that zjk ≥ 0 for all k = 1, 2, , n , the problem (P) has

no feasible solution

Proof By contradiction, we assume that the problem (P) has a feasiblesolution, i.e., there exists x ∈ Rn such that x ≥ 0 and Ax = b Thisimplies that A−1J A = A−1J b or Pn

k=1zjkxk = xj This contradicts theassumption that zjk ≥ 0 and xk ≥ 0 for all k = 1, 2, , n Thus, theproblem (P) has no feasible solution

Remark 2.1.2 From Proposition 2.1.1, assume that J is a dual feasiblebasis, then the condition for the problem (P) has no feasible solution

if there exists an index j ∈ J, xj < 0 such that zjk ≥ 0 for all k =

1, 2, , n

2.2 The dual simplex algorithm

Assume that J = {1, 2, , m} is a dual feasible basis Then wehave the dual simplex tableau corresponding to J as follows:

Basis cJ xJ c1 c2 ck cn

J A1 A2 Ak An

A1 c1 x1 z11 z12 z1k z1n

A2 c2 x2 z21 z22 z2k z2n

Am cm xm zm1 zm2 zmk zmn

f (x) ∆1 ∆2 ∆k ∆n

We note that the column xJ is defined by xJ = A−1J b Then we havesteps of the dual simplex algorithm as follows:

Step 1 Testing the optimality criterion:

1 If xj ≥ 0 for all j ∈ J, then J is an optimal basis and stop

2 Otherwise, i.e., there exists an index j ∈ J such that xj < 0,then go to the Step 2

Step 2 Testing the feasible region of primal problem is empty:

1 If there exists j ∈ J with xj < 0 such that zjk ≥ 0 for all

k = 1, 2, , n, then the the feasible region of primal problem isempty

2 Otherwise, we choose pivot row r and pivot column s as follows:

Remark 2.2.1 We update the new tableau by pivoting at zrs as follows:

• Replace Ar, xr, and cr with As, xs, and cs, respectively