Matrices
Matrices and vectors
Matrices of order m x n over a field K are represented as K^{m x n} Each matrix a ∈ K^{m x n} consists of elements a_{ij} ∈ K for i = 1, , m and j = 1, , n, arranged in a rectangular array with m rows and n columns The dimensions m and n indicate the number of rows and columns, respectively, and the arrangement of these elements uniquely defines the matrix.
The m x 1 and 1 x n matrices in particular are denoted
Km = K{_rl, Kn = K;_ Any x E Km, y E Kn are represented by arrays x~ [Il y~ [y, y.[ with a single column or row; x is distinguished as a column vector, of order m, andy as a row vector
The elements of a E K;:" have rows a(i E Kn (i = 1, ,m) and columns aj) E Km (j = 1, ,n), where
A matrix can also be displayed through its rows, or its columns:
A matrix can be represented in two forms: the row partition form, denoted as 10 MATRICES, and the column partition form, represented as a = [a1, , an] This means a matrix consists of a set of vectors derived from either its rows or its columns The transpose of a matrix, denoted as a', is formed by interchanging its rows and columns, resulting in a new matrix with elements represented as aii t.
If a is m x n, then a' is n x m, and
(a')(k = (ak))', or row k of the transpose a' is obtained by transposing column k of a
A square matrix is defined by having an equal number of rows and columns, denoted as K;: with n rows and n columns The diagonal elements of a square matrix are represented by aij, where the row index i equals the column index j, resulting in n diagonal elements aii (for i = 1 to n) In a diagonal matrix, all non-diagonal elements are zero, meaning that aij equals zero whenever i does not equal j.
A symmetric matrix is identical with its transpose, so it is necessarily a square matrix Hence for any symmetric matrix, a = a', that is, aij = ajiã
Submatrices
For a E K;:', and subsets i= (h, ,ir),j= UI,ããã ,Js), of r rows and s columns, where
1 ::; h < 0 • 0 < ir ::; m, 1 ::;}I < r), i:5r i>r and no further replacement is possible, so that tij = 0 (i,j > r), and the Yj (J > r) are expressed as combinations of the Yi (i:::; r) QED
Rank
Theorem 3.1 The maximal independent subsets of any finite set all have the same number of elements
Given two such sets, of p and q elements, since the p are independent and generated by the q we have p ~ q, by Theorem 2.1, Corollary (iii) Similarly q ~ p, and sop= q
The rank of any finite set of elements in a linear space is defined by the common number of elements in the maximal independent subsets
Theorem 3.2 Any set of r independent elements in a set of rank r is a maximal independent subset
In the presence of an additional element, we can form r+1 combinations from any maximal independent subset of r elements According to Theorem 2.1, this results in a dependent set, indicating that the independent set cannot be expanded further and is therefore maximal.
In linear algebra, the nullity of p elements with rank r is calculated as p - r When the rank equals the number of elements (r = p), the elements are considered to be of full rank, indicating their independence Consequently, the algorithm designed to identify a maximal independent subset effectively serves as an independence test, allowing for the determination of rank.
Elementary operations
There are two elementary operations, with any set of elements in a linear space
(I) Multiply one element by a non-zero scalar,
(II) Add one element to another,
Theorem 4.1 The span of any elements is unaltered by elementary operations
For is equivalent to y = (x1s)(s-1t1) +x2t2 + ããã +xptp (s =I 0), and to
Hence the span of x1, x2, 0 , Xp is unaltered when x1 is replaced by x1s (s =I 0), or by x1 + X2o
Theorem 4.2 Elementary operations are reversed by elementary operations
For the inverse of (I), we have
( , (xt) t-I, ) -+ ( , x, ) (I), which is again elementary For the inverse of (II), there is the elementary sequence
Theorem 4.3 Linear independence is preserved under elementary operations
In the argument for Theorem 4.1, withy= o, the independence of x1 , , Xp requires t1, • , tp = 0 But t1,t2 = 0 # s- 1 t1 , t2 = 0 # t1,t2 - t1 = 0
Theorem 4.4 Elements may be given any permutation by elementary operations
Any permutation being a product of transpositions, it is enough to consider a transposition
That this can be accomplished by elementary operations is seen from the sequence
Dimension
Base
A base for a linear space is any independent set of generators Theorem 1.1 Any finitely generated linear space has a base
A maximal independent subset of generators can be selected, and since all generators are included within its span, this subset forms an independent set of generators for the entire space, effectively serving as a basis.
3 1.5 provides the existence of a maximal independent subset, and a constructive procedure comes from the Replacement Theorem, Corollary ( v )
A base being so identified, arguments about rank are repeated in the following that proceed from the definition of a base
Theorem 1.2 All the bases in a finitely generated linear space have the same number of elements
According to Corollary (iv), if we have two bases consisting of p and q elements, where the first set is independent and the second set serves as generators, it follows that the independent set is contained within the span of the generators By the Replacement Theorem (Corollary iii), this implies that p is less than or equal to q (p ≤ q) Conversely, since q is also a generator for the independent set, we have q less than or equal to p (q ≤ p) Therefore, we conclude that p equals q (p = q).
The common number of elements in bases defines the dimension of the space
Theorem 1.3 Any n independent elements in an n-dimensional linear space form a base
With any further element we have a dependent set, by Corollary
(iii), so this element is in their span This reproduces Corollary (vi)
Dimension
Theorem 2.1 For finite dimensional L, if M < L, then dim M :::; dim L, dimM=dimL => M=L
In other words, the only subspace of L with the same dimension is L itself Any base for M would constitute a set of n independent elements of L, and so be a base also for L
Theorem 2.2 Any p independent elements in a finite dimensional linear space can replace some p elements in any base to form a new base
The replacement can be made without altering the span, or the independence, by the Replacement Theorem, Corollary (ii), so now there are n elements that span the space, or a base
Theorem 2.3 Every subspace of a finite dimensional linear space has a complement
Elements of a base e1 , , en for L may be replaced by the elements fi, ,fp of a base for a subspace M < L, to provide a new base for L, say
Then the subspace N < L given by
N = [ep+l, , en] is a complement of M
Theorem 2.4 For finite dimensional M, N < L, dimM V N = dimM + dim N- dimM 1\ N
Elements in bases x1, •.• ,xP andy1 , • ,yq forM and N can be replaced by the elements z1 , , Zr of a base for the common subspace M 1\ N < M, N to form new bases forM and N, say and
The elements Xr+b, Xp, Yr+l, yp, zl, and Zr span the sets M and N, indicating their independence Any linear relationship among these elements can be expressed as x = y + z, where x, y, and z are combinations of the respective elements This relationship associates an element of M with an element of N, leading to an element in the intersection M ∩ N However, these elements belong to the intersection only when x = 0, y = 0, and consequently z = 0.
To establish the independence of the subsets in these combinations, it is essential that all coefficients in the proposed linear relation equal zero This leads to the conclusion of the required independence Consequently, the dimension of the manifold, denoted as dimMV N, can be expressed as dimMV N = (p - r) + (q - r) + r, simplifying to p + q - r.
Corollary (i) For finite dimensional L and any M, N < L, dimMV N::; dimL, with equality if and only if L = M ~ N
For the equality holds if and only if M, N are independent
Corollary (ii) For finite dimensional L, and any subspaces
M,N, r) if this is a maximal replacement
The Replacement Algorithm starts with an initial table and aims for a specific termination, potentially limiting replacements to a subset of variables This approach has significant applications, particularly in the Replacement Theorem and its implications It is instrumental in identifying a maximal independent subset of elements represented as linear combinations of independent elements Furthermore, the algorithm plays a crucial role in solving systems of linear equations and extends its relevance to systems of linear inequalities, including linear programming (LP), where its broader applications have emerged.
A review of schematics in the tables (i), (ii) and (iii) will assist future reference The linear dependence table has the form y
In the context of linear algebra, the matrix \( t \) represents the coefficients that express the \( y \) vectors as combinations of the \( x \) vectors When \( x \) serves as the fundamental basis in \( K^m \), represented by the columns of the unit matrix \( 1 \) in \( K^{m \times n} \), and \( y \) consists of the columns of a matrix \( a \) in \( K^{n \times p} \), we find that \( t = a \) This relationship arises from the fact that any vector's coordinates relative to the fundamental basis are defined by its own elements, leading to a straightforward representation in this scenario.
Now consider the x and y elements partitioned as x', x" and y' ,y" and so, correspondingly, we have the partitioned table y' y" x' p u x" v d
In this process, we assume that the variables y' will replace the variables x', forming a square submatrix of order corresponding to the number of replacements made Following these substitutions, the table is transformed into y', y", y', 1, u*, x", o, and d* If this transformation represents a maximal replacement, it follows that d* equals o.
In the current scenario, we consider a single replacement scheme where p is defined as 1 x 1 with a non-zero pivot element Consequently, the term p - 1 is applicable, leading to the results: u* = p - 1 u and d* = d - vp - 1 u.
Hence we can represent the replacement as carried out by a matrix premultiplication of the coefficient matrix t, in partitioned form shown by
In the replacement process, a pivot element is chosen from the table, specifically a non-zero element located in the pivot row and column, indicating which generator element will be replaced by a combination This operation involves dividing the pivot row by the pivot element and subsequently subtracting appropriate multiples from other rows to eliminate the elements in the pivot column, resulting in zero.
Matrix rank
An m x n matrix a E K::" has row rank determined as the rank of its mrows a(iEKn(i=1, ,m),
58 REPLACEMENT when Kn is regarded as a linear space over K with these as elements Similarly, the matrix has a column rank, having reference to its n columns aj) EKm (j= 1, ,n)
In conclusion, the matrix rank defined through two distinct methods yields the same value, allowing for a clear and unambiguous definition of a matrix's rank.
The replacement operation on a linear dependence table involves performing a series of elementary row operations As outlined in § 1, these operations do not affect the row rank or row span of the table.
Consider the algorithm by which we would discover the column rank of a matrix a E K:;:' With reference to §1, we start with the scheme a
1 11 v d where a' stands for r independent columns of a, and 1' for the fundamental base vectors to be replaced by them After the replacements we have a' a" a' 1 u* (ii)
In a maximal replacement scenario, we establish that d* equals zero, leading to the determination of the column rank of matrix A as r The original independence of the sets 1' and 1" is maintained through this replacement, confirming that the subset A' remains independent Furthermore, all elements of A" can be expressed as linear combinations of A', with matrix B* serving as the coefficient provider, as indicated in the corresponding table.
This establishes that the a' are a maximal independent set of columns of a, and hence that the column rank is r
The matrix of the final table is
MATRIX RANK 59 which has r independent rows and the rest null, so the row rank is r
But row rank is unaltered by replacements Hence this is the row rank of a, now equal to the column rank
Theorem 2.1 For any matrix, row rank equals column rank
In this argument, consider a matrix with row rank \( r \) and column rank \( s \) If \( a_j \) (where \( j = 1, \ldots, s \)) forms a column basis, then each \( a_k \) can be expressed as a linear combination of these basis vectors, specifically \( a_k = \sum_{j=1}^{s} a_j l_{jk} \) for some coefficients \( l_{jk} \) This implies that the total number of independent combinations cannot exceed \( s \), leading to the conclusion that \( r \leq s \) Conversely, by applying a similar reasoning, we find \( s \leq r \) Therefore, we establish that the row rank \( r \) is equal to the column rank \( s \).
The rank of a matrix can refer to either its row rank or column rank A matrix is considered to be of full rank if it achieves full rank in either its rows or columns For an m x n matrix with rank r, full rank is achieved when r equals m if m is less than n, or r equals n otherwise.
Theorem 2.2 Any square matrix has a left inverse if and only if it has a right inverse, and this is if and only if it is of full rank; then these inverses are identical, and unique
If b is a right inverse of a, then the equation ab = 1 holds true, meaning that for each column j (where j = 1, , n), the columns of the identity matrix 1 can be represented as linear combinations of the columns of matrix a, with the coefficients provided by the corresponding columns of matrix b.
The column rank of matrix \( a \) is at most \( n \), indicating that if \( a \) has a column rank of \( n \), its columns form a basis in \( K^n \) If \( b \) represents the matrix of coordinate vectors corresponding to the columns of \( a \), then the product \( ab = I \) shows that \( b \) is a right inverse of \( a \) Thus, \( a \) possesses a right inverse if and only if its column rank is \( n \) Similarly, \( a \) has a left inverse if and only if its row rank is \( n \), and since row rank equals column rank, these conditions are equivalent Furthermore, if \( b \) and \( c \) serve as both right and left inverses, satisfying \( ab = I \) and \( ca = I \), it follows that \( c \) equals \( b \).
Similarly, were b' another right inverse, we would have b' = c, and hence b' = b; and similarly with the left QED
A matrix a is regular if it has an inverse, a matrix denoted a- 1 such that aa- 1 = 1 = a- 1 a, so it is both a left and right inverse, and unique Otherwise the matrix is singular
Corollary A square matrix is regular if and only if it is of full rank
The relationship between a matrix and its inverse reveals that if matrix b is the inverse of matrix a, then b is also regular and possesses a as its inverse Consequently, since a⁻¹ is the inverse of a, it follows that a is the inverse of a⁻¹.
The procedure for finding the rank of a matrix can, with some modification, also find the inverse, if there is one Instead of with a
1 a we start with the augmented table a 1
To analyze a matrix, we follow a procedure that limits the pivot columns by excluding fundamental base elements, ensuring that the pivot elements remain in the original table If, after maximal replacement under these restrictions, the matrix is regular, we derive the equation 1 = ab, indicating that b serves as the inverse Conversely, if the rank is less than n, the matrix is singular In this scenario, we will have identified a maximal independent set of columns, allowing us to determine the rank of the matrix.
Block replacement
With reference to §4, tables (i)' and (ii)', again consider a partitioned linear dependence table
In block replacement 61, the elements y', y", x', p, and u are arranged such that x' and y' share the same number, indicating a square formation When p is a non-zero single element, it can serve as a pivot, allowing for the replacement of x' with y' This results in the new table configuration of y', y", y', 1, u* and x", o, d*.
In this discussion, we explore the transformation of a square matrix, denoted as p, through a block replacement process that substitutes various x' values with corresponding y' values, ultimately resulting in a new table format It is important to note that this transformation is feasible if p is a regular square matrix.
In an n-dimensional space, where all elements are identified by coordinate vectors based on a common basis, we can represent the group of elements x' as a row matrix with n rows, containing the coordinate vectors of its members Similarly, other groups can be represented in the same manner According to the relationships outlined in table (i), we have the equations y' = x'p + x"v and y" = x'u + x"d, which illustrate the connections between these groups.
If \( p \) is regular with its inverse \( p^{-1} \), then it follows that \( y' p^{-1} = x' p p^{-1} + x'' v p^{-1} \) This leads to the equation \( x' = y' p^{-1} + x'' (-v p^{-1}) \) By substituting these values, we also derive \( y'' = y' p^{-1} u + x'' (d - v p^{-1} u) \) Consequently, we can summarize the results in Table (ii) with \( u^* = p^{-1} u \) and \( d^* = d - v p^{-1} u \).
This shows the block replacement, with p as the pivot block, is possible provided p is regular
To be seen now is that every multiple replacement is equivalent to a single block replacement In the passage from (i) to (ii), the
In the context of matrix operations, when the replacement matrix \( p \) is defined, it is essential to ensure that it is regular Utilizing this as a pivot block allows for a corresponding block replacement that is equivalent to multiple replacements This understanding of correspondence can further enhance the discussion on the topic.
Theorem 3.1 For every multiple replacement, the corresponding single block replacement is feasible, and equivalent to it
By performing a series of single replacements from matrix (i) to matrix (ii), all pivot elements remain within the area of matrix p, which ultimately transforms into the unit matrix 1 The transition from p to 1 involves elementary row operations that maintain the row rank Consequently, since the unit matrix 1 is regular, matrix p is also regular.
An issue that remains is to settle the following
Theorem 3.2 Any block replacement is expressible as a multiple replacement
Block replacement can be achieved through a sequence of elementary replacements, allowing for the one-at-a-time substitution of generators This process is validated by the fact that, with p being regular, replacements involving pivot elements within the range of p will ultimately reduce p to 1.
For any matrix, a critical submatrix is a square submatrix which is regular, while every square submatrix of greater order that contains it is not
Theorem 3.3 A matrix is of rank r if and only if it has a critical submatrix of order r
If matrix \( a \) has a rank of \( r \), then according to the referenced tables and the arguments presented, there exists a regular square submatrix \( p \) of order \( r \) Since the rank of a matrix is equivalent to its column rank, it follows that no columns in \( a \) or any of its submatrices can have a rank exceeding that of \( a \) Consequently, any square submatrix of order greater than \( r \) cannot be regular, establishing \( p \) as a critical submatrix.
If pin table (i) is identified as a critical submatrix of order r, it follows that the rank of matrix a must be at least r The substitution of p as the pivot block results in table (ii) If the rank of a were greater than r, then d* would not equal 0, allowing for further replacements and the discovery of a regular square submatrix that expands p However, since p is critical, this scenario cannot occur, confirming that the rank is indeed r.
Theorem 3.4 If a=[~ ~] where pis critical for a, then
The critical decomposition formula for a matrix is derived from any critical submatrix and its associated rows and columns This formula illustrates that the rows of matrix 'a' can be expressed as linear combinations of the rows from the critical submatrix, while the columns can be similarly expressed when the association is considered in the opposite direction.
With reference to the table (ii) in the last section, with d* = o we have d = vp- 1 u QED
Extended table
Replacing generators in a linear dependence table results in their loss, making the process irreversible To address this issue, the generators should be included among the generated elements We begin with the extended table format of y X x a 1 instead of the standard table y x a.
Taking this in the partitioned form y' y" x' x" x' p u 1' o x" v d o 1 11 where they' are to replace the x', sop is the pivot, the table after replacement is y' y' 1' x" o x' x" p-1 0
Now it is possible to replace generators y' by x', with p- 1 as pivot, and so return to the original table
The extended table containing the unit matrices 1' and 1" includes spurious information, as their positions can always be identified If the columns associated with these matrices were removed, they could be easily restored for replacement routines The crucial elements of both the original and final tables are represented in the condensed tables, which include y' y", x' p u, x" v d, x' y", y' p-1 p-1u, x" -vp-1, and d -vp-1 u.
The principle outlined here demonstrates that transitioning from the first element to the second allows for the restoration of the first when applied back to the second This framework thus introduces the reversibility that was previously absent.
The step from the one table to the other we call a pivot operation This form of operation is associated with A W Tucker
It is distinguished from the ordinary replacement operation, which would instead have produced the table y' y" y' 1 p- 1 u x" o d-vp- 1 u
To inform that this operation is in view, the table involved, which has been envisaged as a condensed extended table, will be called a tableau, again following Tucker's teminology
The pivot operation using a tableau is more cost-effective than the replacement operation involving an extended linear dependence table Additionally, this method exhibits a unique symmetry or duality that enhances its efficiency.
Condensed table
A matrix \( a \in K^{m \times n} \) serves as a representation of the coefficients in a linear dependence table, where \( y \) represents linear combinations of elements \( x \) from a linear space, known as generators The coefficients of these combinations are specified by the matrix \( a \).
Instead now the matrix a is to be associated with relations
CONDENSED TABLE 65 y=ax between variables given as elements of vectors x E Kn, y E Km
The pivot operation, previously discussed as a method for replacement, also holds significant relevance for the relationships defined by t = sa, where s belongs to Km and t belongs to Kn.
In the matrix partitioning, let matrix \( a \) be divided such that \( p \) represents the pivot element or block Similarly, vector \( x \) is partitioned into components \( x' \) and \( x'' \) Consequently, the relationships can be expressed as \( y' = px' + ux'' \) and \( y'' = vx' + dx'' \).
In the context of variable roles, independent variables (x) are classified as base variables, while dependent variables (y) are identified as non-base When we interchange the roles of base variable x' and non-base variable y', assuming p is non-zero, the relationship can be expressed as x' = p - 1 y' + (-p - 1 u)x" Subsequently, by substituting x' into the equation for y", we derive the expression y" = vp - 1 y' + (d - vp - 1 u)x'.
The new coefficient matrix is identical with operation
-p-lu ] d- vp- 1 u would be obtained by the pivot
The way we have put it here, there is some apparent difference
In the condensed table operation, a negative sign is applied to vp-1, while in the Tucker operation, it is applied to p-1u This correspondence necessitates a transposition or dualization, which does not alter the previously stated information, as further clarified in the following remarks.
This further view of the pivot operation as for exchange between base and non-base variables is capable of a double understanding, with reference simultaneously to the coefficient
66 REPLACEMENT matrix and its transpose Consider the similar exchange with the other considered relation, t =sa, related by transposition to the first In partitioned form, this is
The equations t' = s'p + s"v and t" = s'u + s"d illustrate that the new coefficient matrix remains consistent with previous findings This consistency allows for the understanding of pivot operations as exchanges between variables in different systems, specifically between x', y' and u', v' This dual interpretation is crucial for linear programming, particularly in the context of Dantzig's Simplex Algorithm as discussed by A W Tucker, where pivot operations on one linear programming problem correspond to similar operations on its dual counterpart.