Matrix mathematics theory, facts, and formulas, second edition

Copyright c2009 by Princeton University PressPublished by Princeton University Press, 41 William Street, Princeton, New Jersey 08540 In the United Kingdom: Princeton University Press, 6

Range Herm

i t i a

Group In ert

ib Normal

Matrix Mathematics

Theory, Facts, and Formulas

Dennis S Bernstein

PRINCETON UNIVERSITY PRESS

PRINCETON AND OXFORD

Copyright c2009 by Princeton University Press

Published by Princeton University Press,

41 William Street, Princeton, New Jersey 08540

In the United Kingdom: Princeton University Press,

6 Oxford Street, Woodstock, Oxfordshire, 0X20 1TW

All Rights Reserved

Library of Congress Cataloging-in-Publication Data

Bernstein, Dennis S., 1954–

Matrix mathematics: theory, facts, and formulas / Dennis S Bernstein – 2nd ed.

p cm.

Includes bibliographical references and index.

ISBN 978-0-691-13287-7 (hardcover : alk paper)

ISBN 978-0-691-14039-1 (pbk : alk paper)

1 Matrices 2 Linear systems I Title.

QA188.B475 2008

512.9’434—dc22

2008036257 British Library Cataloging-in-Publication Data is available

This book has been composed in Computer Modern and Helvetica.

The publisher would like to acknowledge the author of this volume for providing the

camera-ready copy from which this book was printed.

Printed on acid-free paper. ∞

www.press.princeton.edu

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

To the memory of my parents

vessels, unable to contain the great light flowing into them, shatter and break the remains of the broken vessels fall into the lowest world, where

they remain scattered and hidden

— D W Menzi and Z Padeh,

The Tree of Life, Chayyim Vital’s Introduction to the Kabbalah of Isaac Luria, Jason Aaronson,

Northvale, 1999

Thor placed the horn to his lips He drank with all his might and kept

drinking as long as ever he was able; when he paused to look, he could see that the

level had sunk a little, for the other end lay out in the ocean itself.

— P A Munch, Norse Mythology,

AMS Press, New York, 1970

1.9 Facts on Scalar Identities and Inequalities in One Variable 221.10 Facts on Scalar Identities and Inequalities in Two Variables 301.11 Facts on Scalar Identities and Inequalities in Three Variables 391.12 Facts on Scalar Identities and Inequalities in Four Variables 461.13 Facts on Scalar Identities and Inequalities in Six Variables 471.14 Facts on Scalar Identities and Inequalities in Eight Variables 471.15 Facts on Scalar Identities and Inequalities in n Variables 481.16 Facts on Scalar Identities and Inequalities in 2n Variables 601.17 Facts on Scalar Identities and Inequalities in 3n Variables 671.18 Facts on Scalar Identities and Inequalities in Complex Variables 68

2.7 The Determinant 102

2.9 Facts on Polars, Cones, Dual Cones, Convex Hulls, and

2.11 Facts on the Range, Rank, Null Space, and Defect of

2.12 Facts on the Inner Product, Outer Product, Trace, and Matrix

4.4 Eigenvalues 239

4.7 Rational Transfer Functions and the Smith-McMillan

5.12 Facts on Eigenvalues and Singular Values for Two or More

6.3 Facts on the Moore-Penrose Generalized Inverse for One

7.3 Schur Product 404

8.10 Facts on Identities and Inequalities for Two or More Matrices 4568.11 Facts on Identities and Inequalities for Partitioned Matrices 467

8.18 Facts on Eigenvalues and Singular Values for Two or More

9.11 Facts on Matrix Norms and Eigenvalues Involving One Matrix 5969.12 Facts on Matrix Norms and Eigenvalues Involving Two or More

9.14 Facts on Matrix Norms and Singular Values for Two or More

9.16 Notes 619

11.14 Facts on the Matrix Exponential for Two or More Matrices 68111.15 Facts on the Matrix Exponential and Eigenvalues,

11.16 Facts on the Matrix Exponential and Eigenvalues,

12.5 Detectability 734

12.19 Positive-Semidefinite and Positive-Definite Solutions of the

Preface to the Second Edition

This second edition of Matrix Mathematics represents a major expansion of the

original work While the total number of pages is increased 46% from 752 to 1100,the increase is actually greater since this edition is typeset in a smaller font tofacilitate a manageable physical size

The second edition expands on the first edition in several ways For example,the new version includes material on graphs (developed within the framework ofrelations and partially ordered sets), as well as alternative partial orderings ofmatrices, such as rank subtractivity, star, and generalized L¨owner This edition alsoincludes additional material on the Kronecker canonical form and matrix pencils;realizations of finite groups; zeros of multi-input, multi-output transfer functions;identities and inequalities for real and complex numbers; bounds on the roots ofpolynomials; convex functions; and vector and matrix norms

The additional material as well as works published subsequent to the first editionincreased the number of cited works from 820 to 1503, an increase of 83% Toincrease the utility of the bibliography, this edition uses the “back reference” feature

of LATEX, which indicates where each reference is cited in the text As in the firstedition, the second edition includes an author index The expansion of the firstedition resulted in an increase in the size of the index from 108 pages to 156 pages.The first edition included 57 problems, while the current edition has 73 Theseproblems represent various extensions or generalizations of known results, some-times motivated by gaps in the literature

In this edition, I have attempted to correct all errors that appeared in the firstedition As with the first edition, readers are encouraged to contact me about errors

or omissions in the current edition, which I will periodically update on my homepage

Acknowledgments

I am grateful to many individuals who graciously provided useful advice and terial for this edition Some readers alerted me to errors, while others suggestedadditional material In other cases I sought out researchers to help me understandthe precise nature of interesting results At the risk of omitting those who were help-ful, I am pleased to acknowledge the following: Mark Balas, Jason Bernstein, VijayChellaboina, Sever Dragomir, Harry Dym, Masatoshi Fujii, Rishi Graham, Was-sim Haddad, Nicholas Higham, Diederich Hinrichsen, Iman Izadi, Pierre Kabamba,

ma-Marthe Kassouf, Christopher King, Michael Margliot, Roy Mathias, Peter Mercer,Paul Otanez, Bela Palancz, Harish Palanthandalam-Madapusi, Fotios Paliogiannis,Wei Ren, Mario Santillo, Christoph Schmoeger, Wasin So, Robert Sullivan, YonggeTian, Panagiotis Tsiotras, G¨otz Trenkler, Chenwei Zhang, and Fuzhen Zhang.

As with the first edition, I am especially indebted to my family, who endured threemore years of my consistent absence to make this revision a reality It is clear thatany attempt to fully embrace the enormous body of mathematics known as matrixtheory is a neverending task After committing almost two decades to the project, Iremain, like Thor, barely able to perceive a dent in the vast knowledge that resides

in the hundreds of thousands of pages devoted to this fascinating and incrediblyuseful subject Yet, it my hope, that this book will prove to be valuable to all ofthose who use matrices, and will inspire interest in a mathematical constructionwhose secrets and mysteries know no bounds

Dennis S BernsteinAnn Arbor, Michigandsbaero@umich.eduOctober 2008

Preface to the First Edition

The idea for this book began with the realization that at the heart of the solution

to many problems in science, mathematics, and engineering often lies a “matrixfact,” that is, an identity, inequality, or property of matrices that is crucial to thesolution of the problem Although there are numerous excellent books on linearalgebra and matrix theory, no one book contains all or even most of the vast number

of matrix facts that appear throughout the scientific, mathematical, and engineeringliterature This book is an attempt to organize many of these facts into a referencesource for users of matrix theory in diverse applications areas

Viewed as an extension of scalar mathematics, matrix mathematics provides themeans to manipulate and analyze multidimensional quantities Matrix mathematicsthus provides powerful tools for a broad range of problems in science and engineer-ing For example, the matrix-based analysis of systems of ordinary differential equa-tions accounts for interaction among all of the state variables The discretization ofpartial differential equations by means of finite differences and finite elements yieldslinear algebraic or differential equations whose matrix structure reflects the nature

of physical solutions [1238] Multivariate probability theory and statistical analysisuse matrix methods to represent probability distributions, to compute moments,and to perform linear regression for data analysis [504, 606, 654, 702, 947, 1181].The study of linear differential equations [691, 692, 727] depends heavily on matrixanalysis, while linear systems and control theory are matrix-intensive areas of en-gineering [3, 65, 142, 146, 311, 313, 348, 371, 373, 444, 502, 616, 743, 852, 865, 935,

1094, 1145, 1153, 1197, 1201, 1212, 1336, 1368, 1455, 1498] In addition, matricesare widely used in rigid body dynamics [26, 726, 733, 789, 806, 850, 970, 1026,

1068, 1069, 1185, 1200, 1222, 1351], structural mechanics [863, 990, 1100], tational fluid dynamics [305, 479, 1426], circuit theory [30], queuing and stochasticsystems [642, 919, 1034], econometrics [403, 948, 1119], geodesy [1241], game theory[225, 898, 1233], computer graphics [62, 498], computer vision [941], optimization[255, 374, 953], signal processing [702, 1163, 1361], classical and quantum infor-mation theory [353, 702, 1042, 1086], communications systems [778, 779], statistics[580, 654, 948, 1119, 1177], statistical mechanics [16, 159, 160, 1372], demography[297, 805], combinatorics, networks, and graph theory [165, 128, 179, 223, 235, 266,

compu-269, 302, 303, 335, 363, 405, 428, 481, 501, 557, 602, 702, 844, 920, 931, 1143, 1387],optics [549, 659, 798], dimensional analysis [641, 1252], and number theory [841]

In all applications involving matrices, computational techniques are essential forobtaining numerical solutions The development of efficient and reliable algorithmsfor matrix computations is therefore an important area of research that has been

extensively developed [95, 304, 396, 569, 681, 683, 721, 752, 1224, 1225, 1227, 1229,

1315, 1369, 1427, 1431, 1433, 1478] To facilitate the solution of matrix problems,entire computer packages have been developed using the language of matrices How-ever, this book is concerned with the analytical properties of matrices rather thantheir computational aspects

This book encompasses a broad range of fundamental questions in matrix ory, which, in many cases can be viewed as extensions of related questions in scalarmathematics A few such questions follow

the-What are the basic properties of matrices? How can matrices becharacterized, classified, and quantified?

How can a matrix be decomposed into simpler matrices? A matrixdecomposition may involve addition, multiplication, and partition.Decomposing a matrix into its fundamental components providesinsight into its algebraic and geometric properties For example, thepolar decomposition states that every square matrix can be written

as the product of a rotation and a dilation analogous to the polarrepresentation of a complex number

Given a pair of matrices having certain properties, what can beinferred about the sum, product, and concatenation of these matrices?

In particular, if a matrix has a given property, to what extent does thatproperty change or remain unchanged if the matrix is perturbed byanother matrix of a certain type by means of addition, multiplication,

or concatenation? For example, if a matrix is nonsingular, how largecan an additive perturbation to that matrix be without the sumbecoming singular?

How can properties of a matrix be determined by means of simpleoperations? For example, how can the location of the eigenvalues of amatrix be estimated directly in terms of the entries of the matrix?

To what extent do matrices satisfy the formal properties of the realnumbers? For example, while 0≤ a ≤ b implies that a r ≤ b r for real

numbers a, b and a positive integer r, when does 0 ≤ A ≤ B imply

This book is intended to be useful to at least four groups of readers Sincelinear algebra is a standard course in the mathematical sciences and engineering,graduate students in these fields can use this book to expand the scope of their

linear algebra text For instructors, many of the facts can be used as exercises toaugment standard material in matrix courses For researchers in the mathematicalsciences, including statistics, physics, and engineering, this book can be used as

a general reference on matrix theory Finally, for users of matrices in the appliedsciences, this book will provide access to a large body of results in matrix theory

By collecting these results in a single source, it is my hope that this book will prove

to be convenient and useful for a broad range of applications The material in thisbook is thus intended to complement the large number of classical and modern textsand reference works on linear algebra and matrix theory [10, 376, 503, 540, 541,

558, 586, 701, 790, 872, 939, 956, 963, 1008, 1045, 1051, 1098, 1143, 1194, 1238].After a review of mathematical preliminaries in Chapter 1, fundamental proper-ties of matrices are described in Chapter 2 Chapter 3 summarizes the major classes

of matrices and various matrix transformations In Chapter 4 we turn to mial and rational matrices whose basic properties are essential for understandingthe structure of constant matrices Chapter 5 is concerned with various decompo-sitions of matrices including the Jordan, Schur, and singular value decompositions.Chapter 6 provides a brief treatment of generalized inverses, while Chapter 7 de-scribes the Kronecker and Schur product operations Chapter 8 is concerned withthe properties of positive-semidefinite matrices A detailed treatment of vector andmatrix norms is given in Chapter 9, while formulas for matrix derivatives are given

polyno-in Chapter 10 Next, Chapter 11 focuses on the matrix exponential and stabilitytheory, which are central to the study of linear differential equations In Chapter

12 we apply matrix theory to the analysis of linear systems, their state space alizations, and their transfer function representation This chapter also includes adiscussion of the matrix Riccati equation of control theory

re-Each chapter provides a core of results with, in many cases, complete proofs.Sections at the end of each chapter provide a collection of Facts organized to cor-respond to the order of topics in the chapter These Facts include corollaries andspecial cases of results presented in the chapter, as well as related results that gobeyond the results of the chapter In some cases the Facts include open problems,illuminating remarks, and hints regarding proofs The Facts are intended to providethe reader with a useful reference collection of matrix results as well as a gateway

to the matrix theory literature

Acknowledgments

The writing of this book spanned more than a decade and a half, during whichtime numerous individuals contributed both directly and indirectly I am grate-ful for the helpful comments of many people who contributed technical materialand insightful suggestions, all of which greatly improved the presentation and con-tent of the book In addition, numerous individuals generously agreed to readsections or chapters of the book for clarity and accuracy I wish to thank JasimAhmed, Suhail Akhtar, David Bayard, Sanjay Bhat, Tony Bloch, Peter Bullen,Steve Campbell, Agostino Capponi, Ramu Chandra, Jaganath Chandrasekhar,Nalin Chaturvedi, Vijay Chellaboina, Jie Chen, David Clements, Dan Davison,Dimitris Dimogianopoulos, Jiu Ding, D Z Djokovic, R Scott Erwin, R W Fare-brother, Danny Georgiev, Joseph Grcar, Wassim Haddad, Yoram Halevi, JesseHoagg, Roger Horn, David Hyland, Iman Izadi, Pierre Kabamba, Vikram Kapila,

Fuad Kittaneh, Seth Lacy, Thomas Laffey, Cedric Langbort, Alan Laub, der Leonessa, Kai-Yew Lum, Pertti Makila, Roy Mathias, N Harris McClamroch,Boris Mordukhovich, Sergei Nersesov, JinHyoung Oh, Concetta Pilotto, HarishPalanthandalum-Madapusi, Michael Piovoso, Leiba Rodman, Phil Roe, CarstenScherer, Wasin So, Andy Sparks, Edward Tate, Yongge Tian, Panagiotis Tsiotras,Feng Tyan, Ravi Venugopal, Jan Willems, Hong Wong, Vera Zeidan, Xingzhi Zhan,and Fuzhen Zhang for their assistance Nevertheless, I take full responsibility forany remaining errors, and I encourage readers to alert me to any mistakes, correc-tions of which will be posted on the web Solutions to the open problems are alsowelcome.

Alexan-Portions of the manuscript were typed by Jill Straehla and Linda Smith at HarrisCorporation, and by Debbie Laird, Kathy Stolaruk, and Suzanne Smith at theUniversity of Michigan John Rogosich of Techsetters, Inc., provided invaluableassistance with LATEX issues, and Jennifer Slater carefully copyedited the entiremanuscript I also thank JinHyoung Oh and Joshua Kang for writing C code torefine the index

I especially thank Vickie Kearn of Princeton University Press for her wise ance and constant encouragement Vickie managed to address all of my concernsand anxieties, and helped me improve the manuscript in many ways

guid-Finally, I extend my greatest appreciation for the (uncountably) infinite patience

of my family, who endured the days, weeks, months, and years that this projectconsumed The writing of this book began with toddlers and ended with a teenagerand a twenty-year old We can all be thankful it is finally finished

Dennis S BernsteinAnn Arbor, Michigandsbaero@umich.eduJanuary 2005

R (p 6)

1.3.9)

Chapter 2

real diagonal entries (p 80)

dmin(A)= dn(A) smallest diagonal entry of A ∈ F n ×n having

real diagonal entries (p 80)

of A listed inS1 and the columns of A listed

coS convex hull ofS (p 89)

deleting rowi(A) and colj(A) (p 105)

diag(A1, , A k) block-diagonal matrix

U(n), O(n), U(n, m),

O(n, m), SU(n), SO(n),

SympF(2n), OSympF(2n),

AffF(n), SEF(n), TransF(n)

groups (p 172)

projector I − A corresponding to the idempotent matrix or projector A (p 175)

polynomial matrices with coefficients inF)(p 234)

(p 237)

eigenvalues (p 240)

λ i(A) ith largest eigenvalue of A ∈ F n ×nhaving real

multiplicity having negative, zero, andpositive real part, respectively (p 245)

(p 245)

rational transfer functions) (p 249)

(SISO proper rational transfer functions)(p 249)

rational transfer functions) (p 249)

Fn ×m

proper rational transfer functions) (p 249)

prop(s) (p 249)

(p 251)

(p 251)

mbzeros(G) multiset of blocking zeros of G ∈ F n ×m (s)

(p 251)

Chapter 5

(p 283)

circ(a0 , , a n −1) circulant matrix of a0 , , a n −1 ∈ F (p 355,

α(p 404)

Acol column norm

A 1,1= maxi∈{1, ,m} col i(A) 1(p 556)

(p 556)

induced lower bound of A (p 558)

Chapter 10

Chapter 11

L Laplace transform (p 646)

Conventions, Notation, and Terminology

When a word is defined, it is italicized

The definition of a word, phrase, or symbol should always be understood as an “ifand only if” statement, although for brevity “only if” is omitted The symbol=

means equal by definition, where A = B means that the left-hand expression A is  defined to be the right-hand expression B.

Analogous statements are written in parallel using the following style: If n is (even, odd), then n + 1 is (odd, even).

The variables i, j, k, l, m, n always denote integers Hence, k ≥ 0 denotes a ative integer, k ≥ 1 denotes a positive integer, and the limit lim k →∞ A k is taken

nonneg-over positive integers

The imaginary unit√

−1 is always denoted by dotless j.

The letter s always represents a complex scalar The letter z may or may not

represent a complex scalar

The inequalities c ≤ a ≤ d and c ≤ b ≤ d are written simultaneously as



a b



≤ d.

The prefix “non” means “not” in the words nonconstant, nonempty, nonintegral,nonnegative, nonreal, nonsingular, nonsquare, nonunique, and nonzero In sometraditional usage, “non” may mean “not necessarily.”

“Increasing” and “decreasing” indicate strict change for a change in the argument.The word “strict” is superfluous, and thus is omitted Nonincreasing means nowhereincreasing, while nondecreasing means nowhere decreasing

Multisets can have repeated elements Hence, {x}ms and {x, x}ms are different

The listed elements α, β, γ of the conventional set {α, β, γ} need not be distinct.

For example,{α, β, α} = {α, β}.

The order in which the elements of the set {x1, , x n } and the elements of the

multiset{x1, , x n }ms are listed has no significance The components of the tuple (x1 , , x n) are ordered.

n-The notation (xi) ∞ i=1 denotes the sequence (x1 , x2, ) A sequence can be viewed

as an infinite-tuple, where the order of components is relevant and the componentsneed not be distinct

The composition of functions f and g is denoted by f • g The traditional notation

f ◦ g is reserved for the Schur product.

S1⊂ S2 means that S1is a proper subset of S2, whereas S1⊆ S2 means that S1

is either a proper subset of S2 or is equal toS2 Hence, S1⊂ S2 is equivalent to

The terminology “graph” corresponds to what is commonly called a “simple rected graph,” while the terminology “symmetric graph” corresponds to a “simpleundirected graph.”

di-The range of cos−1 is [0, π], the range of sin −1is [−π/2, π/2], and the range of tan −1

is (−π/2, π/2) The angle between two vectors is an element of [0, π] Therefore, the

inner product of two vectors can be used to compute the angle between two vectors.0!= 1 

For all square matrices A, A0= I In particular, 0  0

n ×n = In With this convention, 

for all−1 < α < 1 Of course, lim x ↓00x= 0, limx↓0 x0= 1, and limx↓0 x x= 1.

Neither ∞ nor −∞ is a real number However, some operations are defined for

these objects as extended real numbers, such as∞ + ∞ = ∞, ∞∞ = ∞, and, for all nonzero real numbers α, α ∞ = sign(α)∞ 0∞ and ∞ − ∞ are not defined See

[68, pp 14, 15]

1/ ∞ = 0 

Let a and b be real numbers such that a < b A finite interval is of the form (a, b), [a, b), (a, b], or [a, b], whereas an infinite interval is of the form ( −∞, a), (−∞, a], (a, ∞), [a, ∞), or (−∞, ∞) An interval is either a finite interval or an infinite inter- val An extended infinite interval includes either ∞ or −∞ For example, [−∞, a)

and [−∞, a] include −∞, (a, ∞] and [a, ∞] include ∞, and [−∞, ∞] includes −∞

Operations denoted by superscripts are applied before operations represented by

preceding operators For example, tr (A + B)2means tr

(A + B)2 and clS∼means

cl(S∼ ) This convention simplifies many formulas.

A vector in Fn is a column vector, which is also a matrix with one column Inmathematics, “vector” generally refers to an abstract vector not resolved in coor-dinates

Sets have elements, vectors have components, and matrices have entries Thisterminology has no mathematical consequence

The notation x(i) represents the ith component of the vector x.

The notation A(i,j) represents the scalar (i, j) entry of A Ai,j or Aij denotes a

block or submatrix of A.

All matrices have nonnegative integral dimensions If at least one of the dimensions

of a matrix is zero, then the matrix is empty

The entries of a submatrix ˆA of a matrix A are the entries of A lying in specified

rows and columns ˆA is a block of A if ˆ A is a submatrix of A whose entries are entries of adjacent rows and columns of A Every matrix is both a submatrix and

block of itself

The determinant of a submatrix is a subdeterminant Some books use “minor.”The determinant of a matrix is also a subdeterminant of the matrix.

The dimension of the null space of a matrix is its defect Some books use “nullity.”

A block of a square matrix is diagonally located if the block is square and thediagonal entries of the block are also diagonal entries of the matrix; otherwise, theblock is off-diagonally located This terminology avoids confusion with a “diagonalblock,” which is a block that is also a square, diagonal submatrix

For the partitioned matrix [A B

and similarly for B, C, and D.

The Schur product of matrices A and B is denoted by A ◦B Matrix multiplication

is given priority over Schur multiplication, that is, A ◦ BC means A ◦ (BC) The adjugate of A ∈ F n ×n is denoted by AA The traditional notation is adj A, while the notation AA is used in [1228] If A ∈ F is a scalar then AA = 1 In

particular, 0A

1×1 = 1 However, for all n ≥ 2, 0A

n ×n= 0n×n .

IfF = R, then A becomes A, A ∗ becomes AT, “Hermitian” becomes “symmetric,”

“unitary” becomes “orthogonal,” “unitarily” becomes “orthogonally,” and

“con-gruence” becomes “T-congruence.” A square complex matrix A is symmetric if

AT= A and orthogonal if ATA = I.

The diagonal entries of a matrix A ∈ F n ×n all of whose diagonal entries are real

are ordered as dmax(A) = d1(A)≥ d2(A)≥ · · · ≥ d n(A) = dmin(A).

Every n ×n matrix has n eigenvalues Hence, eigenvalues are counted in accordance

with their algebraic multiplicity The phrase “distinct eigenvalues” ignores algebraicmultiplicity

The eigenvalues of a matrix A ∈ F n ×nall of whose eigenvalues are real are ordered

as λmax(A) = λ1(A) ≥ λ2(A)≥ · · · ≥ λ n(A) = λmin(A).

The inertia of a matrix is written as

For A ∈ F n ×n , amult A(λ) is the number of copies of λ in the multispectrum of A,

gmultA (λ) is the number of Jordan blocks of A associated with λ, and indA(λ) is the order of the largest Jordan block of A associated with λ The index of A, denoted

by ind A = indA(0), is the order of the largest Jordan block of A associated with

the eigenvalue 0

The matrix A ∈ F n ×n is semisimple if the order of every Jordan block of A is 1, and cyclic if A has exactly one Jordan associated with each of its eigenvalues Defective

means not semisimple, while derogatory means not cyclic

are positive

The min{n, m} singular values of a matrix A ∈ F n ×m are ordered as σmax(A) =

notation σmin(A) is defined only for square matrices.

Positive-semidefinite and positive-definite matrices are Hermitian

A square matrix with entries in F is diagonalizable over F if and only if it can betransformed into a diagonal matrix whose entries are inF by means of a similaritytransformation whose entries are inF Therefore, a complex matrix is diagonalizable

overC if and only if all of its eigenvalues are semisimple, whereas a real matrix isdiagonalizable overR if and only if all of its eigenvalues are semisimple and real.The real matrix 0 1

−1 0 is diagonalizable over C, although it is not diagonalizableover R The Hermitian matrix 1 j

−j 2 is diagonalizable overC, and also has realeigenvalues

An idempotent matrix A ∈ F n ×n satisfies A2= A, while a projector is a Hermitian,

idempotent matrix Some books use “projector” for idempotent and “orthogonalprojector” for projector A reflector is a Hermitian, involutory matrix A projector

is a normal matrix each of whose eigenvalues is 1 or 0, while a reflector is a normalmatrix each of whose eigenvalues is 1 or−1.

An elementary matrix is a nonsingular matrix formed by adding an outer-productmatrix to the identity matrix An elementary reflector is a reflector exactly one

of whose eigenvalues is−1 An elementary projector is a projector exactly one of

whose eigenvalues is 0 Elementary reflectors are elementary matrices However,elementary projectors are not elementary matrices since elementary projectors aresingular

A range-Hermitian matrix is a square matrix whose range is equal to the range ofits complex conjugate transpose These matrices are also called “EP” matrices

The polynomials 1 and s3+ 5s2− 4 are monic The zero polynomial is not monic The rank of a polynomial matrix P is the maximum rank of P (s) overC This

quantity is also called the normal rank We denote this quantity by rank P as distinct from rank P (s), which denotes the rank of the matrix P (s).

The rank of a rational transfer function G is the maximum rank of G(s) over C

excluding poles of the entries of G This quantity is also called the normal rank.

We denote this quantity by rank G as distinct from rank G(s), which denotes the rank of the matrix G(s).

The symbol⊕ denotes the Kronecker sum Some books use ⊕ to denote the direct

sum of matrices or subspaces

The notation|A| represents the matrix obtained by replacing every entry of A by

its absolute value

The notationA represents the matrix (A ∗ A) 1/2 Some books use |A| to denote this

matrix

The H¨older norms for vectors and matrices are denoted by ·  p The matrix norm

induced by ·  q on the domain and ·  p on the codomain is denoted by ·  p,q.

The Schatten norms for matrices are denoted by ·  σp, and the Frobenius norm

is denoted by  · F Hence,  ·  σ ∞ =  ·  2,2 = σmax(·),  ·  σ2 =  · F, and

Defining h(x) = g(x)  − f(x), it follows that (1) is equivalent to

Now, suppose that h has a global minimizer x0 ∈ X Then, (5) implies that

Consequently, inequalities are often expressed equivalently in terms of optimizationproblems, and vice versa

Many inequalities are based on a single function that is either monotonic or convex

For the partitioned matrix [A B

and similarly for B, C, and D.

The Schur product of matrices A and B is denoted by A ◦B Matrix. .. reflector is a normalmatrix each of whose eigenvalues is or−1.

An elementary matrix is a nonsingular matrix formed by adding an outer-productmatrix to the identity matrix An elementary... Every matrix is both a submatrix and< /i>

block of itself