gray - toeplitz and circulant matrices

Abstract In this tutorial report the fundamental theorems on the asymptotic havior of eigenvalues, inverses, and products of “finite section”Toeplitz ma-trices and Toeplitz matrices with

Toeplitz and Circulant Matrices: A review

Stanford UniversityStanford, California 94305

Revised March 2000This document available as anAdobe portable document format (pdf) file athttp://www-isl.stanford.edu/~gray/toeplitz.pdf

c

Robert M Gray, 1971, 1977, 1993, 1997, 1998, 2000.

The preparation of the original report was financed in part by the NationalScience Foundation and by the Joint Services Program at Stanford Since then ithas been done as a hobby

Abstract

In this tutorial report the fundamental theorems on the asymptotic havior of eigenvalues, inverses, and products of “finite section”Toeplitz ma-trices and Toeplitz matrices with absolutely summable elements are derived.Mathematical elegance and generality are sacrificed for conceptual simplic-ity and insight in the hopes of making these results available to engineerslacking either the background or endurance to attack the mathematical lit-erature on the subject By limiting the generality of the matrices consideredthe essential ideas and results can be conveyed in a more intuitive mannerwithout the mathematical machinery required for the most general cases As

be-an application the results are applied to the study of the covaribe-ance matricesand their factors of linear models of discrete time random processes

Acknowledgements

The author gratefully acknowledges the assistance of Ronald M Aarts ofthe Philips Research Labs in correcting many typos and errors in the 1993revision, Liu Mingyu in pointing out errors corrected in the 1998 revision,Paolo Tilli of the Scuola Normale Superiore of Pisa for pointing out an in-correct corollary and providing the correction, and to David Neuhoff of theUniversity of Michigan for pointing out several typographical errors and someconfusing notation

4.1 Finite Order Toeplitz Matrices 23

4.2 Toeplitz Matrices 28

4.3 Toeplitz Determinants 45

5 Applications to Stochastic Time Series 47 5.1 Moving Average Sources 48

5.2 Autoregressive Processes 51

5.3 Factorization 54

5.4 Differential Entropy Rate of Gaussian Processes 57

1

2 CONTENTS

3

4 CHAPTER 1 INTRODUCTION

In addition to the fundamental theorems, several related results that urally follow but do not appear to be collected together anywhere are pre-sented

nat-The essential prerequisites for this report are a knowledge of matrix ory, an engineer’s knowledge of Fourier series and random processes, calculus(Riemann integration), and hopefully a first course in analysis Several of theoccasional results required of analysis are usually contained in one or morecourses in the usual engineering curriculum, e.g., the Cauchy-Schwarz andtriangle inequalities Hopefully the only unfamiliar results are a corollary tothe Courant-Fischer Theorem and the Weierstrass Approximation Theorem.The latter is an intuitive result which is easily believed even if not formallyproved More advanced results from Lebesgue integration, functional analy-sis, and Fourier series are not used

the-The main approach of this report is to relate the properties of Toeplitzmatrices to those of their simpler, more structured cousin — the circulant orcyclic matrix These two matrices are shown to be asymptotically equivalent

in a certain sense and this is shown to imply that eigenvalues, inverses, ucts, and determinants behave similarly This approach provides a simplifiedand direct path (to the author’s point of view) to the basic eigenvalue distri-bution and related theorems This method is implicit but not immediatelyapparent in the more complicated and more general results of Grenander

prod-in Chapter 7 of [1] The basic results for the special case of a finite orderToeplitz matrix appeared in [16], a tutorial treatment of the simplest casewhich was in turn based on the first draft of this work The results were sub-sequently generalized using essentially the same simple methods, but theyremain less general than those of [1]

As an application several of the results are applied to study certain models

of discrete time random processes Two common linear models are studiedand some intuitively satisfying results on covariance matrices and their fac-tors are given As an example from Shannon information theory, the Toeplitzresults regarding the limiting behavior of determinants is applied to find thedifferential entropy rate of a stationary Gaussian random process

We sacrifices mathematical elegance and generality for conceptual plicity in the hope that this will bring an understanding of the interestingand useful properties of Toeplitz matrices to a wider audience, specifically

sim-to those who have lacked either the background or the patience sim-to tackle themathematical literature on the subject

theo-The eigenvalues λ k and the eigenvectors (n-tuples) x k of an n × n matrix

M are the solutions to the equation

Corollary 2.1 Define the Rayleigh quotient of an Hermitian matrix H and

a vector (complex n −tuple) x by

RH (x) = (x ∗ Hx)/(x ∗ x). (2.3)

5

6 CHAPTER 2 THE ASYMPTOTIC BEHAVIOR OF MATRICES

Let η M and η m be the maximum and minimum eigenvalues of H, respectively Then

Lemma 2.1 Let A be a matrix with eigenvalues α k Define the eigenvalues

of the Hermitian matrix A ∗ A to be λk Then

n−1 k=0

λk ≥ n−1 k=0

with equality iff (if and only if ) A is normal, that is, iff A ∗ A = AA ∗ (If A

is Hermitian, it is also normal.)

Proof.

The trace of a matrix is the sum of the diagonal elements of a matrix.The trace is invariant to unitary operations so that it also is equal to thesum of the eigenvalues of a matrix, i.e.,

Tr{A ∗ A } = n−1

k=0

(A ∗ A)k,k =

n−1 k=0

Any complex matrix A can be written as

where W is unitary and R = {rk,j} is an upper triangular matrix [3, p 79].

The eigenvalues of A are the principal diagonal elements of R We have

Tr{A ∗ A } = Tr{R ∗ R } = n−1

k=0

n−1 j=0

|rj,k |2

=

n−1 k=0

pp 102-103].

Let A be a matrix with eigenvalues α k and let λ k be the eigenvalues of

the Hermitian matrix A ∗ A The strong norm A is defined by

A = max x RA ∗ A (x) 1/2 = max

x ∗ x=1 [x ∗ A ∗ Ax] 1/2 (2.10)From Corollary 2.1

A 2= max

k λk = λ∆ M (2.11)

The strong norm of A can be bounded below by letting e M be the eigenvector

of A corresponding to α M , the eigenvalue of A having largest absolute value:

8 CHAPTER 2 THE ASYMPTOTIC BEHAVIOR OF MATRICES

The Hilbert-Schmidt norm is the “weaker”of the two norms since

A 2= max

k λk ≥ n −1 n−1

k=0

λk =|A|2. (2.16)

A matrix is said to be bounded if it is bounded in both norms

Note that both the strong and the weak norms are in fact norms in thelinear space of matrices, i.e., both satisfy the following three axioms:

1 A ≥ 0 , with equality iff A = 0 , the all zero matrix.

Lemma 2.2 Given two n × n matrices G = {gk,j} and H = {hk,j}, then

= n −1

j

h ∗ j G ∗ Ghj ,

(2.20)

Lemma 2.2 is the matrix equivalent of 7.3a of [1, p 103] Note that the

lemma does not require that G or H be Hermitian.

We will be considering n × n matrices that approximate each other when

n is large As might be expected, we will use the weak norm of the difference

of two matrices as a measure of the “distance”between them Two sequences

of n × n matrices An and B n are said to be asymptotically equivalent if

1 A n and B nare uniformly bounded in strong (and hence in weak) norm:

An , Bn ≤ M < ∞ (2.21)and

2 A n − Bn = D n goes to zero in weak norm as n → ∞:

lim

n →∞ |An − Bn| = lim n →∞ |Dn| = 0.

Asymptotic equivalence of A n and B n will be abbreviated A n ∼ Bn If one

of the two matrices is Toeplitz, then the other is said to be asymptoticallyToeplitz We can immediately prove several properties of asymptotic equiv-alence which are collected in the following theorem

10 CHAPTER 2 THE ASYMPTOTIC BEHAVIOR OF MATRICES

4 If A n ∼ Bn and A −1

n , B −1

n ≤ K < ∞, i.e., A −1

n and B n −1 exist and are uniformly bounded by some constant independent of n, then

1 Eqs (2.22) follows directly from (2.17)

2 |An − Cn| = |An − Bn + B n − Cn| ≤ |An − Bn| + |Bn − Cn| n −→ →∞0

3 Applying Lemma 2.2 yields

|AnCn − BnDn| = |AnCn − AnDn + A nDn − BnDn|

The above results will be useful in several of the later proofs

Asymptotic equality of matrices will be shown to imply that eigenvalues,products, and inverses behave similarly The following lemma provides aprelude of the type of result obtainable for eigenvalues and will itself serve

as the essential part of the more general theorem to follow

Lemma 2.3 Given two sequences of asymptotically equivalent matrices A n and Bn with eigenvalues αn,k and βn,k, respectively, then

lim

n →∞ n −1

n−1 k=0

αn,k = lim

n →∞ n −1

n−1 k=0



2

≤ n n−1 k=0

|dk,k|2

≤ n n−1 k=0

n−1 j=0

|dk,j|2 = n2|Dn|2.

.

Dividing by n2, and taking the limit, results in

0≤ |n −1 Tr(D n)|2 ≤ |Dn|2 n −→ →∞ 0. (2.25)which implies (2.24) and hence (2.23)

Similarly to (2.23), if A n and B n are Hermitian then (2.22) and (2.15)imply that

lim

n →∞ n −1

n−1 k=0

α2n,k = lim

n →∞ n −1

n−1 k=0

β n,k2 . (2.26)Note that (2.23) and (2.26) relate limiting sample (arithmetic) averages ofeigenvalues or moments of an eigenvalue distribution rather than individualeigenvalues Equations (2.23) and (2.26) are special cases of the followingfundamental theorem of asymptotic eigenvalue distribution

Theorem 2.2 Let A n and Bn be asymptotically equivalent sequences of trices with eigenvalues αn,k and βn,k, respectively Assume that the eigenvalue moments of either matrix converge, e.g., lim

ma-n →∞ n −1

n−1 k=0

α s n,k exists and is finite for any positive integer s Then

lim

n →∞ n −1

n−1 k=0

α s n,k = lim

n →∞ n −1

n−1 k=0

β n,k s (2.27)

12 CHAPTER 2 THE ASYMPTOTIC BEHAVIOR OF MATRICES

Proof.

Let A n = B n + D n as in Lemma 2.3 and consider A s

n − B s n

B n  s but containing at least one Dn Repeated application of Lemma 2.2 thusgives

eigen-Since (2.26) holds for any positive integer s we can add sums ing to different values of s to each side of (2.26) This observation immedi-

correspond-ately yields the following corollary

Corollary 2.2 Let A n and B n be asymptotically equivalent sequences of trices with eigenvalues αn,k and βn,k, respectively, and let f (x) be any poly- nomial Then

ma-lim

n →∞ n −1

n−1 k=0

f (αn,k) = lim

n →∞ n −1

n−1 k=0

f (βn,k ) (2.30)

Whether or not A n and B n are Hermitian, Corollary 2.2 implies that

(2.30) can hold for any analytic function f (x) since such functions can be expanded into complex Taylor series, i.e., into polynomials If A n and B n

are Hermitian, however, then a much stronger result is possible In thiscase the eigenvalues of both matrices are real and we can invoke the Stone-Weierstrass approximation Theorem [4, p 146] to immediately generalizeCorollary 2.3 This theorem, our one real excursion into analysis, is statedbelow for reference

Theorem 2.3 (Stone-Weierstrass)If F (x) is a continuous complex function

on [a, b], there exists a sequence of polynomials pn (x) such that

lim

n →∞ pn (x) = F (x)

uniformly on [a, b].

Stated simply, any continuous function defined on a real interval can beapproximated arbitrarily closely by a polynomial Applying Theorem 2.3 toCorollary 2.2 immediately yields the following theorem:

Theorem 2.4 Let A n and B n be asymptotically equivalent sequences of mitian matrices with eigenvalues αn,k and βn,k, respectively Since An and

Her-Bn are bounded there exist finite numbers m and M such that

F [αn,k] = lim

n →∞ n −1

n−1 k=0

F [βn,k] (2.32)

if either of the limits exists.

Theorem 2.4 is the matrix equivalent of Theorem (7.4a) of [1] When tworeal sequences{αn,k ; k = 0, 1, , n −1} and {βn,k ; k = 0, 1, , n −1} satisfy

(2.31)-(2.32), they are said to be asymptotically equally distributed [1, p 62].

As an example of the use of Theorem 2.4 we prove the following corollary

on the determinants of asymptotically equivalent matrices

Corollary 2.3 Let A n and Bn be asymptotically equivalent Hermitian ces with eigenvalues αn,k and βn,k, respectively, such that αn,k, βn,k ≥ m > 0 Then

ln α n,k = lim

n →∞ n −1

n−1 k=0

ln β n,k

14 CHAPTER 2 THE ASYMPTOTIC BEHAVIOR OF MATRICES

from which (2.33) follows

With suitable mathematical care the above corollary can be extended to

the case where α n,k, βn,k > 0, but there is no m satisfying the hypothesis of

the corollary, i.e., where the eigenvalues can get arbitrarily small but are stillstrictly positive

In the preceding chapter the concept of asymptotic equivalence of ces was defined and its implications studied The main consequences havebeen the behavior of inverses and products (Theorem 2.1) and eigenvalues(Theorems 2.2 and 2.4) These theorems do not concern individual entries

matri-in the matrices or matri-individual eigenvalues, rather they describe an “average”

behavior Thus saying A −1 n ∼ B −1

n means that that |A −1

n − B −1

n | n −→ →∞ 0 andsays nothing about convergence of individual entries in the matrix In certaincases stronger results on a type of elementwise convergence are possible usingthe stronger norm of Baxter [7, 8] Baxter’s results are beyond the scope ofthis report

The major use of the theorems of this chapter is that we can often studythe asymptotic behavior of complicated matrices by studying a more struc-tured and simpler asymptotically equivalent matrix

Chapter 3

Circulant Matrices

The properties of circulant matrices are well known and easily derived [3, p.267],[19] Since these matrices are used both to approximate and explain thebehavior of Toeplitz matrices, it is instructive to present one version of therelevant derivations here

A circulant matrix C is one having the form

where each row is a cyclic shift of the row above it The matrix C is itself

a special type of Toeplitz matrix The eigenvalues ψ k and the eigenvectors

y (k) of C are the solutions of

ck −m yk = ψ y m ; m = 0, 1, , n − 1. (3.3)Changing the summation dummy variable results in

n −1−m

k=0

ckyk+m+

n−1 k=n −m ckyk −(n−m) = ψ y m ; m = 0, 1, , n − 1. (3.4)

15

16 CHAPTER 3 CIRCULANT MATRICES

One can solve difference equations as one solves differential equations — byguessing an (hopefully) intuitive solution and then proving that it works.Since the equation is linear with constant coefficients a reasonable guess is

yk = ρ k (analogous to y(t) = e sτ in linear time invariant differential

equa-tions) Substitution into (3.4) and cancellation of ρ m yields

n −1−m

k=0 ckρ k + ρ −n

n−1 k=n −m ckρ k = ψ.

Thus if we choose ρ −n = 1, i.e., ρ is one of the n distinct complex n th roots

of unity, then we have an eigenvalue

ψ =

n−1 k=0

where the normalization is chosen to give the eigenvector unit energy

Choos-ing ρ j as the complex n th root of unity, ρ j = e −2πij/n , we have eigenvalue

ψm =

n−1 k=0

To verify (3.8) we note that the (k, j) th element of C, say a k,j, is

ak,j = n −1

n−1 m=0

e 2πim(k −j)/n ψm

= n −1

n−1 m=0

e 2πim(k −j)/n n−1

r=0 cre 2πimr/n

= n −1

n−1 r=0 cr

n−1 m=0

e 2πim(k −j+r)/n

(3.9)

But we have

n−1 m=0

Since C is unitarily similar to a diagonal matrix it is normal Note that

all circulant matrices have the same set of eigenvectors This leads to thefollowing properties

Theorem 3.1 Let C = {ck −j } and B = {bk −j } be circulant n × n matrices with eigenvalues

ψm =

n−1 k=0 cke −2πimk/n

βm =

n−1 k=0 bke −2πimk/n , respectively.

1 C and B commute and

CB = BC = U ∗ γU , where γ = {ψmβmδk,m}, and CB is also a circulant matrix.

18 CHAPTER 3 CIRCULANT MATRICES

2 C + B is a circulant matrix and

asymp-Chapter 4

Toeplitz Matrices

In this chapter the asymptotic behavior of inverses, products, eigenvalues,and determinants of finite Toeplitz matrices is derived by constructing anasymptotically equivalent circulant matrix and applying the results of theprevious chapters Consider the infinite sequence{tk ; k = 0, ±1, ±2, · · ·} and

define the finite (n × n) Toeplitz matrix Tn = {tk −j } as in (1.1) Toeplitz

matrices can be classified by the restrictions placed on the sequence t k If

there exists a finite m such that t k = 0, |k| > m, then Tn is said to be a

finite order Toeplitz matrix If t k is an infinite sequence, then there are two

common constraints The most general is to assume that the t k are squaresummable, i.e., that

as-tk are absolutely summable, i.e.,

19

20 CHAPTER 4 TOEPLITZ MATRICES

Not only does the limit in (4.3) converge if (4.2) holds, it converges uniformly for all λ, that is, we have that







≤ −n−1k= −∞

|tk| + ∞ k=n+1

|tk|

,

where the righthand side does not depend on λ and it goes to zero as n → ∞

from (4.2), thus given / there is a single N , not depending on λ, such that

Since f (λ) is the Fourier series of the sequence t k, we could alternatively

begin with a bounded and hence Riemann integrable function f (λ) on [0, 2π]

(|f(λ)| ≤ M|f| < ∞ for all λ) and define the sequence of n × n Toeplitz

as-except possibly at a countable number of points Which assumption is made

depends on whether one begins with a sequence t k or a function f (λ) —

either assumption will be equivalent for our purposes since it is the Riemann

integrability of f (λ) that simplifies the bookkeeping in either case Before finding a simple asymptotic equivalent matrix to T n, we use Corollary 2.1

to find a bound on the eigenvalues of T n when it is Hermitian and an upperbound to the strong norm in the general case

Lemma 4.1 Let τ n,k be the eigenvalues of a Toeplitz matrix Tn (f ) If T n (f )

22 CHAPTER 4 TOEPLITZ MATRICES

so that

x ∗ Tnx =

n−1 k=0

n−1 j=0

tk −j xk xj¯

=

n−1 k=0

n−1 j=0







n−1 k=0 xke ikλ

|xk|2 = (2π) −1

 2π

0 dλ | n−1

k=0 xke ikλ |2. (4.10)Combining (4.9)-(4.10) results in





 2

which with (4.8) yields (4.6) Alternatively, observe in (4.11) that if e (k) is

the eigenvector associated with τ n,k , then the quadratic form with x = e (k)

yields x ∗ Tnx = τn,k#n −1

k=0 |xk|2 Thus (4.11) implies (4.6) directly

We have already seen in (2.13) that if T n (f ) is Hermitian, then Tn (f ) =

maxk |τn,k| ∆

=|τn,M |, which we have just shown satisfies |τn,M | ≤ max(|Mf |, |mf |)

which in turn must be less than M |f|, which proves (4.7) for Hermitian

ma-trices Suppose that T n (f ) is not Hermitian or, equivalently, that f is not real Any function f can be written in terms of its real and imaginary parts,

f = fr + if i , where both f r and f i are real In particular, f r = (f + f ∗ )/2 and f i = (f − f ∗ )/2i Since the strong norm is a norm,

Tn (f ) = Tn (f r + if i)

= Tn (f r ) + iT n (f i)

≤ Tn (f r) + Tn (f i)

≤ M |f | + M |f |

4.1 FINITE ORDER TOEPLITZ MATRICES 23

Since|(f ±f ∗ )/2 ≤ (|f|+|f ∗ |)/2 ≤ M |f| , M |f r | + M |f i | ≤ 2M |f|, proving (4.7)

Note for later use that the weak norm between Toeplitz matrices has a

simpler form than (2.14) Let T n={tk −j } and T 

n−1 j=0

|tk −j − t 

k −j |2

= n −1

n−1 k= −(n−1)

(n − |k|)|tk − t  k|2

=

n−1 k= −(n−1)

(1− |k|/n)|tk − t  k|2

. (4.12)

We are now ready to put all the pieces together to study the asymptotic

behavior of T n If we can find an asymptotically equivalent circulant matrixthen all of the results of Chapters 2 and 3 can be instantly applied Themain difference between the derivations for the finite and infinite order case

is the circulant matrix chosen Hence to gain some feel for the matrix chosen

we first consider the simpler finite order case where the answer is obvious,and then generalize in a natural way to the infinite order case

Let T n be a sequence of finite order Toeplitz matrices of order m + 1, that is,

ti = 0 unless|i| ≤ m Since we are interested in the behavior or Tn for large n

we choose n >> m A typical Toeplitz matrix will then have the appearance

of the following matrix, possessing a band of nonzero entries down the centraldiagonal and zeros everywhere else With the exception of the upper left and

lower right hand corners that T n looks like a circulant matrix, i.e each row

24 CHAPTER 4 TOEPLITZ MATRICES

is the row above shifted to the right one place

4.1 FINITE ORDER TOEPLITZ MATRICES 25

asymptotically equivalent and simple

Lemma 4.2 The matrices T n and C n defined in (4.13)and (4.14)are totically equivalent, i.e., both are bounded in the strong norm and.

asymp-lim

Proof The t k are obviously absolutely summable, so T n are uniformly

bounded by 2M |f| from Lemma 4.1 The matrices C n are also uniformly

bounded since C n ∗ Cn is a circulant matrix with eigenvalues |f(2πk/n)|2 ≤

≤ mn −1m

k=0

(|tk|2 +|t −k |2)n −→ →∞0

.

26 CHAPTER 4 TOEPLITZ MATRICES

The above Lemma is almost trivial since the matrix T n − Cn has fewer

than m2 non-zero entries and hence the n −1in the weak norm drives|Tn −Cn|

to zero

From Lemma 4.2 and Theorem 2.2 we have the following lemma:

Lemma 4.3 Let T n and Cn be as in (4.13)and (4.14)and let their values be τn,k and ψn,k, respectively, then for any positive integer s

eigen-lim

n →∞ n −1

n−1 k=0

τ n,k s = lim

n →∞ n −1

n−1 k=0

The above two lemmas show that we can immediately apply the results

of Section II to T n and C n Although Theorem 2.1 gives us immediate hope

of fruitfully studying inverses and products of Toeplitz matrices, it is not yetclear that we have simplified the study of the eigenvalues The next lemmaclarifies that we have indeed found a useful approximation

Lemma 4.4 Let C n (f ) be constructed from T n (f ) as in (4.14)and let ψ n,k

be the eigenvalues of Cn (f ), then for any positive integer s we have

lim

n →∞ n −1

n−1 k=0

ψ s n,k = (2π) −1

 2π

0 f s (λ) dλ. (4.19)

4.1 FINITE ORDER TOEPLITZ MATRICES 27

If T n (f ) and hence C n (f ) are Hermitian, then for any function F (x)

contin-uous on [mf , Mf ] we have

lim

n →∞ n −1

n−1 k=0

ψ n,k s = lim

n →∞ n −1

n−1 k=0

f (2πk/n) s

= lim

n →∞

n−1 k=0

f (λk)s ∆λ/(2π)

= (2π) −1

 2π

0 f (λ) s dλ, (4.22)

where the continuity of f (λ) guarantees the existence of the limit of (4.22)

as a Riemann integral If T n and C n are Hermitian than the ψ n,k and f (λ)

are real and application of the Stone-Weierstrass theorem to (4.22) yields

(4.20) Lemma 4.2 and (4.21) ensure that ψ n,k and τ n,kare in the real interval

[m f , Mf ].

Combining Lemmas 4.2-4.4 and Theorem 2.2 we have the following specialcase of the fundamental eigenvalue distribution theorem

28 CHAPTER 4 TOEPLITZ MATRICES

Theorem 4.1 If T n (f ) is a finite order Toeplitz matrix with eigenvalues τ n,k , then for any positive integer s

lim

n →∞ n −1

n−1 k=0

F (τn,k ) = (2π) −1

 2π

0 F [f (λ)] dλ; (4.24)

i.e., the sequences τn,k and f (2πk/n) are asymptotically equally distributed.

This behavior should seem reasonable since the equations T nx = τ x and Cnx = ψx, n > 2m + 1, are essentially the same n thorder difference equationwith different boundary conditions It is in fact the “nice”boundary condi-

tions that make ψ easy to find exactly while exact solutions for τ are usually

intractable

With the eigenvalue problem in hand we could next write down theorems

on inverses and products of Toeplitz matrices using Lemma 4.2 and the results

of Chapters 2-3 Since these theorems are identical in statement and proofwith the infinite order absolutely summable Toeplitz case, we defer thesetheorems momentarily and generalize Theorem 4.1 to more general Toeplitzmatrices with no assumption of fine order

Obviously the choice of an appropriate circulant matrix to approximate aToeplitz matrix is not unique, so we are free to choose a construction withthe most desirable properties It will, in fact, prove useful to consider twoslightly different circulant approximations to a given Toeplitz matrix Say

we have an absolutely summable sequence {tk ; k = 0, ±1, ±2, · · ·} with

f (λ)e −ikλ

4.2 TOEPLITZ MATRICES 29

Define C n (f ) to be the circulant matrix with top row (c (n)0 , c (n)1 , · · · , c (n)

n −1)where

c (n) k = n −1

n−1 j=0

and hence the c (n) k are simply the sum approximations to the Riemann integral

giving t −k Equations (4.26), (3.7), and (3.9) show that the eigenvalues ψ n,m

of C n are simply f (2πm/n); that is, from (3.7) and (3.9)

ψn,m =

n−1 k=0

c (n) k e −2πimk/n

=

n−1 k=0

f (2πj/n)



n −1

n−1 k=0

22πik(j −m)/n

%

= f (2πm/n)

. (4.28)

Thus, C n (f ) has the useful property (4.21) of the circulant approximation

(4.15) used in the finite case As a result, the conclusions of Lemma 4.4

hold for the more general case with C n (f ) constructed as in (4.26) Equation (4.28) in turn defines C n (f ) since, if we are told that C nis a circulant matrix

with eigenvalues f (2πm/n), m = 0, 1, · · · , n − 1, then from (3.9)

c (n) k = n −1

n−1 m=0 ψn,me 2πimk/n

= n −1

n−1 m=0

f (2πm/n)e 2πimk/n