Algebraic structures and operator calculus special functions and computer science

Preface ixIntroductionChapter 1 Basic Data Structures Chapter 2 Data Structures and Orthogonal Polynomials Chapter 3 Applications of Bessel Functions and Lommel Polynomials Chapter 4 Fou

and Operator Calculus

Southern Illinois University,

Carbondale, Illinois, U.S.A

KLUWER ACADEMIC PUBLISHERS

DORDRECHT / BOSTON / LONDON

ISBN 0-7923-2921-X

Published by Kluwer Academic Publishers,

P.O Box 17, 3300 AA Dordrecht, The Netherlands

Kluwer Academic Publishers incorporates

the publishing programmes of

D Reidel, Martinus Nijhoff, Dr W Junk and MT? Press

Sold and distributed in the U S A and Canada

by Kluwer Academic Publishers,

101 Philip Drive, Norwell, MA 02061, U S.A

In all other countries, sold and distributed

by Kluwer Academic Publishers Group,

P.O Box 322, 3300 AH Dordrecht, The Netherlands

Printed on acid free paper

All Rights Reserved

® 1994 Kluwer Academic Publishers

No part of the material protected by this copyright notice may be reproduced orutilized in any form or by any means, electronic or mechanical,

including photocopying, recording or by any information storage and

retrieval system, without written permission from the copyright owner

Printed in the Netherlands

To our parents

In memoriam : Leo and Cecilia, Joseph and Anne

Preface ixIntroduction

Chapter 1 Basic Data Structures

Chapter 2 Data Structures and Orthogonal Polynomials

Chapter 3 Applications of Bessel Functions and Lommel Polynomials

Chapter 4 Fourier Transform on Finite Groups and Related Transforms

III Krawtchouk polynomials and finite probability theory 91

IV Young tableaux and combinatorial enumeration in parallel processing 136

vii

In this volume we will present some applications of special functions in computerscience This largely consists of adaptations of articles that have appeared in the literature Here they are presented in a format made accessible for the non-expert by providing somecontext The material on group representations and Young tableaux is introductory innature However, the algebraic approach of Chapter 2 is original to the authors and hasnot appeared previously Similarly, the material and approach based on Appell states, soformulated, is presented here for the first time

As in all volumes of this series, this one is suitable for self-study by researchers It is

as well appropriate as a text for a course or advanced seminar

The solutions are tackled with the help of various analytical techniques, such as erating functions, and probabilistic methods/insights appear regularly An interestingfeature is that, as has been the case in classical applications to physics, special functionsarise - here in complexity analysis And, as in physics, their appearance indicates anunderlying Lie structure

gen-Our primary audience is applied mathematicians and theoretical computer scientists

We are quite sure that pure mathematicians will find this volume interesting and useful aswell

We expect this volume to have a utility between a reference and a monograph Wewish to make available in one volume a group of works and results scattered in the literaturewhile providing some background to the mathematics involved which the reader will nodoubt find appealing in its own right

The authors gratefully acknowledge AFCET, Universite de Nancy I, Southern IllinoisUniversity, INRIA, and NATO, for support while various parts of this work were carriedout

ix

I N T R O D U C T I O N

G e n e r a l r e m a r k s

In this volume, we present some applications of algebraic, analytic, and probabilistic methods to some problems in computer science Special functions, notably orthogonal polynomials, and Bessel functions will appear

In Chapter J, we present the basic d a t a structures we will be studying and introduce the ideas guiding the analysis Chapter 2 presents applications of orthogonal polynomials

and continued fractions to the analysis of dynamic data structures in the model introduced

by J Frangon and D.E Knuth, called Knuth's model The approach here is original with this exposition The underlying algebraic/analytic structures play a prominent role

Chapter 3 presents some results involving Bessel functions and Lommel polynomials arising

from a study of the behavior of the symbol table Then, in another direction, theta functions come up as solutions of the heat equation on bounded domains, describing the

limiting behavior of the evolution of stacks and the banker's algorithm In Chapter 4 we

present some basic material on representations of finite groups, including Fourier transform Then Fourier transform is considered in more detail for abelian groups, particularly cyclic groups Next we look at Krawtchouk polynomials, which arise in a variety of ways, e.g.,

in the study of random walks The concluding Chapter 5 discusses representations of the

symmetric group and connections with Young tableaux Then, a variety of applications of Young tableaux are shown, including examples related to questions in parallel processing Chapters 4 and 5 mainly include material from the literature that we find particularly interesting as well as some new approaches It is hoped that the reader will find these chapters to be useful as background for further study as well as providing examples having effective illustrative and reference value

R e m a r k The remainder of this introductory chapter provides some basic information

t h a t the reader may find useful, and is included so t h a t the volume is fairly self-contained Here one can find the notational conventions used in this volume; they are consistent with those of Volume 1 Many formulas and much information may be found in the Handbook by Abrajnowitz&Stegun [2], an overall very handy reference for the material of this chapter and for properties of the special functions used throughout this volume Also, see Rainville[73]

I I S o m e n o t a t i o n s

1 First, we recall the gamma and beta functions, given by the integrals

r ( x ) = / t ^ - i f i - ' d t

Jo r(x + y) Jo

-4 For Stirling numbers, we use Sn,k and s^^k for Stirling numbers of the first and

second kinds, respectively They may be defined by the relations:

5 For finite sets E, we denote the cardinality of E by \E\ or # E

I I I O r t h o g o n a l p o l y n o m i a l s a n d c o n t i n u e d fractions

Orthogonal polynomials arise frequently as solutions to three-term recurrence relations

A standard reference on orthogonal polynomials is Szego[81] For our work Chihara[10] is

a useful reference as well

3 1 ORTHOGONAL POLYNOMIALS

Given a bounded, nondecreasing function F{x) on R , we define a corresponding Stieltjes

measure dF{x) The moments of dF are defined by

We will take F to be normalized so t h a t ^o = 1- It is assumed that the moments exist for

all n > 0 Integration with respect to dF, m,athem,atical expectation or expected value ,

is denoted by angle brackets ( • )

{f(X))= r f(x)dF(x) (3.1.1)

J — CO

in particular, we can write the moments as Hn = {X") In the discrete case, we have a

probability sequence (discrete distribution), {pk }k>o, satisfying p* > 0, ^ p ^ t = 1, with

corresponding moments

/^n = Y^(ak)"pk

k=0

where at denotes the value taken on with corresponding probability pk- In the (absolutely)

continuous case, we have a nonnegative density function p{x), and

fJ'n = I x" p{x)dx

J — oo

We define an inner product (•, •} on the set of polynomials, which is the mathematical

expectation of their product On the basis { x " }, this is given in terms of the moments:

{X",X"'} = ( X " X ' " ) = / X^-X"" dF{x) = fXn+m

J — oo

The corresponding sequence of orthogonal polynomials, { <j>„{x) }n>Oi say, may be

deter-mined by applying the Gram-Schmidt orthogonalization procedure to the basis { a;" } The

polynomials { <f>n{x) } satisfy

M^)^m{x)dF{x) = n"' ^ , ^ (3.1.2)

where 7„ are t h e squared norms We normalize 70 t o 1

With (f>„{x) = x" + • • •, monic polynomials, the { <j>„{x) } satisfy a three-term

recur-rence relation of the form:

X<f>Ti= <i)n+\+ 0,n<t>n-'rhn4>„-l (3.1.3)

with ?io = 1, (^1 = X — ao Orthogonality, comparing (a;i^„,(^„_i) = 6n7n-i with

{4>n,x4in-\) = 7n, yields the relation:

ln=Wh -hn (3.1.4)

3 2 CONTINUED FRACTIONS

A general reference for continued fractions is Jones&Thron[47] Two sequences of (real

or complex) numbers { a„ }n>i> { ^n }n>o determine a continued fraction:

6o -I- ai hi -f- 0 2 / 6 2 + + a„/fe„ -H

which is the limit as n —» oo of the approximants:

—^ = 60 + 0.1 ^bi + 02/62 + • - • + a„/bn

P„ and Q„ are the n"" partial numerators and partiaJ denominators, respectively They

satisfy the recurrence relations:

U„ = b„Un-l + anUn-2 (3.2.1)

with initial conditions

P _ i = l , P o = 6 o , g _ i = 0 , Qo = l

respectively One may remark t h a t this setup corresponds t o t h e matrix relations:

5n Pn J \an b„J \ai bi J \1 bo J

Q

3.2.1 Connection with orthogonal polynomials

Consider the continued fraction

1 / l - aox - Wx^ j l -axx- hix'^ j j l - a „ _ i z - 6„a;^ / (3.2.1.1)

From equation (3.2.1), the partial numerators and denominators satisfy

"n+i = (1 - a.nx)un - &„a;^t<„_i for n > 1, where the numerator poJynomiais, P„{x), satisfy the initial conditions PQ = 0,

Pi = 1, and the denominator polynomials, Qn{x), satisfy Qo = 1, Qi = 1 — aox

The reciprocal polynomials, j/'„(a;), defined as

M^) = ^"Qn{l/x)

satisfy t h e three-term recurrence

xtj^n = ipn+i + a„tl;„ + b„'ipn-.t (3.2.1.2)

with initial conditions I/JO = 1, V"! = ^^ — tio- These give a sequence of orthogoned

polyno-mials with squared norms 7n = &1&2 • • • 6„, n > 0

I V B e s s e l f u n c t i o n s , L o m m e l p o l y n o m i a l s , t h e t a f u n c t i o n s

4 1 BESSEL FUNCTIONS AND LOMMEL POLYNOMIALS

The reader is referred to G.N Watson's [84] treatise for an extensive presentation of

the theory of Bessel functions Here we indicate the features t h a t are important for our

In the present study, a principal feature of Bessel functions is the fundamental recurrence:

2iyz-'j^{z) = J^+i{z) + J.-i{z) (4.1.3)

Iterating this recurrence, Lommel polynomials come up as the coefficients expressing

J^j^„(z) as a linear combination of Jv{z) and Jv-i(z)'

This holds for nonnegative integer n F o r n = 1, wehave/li,i,(z) = 2i^/2r, while i?o,i/(-2) =

1-The Lommel polynomials satisfy t h e recurrence

which has as general solution

i>„ - CiJ?„,_£^{-2e) - C2iZn-i,i-£2:(-2e)

i.e., with initial values ^ _ i = C2, </'o — c\- Eq (4.1.4) can be interpreted as expressing the

solution to x-\\)n — ^n+\ + e~^nipri + «/'n-i in the form

Jn^ex(~2e) = J-^^{-2e)R„,-exi-2s) - J-i^,^{~2e)Rn-i,i-exi-2£) (4.1.6)

which combines features of both eqs (4.1.3), (4.1.4)

V S o m e analytic t e c h n i q u e s

Here we indicate some basic analytic techniques that will be used Included are:

asymp-totic behavior of coefficients of a power series, central limit approximation of sums involving

binomial coefficients, and Lagrange inversion

the coefficients an decrease exponentially fast

Proof: The terms of the series 2 ^ Q |c(„|(iJ — e)" converge to zero for any e > 0

Choose e so that R~ e > 1 •

Because of this, one looks for the singularities of functions when it is known that the

coefficients a„ grow as n —» oo For our purposes, the following suffices

5.1.2 L e m m a Let f{z) = S ^ o * * " ^ " ^'^ meromorphic, analytic in a neighborhood of

0 in the complex plane Let ^ denote the pole of f nearest to the origin If ( is a poJe of

order p, then we have the estimate

, , A nP-i

r(p) c"+p

where A = lim f(z){z — C)*"

Proof: Write g{z) = f{z) • {z - CY- So A = g{0 and

where the second term has at most a pole of order < p Now, for \z\ < |C|, the binomial

theorem implies

At a pole of larger modulus, the same estimate gives a rate of growth exponentially slower,

whereas terms of lower order poles grow at correspondingly lower powers of n •

See Flajolet-Vitter[28] for more along these lines

5 2 CENTRAL LIMIT A P P R O X I M A T I O N

The central limit approximation is a detailed application of the theorem of

DeMoivre-Laplace t h a t one can approximate sums involving binomial coefficients, interpreted as

taking expectations with respect to binomial distributions, by Gaussian integrals Here is

a useful version

5.2.1 L e m m a Let f be a polynomial Then, as N —* oo,

Another formulation of the same statement is:

N \ /' N

k + N/2) ^ \N/2 where k = avN In particular, we have

via the substitution t = 2ậ The result follows, using r ( | ) = y/n •

See Canfield[9] for more information on this topic

5 3 LAGRANGE INVERSION

Given a function, / , analytic in a neighborhood of XQ, that is locally one-to-one, write

y = f{x) Lagrange inversion provides a way of finding expansions of functions of x in

terms of the variable ỵ In particular, the relation x = f~^{y) is the inverse of / The idea

is t h a t we want the expansion in terms of y of any function g{x) analytic in a neighborhood

of xọ

5.3.1 P r o p o s i t i o n Let y = f{x) be anaJyfic near XQ, yo = f{xo), with / ' ( x o ) 7^ 0

For g analytic near xo, the Lagrange inversion formula gives the expansion

2 The moment generating function denotes a generating function for the sequence

of moments {^n }• Note that for symmetric distributions, where the odd moments

vanish, we often write the generating function for { n^n }•

6 1 POLYNOMIALS CORRESPONDING TO DISCRETE DISTRIBUTIONS

Here we list basic information on polynomials orthogonal with respect to the binomial and Poisson distributions, respectively

• Binomial distribution — Krawtchouk polynomials { Kn{x) }

Recurrence formula

X Kn - A'„+i + n{N - n + l)A'„_i

with A'o = 1, K\ = X

Measure of orthogonality and squared norms

6 2 POLYNOMIALS CORRESPONDING TO CONTINUOUS DISTRIBUTIONS

Here we have Tchebychev and Hermite polynomials Recall that the Tchebychev

poly-nomials are determined according to

T„(cos $) = cos nO, Un{cos 9) = sin(n + \)9/ sin 6

The Hermite polynomials are orthogonal with respect to the Gaussian distribution

• Tchebychev polynomials of the first kind { T„(a;) }

Formula for the Tchebychev polynomials of the second kind

C h a p t e r 1 BASIC DATA S T R U C T U R E S

In this Chapter are introduced the basic d a t a structures t h a t will be the principal ject of the first two chapters (and part of Chapter 3) First, we introduce basic terminology and constructions In Chapter 2, detailed computations and results will be presented The structures discussed here are representations of files or lists of data stored in

sub-a computer The idesub-a is to orgsub-anize the d sub-a t sub-a for resub-ady sub-access sub-and effective use of the information

I B a s i c d a t a s t r u c t u r e s

We start with a universe of elements, items It is assumed that one can equip the universe with a total order Once given an order structure, the items are referred to as keys

1.1 D e f i n i t i o n A list is a sequence of items

When the items can be ordered, i.e., when the list consists of keys, then one has a sorted or

unsorted list accordingly Particular data types are determined according to operations

used to construct corresponding lists A sequence of operations performed on a given

structure is represented as a word in a language with alphabet corresponding to the

operations allowed to be performed on the d a t a structure Here we consider structures

which permit insertions denoted by J, deletions , denoted D, and record searches (examining items for information) called queries denoted Q"*" or Q~ according as the

result of the search was positive (successful) or negative (unsuccessful) This gives a basic

alphabet {I,D,Q+,Q-}

R e m a r k Since the words can be of arbitrary length, one could consider the set of infinite

sequences of letters, restricting to the subclass of tame sequences, having all but finitely

many components equal to e the 'empty symbol', denoting a blank, which thus would be

adjoined (formally) as part of the alphabet

1 1 LINEAR LISTS AND DICTIONARIES

A basic structure of interest is the linear list This is a list which is constructed using

only insertions and deletions

1.1.1 D e f i n i t i o n A linear list is a list that admits the operations / and D only

For purposes of illustration, we denote the items by lowercase letters { a, 6, c , } and the

corresponding operations as a word using the alphabet { I,D} For example, starting from

abc, inserting d, e, / , then deleting a and / might give successively: dabc, dabec, dabfec, dbfec, dbec, corresponding to the word IIIDD It is clear t h a t many different lists may

correspond to a given sequence of operations A main feature of the analysis is how to deal with precisely this fact in determining the complexity of a particular structure Probably the simplest and most well-known example of a lineax list is the stack

1.1.2 D e f l n i t i o n A stack is a linear list for which the insertions and deletions are

all made at only one end of the list

For a stack, a given word corresponds to only one possible set of consequences In the

above example, after three insertions, the deletions would necessarily indicate removal of

/ and then e from the list

1.1.1 Queues

Another important class of structures are queues

1.1.1.1 D e f i n i t i o n A queue is a linear list for which insertions are made at one end

of the list and deletions are made at the other end

As with a stack, a series of operations corresponds to only one possible set of consequences

An important modification is the priority queue We assume that the items are keys,

i.e., come from a totally ordered universe In the priority queue, only the highest priority

item can be deleted, with priority according to the ordering of the keys

1.1.1.2 D e f i n i t i o n A priority queue is a linear list of keys for which only the minimal

key may be deleted Insertions are unrestricted

1.1.2 Dictionaries

Dictionaries are modelled on the everyday notion of dictionary, i.e., queries, corresponding

to looking up information, arise Thus, all four operations I, D, Q"*", Q~ are admitted

The elements are keys

1.1.2.1 Definition A dictionary is a list of keys admitting the four o p e r a t i o n s / , Z),

Q+,

Q A queue/dictionary structure that gives rise to an especially interesting mathematical

analysis is the symbol table Here positive queries are admitted, but only the last key

inserted may be deleted I.e.,

1.1.2.2 D e f i n i t i o n A symbol table is a list of keys admitting the three operations

/ , D, Q"*" Deletions operate only on the last (most recent) key inserted

At this point it is not clear which structure is the most effective in a given situation

The question is how to determine what practical implementations are most efficient In

the next section we consider some basic data organizations — how the structures are

implemented in the machine The mathematical analysis, average case analysis, gives one

an idea of the relative efficiency of the various d a t a organizations

1.2 IMPLEMENTATIONS

We will be concerned primarily with linear lists, priority queues, dictionaries, and symbol tables Each of these data types may be implemented as sorted lists, for keys, as well as unsorted lists Additionally there are various implementations available for these

structures Below, we illustrate the use of pagodas, a structure developed by Fran^on,

Viennot& Vuillemin [35]

R e m a r k Sorting algorithms are a good example of the differences t h a t can arise between worst-case and average-case behavior Consider the most efficient sorting algorithm, 'quick-sort', developed by C.A.R Hoare Analysis shows that this algorithm sorts a sequence of

n elements in an average time of the form O ( n l o g n ) , i.e., bounded by a constant times

n log 77 Comparison among all sorting algorithms yields the conclusion that quicksort is

the most efKcient on the average However, the worst case, when the data is almost in order at t h e start, is quite poor, taking on the order of n^ steps Fortunately these worst cases are rare See Knuth[53], v 3, p 114ff

For priority queues, in addition to sorted and unsorted lists, there are implementations

as binary tournaments and pagodas The idea of these structures is to take advantage of the priorities among the items

Here we identify a priority queue of size n with a permutation of the integers from 1

to n, the keys To these correspond labelled, rooted trees The labels are the keys 1 , , n First we recall the definition of a binary tree A binary tree T is given recursively as

a triple T = {l{T),v{T),r{T)) where l(T) and r{T) are binary trees, the left and right subtrees respectively of T, and v(T) is the label (value) of the root of T The empty tree is included in the definition of a binary tree Similarly, a binary tournament is

defined recursively by the condition that the left and right subtrees of the root are binary

tournaments satisfying v{T) < min{ v{(): ( a node of 1{T) or r{T) } In other words, for any node (, its sons (' satisfy v{() < v{^'), priority increasing with the depth of the node

R e m a r k Various types of tournaments have been designed depending on the constraints imposed Heaps, AVL trees, 3-2 trees, binomial queues are among the most well known See Aho, Hopcroft&; Ullman[3], Knuth[53], Mehlhorn&Tsakalidis[68] for full discussions For example, consider the queue (permutation) 573961428 Successively taking out the minimal values, partitioning accordingly, and otherwise keeping order, leads to the following binary tournament:

1 / \

or, adjoining O's so t h a t every node has left and right subtrees, this can be represented by

the set of triples

Now we are ready for the construction of pagodas Start with a binary tournament T,

given as a set of triples (as above) The pagoda representation is a set of triples determined

as follows For each subtree P, determine the left inverse subtree l''^{P) as the unique

subtree, Q, say, such that 1{Q) = P in the triple corresponding to Q For example, in the

tournament of the previous section, we have /~^(e) = c Similarly, determine the right

inverse subtree of P Next, define recursively the leftmost descendant of a subtree P as

follows

i*( p> _ / -P' if KP) is empty

I ^*{KP)), otherwise

r*{P) is defined correspondingly (Here, empty subtrees correspond to the O's in the triples

used in the representation given above for tournaments.) Now we have

1.2.1.1 Definition Let T be a binary tournament A •pagoda is a representation of

T as a set of triples (A(P), w(P), p{P)), one for each nonempty subtree P of T, such that

Briefly, for each subtree locate its (left or right) ancestor if there is one; if none, then

give its furthest descendant For example, the pagoda representation for the tournament

considered above is given by:

a = ( e , l,/i) d = ( 6 , 4 , d ) g = {g,7,e}

b = (d, 2,a) e = (c, b,g) h = (h, 8, ft)

c = ( a , 3 , / ) f = {i,6,c) t = {f,9,i)

Frangon, Viennot&Vuillemin[35] proved t h a t , for implementing priority queues, the

pagoda has the best average complexity (see §1.5 of Chapter 2)

I I D y n a m i c d a t a s t r u c t u r e s

The motivation for studying dynamic data structures is to compare the efficiency of the various d a t a structures as implemented How is one to analyze the behavior of these

systems? Two basic ideas are time costs and space costs, involving respectively the

com-plexity due to the number of operations involved and the comcom-plexity due to storage — memory — requirements Here we study time costs

2 1 HISTORIES

We present a useful formalism Let O denote the alphabet of operations, in our case we have O C {I,D,Q'^,Q~ } Thus, a word means a sequence of elements belonging to O

We denote the set of words over the alphabet O hy O* We want to count the number of

operations involved when the operations are performed on files under constraints imposed according to the various d a t a structures First, we make a general definition, corresponding

to the necessary condition that a file cannot be of negative size

2.1.1 D e f i n i t i o n For w £ O* and O £ O, denote by \w\o the number of times O

2.1.3 D e f i n i t i o n A schema is a word w = O1O2 • • • On € O* such that the sequence

of subwords Wi = O1O2 • • -Oi satisfies h(wi) > 0, for all i, 1 < i < n

When counting file operations involving keys, we consider only their relative order The method of counting the number of ways an operation can be applied to a given file in a given situation is determined by a sequence of nonnegative integers, ranks, corresponding

to (the relative ordering of) keys

2.1.4 D e f i n i t i o n Let ri, ,r„ be a sequence of nonnegative integers, ranks A

history is a sequence of ordered pairs { ( O i , r i ) , ,{0„,r„) } such that O1O2 • • On is a

schema and r j , , r „ are admissible ranks (for a given model)

The ranks allowed determine the particular models Below, we discuss two models that

have been studied: the maricovian model and Knuth's model So a model is determined

by the alphabet O and, for each data structure, the specification of allowed histories (In

Chapter 2, detailed results for Knuth's model will be presented.)

Here we continue with a sketch of the general approach, assuming a model has been given

2.1.5 D e f i n i t i o n For an operation O, the number of possibilities N p o s ( 0 , fc) is the

number of ranks r for which O is defined on a file of size k

If an operation is not allowed at a given time, we set Npos (0,k) ~ 0 This determines

the count of the number of operations of a given type t h a t are permitted to be performed

on a file

2 2 TIME COSTS

First some notation

2.2.1 Definition For a given class of histories:

1) '^k,t,n denotes the set of histories of length n with initial height k and final height /

Denote the cardinality |?it,;,„| by

Hk,i,n-2) Hn denotes the set of histories starting and ending with an empty file, i.e.,

Tin = 'Ho,Q,n- Denote the cardinality |7f„| by H„

Suppose we are given a cost functional on the set of histories: cost (h), some measure of

the complexity of the histories Then we have

2.2.2 Definition The integrated cost ov6r 7^n is

1

iiTn = — ^ cost (h)

In order to analyze time costs, it is convenient t h a t the d a t a structure satisfy a condition of

stationarity Many implementations commonly used in computer science have the required

property

Here is one way of stating what is needed Denote by Ek the set of all states of a structure

of size k This depends on the particular implementation For example, for a sorted list,

there is only one state for each fc, while there axe A;! possible states of size k for an unsorted

list As another example, there are k\ pagodas of size k One defines

2.2.3 Definition The standard probability distribution on Ek is determined by giving

equal weight to all states of size k corresponding to the schema / * We denote by Ps(e)

the corresponding probability of a state e £

Ek-2.2.4 Definition A data structure is stationary if for all k, probabilities induced on

Ek by histories corresponding to schema /*"^^D, l''Q~, and l''Q'^ (respectively) coincide

with the standard probability distribution p,

The structures we will consider are known to be stationary This property permits the

calculation of costs in a particularly effective manner Namely, we can use the notion of

individual costs

2.2.5 D e f i n i t i o n T h e individual cost of operation O on a file of size k is given by

COk= ^ ps{e) cost {0,e) e€Ek

where cost (O, e) is the cost of performing operation O on the state e

Here costs are determined by the average number of comparisons of keys required for the operation The following table gives the average costs of operations for various implemen-tations,

(In this table we use the notation /li = 1 + | + • • • + p )

Data type Implementation C I ; C D , CQt CQ,

Sorted list Unsorted list Binary tournament Pagodas

Sorted list Unsorted list

Sorted list Unsorted list

insertion in a dictionary of size k is (k + 2)/2 if the hst is sorted, 0 if unsorted The average

cost for deletions and positive queries is independent of whether the list is sorted, while

the average cost of a negative query is (A;-|-2)/2 for sorted lists, k for unsorted Comparing

these costs, it is unclear how to determine which list structure is more efficient This is where the idea (of Frajigon, Knuth, et al.) to consider the dynamic behavior of these structures comes from

To compute the average (integrated) cost we use the individual costs COk- First we

have

2.2.6 D e f i n i t i o n The level crossing number NOk,n is the number of operations of

type O performed on a file of size k in the course of the histories W„

We can thus write

2.2.7 P r o p o s i t i o n For stationary structures, the integrated cost is given by

^'" = TT E E '^^>= • ^^*."

0 € 0 k

2 3 POSSIBILITY F U N C T I O N S : MARKOVIAN MODEL, KNUTH'S MODEL

The models presented here differ principally in the way one counts insertions (and

negative queries) In [54], Knuth discusses two types of insertions in a d a t a structure

Take the interval [0,1] as the set of keys Consider inserting the next key into a file of k

keys:

/o means insertion of a random number by order in the sense that the new number is

equally likely to fall in any of the k + 1 intervals determined by the keys in the file

/ means the insertion of a random number uniformly distributed on the interval [0,1],

independent of previously inserted numbers

G.D Knott[52] showed that Ig differs from / Considering insertions of type IQ gives the

marJcovian model (studied in Fran5on[29])

2.S.1 Markovian model

For this model the specification of histories is as follows (recall t h a t h{w) is the height

of the word w, i.e., it is the size of the corresponding file):

1 For linear lists, the alphabet of operations O = {I,D] with admissible ranks

satisfy-ing:

0<rj <h{Oi Oj-i) Oj = D Q<rj <h{0i 0j-i) Oj = I

2 For priority queues, C = { / , Z? } with admissible ranks satisfying:

r , = 0 0,=D

0<rj <h{0i 0j-i) Oj=I

3 For dictionaries, O = {I,D, Q"*", Q~ ] with admissible ranks satisfying:

0<rj <h(Oi Oj-i) Oj=DoiQ+

0<rj <h(Oi Oj-i)

OJ=IOTQ-4 For symbol tables, O = { I,D,Q^ } with admissible ranks satisfying:

Tj = 0 Oj=D 0<r, <h{0i 0j_i) Oj = Q+

0<rj <h(0i 0j-i) Oj=I

Thus, t h e possibility functions for this model are:

D a t a t y p e Npos(7,A;) Npos(Z?,fc) Npos(Q+,A:) 'Npos{Q-,k)

k + 1

2.S.2 Knuth's model

Considering insertions of type / gives Knuth's model Here is an example of how this

works for a linear list Consider the sequence of operations IIIDI, starting from an empty

file Let X < y < z he the first three keys inserted We have x, y a n d z equally likely to be

deleted Let w be the fourth key inserted Whichever key has been deleted, considering

all four keys, the four possibilities: w<x<y<z, x<w<y<z,x<y<w<z.,

X < y < z < w — are equally likely Thus, we count 4 possibilities for the fourth

insertion, even though the file consists of only two items In general, consider a sequence

of operations Oi 0 „ on aji initially empty file Let On be the z'-th / of the sequence

Let x\ < • • • < l i - i be t h e keys inserted during the first n — 1 operations Let w be the

2-th key inserted All of the possibilities w < xi < • • • < Xi_i, , Xi < • • • < Xj_i < w

are equally likely, whatever t h e keys deleted In combinatorial terms; after n operations,

consisting of i J ' s and n — i Z)'s, the data structure thus being of size k = 2i —n, the keys

are considered as a subset of a set of size i where Etny of the (^) subsets of size k are equally

likely Thus, t h e number of possibilities for the i-th I equals i — regardless of the size of

the structure at that point — while in the markovian model, t h e number of possibilities

for an insertion is A; + 1 for a d a t a structure of size k For dictionaries, a similar argument

applies t o negative queries as for insertions

For Knuth's model the specification of histories is as follows:

1 For linear lists, O = {I,D] with admissible ranks satisfying:

2 For priority queues, O = {I,D} with admissible ranks satisfying:

rj = 0 Oj = D 0<rj <\Oi Oj-i\i Oj = I

3 For dictionaries, O = {I,D, Q"*", Q~ } with admissible ranks satisfying:

0 <rj < h{Oi Oj-i) Oj=D ov Q+

Thus, the possibility functions are given by:

D a t a t y p e Nposfi"" 7 or Q - ) Npos(Z?,/fc) Npos(Q+,A:)

Linear list i k

Priority queue i 1

Dictionary i k k

We will discuss results for this model in Chapters 2 and 3

R e m a r k As basic references for the subject;

Gormen, Leiserson&Rivest[12], Gonnet&Baeza-Yates[39],

Graham, Knuth&Patashnik[40], Greene&Knuth[41], Hofri[45], Kemp[50], Mehlhorn[67],

Mehlhorn&;Tsakahdis[68], and Sedgewick[80] A basic text is Aho, Hopcroft&Ullman[3]

As a general reference, see Fair, Mohr&Schott[71] For an overview, see the handbook

edited by van Leeuwen[56] Of course, Knuth[53] is a general reference

C h a p t e r 2

DATA S T R U C T U R E S & O R T H O G O N A L P O L Y N O M I A L S

In this Chapter, we first present the analysis involved in finding the behavior of data

structures in Knuth's model Then we indicate some features of a finite universe model,

which has perhaps a more realistic flavor t h a n the models first discussed Then we

con-sider mutual exclusion This models a system of parallel processes accessed, at random,

by various users Each process can be accessed by only one user at a time, hence the description A probabilistic model again leads to recurrences of the type studied in this Chapter We conclude with some remarks on a general approach to duality, relating the

space-time recurrence with the dual action of the operator of multiplication by X

I D y n a m i c d a t a s t r u c t u r e s : K n u t h ' s m o d e l

Here we will analyze the behavior of the data structures of Chapter 1, §2.3.2 We present the approach based on p a t h counting, used as well in the study of random walks Using the

notion of dual recurrences this leads to orthogonal polynomials and an associated operator

calculus We then employ the operator calculus in our analysis of the data structures

We will look in detail at the behavior of histories and corresponding integrated costs for linear lists, priority queues, and dictionaries, following the indications of Ch 1, §2.2 First, we enumerate the histories Then we obtain expressions for the integrated costs using level crossing numbers and the individual costs This puts us in a position to compare the relative efficiency of different implementations

1 1 ENUMERATION O F HISTORIES AND O R T H O G O N A L POLYNOMIALS

Before looking at K n u t h ' s model, we introduce the basic technique in the context of the markovian model, where the approach is directly applicable Then we will return to Knuth's model

Here we use the approach used in the study of random walks, where transitions from state (position) to state are governed by certain probabilities For counting paths, the idea is the same, but we use the number of possible transitions, rather than probabilities Consider the markovian model of Ch 1, §2.3.1 Let c„t denote the number of paths, i.e.,

histories, starting from height 0 that are at level k at time n This is, in fact, the same as

Ho,k,n- We think of the height, file size, as the location of a particle (random walker) on

the line It jumps right or left or sits at each time step according to whether the operation

7, D, or a query is applied to the file Denote

ifc = N p o s ( / , A:), qii =Npos{Q,k), dk — Npos{D,k) Then we have, considering the one-step transition from time n to time n -(- 1,

1.1.1 P r o p o s i t i o n Let Cnk denote the niimber of ways starting from level 0 to reach

level k in n steps The c„jt satisfy the recurrence

Cn+l k = U - l Cnk-l + qk Cnk + ^ * + l C„ k+\

with Coo = 1, Cot = 0, A: > 0

We can identify this recurrence with the transition matrix

where the kl entry indicates the number of ways of going from level k to level / in one step,

fc, ? > 0 The n'*' power of this matrix thus gives the number of ways of going from one

level to another in n steps

Now the idea is to look at the duaJ recurrence Think of the matrix in (1.1-1) as

an operator JT on a space with basis { <^o, 0 i , , 4>k-, • • • } Consider the spectral problem where X acts as multiplication by the variable x Writing $ = (<^0i i^ij • • •) we want

This turns into the three-term recurrence relation for the sequence {i^t }:

^4>k= ik'i>k-^\ + (Ik^k + dk<^k-\ (1.1.2)

with initial conditions 4>-\ = 0, <j>o = 1 Thus, the {<^jt } are identified as orthogonal

polynomials in the variable x As in eq (3.1.1) of the Introduction, we denote integration

with respect to the corresponding distribution, expected value, by ( • )

The operator X is the one-step transition operator and may be thought of as a position

operator (as in quantum mechanics) Another way to see t h a t the recurrence relation (1.1.2) is dual to that of Prop 1.1.1 is given by:

1.1.2 P r o p o s i t i o n The action of X on the basis { I/>A } satisfies

X"<l>0 = /^Cnk<l>k

fc=0

Proof: Inductively, assume the relation, which holds for n = 0 Then

This shows X " as the n-step transition operator Shifting all indices by k

1.1.3 Corollary Denoting hy Cni,k the number of ways of going from level k to level I

in n steps, we have

n

X''^k = Y,Cnl,k(f>l

1=0

with coi,k = 1, ifi = k,0 otherwise

We can introduce the monic polynomials J/J^ = </>;t ioh • • • ik-i- They satisfy the recurrence

relation

xil^k = i>k+i + Qki^k + ik-idki>k-i

Equation (3.2.1.2) of the Introduction indicates the link with the continued fraction proach of section §1.6 As follows by equations (3.1.3), (3.1.4) of the Intro., the squared

ap-norm of ^k is

Ik — toil • • • ik-i didi •• -dk

As in §3.1 of the Intro., eq (3.1.2), for the inner product we have

1.1.4 P r o p o s i t i o n The {(fek) satisfy

1.1.5 T h e o r e m The number of histories Hk,i,n is given by the expected value

And we have the observation, from the point of view of operator calculus, via the first

statement of the Theorem,

1.1.6 Corollary The exponential generating functions

V ^Hk,i,n = (e'^ <f>k(t>i)/ei

are the matrix elements of the operator e'^ in the basis

{(i>k}-R e m a r k This feature is a connection with Volume 1 [21] of this series, Chapter 7 in

particular, that will be useful in our analysis

It turns out that, appropriately modified, these techniques can be applied to the study

of Knuth's model as well

1.1.1 Enumeration of histories in Knuth's model

Here we introduce c^j., the number of paths starting from 0 that are of height k after n

steps, with s the number of insertions and negative queries combined P r o p 1.1.1 becomes

1.1.1.1 P r o p o s i t i o n The c^f satisfy the recurrence

< + i k = ^ < " * - ! + s c^-1 + qk c;^ + dk+i c'„ ;t+i

with CQO = 1, CQJ = 0, A:,3 > 0

The idea here is to introduce the generating function

Cnk{t) = Xll"*^"*

8 = 0

Thus, the s is summed out, leading to a situation similar to the maxkovian model We see

immediately t h a t

1.1.1.2 P r o p o s i t i o n The C„kit) satisfy the recurrence

Cn+l k = tCn k-1 +{t + Qk) C„k + dk+\ Cn k+\

with Coo(t) = 1, Co kit) = 0, ^ > 0 The t indicates t in the case when there axe negative

queries, otherwise it is omitted

Here we introduce the operator Xt acting on a space with basis { (j>k{x,t) } such that

n

k=0

as in Prop 1.1.2 Dual to Prop 1.1.1.2 is

1.1.1.3 P r o p o s i t i o n The polynomials { 4>kix,t) } satisfy the recurrence

X(j>k = t(f>k+i + (t + qk)4>k + dk4>k-l with <^_i = 0, (/"o = 1 The squared norms are given by

ekit):=Ukf =t-''did2 -dk

The corresponding monic polynomials {«/"*:} are given by tpk = <* <f>k satisfying the

recur-rence

xi/^k = V't+i +(i+qk)tpk -\-tdk^k-\ (1.1.1.1)

The n-step transitions satisfy

n

As in Theorem 1.1.5 we have

1.1.1.4 P r o p o s i t i o n Let H^ ; „ denote the number of histories from level k to level I

in n steps with s insertions and negative queries (combinedj The generating function

°° t"

3 = 0

satisfies the relations of Theorem 1.1.5 with t-dependent X and { 4>k

}-To get back to the number of histories from HI, „(<) we observe the following

1.1.1.5 P r o p o s i t i o n The number of histories Hk,!,n is given by the integral

Proof: In the above Proposition, integrate both sides with respect to e~* from 0 to

oo using the relation

which takes into account all possible insertions and deletions •

1 2 DATA S T R U C T U R E S AND ORTHOGONAL POLYNOMIALS IN KNUTH'S MODEL

Now we apply the above results to some particular data structures

1.2.1 Priority queues

As noted above in C h l , §2.3.2, there are no queries, and dk = I, for all k > 1 Prop

1.1.1.3 gives, where we rec?i,ll that the middle term, involving (pk, corresponds to queries:

1.2.1.1 P r o p o s i t i o n For priority queues we have:

1 The recurrence relations

X<j>k = i(f>k+l + <i>k-\

with the squared norms ek{t) = <~*

2 We have the identification

4>k{x,t)=t-''I^Uk{xl2yft)

<" f2n

Tchebychev polynomials of the second kind, and the moments /^2n(0 =

n + 1 \n

Proof: The recurrence formula follows directly from Prop 1.1.1.3, modified

accord-ing to the absence of queries Then # 2 follows from the recurrence formula for the Un

given in Intro., §6.2 The density function for the orthogonality relations for the U„ is

- \/l — x'^ on [—1,1] Changing variables x —> x/(2y/i) gives the density

TTV^ V it

on [—2\rt,2\/t] The moments scale inversely as x, yielding iJ.2n{t) as indicated

For the histories;

1.2.1.2 P r o p o s i t i o n The generating function for the number of histories, Ho,k,nii),

with Coo{t) = 1, Cok{t) = 0, k > 0 (Notice that this is very close to Pascal's triangle.)

Inserting the formula in the statement of the Proposition, one sees that it must be verified, after cancelling common factors of i("+*:+i)/2^

n + 2 \k + l ( n + 1 \ ^ ( " + 1 \k^-2

^(n + 1 - fc)/2y n + 2 V(" + 1 - ^ ) / 2 / « + 1 \ { n - \ - k)l2J n + 1

Writing out in terms of factorials, the result readily follows •

1.2.2 Linear lists

Here, a.s in the previous section, there are no queries, while dk = k, for A: > 1 Thus,

1.2.2.1 P r o p o s i t i o n For linear hsts we have:

1 The recurrence reiations

x4>k =t 4>k+i + k4'k-i with the squared norms £fc(<) = <^* k\

2 We have the identification

<j>kix,t)=t-'=Hk{x,t) Hermite polynomials, and the moments fi2n{t) — (2i)" ( | ) T I

Proof: The recurrence formula follows from Prop 1.1.1.3, modified according to the

absence of queries, and # 2 follows from the recurrence formula and facts given in Intro.,

1.2.3 Dictionaries

Proceeding as in the above two sections, we find

1.2.3.1 P r o p o s i t i o n For dictionaries we have:

1 The recurrence relations

x(j>k-t (j>k+\ + (< + k)(jik + kcpk-\

•with the squared norms et(<) = t~'' k\

2 We have the identification

Mx,t) = t~''Pk{x,t) Poisson-Charher polynomials, and the moments Hn{t) = Hn = 2 •*">* * •

1.3 OPERATOR CALCULUS APPROACH

Look at the dual recurrence showing the operator X acting on the basis vector (fije

X4ik = «*<j^jfc+i + <ik<i>k + dk4>k-\

Each term of this recurrence corresponds to an action on the file: insertion, query, deletion

We define corresponding operators /Z, the raising operator, N, or neutral operator, and L,

lowering operator, named according to their effect on the index k:

R<j)k = ik<f>k+i N4>k = qk(l>k L<^k = dk4>k-i

A useful operator V is defined by

V(l>k = {klik-i)4>k-\

so t h a t

RVH = kcj>k

is the nunjber operator, multiplying by the level k

R e m a r k Some of the Lie algebras of dimensions three and four correspond to various

choices of the coefficients ijt, qk, dk- It turns out t h a t our basic d a t a structures

correspond-ing to classical orthogonal polynomials correspond as well to fundamental Lie algebras The

latter connection is a major theme of volume 1 In particular, the Poisson distribution,

Poisson-Charher polynomials, corresponds to the oscillator algebra with ik = I, gk = t + k,

dk = k, and the Gaussian distribution, Hermite polynomials, corresponds to the

iJeisen-berg algebra with ik = I, qk = 0, dk = k The discussion here applies to the Meixner

polynomials class in general, see Chapter 4 for complete details, but in fact we will apply

it here explicitly for just the Poisson and Gaussian cases

Observe that

1.3.1 P r o p o s i t i o n On the monic polynomials, ipn{x) = io • • • in-i4'n{x),

Thus, iPn

^R"<t>o-Now we define the coherent states that are exponential generating functions for the

1.3.2 P r o p o s i t i o n On the coherent states we have:

-RV'a = -g- ^ a , l^'Aa = a ^Aa

Proof: These follow from the definition and the Proposition above describing the

action on the tpn- •

From the general theory of Meixner polynomials, we have the following structure

Given t h a t the moment generating function for Xt is

where e*^^'' is the moment generating function of Xf In particular,

for the Poisson distribution: H{s) = e' — 1, Ho{s) = e' — I — s, V(s) = e' — 1, U(v) =

log(l + v)

for the Gaussian distribution: H(s) = 5^/2, with V(s) = s — U(s)

1.3.4 P r o p o s i t i o n For the Poisson and Gaussian cases, these functions satisfy the

relation

H{a + 6) - H(a) - H{b) = V{a)V{b) This is called the cocycle identity

Notice that by orthogonality, the inner product of two coherent states is a generating

function for the squared norms ||^„(p However, using the observations above, this can be

calculated directly as follows

1.3.5 P r o p o s i t i o n For the Poisson and Gaussian cases, the inner product of the

coherent states is given by

Proof: We have ( e " ^ ' e ' ' ^ ' ) = (e(«+'')-^') = exp(tH{a + b)) and thus by the cocycle

identity, using the formulation of Prop 1.3.3,

V,„i, = gtlHiU(a)+U(b))-HiU{a))-H(U{b))]

^ ^tViU{a))V(Uib)) ^^tab

l.S.l Coherent state representations

Playing the role of a Fourier transform for this operator calculus is the CSR, coherent

state representation For an operator, Q, this is given by the normalized matrix elements

( Q ) a 6 - ( V ' a , W 6 ) / ( V ' a , ^ 6 )

Note that the adjoint Q* satisfies (Q*)ab ~

{Q)ba-1.3.1.1 T h e o r e m For the Poisson and Gaussian cases, we have for R and V

and for the number operator

The joint coherent state representation, for the product, is

(eP^ e»^ A^^)„6 = exp(apt + qb\ + a6t(A - 1))