The numerical solution of delay-differential equations P1

1.1.3 the propagation of derivative discontinuitiesthrough systems of delay-differential equations 31... In 1.1.3 a model for the propagation of derivative discontinuities through system

A thesis submitted to the University of

Manchester for the degree of Doctor of

Philosophy in the Faculty of Science.

October 1989

1.1.3 the propagation of derivative discontinuities

through systems of delay-differential equations 31

Chapter two - DDE methods

2.2 linear multistep and predictor corrector methods 75

2.5 extended ODE-techniques and the method of steps 114

2.6 an alternative scheme for error control 117

Chapter three - DELSOL

3.1.2.2 back-approximation and lag evaluation 188

3.2.4 error estimation, stcpsize and order control 222

3.2.5 stcpsize selection following secondary

B the effect of spurious derivative

0 Abstract

Delay-differential equations (ĐÉs) arise in many fields of

science and engineering In this thesis we consider the development

of numerical software for the solution of such problems

Our discussion opens with a brief introduction to the theory ofdelay-differential equations Attention is paid to features relevant

to numerical codes In particular a model for the propagation of

derivative discontinuities through systems of equations is presented

Following a short resumd of standard techniques for the solution

of ordinary differential equations (ODÉs), we then consider the

application of ODE software to evolutionary ĐÉs Special attention

is paid to the occurrence, effect and accommodation of derivative

discontinuitics and the approach is illustrated for linear multistepand predictor-corrector methods After discussing stability, someproblems specific to state-dependent delay-problems are consideredbefore a brief comparison with the 'method of steps' as described byEl'sgorts An new alternative error control strategy for ODE andĐE schemes based upon a variational-type error analysis is then

presented, followed by a discussion of the problems inherent in

detecti ng derivative discontinuities

We conclude by presenting a variable-order variable-step numericalroutine, derived from an existing reverse communication Adams PECE ODEcode, suitable for the solution of systems of delay-differential

equations A novel representation for the differential equation isused to acknowledge structural differences between delay- and ordinarydifferential systems Special attention is also paid to the

organisation of lag function evaluations, back-solution approximationand order and stepsize controls Finally we present a selection ofnumerical examples and a discussion of the codes application to moregeneral delay- and to neutral-differential problems

0 Declaration

No portion of the work referred to in this thesis has been submitted

in support of an application for another degree or qualification ofthis or any other university or institute of learning

o Statement

Since obtaining a BSc in Mathematics in 1985, David Wille has

studied under Prof C T H Baker in the department of mathematics

at the university of Manchester In 1986 he obtained the degree ofMSc in Numerical Analysis and Computation and held the position oftemporary lecturer for the academic year 1988-89 within the above

department He is currently employed as a research associate

to my mum, my dad and my sister Sian

0 Acknowledgments

I would like to start by expressing my thanks to Prof C T H Baker who's constructive comments and lively interest have been

invaluable throughout the preparation of this thesis.

Many thanks also to all those in Manchester and at NAG who have helped me through the course of my work In particular I would like

to thank the Numerical Algorithms Group for making available a copy

of their code DO2QFQ and Julia and Lynn for their expert and rapid typing.

This work was supported in part through a CASE award from the

Science and Engineering Research Council in collaboration with the Numerical Algorihms Group (UK) Ltd.

to one specific existing ODE code Thus although we recommend our

routine for general use, we recognise that for certian specific

problems it may be out-performed by simpler and less general codes We consider, however, the development of general purpose robust software

to be a suitable consideration for numerical analysts.

the Numerical Algorithms Group Plc.

Z+ the set of positive integers {ieZ:i  l}

C the set of complex numbers f x + iy : x,y e R 1

C ( A 4 B ) the set of continuous mappings from A to B

C k ( A 4 B ) the set of k-times differentiable mappings from

A to B.

Cc °( A 4 B ) the set of analytic mappings from A to B.

1r( A 4 B ) the set of piecewise-analytic mappings from

A to B.

For simplicity the following abbreviations are also used

C k (A)

k C

Moreover if C k (A) for some vaI

l neighbourhood A

A

of t - N E (t), e  0 - we say that f is Ck at t.

D_ and D.I are used to denote the left and right hand derivative

th>0

Where defined, D then denotes the two-sided derivative operator

D f F4 f' = af f = D_f

• Equation and reference numbers

The numbering of figures, tables and equations is restricted to the sections or subsections in which they are defined Where references arc required to equations from other sections, the equation number is then prefixed by the appropiate section number Thus

(1.1.3:14)

denotes equation (14) in section (1.1.3) The use of square

brackets, eg [25], is reserved for references.

o Graphs

Graphs are refe rred to in the text as cartesian products Thus

t x y(t) denotes the graph of y as a function of t The first

variable is always plotted along the horizontal axis, and the second along the vertical.

Chapter one

introduction and ODE methods

Chapter one - foreword

In chapter one we present some of the background theory and

numerical methods necessary for the numerical treatment of

delay-differential equations.

We start - in (1.1.1) - with a short introduction to the theory and background of delay-equations We present an existence and

uniqueness proof in section (1.1.2) In (1.1.3) a model for the

propagation of derivative discontinuities through systems of delay equations is given This is used in later chapters for stepsize and order control.

In section (1.2) we present a brief resume of numerical methods suitable for ordinary differential equations In chapter two these are adapted for delay-problems We conclude (1.2) with a discussion

of error-per-unit-step error control strategies which we refer to in

a later chapter in section (2.6).

Most of the results presented in chapter one are taken from the existing literature with the exception of section (1.1.3), which we believe to be an original contribution to the field Our results may

be distinguished from those of Feldstein (29] The material from

(1.1.3) has also appeared in a Manchester numerical analysis technical report (53].

section 1.1

an introduction to delay-differential equations

1.1.1 Introduction

p Delay-differential equations

In this thesis we are concerned with the numerical solution of

delay-differential equations (ĐÉs) Delay-differential equations

may best be regarded as extensions of ordinary differential equations(ODÉs) in which the solution derivative y.(t) is allowed to

depend not only on the current solution point (t,y(t)), but also onvalues of the solution at previous points Thus the equation

past solution points, or lag points These are themselves defined by lag functions (see below) The lag points may vary in position not

only with the current time, but also the current solution y(t) Ageneral form for a first-order scalar ĐE is thus

Y i ( t ) = f(t,y(t) y(al(t,y(t))) Y(ak(td(t)))) (2)

y e R for t  0 where the lag points [ai(t,y(t))] all satisfy

ai(t,y(t)) 5 t and k is finitẹ In this equation the lag functionsare fai(t,y(t))} Alternatively, ĐÉs may be defined in terms of

ai(t,y(t)) = t - ti(t,y(t))

for suitable fti(t,y(t))}, (2) may be re-written as

y (t) = f(t,y(t),y(t-TI(t,y(t))) y(t-Tk(t,y(t)))) (3)

For t  0 The differences ti(t,y(t)) = t - ai(t,y(t)) are known

as delays' and may be defined in terms of delay-functions, {TỌ

In general, of course, lag point positions fa1(t,y(t))1

may depend not only on the current value of t, but also on y(t).

If a lag function ai(t,y(t)) varies with y(t) it is said to be

state-dependent, and if not state-independent If all the lags are state-independent then the delay-equation too is said to be state-

independent.

0 Terminology

Unfortunately, the terminology for delay-differential equations has yet to be standardized Some authors, for example, refer to

delays, Ti = t-a, as [time] lags Alternative names for ĐÉs

include 'ordinary differential equations with time lags' or 'retarded

ordinary differential equations' (RODES) A sensible generic term is

'ordinary differential equations with retarded arguments'.

NB 1 this term is later also used to denote 'delayed-terms' but

its interpretation is always clear from its context.

LI Occurrence

Examples of delay differential equations arise in many areas ofscience and engineering where dynamic processes depend on states atprevious times In control theory and population dynamics, for

example, delays can be presented physically in feedback loops Otherexamples can be found in fields as diverse as physics, engineering, biology and economics Indeed, a study of differences in behaviourbetween ordinary and delay-differential equations suggests that wherephysical delays are present, ĐÉs may provide the only realistic

models availablẹ For a more detailed introduction to their

applications the reader is referred to SCHMITT [1] A selection ofĐÉs, together with numerical results, is presented in Chapter 3

0 The initial set

Many common ĐÉs - and all those considered in this thesis - areexamples of evolutionary problems Given an initial state (ẹg a

boundary condition) they uniquely determine the evolution of their

solution over all subsequent timẹ For an ODE, a suitable initial

state might be a starting value, but for a ĐE it may be more

involved Consider for example,

,

y (t) = y(t-1), t  0 (4)

For such an equation it is clear that the solution is no longer

uniquely defined by a single value at t = 0, but rather by a range

of initial values specified over some initial range in t. In

example (4) for instance y is required over an initial interval

[-1,0] Such a range is known as an initial set and the associated solution values, as an initial function For equation (4) the full evolutionary problem can then be expressed:

y ' (t) = y(t-1), t  0

(5) y(t) = y(t) t E [-1,0]

for some suitable initial function 9 : [-1,0] -4 R defined on the

initial set [-1,0] Not all delay problems, however, require an

initial function in order to be well defined The equation

,

y ( t) = y( t _t/ 2 ) , y(t/2) t  0

has an initial set of measure zero and so as for ODÉs requires

only a value at a single point y(0) = a as a boundary condition.

Equations of this type are sometimes known as initial value

delay-differential equations (IVĐÉs) [2] as distinct from initial

function delay-differential equations (IFĐÉs) [2] which require

initial functions.

0 Notes

(i) Unless otherwise stated we shall always assume that all

integration is done with respect to increasing timẹ Thus

the initial set and the lag points will always be presumed

to lie to the left of the current time t

ai(t,y(t)) 5 t V i,t

Alternatively the delays are always presumed to be positive

Ti(t,y(t))  0 V i,t

(ii) As for evolutionary ODE problems, we shall always assume

ĐÉs to be defined in terms of their right-hand solution

derivatives ỵ;(t) Thus for

y i (t) = f(t,y(t),y(t-T)) for example we shall always read

y(t) f(t,y(t),y(t-T))

where ỵ;.(t) = lim {Y(t4-8)-Y(0}/8

840

0 Derivative discontinuities

The introduction of an initial set for ĐÉs can play an

critical röle in determining the solutions continuitỵ Where an

initial set of non-zero measure is present, the solution may only becontinuous in all its derivatives if the right and left hand

derivatives all agree and are defined at the initial point This

however is not in general true and ĐE solutions are in general onlypiecewise continuous in their [higher] derivatives Consider, for

example, equation (5) where y(t) a 1 :

y ' (t) = y(t-1) t  0

(6)y(t) 1, t c [-1,0]

Over t e (-1,0] this has solution

that the left hand derivative y_(0), obtained by differentiating

,the initial function, and the right hand derivative y+(0), defined

by the differential equation, clearly differ Thus the two sided

derivative y ' (0) does not exist and y ' (t) has a jump-discontinuity

at t = 0

Once occurred, this initial derivative discontinuity can be

propagated through the delayed-term y(t-1) to later times Considerfor example the point t = k where k E Z Differentiating (6)

k-times we obtain

(k+1)(t) = y(k)

which implies, by induction,

(k+1)([) = y (t-k)

y

thus showing that (k+1) has a jump discontinuity at t = k

We say y has a (k+O th order derivative discontinuity at the

point t = k.

Such arguments are infact quite general and, with minor

modifications, may be extended to a wide range of ĐÉs Where thefunctions f and (ai) are analytic, all discontinuities can be

shown to originate from the initial point or set (1.1.3) A

dis-continuity in y(k+1) at a point t can only arise if y( k ) has aprevious discontinuity at some point t ' < t

t,

= t-Ti(t,y(t))

for some Ti Thus, given discontinuies at the initial point or inthe initial set, to understand their subsequent distribution it is

sufficient to understand how they are propagated through the

solution This is is the subject of section (1.1.3)

Information about the solutions continuity is of crucial

importance to numerical methods A failure to correctly accommodatederivative discontinuities can undermine the numerical formulae on

which by the methods are based

Throughout this thesis, for ease of notation, the terms

'derivative discontinuity' and 'discontinuity' are used

inter-changeably Moreover if y is itself discontinuous, then we

say that it has a 'discontinuity of order-zero'.

initial set, and the initial point is discontinuity-free - ịẹ

continuous in all derivatives - then the solution of the ĐE is

merely the analytic continuation of the initial set For equation

(7) that is simply y(t) = sin(t)

For initial value delay-differential equations (IVĐÉs) the

solutions can also be analytic Where the initial set has zero

measure there can be no inconsistency between left and right initialderivatives Thus, providing f and (at) are analytic, then so isthe solution - c.f for ODÉs [3]

0 An observation

Discontinuities at the initial point essentially arise from

inconsistencies between f, (at) and the local behaviour of 9 atvarious points in the initial set For the equation

Dik+1) y(t) = Di_ f(t,y(t),9(y(ăt,y(t)))

for various k Here D_ and D i_ are the left and right derivativeoperators with respect to t The continuity at t = 0 is thus

determined by f, a and the local behaviour of 9 at the points

t = 0 and ă0,y(0)) < 0

a Comment

ĐÉs can sometimes introduce derivative discontinuities into

mathematical models where they are not physically expected For

y'(t) = ay(1)fb-y(t-T)}

Y(1) = 1for example - arising from a delay-logistic population model [4] -

the discontinuities at t = 0,T,2T may not be physically

meaningful The discontinuity at t = 0 arises from the simplifiedform of the initial function whereas its propagation is a result ofsimplified dependence on y at t and t-T alonẹ Indeed the

whole equation is a continuous simplification of a discrete process

If y(L-T) were replaced by the more realistic term [5]

K(s,y(s)) ds0

for some K : C00 (R 2 4 R) the solution to the corresponding Volterraintegro-differential equation - sec (1.1.2) below - would be analytic

on (0, ) ĐÉs are still often never-the-less useful in modelling

systems with delayed processes A simplified model with a delay may

be better than a simplified model without

y ' (t) f(t,y(t), y(al(t,y(t)) y(ak(t,y(t)))

can be readily generalised to the system or vector equation

.

y (t) = f(t, y (t), Y(al(t,Y(t)) Y(ak(t,Y(t))), y e Rn

where y e Rn , ai(t,y(t))  t Vi and k is finite This is the

most general form of DDE considered in this thesis As is shown in (1.1.3) however, the behaviour of systems (9) may differ slightly from scalar forms (8) Thus is useful to re-write (9) into the alternative form

Yi = f i ( t Y

n -11

Y n , 1k1

Ym ( aii ( t Y))

-11 11

Y (a (t,Y))) m1F1 —1.E1

Volterra functional differential equations, existence and uniqueness

To establish existence and uniqueness it is convenient to consider ĐÉs as members of the wide class of Volterra functional differential equations (VFDÉs) [6] In VFDÉs, the derivative y ' (t) is allowed

to vary not only with t, y(t) and a finite number of past values, but the entire past solution : f y(s) : s  t } Thus, integrating forwards across [a,b] we can write

y i (t) = F(t,y(.)) , y e Rn

where F is a [Volterra] functional

F [a,b] x Cn[a,t] -4 Rn

and y(.) is the restriction yl

[a,t] Here, [a,a] is the initial set Existence is then established by the following theorem due to Driver [7].

THEOREM 1 — Driver

"Let F be a functional which maps

[a,b] x Cn[a,b] -4 Rn such that F(t,y) is

(i) continuous in t and (ii) locally Lipschitz continuous in w

for t E [a,b] and y E Cn[a,b].

Then for any initial function

y[a,a] 4 D g Rn

for D compact there exists a unique function

g : [ a,P) -4 Rn

which solves the functional differential equation

,

y (t) = F(t,y(.)) over the interval [a,[3] where p c (a,y) for y < b."

a Related equations

Other examples of VFDÉs include Volterra integro-differential

equations [8] (VIDÉs)

t ,

y i (t) = f(t,y(t),y(t+T))

for example, are also omitted For a formal introduction to and

a discussion of both these and ĐÉs the reader is directed to

EL'SGOL'TS [10] Neutral-, delay- and advanced- equations are

sometimes collectively referred to as 'differential equations with

deviating arguments'.

1.1.3 The propagation of derivative discontinuities in systems of

delay-differential equations

0 Introduction

Since the presence of derivative discontinuities can clearly

affect many of the schemes used to integrate ĐÉs (2.1.3), for

numerical methods it is often useful to know the position and nature

of all such discontinuities Since it can be shown, for sufficientlysmooth derivative and delay functions, that all derivative dis-

continuities outside the initial point or set must arise from

previous discontinuities (see 13 below), it is sufficient to

understand the way they are propagated through the solution This

is the subject of this section In particular we discuss how

discontinuities arc propagated through systems of equations

0 Representation

Since the propagation of derivative discontinuities turns out to

be related to the coupling between solution components the expandedcomponent-wise representation (10)

y m2 (a (t,y)))t2 22

Yn = fn(t' Yn

nl — Ynnk2' Y mnl(ain

nl( " )) —

Y m ( a/T1 (t4)))Ilk-2 112

turns out to be more useful than the vector form (9)

y(t) = f(t , Y(t), Y(al(t,Y(t))) Y(ak(t,Y(t))) , Y c Rn For this reason it will be preferred for the remainder of this section.

0 Strong and Weak Coupling

In order to build up a picture of the way in discontinuities are propagated through systems of ĐÉs, two new notions are intro- duced; those of strong and weak coupling Consider, for example the

Oh component in (11) For analytic fi and {ai} we show that yi can only have a discontinuous derivative if

(i) it inherits a discontinuity from some other component of y(t) or

(ii) one of its lag points a— (t,y(t)) lies on a past

mij discontinuitỵ

Writing z = y (a ) we now show this analyticallỵ Using the

mij mij mij continuity of fi and {oci} , and the notation of (11), we by first multiple applications of the chain rule, and then by the mean value theorem obtain

r ii *—

+ E EL r iz (7).01 ) - z1(11P.(t2)1

m i m ij ii7.0 j=1 i

These two modes of propagation (i) and (ii) will correspond to ourdefinitions of strong and weak coupling Consider case (i) first.

If the i th component, y(t) of (11), is to inherit a discontinuity(directly) from some other component, the j th say, then it is clearfrom the above equations that yj(t) must appear and be referenced

in either the argument list of fi, or of a lag function fa– 1

mijreferenced by fi Whenever this is true we say that i is stronglycoupled to j, a definition which can be used to formally define alogical relation S, and then a graph which models the strongcoupling Formally we write

Representing each component-index as a node, we then represent thecouple jSi by a directed path of length one from j to i

making a similar expansion for the fz(r)I themselves, we see that

S i

•

j ,

4

Repeating this for all possible i and j we obtain a for a

n-dimensional system, a n-node graph which fully depicts the systems' strong dependence We shall refer to such a graph as the strong

dependency network Consider for example the following system:

1

This fully summarises the strong dependence of the system It

should be noted (of course) that two nodes are strongly dependent only

if there is a direct link between them Although, for example,

in figure (2) a discontinuity in y2 may affect the continuity of y4 the indices 2 and 4 are not strongly coupled as the discontinuity must cross more than one link to pass between them Thus, to test whether

some component, yj, can affect some other component, yi, we should ask whether there is a directed path between them rather than a single arrow Formally we say that we are interested in the transitive

closuré S of S rather than S itself.

In ĐÉs however, strong coupling does not represent the only

means by which discontinuities may be passed between different

components As in (ii) above, an alternative source of such

dis-continuities is from the delay-terms fy (a—(t,y))) We shall

mu mij

refer to this as weak coupling We say that a component i is weakly

coupled to j if fi depends (directly) on the component yi at

some previous timẹ This relation may be formalised as before into a new logical relation W which will represent weak coupling That

is we write

jWi 4 i is weakly coupled to j

and shall refer to the corresponding graph which it defines as the

weak dependency network By convention we shall represent weak links

by dotted lines In equation (13) for example there were two weak

links 1W1 and 4W3 which gives a weak dependency network of

NO that is jii 3(ti) i  o n : j s., i = sn and t 1 St1 + 1 , i =

figure 3 — the weak dependency network for system (13)

If we now combine this graph with the strong network given above we obtain the following dependency network which fully depicts both the strong and weak coupling of the system:

By (12) this diagram fully describes all the propagation modes by

which discontinuities may be propagated through the solution.

0 Extensions

From (12) it is also clear that even if some delayed component

does lie on a discontinuity, the order of the discontinuity inherited must be at least one degree higher than the original In terms

of the dependency network, the order of a propagated discontinuity

increases by at least one for each arc it crosses Extending this

principal across many arcs it can be shown that the minimum degree of smoothing between any two points is equal to the minimum distance

between them taken along the directed graph Suppose for example

that yi in (13) encountered a discontinuity in the past solution

which constricted its order of continuity to 7 Then from figure 4, Y2 has an order of continuity potentially as low as 8, y3 as 9

and so on There is no way, of course, of telling from the dependency network whether these minimum bounds are in fact achieved, as that

will in general depend on the precise nature of { f i} and fail.

Indeed such high order or generalised smoothing is well known through delayed-terms in scalar equations and for further details the reader

is referred to NEVES and FELDSTEIN [11] For systems of equations,

as are considered here, such higher order effects could be taken into into account by using a weighted dependency network Such extensions are however often numerically ill-conditioned - see (2.7) - and so are not considered in this thesis.

0 Practical application

The above theory can now be used to construct a practical

algorithm to bound the continuity of the solution components Since

it can be shown for analytic {fa} and fail, that derivative

discontinuities ultimately arise from components in which lag

lag functions actually lie on past discontinuities, it is clear that

in order to obtain a bound, ci, on the continuity of yi, it is

sufficient to take the minimum of the propagated discontinuities

originating from these points Furthermore, by the above argument,the degree of smoothing of such discontinuities from their source

to their point of effect is given by the distance they traverse acrossthe dependency network Denoting the set of components which have

lag terms lying on past discontinuities by H, and the bounds imposed

by their lag terms on their continuity by MO, we can write

ci = min fhi + d(j,i)} (14)

ieH

where d(j,i) denotes the minimum distance from j to i across

the dependency network This provides the basis of a practical

algorithm to bound the continuity of each component so long as we

keep an accurate record of the solutions continuity starting from theinitial point and a check on whenever a lag term crosses a dis-

continuity This is discussed in sections (2.1.3) and (3.1.2.3)

A detailed proof of the above result is given in the appendix A

By convention we assume that d(i,i) = 0 for all i and write

hi = if fi inherits no discontinuities from any (previous) lagterms

a Comment

The notion of strong and weak coupling can be used to gain some insight into the nature of delay-equations Strong dependence is in

no way new to delay-equations, occurring for example in coupled

systems of ODÉs, but weak coupling is new and sets the delay-case apart from that for ODÉs Whereas strong coupling governs the propa- gation of discontinuities within a given time instant, it is weak

coupling which is reponsible for the propagation of information

directly from one time instant to another and thus gives rise to the characteristic structure of ĐÉs.

Consider for example the following system

3 3 3

1

\ -"k 2 \

N

Expanding this figure along the time axis we obtain the following

picture

figure 6 — the role of strong and weak coupling

where each strong dependence network lies in a plane perpendicular

to the :-axis Strong coupling is then seen to propagate

dis-continuities within specific time instances whereas weak coupling

propagates them between different times From this picture it is

also clear that the order of any inherited discontinuity depends notmerely on the overall order of the solution at the previous time butrather on the order of a specific component In practical terms thismeans that unlike for ODE codes we are interested in the component-wise continuity and not the overall continuity of the solution This

is the motivation for the theory presented in this section In theabove example, for instance, it is clear that the minimum order ofsmoothing between successive discontinuities is of order two ratherthan one, which might have otherwise have been expected