Numerical Methods for Ordinary Dierential Equations Episode 8 ppsx

Pad´e approximations to the exponential function are especially interesting to us, because some of them are equal to the rational functions of someimportant Gauss, Radau and Lobatto meth

RUNGE–KUTTA METHODS 229Lobatto IIIB (s = 5, p = 8),

21 2520

56−15 √21 315

343−69 √21

2520 01

2 1 20 49+12√

12 360

8 45

49−12 √12

7+√

21 14 1 20 343+69√

21 2520

56+15√

21 315

119+3√

21

1 20

49 180

16 45

49 180

1 20

7− √21

14

1 20

29 180

47−15 √21 315

203−30 √21

1260 − 3

140 1

2 1 20 329+105√

21 2880

73 360

329−105 √21 2880

3 160 7+√

21 14

1 20 203+30√

21 1260

47+15√

21 315

16 45

49 180

1 20 1

20

49 180

16 45

49 180

1 20

Exercises 34

34.1 Show that there is a unique Runge–Kutta method of order 4 with s = 3

for which A is lower triangular with a11= a33= 0 Find the tableau forthis method

34.2 Show that the implicit Runge–Kutta given by the tableau

1 4 1 8 1

7

10 − 1 100 14 25 3

20 0

1 14 32 81 250 567 5 54has order 5

34.3 Find the tableau for the Gauss method with s = 4 and p = 8.

34.4 Show that Gauss methods are invariant under reflection.

35 Stability of Implicit Runge–Kutta Methods

350 A-stability, A(α)-stability and L-stability

We recall that the stability function for a Runge–Kutta method (238b) is therational function

where λ ≈ 0.158984 is a zero of 6λ3− 18λ2+ 9λ − 1 This value of λ was

chosen to ensure that (350b) holds, even though the method is not A-stable

It is, in fact, A(α)-stable with α ≈ 1.31946 ≈ 75.5996 ◦.

351 Criteria for A-stability

We first find an alternative expression for the rational function (350a)

Lemma 351A Let (A, b, c) denote a Runge–Kutta method Then its stability

function is given by

R(z) = det (I + z(1b − A))

det(I − zA) .

Figure 350(i) A(α) stability region for the method (350c)

Proof Because a rank 1 s × s matrix uv has characteristic polynomial det(Iw −uv ) = w s −1 (w −v u), a matrix of the form I +uv has characteristic polynomial (w −1) s −1 (w −1−v u) and determinant of the form 1+v u Hence,

R(z) = N (z)

D(z)

and define the E-polynomial by

E(y) = D(iy)D( −iy) − N(iy)N(−iy).

Theorem 351B A Runge–Kutta method with stability function R(z) =

N (z)/D(z) is A-stable if and only if (a) all poles of R (that is, all zeros

of D) are in the right half-plane and (b) E(y) ≥ 0, for all real y.

Proof The necessity of (a) follows from the fact that if z ∗ is a pole then

limz →z ∗ |R(z)| = ∞, and hence |R(z)| > 1, for z close enough to z ∗ The

necessity of (b) follows from the fact that E(y) < 0 implies that |R(iy)| > 1,

so that

these conditions follows from the fact that (a) implies that R is analytic in

the left half-plane so that, by the maximum modulus principle,|R(z)| > 1 in

this region implies|R(z)| > 1 on the imaginary axis, which contradicts (b)

352 Pad´ e approximations to the exponential function

Given a function f , assumed to be analytic at zero, with f (0) = 0, and given non-negative integers l and m, it is sometimes possible to approximate f by

a rational function

f (z) ≈ N (z) D(z) , with N of degree l and D of degree m and with the error in the approximation equal to O(z l+m+1 ) In the special case m = 0, this is exactly the Taylor expansion of f about z = 0, and when l = 0, D(z)/N (z) is the Taylor expansion of 1/f (z).

For some specially contrived functions and particular choices of the degrees

l and m, the approximation will not exist An example of this is

f (z) = 1 + sin(z) ≈ 1 + z −1

6z

3+· · · , (352a)

with l = 2, m = 1 because it is impossible to choose a to make the coefficient

of z3 equal to zero in the Taylor expansion of (1 + az)f (z).

When an approximation

f (z) = N lm (z)

D lm (z) + O(z

l+m+1)

exists, it is known as the ‘(l, m) Pad´ e approximation’ to f The array of Pad´e

approximations for l, m = 0, 1, 2, is referred to as ‘the Pad´e table’ for the

function f

Pad´e approximations to the exponential function are especially interesting

to us, because some of them are equal to the rational functions of someimportant Gauss, Radau and Lobatto methods We show that the full Pad´etable exists for this function and, at the same time, we find explicit values

for the coefficients in N and D and for the next two terms in the Taylor series for N (z) − exp(z)D(z) Because it is possible to rescale both N and

D by an arbitrary factor, we specifically choose a normalization for which

N (0) = D(0) = 1.

N lm (z) −exp(z)D lm (z)+C lm z l+m+1+l+m+2 m+1 C lm z l+m+2 = O(z l+m+3 ) (352d)

Proof In the case m = 0, the result is equivalent to the Taylor series for

exp(z); by multiplying both sides of (352d) by exp( −z) we find that the result

is also equivalent to the Taylor series for exp(−z) in the case l = 0 We now suppose that l ≥ 1 and m ≥ 1, and that (352d) has been proved if l is replaced

by l − 1 or m replaced is by m − 1 We deduce the result for the given values

of l and m so that the theorem follows by induction.

Because the result holds with l replaced by l − 1 or with m replaced by

l

l + m D l −1,m (z) + l + m m D l,m −1 (z) − D lm (z) = 0. (352h)

Table 352(I) Pad´e approximations Nlm /D lm for l, m = 0, 1, 2, 3

so that (352g) follows The verification of (352h) is similar and will be omitted

It now follows that

N lm (z) −exp(z)D lm (z)+C lm z l+m+1+l+m+2 m+1 Clm z l+m+2 = O(z l+m+3 ), (352i)

and we finally need to prove that C lm = C lm Operate on both sides of (352i)

with the operator (d/dz) l+1 and multiply the result by exp(−z) This gives

The formula we have found for a possible (l, m) Pad´e approximation to

exp(z) is unique This is not the case for an arbitrary function f , as the example of the function given by (352a) shows; the (2, 1) approximation is

not unique The case of the exponential function is covered by the followingresult:

Theorem 352B The function Nlm /D lm , where the numerator and nator are given by (352b) and (352c), is the unique (l, m) Pad´ e approximation

denomi-to the exponential function.

Proof If N lm /  D lm is a second such approximation then, because these

functions differ by O(z l+m+1),

N D −  N D = 0,

because the expression on the left-hand side is O(z l+m+1), and is at the same

time a polynomial of degree not exceeding l +m Hence, the only way that two

distinct approximations can exist is when they can be cancelled to a rational

function of lower degrees This means that for some (l, m) pair, there exists

a Pad´e approximation for which the error coefficient is zero However, since

exp(z) is not equal to a rational function, there is some higher exponent k and

a non-zero constant C such that

N lm (z) − exp(z)D lm (z) = Cz k + O(z k+1 ), (352k)

with k ≥ l + m + 2 Differentiate (352k) k − m − 1 times, multiply the result

by exp(−z) and then differentiate a further m + 1 times This leads to the

Expressions for the (l, m) Pad´e approximations are given in Table 352(I) for

l, m = 0, 1, 2, 3 To extend the information further, Table 352(II) is presented

to give the values for l = m = 0, 1, 2, , 7 Similar tables are also given for

the first and second sub-diagonals in Tables 352(III) and 352(IV), respectively,and error constants corresponding to entries in each of these three tables arepresented in Table 352(V)

Table 352(III) First sub-diagonal members of the Pad´e table Nm −1,m /D m −1,m

V l,m −1 (z) and V l,m (z) are related by

lV l −1,m (z) + mV l,m −1 (z) = (l + m)V l,m (z).

Many similar relations between neighbouring members of a Pad´e table exist,and we present three of them In each case the relation is between three Pad´evectors of successive denominator degrees

It is easy to verify that the coefficients of z0, z1 and z2 vanish in both

components of V (z) We also find that

Table 352(V) Error constants for diagonal and first two sub-diagonals

353 A-stability of Gauss and related methods

We consider the possible A-stability of methods whose stability functionscorrespond to members on the diagonal and first two sub-diagonals of thePad´e table for the exponential function These include the Gauss methodsand the Radau IA and IIA methods as well as the Lobatto IIIC methods

A corollary is that the Radau IA and IIA methods and the Lobatto IIICmethods are L-stable

Theorem 353A Let s be a positive integer and let

R(z) = N (z)

D(z) denote the (s − d, s) member of the Pad´e table for the exponential function, where d = 0, 1 or 2 Then

|R(z)| ≤ 1, for all complex z satisfying Rez ≤ 0.

RUNGE–KUTTA METHODS 239

Proof We use the E-polynomial Because N (z) = exp(z)D(z) + O(z 2s −d+1),

we have

E(y) = D(iy)D( −iy) − N(iy)N(−iy)

= D(iy)D( −iy) − exp(iy)D(iy) exp(−iy)D(−iy) + O(y 2s −d+1)

= O(y 2s −d+1 ).

Because E(y) has degree not exceeding 2s and is an even function, either E(y) = 0, in the case d = 0, or E(y) = Cy 2s with C > 0, in the cases d = 1 and d = 2 In all cases, E(y) ≥ 0 for all real y.

To complete the proof, we must show that the denominator of R has no zeros in the left half-plane Without loss of generality, we assume that Re z < 0 and we prove that D(z) = 0 Write D0, D1, , D sfor the denominators ofthe sequence of Pad´e approximations given by

where the constants α and β will depend on the value of d and s However,

α = 0 if d = 0 and α > 0 for d = 1 and d = 2 In all cases, β > 0.

Consider the sequence of complex numbers, ζ k , for k = 1, 2, , s, defined

The fact that D s (z) cannot vanish now follows by observing that

D s (z) = ζ1ζ2ζ3· · · ζ s Hence, D = D sdoes not have a zero in the left half-plane
Alternative proofs of this and related results have been given byAxelsson(1969, 1972), Butcher (1977), Ehle (1973), Ehle and Picel (1975), Watts andShampine (1972) and Wright (1970)

354 Order stars

We have identified some members of the Pad´e table for the exponentialfunction for which the corresponding numerical methods are A-stable Wenow ask: are there other members of the table with this property? It will be

seen that everything hinges on the value of m −l, the degree of the denominator minus the degree of the numerator It is clear that if m − l < 0, A-stability is

impossible, because in this case

|R(z)| → ∞,

as z → ∞, and hence, for some z satisfying Rez < 0, |R(z)| is greater than

1 For m − l ∈ {0, 1, 2}, A-stability follows from Theorem 353A Special cases with m −l > 2 suggest that these members of the Pad´e table are not A-stable.

For the third sub-diagonal, this was proved by Ehle (1969), and for the fourthand fifth sub-diagonals by Nørsett (1974) Based on these observations, Ehle

(1973) conjectured that no case with m − l > 2 can be A-stable This result

was eventually proved by Wanner, Hairer and Nørsett (1978), and we devotethis subsection to introducing the approximations considered in that paperand to proving the Ehle conjecture

In Subsection 216, we touched on the idea of an order star Associated with

the stability function R(z) for a Runge–Kutta method, we consider the set of

points in the complex plane such that

| exp(−z)R(z)| > 1.

This is known as the ‘order star’ of the method, and the set of points suchthat

| exp(−z)R(z)| < 1

is the ‘dual order star’ The common boundary of these two sets traces out

an interesting path, as we see illustrated in Figure 354(i), for the case of the

(1, 3) Pad´e approximation given by

1

4z

1−3z +1z2− 1z3.

RUNGE–KUTTA METHODS 241

−2

−2i

2i

Figure 354(i) Order star for the (1, 3) Pad´e approximation to exp

In this diagram, the dual order star, which can also be described as the

‘relative stability region’, is the interior of the unshaded region The orderstar is the interior of the shaded region

In Butcher (1987) an attempt was made to present an informal survey

of order stars leading to a proof of the Ehle result In the present volume,the discussion of order stars will be even more brief, but will serve as anintroduction to an alternative approach to achieve similar results In addition

to Wanner, Hairer and Nørsett (1978), the reader is referred to Iserles andNørsett (1991) for fuller information and applications of order stars

The ‘order star’, for a particular rational approximation to the exponentialfunction, disconnects into ‘fingers’ emanating from the origin, which may bebounded or not, and similar remarks apply to ‘dual fingers’ which are theconnected components of the dual star The following statements summarizethe key properties of order stars for applications of the type we are considering.Because we are including only hints of the proofs, we refer to them as remarks

rather than as lemmas or theorems Note that S denotes the order star for a specific ‘method’ and I denotes the imaginary axis.

Remark 354A A method is A-stable if and only if S has no poles in the

negative half-plane and S ∪ I = ∅, because the inclusion of the exponential factor does not alter the set of poles and does not change the magnitude of the stability function on I.

Remark 354B There exists ρ0 > 0 such that, for all ρ ≥ ρ0, functions

θ1(ρ) and θ2(ρ) exist such that the intersection of S with the circle |z| = ρ

is the set {ρ exp(iθ) : θ1 < θ < θ2} and where lim ρ →∞ θ1(ρ) = π/2 and lim ρ →∞ θ2(ρ) = 3π/2, because at a great distance from the origin, the

Figure 354(ii) Relation between order arrows and order stars

behaviour of the exponential function multiplied by the rational function on which the order star is based is dominated by the exponential factor.

Remark 354C For a method of order p, the arcs {r exp(i(j +1

2)π/(p + 1)) :

0 ≤ r}, where j = 0, 1, , 2p + 1, are tangential to the boundary of S at

0, because exp( −z)R(z) = 1 + Cz p+1 + O( |z| p+2 ), so that | exp(−z)R(z)| =

1 + Re(Cz p+1 ) + O( |z| p+2 ).

It is possible that m bounded fingers can join together to make up a finger

of multiplicity m Similarly, m dual fingers in S can combine to form a dual finger with multiplicity m.

Remark 354D Each bounded finger of S, with multiplicity m, contains

at least m poles, counted with their multiplicities, because, by the Cauchy– Riemann conditions, the argument of exp( −z)R(z) increases monotonically

as the boundary of the order star is traced out in a counter-clockwise direction.

In the following subsection, we introduce a slightly different tool forstudying stability questions The basic idea is to use, rather than the fingersand dual fingers as in order star theory, the lines of steepest ascent and descent

from the origin Since these lines correspond to values for which R(z) exp( −z)

is real and positive, we are, in reality, looking at the set of points in thecomplex plane where this is the case

We illustrate this by presenting, in Figure 354(ii), a modified version ofFigure 354(i), in which the boundary of the order star is shown as a dashedline and the ‘order arrows’, as we call them, are shown with arrow headsshowing the direction of ascent

RUNGE–KUTTA METHODS 243

355 Order arrows and the Ehle barrier

For a stability function R(z) of order p, define two types of ‘order arrows’ as

follows:

Definition 355A The locus of points in the complex plane for which φ(z) =

R(z) exp( −z) is real and positive is said to be the ‘order web’ for the rational function R The part of the order web connected to 0 is the ‘principal order web’ The rays emanating from 0 with increasing value of φ are ‘up arrows’ and those emanating from 0 with decreasing φ are ‘down arrows’.

The up and down arrows leave the origin in a systematic pattern:

Theorem 355B Let R be a rational approximation to exp of exact order p,

so that

R(z) = exp(z) − Cz p+1 + O(z p+2 ), where the error constant C is non-zero If C < 0 (C > 0) there are up (down) arrows tangential at 0 to the rays with arguments k2πi/(p + 1),

k = 0, 1, , p, and down (up) arrows tangential at 0 to the rays with arguments (2k + 1)πi/(p + 1), k = 0, 1, , p.

Proof If, for example, C < 0, consider the set {r exp(iθ) : r > 0, θ ∈ [k2πi/(p + 1)

k ∈ {0, 1, 2, , p} We have

R(z) exp( −z) = 1 + (−C)r p+1 exp((p + 1)θ) + O(r p+2 ).

small, the real part of (−C)r p+1 exp((p + 1)θ)) is positive The imaginary

part changes sign so that an up arrow lies in this wedge The cases of the

down arrows and for C > 0 are proved in a similar manner.
Where the arrows leaving the origin terminate is of crucial importance

Theorem 355C The up arrows terminate either at poles of R or at −∞ The down arrows terminate either at zeros of R or at + ∞.

Proof Consider a point on an up arrow for which |z| is sufficiently large

to ensure that it is not possible that z is a pole or that z is real with (d/dz)(R(z) exp( −z)) = 0 In this case we can assume without loss of generality that Im(z) ≥ 0 Write R(z) = Kz n + O( |z| n −1) and assume that

K > 0 (if K < 0, a slight change is required in the details which follow) If

Because θ cannot leave the interval [0, π], then for w to remain real, y is bounded as z → ∞ Furthermore, w → ∞ implies that x → −∞.

The result for the down arrows is proved in a similar way

We can obtain more details about the fate of the arrows from the followingresult

Theorem 355D Let R be a rational approximation to exp of order p with

numerator degree n and denominator degree d Let n denote the number of down arrows terminating at zeros and  d the number of up arrows terminating

at poles of R Then

n +  d ≥ p.

Proof There are p + 1 − n down arrows and p + 1 −  d up arrows terminating

at +∞ and −∞, respectively Let θ and φ be the minimum angles with the

properties that all the down arrows which terminate at +∞ lie within θ on

either side of the positive real axis and all the up arrows which terminate at

−∞ lie within an angle φ on either side of the negative real axis Hence

2θ ≥ (p − n)2π

p + 1 , 2φ ≥ (p −  d)2π

p + 1 .

Because up arrows and down arrows cannot cross and, because there is a

wedge with angle equal to at least π/(p + 1) between the last down arrow and the first up arrow, it follows that 2θ + 2φ + 2π/(p + 1) ≤ 2π Hence we obtain

the inequality

2p + 1 − n −  d

p + 1 2π ≤ 2π,

For Pad´e approximations we can obtain precise values ofn and  d.

Theorem 355E Let R(z) denote a Pad´ e approximation to exp(z), with degrees n (numerator) and d (denominator) Then n of the down arrows terminate at zeros and d of the up arrows terminate at poles.

Proof Because p = n + d, n ≥ n and d ≥  d, it follows from Theorem 355D

that

p = n + d ≥ n +  d ≥ p and hence that (n − n) + (d −  d) = 0 Since both terms are non-negative they

Before proving the ‘Ehle barrier’, we establish a criterion for A-stabilitybased on the up arrows that terminate at poles

RUNGE–KUTTA METHODS 245

Theorem 355F A Runge–Kutta method is A-stable only if all poles of the

stability function R(z) lie in the right half-plane and no up arrow of the order web intersects with or is tangential to the imaginary axis.

Proof The requirement on the poles is obvious If an up arrow intersects or

is tangential to the imaginary axis then there exists y such that

Theorem 355G Let R(z) denote the stability function of a Runge–Kutta

method If R(z) is an (n, d) Pad´ e approximation to exp(z) then the Runge– Kutta is not A-stable unless d ≤ n + 2.

Proof If d ≥ n + 3 and p = n + d, it follows that d ≥ 1

2(p + 3) By Theorem 355E, at least d up arrows terminate at poles Suppose these leave zero in

directions between−θ and +θ from the positive real axis Then

2θ ≥ 2π(d − 1)

p + 1 ≥ π,

and at least one up arrow, which terminates at a pole, is tangential to theimaginary axis or passes into the left half-plane If the pole is in the left half-plane, then the stability function is unbounded in this half-plane On the otherhand, if the pole is in the right half-plane, then the up arrow must cross theimaginary axis In either case, the method cannot be A-stable, by Theorem

where z = hq Even though this analysis provides useful information about

the behaviour of a numerical method when applied to a stiff problem, evenmore is learned from generalizing this analysis in two possible ways The first