Numerical Methods for Ordinary Dierential Equations Episode 9 ppsx

264 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONSwhere the coefficient of y n −1is seen to be the stability function value and error behaviour of order p can be verified term by ter

264 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

where the coefficient of y n −1is seen to be the stability function value

and error behaviour of order p can be verified term by term.

On the other hand, if hL is large, a more realistic idea of the error is found

using the expansion

(I − hLA) −1=− 1

hL A

−1 − 1

h2L2A −2 − · · · , and we obtain an approximation to the error, g(x n)− y n, given by

b In this special case, the term b A −1 0.

In other cases, the contributions from b A −1 0, if the

stage order is less than the order

Define

η + hLb (I − hLA) −1 n > 0,

with η0 defined as the initial error g(x0)− y0 The accumulated truncation

error after n steps is equal to



i=0 R( ∞) n −i η

i

There are three important cases which arise in a number of widely use

methods If R( ∞) = 0, as in the Radau IA, Radau IIA and Lobatto IIIC

methods, or for that matter in any L-stable method, then we can regard theglobal truncation error as being just the error in the final step Thus, if the

local error is O(h q+1 ) then the global error would also be O(h q+1) On the

other hand, for the Gauss method with s stages, R( ∞) = (−1) s For the

methods for which R( ∞) = 1, then we can further approximate the global error as the integral of the local truncation error multiplied by h −1 Hence,

a local error O(h q+1 ) would imply a global error of O(h q) In the cases for

which R( ∞) = −1 we would expect the global error to be O(h q+1), because

of cancellation of η i over alternate steps

We explore a number of example methods to see what can be expected forboth local and global error behaviour

which, if|hL| is large, dominates (362e).

We also consider the important case of the Radau IIA methods In this case

!

g (2s) (x n −1 ) + O(h 2s+1)

=− h 2s s!(s − 1)!32(2s − 1)!3 g (2s) (x n −1 ) + O(h 2s+1 ).

266 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

As we have remarked, for |hL| large, this term is cancelled by −b A −1

Hence, the local truncation error can be approximated in this case by

To summarize: for very stiff problems and moderate stepsizes, a combination

modelled for the Prothero–Robinson problem by a high value of hL, the stage

order, rather than the classical order, plays a crucial role in determiningthe error behaviour For this reason, we consider criteria other than super-convergence as important criteria in the identification of suitable methods forthe solution of stiff problems In particular, we look for methods that arecapable of cheap implementation

363 Singly implicit methods

We consider methods for which the stage order q and the order are related by

p = q = s To make the methods cheaply implementable, we also assume that

where c k denotes the component-by-component power

We can now evaluate A k −11 by induction In fact,

Table 363(I) Laguerre polynomials L s for degrees s = 1, 2, , 8

(−λ) s −i A i 1 = 0.

(−λ) s −i1

(−1) i(−ξ) i

i! . However, this is just the Laguerre polynomial of degree s, usually denoted by

L s (ξ), and it is known that all its zeros are real and positive For convenience,

expressions for these polynomials, up to degree 8, are listed in Table 363(I) andapproximations to the zeros are listed in Table 363(II) We saw in Subsection

361 that for λ = ξ −1 for the case of three doubly underlined zeros of orders

2 and 3, L-stability is achieved Double underlining to show similar choicesfor other orders is continued in the table and these are the only possibilitiesthat exist (Wanner, Hairer and Nørsett, 1978) This means that there are

no L-stable methods – and in fact there is not even an A-stable method –

with s = p = 7 or with s = p > 8 Even though fully L-stable methods are

confined to the eight cases indicated in this table, there are other choices of

λ = ξ −1 that give stability which is acceptable for many problems In each ofthe values of ξ for which there is a single underline, the method is A(α)-stable with α ≥ 1.55 ≈ 89 ◦.

268 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Table 363(II) Zeros of Laguerre polynomials for degrees s = 1, 2, , 8

6 0.2228466042 1.1889321017 2.9927363261 5.7751435691 9.8374674184 15.9828739806

7 0.1930436766 1.0266648953 2.5678767450 4.9003530845 8.1821534446 12.7341802918 19.3957278623

8 0.1702796323 0.9037017768 2.2510866299 4.2667001703 7.0459054024 10.7585160102 15.7406786413 22.8631317369

The key to the efficient implementation of singly implicit methods is thesimilarity transformation matrix that transforms the coefficient matrix to

lower triangular form Let T denote the matrix with (i, j) element

t ij = L j −1 (ξ i ), i, j = 1, 2, , s.

The principal properties of T and its relationship to A are as follows:

Theorem 363A The (i, j) element of T −1 is equal to

Proof To prove (363d), use the Christoffel–Darboux formula for Laguerre

polynomials in the form

where we have used known properties of Laguerre polynomials The value of

For convenience we sometimes write

270 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

We now consider the possible A-stability or L-stability of singly implicitmethods This hinges on the behaviour of the rational functions

R(z) = N (z)

(1− λz) s , where the degree of the polynomial N (z) is no more than s, and where

1

λ



z i ,

where L (m) n denotes the m-fold derivative of L n, rather than a generalized

Laguerre polynomial To verify the L-stability of particular choices of s and

λ, we note that all poles of N (z)/(1 − λz) s are in the right half-plane Hence,

it is necessary only to test that |D(z)|2− |(1 − λz) s |2≥ 0, whenever z is on the imaginary axis Write z = iy and we find the ‘E-polynomial’ defined in

this case as

E(y) = (1 + λ2y2)s − N(iy)N(−iy), with E(y) ≥ 0 for all real y as the condition for A-stability Although A- stability for s = p is confined to the cases indicated in Table 363(II), it will

be seen in the next subsection that higher values of s can lead to additional

possibilities

We conclude this subsection by constructing the two-stage L-stable singlyimplicit method of order 2 From the formulae for the first few Laguerrepolynomials,

2− √2 4

In the implementation of this, or any other, singly implicit method, theactual entries in this tableau are not explicitly used To emphasize thispoint, we look in detail at a single Newton iteration for this method Let

M = I − hλf  (y n −1 ) Here the Jacobian matrix f  is supposed to have been

evaluated at the start of the current step In practice, a Jacobian evaluated

at an earlier time value might give satisfactory performance, but we do notdwell on this point here If the method were to be implemented with no specialuse made of its singly implicit structure, then we would need, instead of the

N × N matrix M, a 2N × 2N matrix 4 M given by

In this ‘fully implicit’ situation, a single iteration would start with the input

approximation y n −1and existing approximations to the stage values and stagederivatives Y1, Y2, hF1 and hF2 It will be assumed that these are consistentwith the requirements that

Y1= y n −1 + a11hF1+ a12hF2, Y2= y n −1 + a21hF1+ a22hF2,

and the iteration process will always leave these conditions intact

364 Generalizations of singly implicit methods

In an attempt to improve the performance of existing singly implicit methods,Butcher and Cash (1990) considered the possibility of adding additional

diagonally implicit stages For example, if s = p + 1 is chosen, then the

coefficient matrix has the form

An appropriate choice of λ is made by balancing various considerations.

The first of these is good stability, and the second is a low error constant.Minor considerations would be convenience, the avoidance of coefficients withabnormally large magnitudes or with negative signs, where possible, and a

preference for methods in which the c i lie in [0, 1] We illustrate these ideas for the case p = 2 and s = 3, for which the general form for a method would

272 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

−0.04

−0.02

0.00 0.02

Figure 364(i) Error constant C(λ) for λ ∈ [0.1, 0.5]

The only choice available is the value of λ, and we consider the consequence

of making various choices for this number The first criterion is that themethod should be A-stable, and we analyse this by calculating the stabilityfunction

and the E-polynomial

E(y) = |D(iy)|2− |N(iy)|2=

√

32(3− √3) ,

or that λ lies in the interval [0.180425, 2.185600] The error constant C(λ), defined by exp(z) − R(z) = C(λ)z3+ O(z4), is found to be

C(λ) = 1

6 −3

2λ + 3λ

2− λ3,

and takes on values for λ ∈ [0.1, 0.5], as shown in Figure 364(i).

The value of b1 is positive for λ > 0.125441 Furthermore b2is positive for

λ < 0.364335 Since b1+ b2+ λ = 1, we obtain moderately sized values of all components of b if λ ∈ [0.125441, 0.364335] The requirement that c1and c2lie

in (0, 1) is satisfied if λ < (2 − √2)−1 ≈ 0.292893 Leaving aside the question

of convenience, we should perhaps choose λ ≈ 0.180425 so that the error

constant is small, the method is A-stable, and the other minor considerations

are all satisfied Convenience might suggest an alternative value λ = 1

365 Effective order and DESIRE methods

An alternative way of forcing singly implicit methods to be more appropriatefor practical computation is to generalize the order conditions This has to bedone without lowering achievable accuracy, and the use of effective order isindicated Effective order is discussed in a general setting in Subsection 389but, for methods with high stage order, a simpler analysis is possible.Suppose that the quantities passed from one step to the next are notnecessarily intended to be highly accurate approximations to the exactsolution, but rather to modified quantities related to the exact result by

weighted Taylor series For example, the input to step n might be an

as a satisfactory alternative to a method of classical order p.

We explore this idea through the example of the effective ordergeneralization of the L-stable order 2 singly implicit method with the tableau(363g) For this method, the abscissae are necessarily equal to 3− 2 √2 and

1, which are quite satisfactory for computation However, we consider other

choices, because in the more complicated cases with s = p > 2, at least one

of the abscissae is outside the interval [0, 1], for A-stability.

If the method is required to have only effective order 2, then we can assume

that the incoming and outgoing approximations are equal to

respectively Suppose that the stage values are required to satisfy

Y1= y(x n −1 + hc1) + O(h3), Y2= y(x n −1 + hc2) + O(h3),

with corresponding approximations for the stage derivatives In deriving the

order conditions, it can be assumed, without loss of generality, that n = 1.

The order conditions for the two stages and for the output approximation

y = y are

274 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

These can be converted into algebraic relations on the various free parameters

by expanding by Taylor series about x0 and equating coefficients of hy  (x

a11+ a22= 2λ,

a11a22− a21a12= λ2 Assuming that c1 and c2 are distinct, a solution to these equations alwaysexists, and it leads to the values

Combine the effective order idea with the diagonal extensions introduced

in Subsection 364, and we obtain ‘DESIRE’ methods (diagonally extendedimplicit Runge–Kutta methods using effective order) These are exemplified

by the example with p = 2, s = 3 and λ = 15 For this method, α1 =−3

49

200 0

1 20071 119200 15

103 250 119 250 14 125

Exercises 36 36.1 Derive the tableau for the two-stage order 2 diagonally implicit method

satisfying (361a), (361b) with λ = 1 −1

36.3 Show that the method derived in Exercise 36.2 has stage order 2.

36.4 Derive a diagonally implicit method with s = p = 3 and with λ = c2=

36.6 Show that for an L-stable method of the type described in Subsection

364 with p = 3, s = 4, the minimum possible value of λ is approximately 0.2278955169, a zero of the polynomial

185976λ12− 1490400λ11+ 4601448λ10− 7257168λ9+ 6842853λ8

−4181760λ7+1724256λ6−487296λ5+94176λ4−12192λ3+1008λ2−48λ+1.

37 Symplectic Runge–Kutta Methods

370 Maintaining quadratic invariants

We recall Definition 357B in which the matrix M plays a role, where the elements of M are

m ij = b i a ij + b j a ji − b i b j (370a)Now consider a problem for which

276 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

for all y It is assumed that Q is a symmetric matrix so that (370b) is equivalent to the statement that y(x) Qy(x) is invariant.

We want to characterize Runge–Kutta methods with the property that

y n Qy n is invariant with n so that the the numerical solution preserves the conservation law possessed by the problem If the input to step 1 is y0, thenthe output will be

with m ij given by (370a)

Thus M = 0 implies that quadratic invariants are preserved and, in

particular, that symplectic behaviour is maintained Accordingly, we have thefollowing definition:

Definition 370A A Runge–Kutta method (A, b , c) is symplectic if

M = diag(b)A + A diag(b) − bb

is the zero matrix.

The property expressed by Definition 370A was first found by Cooper (1987)and, as a characteristic of symplectic methods, by Lasagni (1988), Sanz-Serna(1988) and Suris (1988)

371 Examples of symplectic methods

A method with a single stage is symplectic only if 2b1a11 − b2 = 0 For

consistency, that is order at least 1, b1 = 1 and hence c1 = a11 = 12; this

is just the implicit mid-point rule We can extend this in two ways: by either

looking at methods where A is lower triangular or looking at the methods with stage order s.

For lower triangular methods we will assume that none of the b i is zero.

The diagonals can be found from 2b i a ii = b2i to be a ii= 12b i For the elements

of A below the diagonal we have b i a ij = b i b j so that a ij = b j This gives atableau

For methods with order and stage order equal to s, we have, in the notation

i = 0 for i = s + 1, s + 2, , 2s This follows from the observation that V M V = 0 Thus, in addition to B(s), B(2s) holds Hence, the abscissae of the method are the zeros of P ∗

s and the method is the s-stage

=

s



i,j=1 (b i b j )φ i ψ j

= Φ(t)Φ(u).

278 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Assuming the order conditions Φ(t) = 1/γ(t) and Φ(u) = 1/γ(u) are satisfied,

then

Φ(tu) − 1

γ(tu) + Φ(ut) − 1

Using this fact, we can prove the following theorem:

Theorem 372A Let (A, b , c) be a symplectic Runge–Kutta method The

method has order p if and only if for each non-superfluous tree and any vertex

in this tree as root, Φ(t) = 1/γ(t), where t is the rooted tree with this vertex.

Proof We need only to prove the sufficiency of this criterion If two rooted

trees belong to the same tree but have vertices v0, v say, then there is a sequence of vertices v0, v1, , v m = v, such that v i −1 and v i are adjacent

for i = 1, 2, , m This mean that rooted trees t, u exist such that tu is the rooted tree with root v i −1 and ut is the rooted tree with root v i We areimplicitly using induction on the order of trees and hence we can assume that

Φ(t) = 1/γ(t) and Φ(u) = 1/γ(u) Hence, if one of the order conditions for the trees tu and ut is satisfied, then the other is By working along the chain of possible roots v0, v1, , v m, we see that the order condition associated with

the root v0 is equivalent to the condition for v In the case of superfluous trees, one choice of adjacent vertices would imply that t = u Hence, (372a) is equivalent to 2Φ(tt) = 2/γ(tt) so that the order condition associated with tt

is satisfied and all rooted trees belonging to the same tree are also satisfied

373 Experiments with symplectic methods

The first experiment uses the simple pendulum based on the Hamiltonian

H(p, q) = p2/2 − cos(q) and initial value (p, q) = (1, 0) The amplitude is found to be π/3 ≈ 1.047198 and the period to be approximately 6.743001.

Numerical solutions, displayed in Figure 373(i), were found using the Euler,implicit Euler and the implicit mid-point rule methods Only the last of these

is symplectic and its behaviour reflects this That is, like the exact solutionwhich is also shown, the area of the initial set remains unchanged, even thoughits shape is distorted

The second experiment is based on problem (122c), which evolves on the

unit sphere y2+ y2 + y2 = 1 The value of y2 + y2+ y2 is calculated bythe Euler method, the implicit Euler method and the implicit mid-point rulemethod Only the last of these is symplectic The computed results are shown

in Figure 373(ii) In each case a stepsize h = 0.1 was used Although results

are shown for only 500 time steps, the actual experiment was extended much

further There is no perceptible deviation from y2+ y2+ y2= 1 for the firstmillion steps

Figure 373(i) Solutions of the Hamiltonian problem H(p, q) = p2/2 − cos(q).

Left: Euler method (grey) and implicit Euler method (white) Right: exact solution(grey) and implicit mid-point method (white) The underlying image depicts the

takahe Porphyrio hochstetteri, rediscovered in 1948 after many years of presumed

Figure 373(ii) Experiments for problem (122c) The computed value of n 2is

shown after n = 1, 2, , steps.

Exercises 37 37.1 Do two-stage symplectic Runge–Kutta methods exist which have order

3 but not order 4?

37.2 Do three-stage order 3 symplectic Runge–Kutta methods exist for which

A is lower triangular?

280 NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

38 Algebraic Properties of Runge–Kutta Methods

380 Motivation

For any specific N -dimensional initial value problem, Runge–Kutta methods

can be viewed as mappings from RN to RN However, the semi-groupgenerated by such mappings has a significance independent of the particularinitial value problem, or indeed of the vector space in which solution values

lie If a method with s1 stages is composed with a second method with s2stages, then the combined method with s1+ s2 stages can be thought of asthe product of the original methods It turns out that this is not quite the bestway of formulating this product, and we need to work with equivalence classes

of Runge–Kutta methods This will also enable us to construct a group, ratherthan a mere semi-group

It will be shown that the composition group of Runge–Kutta equivalentclasses is homomorphic to a group on mappings from trees to real numbers

In fact the mapping that corresponds to a specific Runge–Kutta method isjust the function that takes each tree to the associated elementary weight.There are several reasons for introducing and studying these groups.For Runge–Kutta methods themselves, it is possible to gain a betterunderstanding of the order conditions by looking at them in this way.Furthermore, methods satisfying certain simplifying assumptions, notably the

C and D conditions, reappear as normal subgroups of the main group An

early application of this theory is the introduction of the concept of ‘effectiveorder’ This is a natural generalization from this point of view, but makes verylittle sense from a purely computational point of view While effective orderwas not widely accepted at the time of its discovery, it has been rediscovered(L´opez-Marcos, Sanz-Serna and Skeel, 1996) and has now been seen to havefurther ramifications

The final claim that is made for this theory is that it has applications to theanalysis of the order of general linear methods In this guise a richer structure,incorporating an additive as well as a multiplicative operation, needs to beused; the present section also examines this more elaborate algebra

The primary source for this theory is Butcher (1972), but it is also widelyknown through the work of Hairer and Wanner (1974) Recently the algebraicstructures described here have been rediscovered through applications intheoretical physics For a review of these developments, see Brouder (2000).Before proceeding with this programme, we remark that the mappings fromtrees to real numbers, which appear as members of the algebraic systemsintroduced in this section, are associated with formal Taylor series of theform

a( ∅)y(x) +

t ∈T

a(t) σ(t) h r(t) F (t)(y(x)). (380a)

Such expressions as this were given the name B-series by Hairer and Wanner