Numerical Methods for Ordinary Dierential Equations Episode 3 pot

To conclude this subsection, we present a pictorial illustration of the use ofthe Euler method, for the scalar initial value problem yx = 1 + x C + x2, for C an arbitrary constant, and t

y  (x) = f (y(x)), y(x

0) = y0,

or in one of the expanded forms

To conclude this subsection, we present a pictorial illustration of the use ofthe Euler method, for the scalar initial value problem

y(x) = 1 + x

C + x2,

for C an arbitrary constant, and the exact and approximate solutions are

shown in Figure 200(i)

201 Some numerical experiments

To see how the Euler method works in practice, consider the initial valueproblem

We can now take the calculation a second step forward, to find an

approximation at x = 0.2 using the formula y(0.2) ≈ y(0.1) + 0.1y  (0.1). For the value of y(0.1), we can use the result of the first Euler step and for the value of y  (0.1), we can use (201a) with the approximate value of

y(0.1) substituted This gives y  (0.1) ≈ (1.1 + 0.1)/(1.1 − 0.1) = 1.2 Hence, y(0.2) ≈ y(0.1) + 0.1y  (0.1) ≈ 1.1 + 0.12 = 1.22.

In Table 201(I) these calculations are continued as far as x = 0.5 Steps of size 0.1 are taken throughout but, for comparison, the same results are also given for steps of sizes 0.05 and 0.025, respectively For the three columns of approximations, the headings h = 0.1, h = 0.05 and h = 0.025 denote the

sizes of the steps used to arrive at these approximations The exact values of

y are also given in the table.

It is interesting to compare the errors generated in the very first step, for

the three values of h that we have used For h = 0.1, the exact solution minus the computed solution is 1.109950 − 1.100000 = 0.009950; for h = 0.05, the

corresponding difference is 1.052497 −1.050000 = 0.002497; for h = 0.025, the

difference is 1.025625 − 1.025000 = 0.000625 It is seen that, approximately,

when h is multiplied by a factor of 12, the error in the first step is multiplied by

a factor of 14 This is to be expected because, according to Taylor’s theorem,

the exact answer at x = h is y(h) ≈ y(0) + hy  (0) + (h2/2)y (0) The first

two terms of this approximation are exactly what is calculated by the Euler

method, so that the error should be close to (h2/2)y (0) We can check thismore closely by evaluating y (0) = 2.

Of greater interest in understanding the quality of the numerical

approximation is the error accumulated up to a particular x value, by a sequence of Euler steps, with varying value of h In the case of x = 0.5,

we see that, for the three stepsizes we have used, the errors are respectively

1.724745 − 1.687555 = 0.037190, 1.724745 − 1.706570 = 0.018175 and

1.724745 − 1.715760 = 0.008985 These error values approximately drop by a

factor 12 when h is reduced by this same factor The reason for this will be

discussed more fully in Subsection 212, but it can be understood informally.Note that there is a comparable error produced in each of the steps, but there

Table 201(I) Euler method: problem (201a)

so that the total error should be about 0.40825h This very crude argument

leads to a prediction that is incorrect by a factor of only about 10% In thesolution of practical problems using the Euler method, or indeed a differentmethod, it is not really feasible to estimate the total accumulated error, but it

is important to know the asymptotic form of the error in terms of h This will

often make it possible to gauge the quality of approximations, by comparing

the values for differing h values It will also often make it possible to make

realistic decisions as to which of various alternative numerical methods should

be used for a specific problem, or even for a large class of problems

Table 201(II) Euler method: problem (201d) with e = 0

It is equally straightforward to solve problems in more than one dependentvariable using the Euler method Given the problem of inverse-square lawattraction in two dimensions

where Y2+ Y2, it is necessary to first write the problem as a system

of first order equations This is done by writing y1 and y2 for the space

coordinates Y1 and Y2, and writing y3 and y4 for the velocity coordinates,

given as the first derivatives of Y1and Y2 With this reformulation, the system

of differential equations is written in the form

The initial value, written as a vector y(0) = [1, 0, 0, 1] , defines the solution

y(x) = [cos(x), sin(x), − sin(x), cos(x)] The first step of the Euler method

gives a numerical result y(h) ≈ [1, h, −h, 1] ; this differs from the exact

the norm of this vector and consider its behaviour as a function of h.

It will be interesting to perform many steps, sufficient to complete, forexample, half of one orbit and to compare the (Euclidean) norm of the error

for differing values of h For various values of h, decreasing in sequence by a

factor 12, some calculations are presented for this experiment in Table 201(II)

The approximate halving of the error, when h is halved, is easily observed in

this table

If the same problem is solved using initial values corresponding to an elliptic,

rather than a circular, orbit, a similar dependence of the error on h is observed,

but with errors greater in magnitude Table 201(III) is for an orbit with

eccentricity e = 12 The starting value corresponds to the closest point onthe orbit to the attracting force, and the exact value at the end of a halfperiod is

00

− √1 3





.

When the eccentricity is further increased to e = 34, the loss of accuracy

in carrying out the computation is even more pronounced Results for

e = 34 are given in Table 201(IV), where we note that, in this case,

y(π) = [ −7

4, 0, 0, −1/ √7]

202 Calculations with stepsize control

The use of the Euler method, with constant stepsize, may not be efficient forsome problems For example, in the case of the eccentric orbits, discussed inthe previous subsection, a small step should be taken for points on the orbit,close to the attracting force, and a larger step for points remote from theattracting force In deciding how we might attempt to control the stepsizefor a general problem, we need to consider how the error committed in eachstep can be estimated First, however, we consider how the stepsize in a stepshould be chosen, to take account of this error estimate

Because the total error is approximately the sum of the errors committed inthe individual steps, at least for a limited number of steps, we look at a simple

model in which the interval of integration is divided up into m subintervals, with lengths δ1, δ2, , δ m We assume that the norms of the errors in steps

carried out in these intervals are C1h2, C2h2, , C m h2

m, respectively, where

h1, h2, , h mare the constant stepsizes in these subintervals Assume that a

total of N steps of integration by the Euler method are carried out and that

a fraction ti of these are performed in subinterval i = 1, 2, , m This means that ti N steps are carried out in subinterval i and that h i = δi /t i N The total

error committed, which we assume, in the absence of further information, to

be the sum of the individual errors, is approximately

E =

m



i=1 (ti N )C i

where δi /t i N is the stepsize used for every step in subinterval number i By

the Cauchy–Schwarz inequality, the minimum value of (202a) is achieved by

and it follows that optimality occurs when Ci h2

i is maintained constant overevery subinterval We interpret this result to mean that the estimated values

of the error should be kept as close as possible to some pre-assigned value

This pre-assigned value, which is under control of the user, will be regarded

as the user-imposed tolerance

To actually estimate the error committed in each step, we have a natural

resource at our disposal; this is the availability of approximations to hy  (x) at the beginning and end of every step At the beginning of step n, it is, of course, the value of hf (xn −1 , y n −1) used in the computation of the Euler step itself

At the end of this step we can calculate hf (x n , y n) This might seem to be an

additional calculation of the function f , but this computation needs to be done anyway, since it is needed when the following step is eventually carried out From these approximations to hy  (xn −1 ) and hy  (xn) we can recalculate the step from yn −1using the more accurate trapezoidal rule to yield the improvedapproximation to y(x n), given by

and we can use the difference between this approximation to y(xn), and the

result computed by the Euler step, as our local error estimate

Hence we have, as an estimate of the norm of the error,

1

2hf (x n −1 , y(x n −1))− hf(x n , y(x n)).

As an illustration of how variable stepsize works in practice, the calculations

of gravitational orbits with eccentricities 0.5 and 0.75 have been repeated using

variable stepsize, but with the tolerances set at values that will give a totalnumber of steps approximately the same as for the constant stepsize casesalready investigated A summary of the results is shown in Figure 202(i)

To make the comparisons straightforward, only norms of errors are plottedagainst stepsize (or mean stepsize in the variable stepsize cases)

Figure 203(i) Norm error against n −1 for the ‘mildly stiff’ problem (203a)

203 Calculations with mildly stiff problems

Consider the initial value problem

for which the exact solution is y1(x) = cos(x), y2(x) = sin(x) We attempt to

solve this problem using the Euler method First, we use constant stepsize

Specifically, we perform n steps with h = π/n and with n taking on various integer values This yields a sequence of approximations to y(π), and results

for the norm of the error are given in Figure 203(i)

The results shown here have a disturbing feature Even though theasymptotic first order behaviour is clearly seen, this effect is recognizable

only below a certain threshold, corresponding to n = 38 For h above the corresponding value of π/38, the errors grow sharply, until they dominate the

solution itself We consider what can be done to avoid this extreme behaviourand we turn to variable stepsize as a possible remedy We need to be moreprecise than in Subsection 202, in deciding how we should apply this approach.After a step has been completed, we have to either accept or reject the step,and rejecting requires us to repeat the step, but with a scaled-down stepsize

In either case we need a policy for deciding on a stepsize to use in the newattempt at the failed step, or to use in the succeeding new step

Because the local truncation error is asymptotically proportional to the

square of h, it makes sense to scale the stepsize in the ratio T /

is the error estimate and T is the maximum permitted value of

it is essential to insert a ‘safety factor’ S, less than 1, into the computation,

Figure 203(ii) Norm error against tolerance T for the ‘mildly stiff’ problem

(203a) with variable stepsize

to guard against a rejection in a new step, because of slight variations inthe magnitude of the error estimate from step to step It is also wise to use

two further design parameters, M and m, representing the maximum and minimum stepsize ratios that will be permitted Typically M = 2, m = 1

2

and S = 0.9, and we adopt these values Fortunately, this experiment of using

variable stepsize is successful, as is seen from Figure 203(ii)

There is a loss of efficiency, in that unstable behaviour typically results

in wide variations of stepsize, in sequences of adjacent steps However, thereare relatively few steps rejected, because of excessive error estimates For the

special choice of the tolerance T = 0.02, 38 successful steps were taken, in addition to 11 failed steps The value of the stepsize h as a function of the value of x, at the beginning of each of the steps, is shown in Figure 203(iii).

The phenomenon experienced with this example goes under the name of

‘stiffness’ To understand why this problem is stiff, and why there seems to

be a value of h such that, for values of the stepsize above this, it cannot

be solved by the Euler method, write v1(x) and v2(x) for the deviations of

y1(x) and y2(x) from the exact solution That is, y1(x) = cos(x) + v1(x) and

y2(x) = sin(x) + v2(x) Because the system is linear, it reduces in a simple

To simplify the discussion further, find the eigenvalues, and corresponding

eigenvectors, of the matrix A occurring in (203b), where

Figure 203(iii) Stepsize h against x for the ‘mildly stiff’ problem (203a) with

variable stepsize for T = 0.02

The eigenvalues of A are λ1 = 0 and λ2 =−25 and the eigenvectors are the

columns of the matrix

the equation for w2 is solved numerically, difficulties arise In a single step of size h, the exact solution for w2 should be multiplied by exp(−25h), but the

numerical approximation is multiplied by 1− 25h Even though | exp(−25h)|

is always less than 1 for positive h, |1 − 25h| is greater than 1, so that its

powers form an unbounded sequence, unless h ≤ 2

25.This, then, is the characteristic property of stiffness: components of thesolution that should be stable become unstable when subjected to numericalapproximations in methods like the Euler method

Table 204(I) Comparison of explicit and implicit Euler methods:

204 Calculations with the implicit Euler method

As we have pointed out, the Euler method approximates the integral of

y  (x), over each subinterval [xn

multiplied by an approximation to the height of the integrand at the left-handend We can consider also the consequences of using the width of this interval,

multiplied by the height at the right-hand end.

This would mean that the approximation at x1 would be defined by

y(x1) ≈ y1, where y1 = y0+ hf (x1, y1) This results in what is known asthe ‘implicit Euler method’ The complication is, of course, that the solutionapproximation at the end of the step is defined not by an explicit formula,but as the solution to an algebraic equation

For some problems, we can evaluate y1 by simple (‘fixed point’) iteration

That is, we calculate a sequence of approximations Y[0], Y[1], Y[2], using

the formula

Y [k] = y0+ hf (x1, Y [k −1] ), k = 1, 2, 3,

Assuming that the sequence of approximations converges, to within a required

tolerance, to a limiting value Y , then we take this limit as the value of y1 Thestarting value in the sequence may be taken, for simplicity and convenience,

as y0

Some results for this method, as applied to the initial value problem (201a),are given in Table 204(I) In this table, all approximations are made for the

solution at x = 0.5 and, for each number of steps n, the calculation is carried

out using both the Euler method and the implicit form of the Euler method.The total errors for the two methods are shown In the case of the implicitmethod, the total number of iterations to achieve convergence, to within a

Figure 204(i) Norm error against n −1 for the ‘mildly stiff’ problem (203a) using

the method (204a)

tolerance of 10−6, is also given If a tolerance as high as 10−4 had been

specified, there would have been only about two, rather than three, iterationsper step, but the cost would still be approximately twice as great as for theexplicit Euler method

As we see from these results, there is no advantage in the implicit form

of the Euler method, in the case of this problem On the contrary, there is

a serious disadvantage, because of the very much greater computing cost, as

measured in terms of f evaluations, for the implicit as compared with the

explicit form of the method

For stiff problems, such as that given by (203a), the implicit Euler methodshows itself to advantage Since this problem is linear, it is possible to writethe answer for the approximation computed at the end of a step explicitly In

the step going from x0 to x1 = x0+ h, with solution approximations going from y0= [(y0)1, (y0)2] to y1= [(y1)1, (y1)2] , we have the relations betweenthese quantities given by

and the new approximation is found using a linear equation solution

The results for this calculation, presented in Figure 204(i), show that thismethod is completely satisfactory, for this problem Note that the largest

stepsize used is π, so that only a single step is taken.

Exercises 20 20.1 On a copy of Figure 200(i), plot the points corresponding to the solution

computed by the Euler method with y(0) = 1

Using this alternative formulation, recalculate the solution, using five

equal steps of the Euler method, from t = 0 to t = ln 2 Plot the solution points after each step on a graph in the (x, y) plane.

20.3 Continue the calculations in Table 201(I) to the point x = 1.

2(y2+ y2)− 1/ y2

1+ y2

2, the total energy, and

A = y1y4− y2y3, the angular momentum, are invariants of the system

(201d); that is, for any value of x the values of each of these will be

equal respectively to the values they had at the initial time The quality

of a numerical method for solving this problem can be measured bycalculating by how much these theoretical invariants actually change inthe numerical computation Repeat the calculations in Tables 201(II),201(III) and 201(IV) but with the deviation in the values of each ofthese quantities used in place of the errors

210 Formulation of the Euler method

Consider a differential equation system

containing x0, and we can extend the region where a Lipschitz condition holds

to the entire N -dimensional vector space, secure in the knowledge that no

practical difference will arise, because the solution will never extend beyondvalues in some compact set

We assume that the solution to (210a) is required to be approximated at a

point x, and that a number of intermediate step points are selected Denote these by x1, x2, , x n = x Define a function, y, on [x0, x] by the formula

y(x) = y(xk −1 ) + (x − x k −1 )f (xk −1 , y(xk −1 )), x ∈ (x k −1 , x k], (210b)

for k = 1, 2, , n If we assume that y(x0) = y(x0) = y0, then y exactly

agrees with the function computed using the Euler method at the points

x = x k , k = 1, 2, , n The continuous function y, on the interval [x0, x], is a

piecewise linear interpolant of this Euler approximation

We are interested in the quality of y as an approximation to y This will clearly depend on the values of the step points x1, x2, , and especially on

the greatest of the distances between a point and the one preceding it Denote

the maximum of x1− x0, x2− x1, , x n − x n −1 by H.

We would like to know what happens to

that 0)− y(x0)

uniform norm of 0, x] Under very general conditions,

we show that y converges uniformly to y, as the mesh is refined in this way.

211 Local truncation error

In a single step of the Euler method, the computed result, y0+ hf (x0, y0),differs from the exact answer by

y(x0+ h) − y(x0)− hf(x0, y(x0)) = y(x0+ h) − y(x0)− hy  (x

0) Assuming y has continuous first and second derivatives, this can be written

in the form

h2

 1 0

(1− s)y  (x

For i = 1, 2, , N , component i can be written, using the mean value

theorem, as 12h2times component i of y  (x0+ hs ∗ ), where s ∗ is in the interval(0, 1) Another way of writing the error, assuming that third derivatives also

exist and are bounded, is

1

2h

2y  (x

This form of the error estimate is quite convenient for interpreting

numerically produced results, because if h is sufficiently small, the local error will appear to behave like a constant vector multiplied by h2 It is also usefulfor determining how stepsize control should be managed

212 Global truncation error

After many steps of the Euler method, the errors generated in these steps willaccumulate and reinforce each other in a complicated manner It is important

to understand how this happens We assume a uniform bound h2m on the

norm of the local truncation error committed in any step of length h We

aim to find a global error bound using a difference inequality We make the

standard assumption that a Lipschitz condition holds, and we write L as the

Lipschitz constant

Recall thaty(x) denotes the computed solution on the interval [x0, x] That

is, at step values x0, x1, , xn = x, y is computed using the equation

y(xk) = yk = yk −1 + (xk − x k −1 )f (xk −1 , y k −1) For ‘off-step’ points, y(x)

is defined by linear interpolation; or, what is equivalent, y(x) is evaluated

using a partial step from the most recently computed step value That is, if

x ∈ (x k −1 , x k), then

y(x) = yk −1 + (x − x k −1 )f (x k −1 , y k −1 ). (212a)Let α(x) and β(x) denote the errors in y(x), as an approximation to y(x), and in f (x, y(x)), as an approximation to y  (x), respectively That is,

so that, by the Lipschitz condition,

(212d)

Define E(x) so that the exact solution satisfies

y(x) = y(x k −1 ) + (x − x k −1 )f (xk −1 , y(x k −1 )) + (x − x k −1)2E(x),

x ∈ (x k −1 , x k], (212e)

and we assume that

Subtract (212a) from (212e), and use (212b) and (212c), so that

where we have used (212d) and assumed that no step has a length greater

than H We distinguish two cases If L = 0, then it follows that

and if L > 0, it follows that



Hm L

Theorem 212A Assuming that f satisfies a Lipschitz condition, with

constant L, the global error satisfies the bound

213 Convergence of the Euler method

We consider a sequence of approximations to y(x) In each of these

approximations, a computation using the Euler method is performed, starting

from an approximation to y(x0), and taking a sequence of positive steps

Denote approximation number n by y n

The only assumption we will make aboutyn, for each specific value of n, is that the initial error y(x0)− y n(x0) is bounded in norm by Kn and that the

greatest stepsize is bounded by Hn It is assumed that, as n → ∞, H n → 0

and Kn → 0 As always, we assume that f satisfies a Lipschitz condition.

Denote by Dn the value of n(x)

as n → ∞.

Proof This result follows immediately from the bound on accumulated errors

The property expressed in this theorem is known as ‘convergence’ Insearching for other numerical methods that are suitable for solving initial valueproblems, attention is usually limited to convergent methods The reason forthis is clear: a non-convergent method is likely to give increasingly meaninglessresults as greater computational effort is expended through the use of smallerstepsizes

Because the bound used in the proof of Theorem 213A holds not only for

x = x, but also for all x ∈ [x0, x], we can state a uniform version of this result.

Theorem 213B Under the conditions of Theorem 213A,

sup

n(x)

as n → ∞.

Table 214(I) An example of enhanced order for problem (214a)

It is interesting to know not only that a numerical result is convergent, but

also how quickly it converges In the case of a constant stepsize h, the bound

on the global error given in Theorem 212A is proportional to h We describe

this by saying that the order of the Euler method is (at least) 1

That the order is exactly 1, and that it is not possible, for a general

differential equation, to obtain error behaviour proportional to some higher

power of h, can be seen from a simple example Consider the initial value

problem

y  (x) = 2x, y(0) = 0, with exact solution y(x) = x2 If x = 1, and n steps are performed with stepsize h = n −1, the computed solution is

This differs from the exact solution by 1/n = h.

In spite of the fact that the order is only 1, it is possible to obtainhigher order behaviour in special specific situations Consider the initial valueproblem

y  (x) = −y(x) tan(x) − 1

with solution y(x) = cos(x) − sin(x) Because of an exact cancellation of

the most significant terms in the error contributions, at different parts of the