Introduction to Numerical Methods for Diferentia

Lecture 3 Introduction to Numerical Methods for Differentialand Differential Algebraic Equations... Newton’s Law: m¨x = −Flxm¨y = mgFlyConservation of mechanicalenergy: x2+ y2 = l2 1 Ini

Lecture 3 Introduction to Numerical Methods for Differential

and Differential Algebraic Equations

Newton’s Law:

m¨x = −Flxm¨y = mgFlyConservation of mechanicalenergy: x2+ y2 = l2

1 Initial Value Problems (IVPs)

Single Step Methods Multi-step Methods

2 Boundary Value Problems

Finite Difference Methods Single- and Multiple-Shooting Methods

DAE models in engineering applications Peculiarities of DAEs

Index notions for DAEs Backward Difference Formula (BDF) Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

Initial Value Problems

General form of an initial value problem (IVP) is

If f is (explicitly) time independent; i.e f (t, x(t)) = f (x(t)), thenthe system (1) - (2) is said to be autonomous

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

x(t0) = x0 is the initial condition (start time of the motion)The expression ˙x(t) represents the first order derivative of x(t)with respect to time; i.e ˙x(t) = dtdx(t) Thus, the IVP (1) - (2)represents a system of first order differential equations.

If f (t, x(t)) = A(t)x(t), then we have



=

ω

−Lgsin(θ)

,θ(0) =

π 4

ω(0) = 0

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Commonly, numerical methods are developed for systems of firstorder differential equations.

⇒ any higher order differential equation should be written as asystem of first order differential equations

NB: The Matlab ODE Toolbox works only with systems of first orderdifferential equations

Strategy: Convert any n-th order initial value problem into a system

of first-order IVP by introduction new variables

Then the n-th order IVP reduces to

equation (1); i.e x∗(t) = f (t, x∗(t)) and the initial condition (2); i.e

Basic idea of numerical solution of differential equations:

Select a set of equally spaced time points t1, t2, , tN from theinterval [t0, tf] (tf freely chosen for IVPs) That is, t1 = t0+ h ,

t2 = t1+ h, and so on; so that h = tk+1− tk, k = 0, 1, , N − 1.Approximate the continuous solution x∗(t) of (1) & (2) by values

at the discrete time points t1, t2, , tN ; i.e

x(t1) = x1, x(t2) = x2, , x(tN) = xN

determine the values x1, x2, , xN through the IVP (1) - (2).Note:

more discretization points from [t0, tf] implies

better accuracy of the approximation; but

demands more computation effort,

depending on the algorithm used, it may be necessary to solve asystem of linear or nonlinear equations

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

t=t 0

is the slope ofthe tangent line to the graph of x(t) at the point (t0, x0)

Figure: Euler’s Method

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Since the function x(t) is not yet known, the derivative (slope) can beapproximated by

˙x(t0) ≈ x(t0+ h) − x(t0)

hThus, using ˙x(t0) = f (t0, x0), we have

xk+1= xk+ hf (tk, xk), k = 0, 1, , m − 1

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Euler’s Algorithm

Select N

Set h = (tf −t 0 )

N ; x0= x(t0)FOR k = 0 : (n − 1)

Simple Pendulum with Euler’s Method

Simple Pendulum with Euler’s Method

for k=1:N

oomega(k+1)=oomega(k)+h*

(-beta/m/L*oomega(k)-g/L*sin(ttheta(k)));ttheta(k+1)=ttheta(k)+h*oomega(k);

end

time=cputime-t;

plot(tt,oomega,tt,ttheta)

title(’Simple pendulum using Euler’’s method’)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Advantages/Disadvantages of Euler’s Method

Advantages:

Euler’s Method is simple and direct

Can be used for nonlinear IVPs

Disadvantages:

it is less accurate and numerically unstable

Approximation error is proportional to the step size h Hence,good approximation is obtained with a very small h Thisrequires a large number of time discretization leading to a largercomputation time

Usually applicable to explicit differential equations

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Improvements on Euler’s Method

(A) The Mid-Point Method

Let k1= hf (ti, xi)

Note that: k1= hf (ti, xi) = xi+1− xi But to compute k1, it is not

Figure: Use also the mid point Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Improvements on Euler’s Method

• The point (ti+12ti, xi+12k1) is the mid-point of the segment thatconnects the points (ti, xi) and (ti+1, xi+1)

• Now, instead of the single slope at (ti, xi), we use the slopes at(ti, xi) and at (ti+12ti, xi+ 12k1) to approximate xi+1

⇒ This leads to the Mid-Point Runge-Kutta Algorithm:

The mid-point Runge-Kutta Method

Improvements on Euler’s Method

(B) Methods that use Intermediate Points

Question: Can we improve the mid-point RK method by freely

(ti+1, xi+1)?

Figure: Choose freely any intermediate point

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

• A weighted sum of f (ti, xi) and f (ti+ ch, xi+ mk1) provideds

more slope information than f (ti, xi) alone

Improvements on Euler’s Method

(ii) 2nd- order Taylor approximation of x(t + h):

A system of three equations with four unknowns ⇒ This system hasinfinitely many solutions

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Improvements on Euler’s Method

system of equations for the other three

⇒ Hence, we can construct several RK2 methods

Example:

reduces to the mid-point RK2 method

1 2

1

1 3

2 3

3 4

Runge-Kutta Methods

The above schemes are known as second-order Runge-Kuttamethods, since x(t + h) has been approximated by a second degreeTaylor polynomial

I For an accurate approximation use more intermediate pointsbetween (ti, xi) and (ti+ h, xi+ k1) and include more slopeinformation

I Requires higher degree Taylor approximations of x(t + h)

Figure: Use several intermediate points Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

• The relation between ω1, , ωm, α1, , αm, β1, , βm is

determined by comparing the m-th order Talyor approximation ofx(t + h) and the scheme RK(m)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

I Hence, the resulting method is called an m-th order Runge-Kuttamethod.

I There are third-order Runge-Kutta (RK3), fourth-order

Runge-Kutta methods (RK4), etc

The classical 4-th order Runge-Kutta Method has the form:

The classical 4th order Runge-Kutta Method

%The classic 4th order Runge Kutta method

for k=1:N

k1=h*(-beta/m/L*oomega(k)-g/L*sin(ttheta(k)));

m1=h*oomega(k);

k2=h*(-beta/m/L*(oomega(k)+k1/2)-g/L*(sin(ttheta(k)+m1/2)));m2=h*(oomega(k)+m1/2);

k3=h*(-beta/m/L*(oomega(k)+k2/2)-g/L*(sin(ttheta(k)+m2/2)));m3=h*(oomega(k)+k2/2);

k4=h*(-beta/m/L*(oomega(k)+k3/2)-g/L*(sin(ttheta(k)+m3/2)));m4=h*(oomega(k)+k3);

title(’Simple pendulum solved with 4th order Runge Kutta’)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

In general the 4-th order Runge-Kutta Methods take the form:

Question: When do RK methods fail?

The state x1(t) = x1(0)e−100t and x2(t) = x2(0) = e−0.1t x2(t)

systems with the same step length h

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Advantages and Disadvantages of Simple RK Methods

Advantages

they are easy to implement

the are stable

Disadvantages

they require relatively large computer time

error estimation are not easy to be done

the above simple RK methods do not work well for stiff

differential equations (eg linear differential equations with widelyspread eigenvalues)

In particular, they are not good for systems of differential

equations with a mix of fast and slow state dynamics

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Adaptive Runge-Kutta Methods

The choice of a good step size h is crucial In particular, this isimportant for stiff differential equations:

• too large h may lead to inaccurate approximation of the solution;

• too small h forces the algorithm use large computation resources

⇒ The selection of the step length h is a compromise betweennumerical accuracy and speed of computation

I Adaptive adjustment of the step size h reduces (local) truncationerror for approximating x(ti+1) by xi+1:

kx(ti+1) − xi+1kRecall the Taylor approximation:

x(ti+ h) = x(ti) + h ˙x(ti) + h

2

2 x(t¨ i) +

h33!

Adaptive Runge-Kutta Methods

• Adaptive strategies adjust the step-length from iteration to iteration

by testing the accuracy iterates

• Usually this test of accuracy is done by comparing lower and higherorder RK algorithms

⇒ Hence, adaptive RK methods combine lower and higher order

RK methods to adjust the step-size h

i+1 and xRK4i+1 are approximations for x(ti+1).

Thus the local truncation errors (using the Taylor approximation ofx(t + h) with ti+1= ti+ h):

from (RK3) is x(ti+1) − xRK3i+1 (ti+1) = C3h3+

from (RK4) x(ti+1) − xRK4i+1 = C4h4+

⇒ (subtracting the second from the first)

xRK4i+1 − xRK3i+1 ≈ C3h3

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

I Hence, the truncation error can be estimated in two useful ways:(i) on the hand, the local truncation error is can be approximated by

In the next iteration step we would like to choose a new step-size

tolerance level ErrTol

I The step-size should be adjusted relative to the truncation errorcommitted in the previous step and the given error tolerance for thenext step (taking ratios of terms in equations (7) and (8))

kC1kh3 new

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Adaptive Runge-Kutta Methods

⇒

hnew< h

ErrT olErrT runc

1/3

In particular the new h can be computed adaptively by

ErrT olErrT runc

1/3

for a fixed γ such that 0 < γ < 1

(ii) on the other hand, comparing the algorithms (RK3) and (EK4s)s,

ErrT runc = |xRK4i+1 − xRK3i+1 | = h

6 [−2k2+ k3+ k4]These lead to an adaptive 3-4 Runge-Kutta algorithm

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Adaptive Runge-Kutta Methods an RK34

• Set t0,x0, initial step size h, ErrorT ol, tf and γ,while t < tf do

Common adaptive methods are the Runge-Kutta-Fehlberg(RKF)formulas.

• RKF23 (a combination of 2nd and 3rd order RKs), RKF45, etc.Matlab ode functions:

I ode23 is an adaptive Runge-Kutta methods and a combination of a2nd and 3rd order Runge-Kutta methods

I ode45 is an adaptive RK known as the Dormand-Prince method.Example: Mass-springer dumper system:

M ¨x + C ˙x + kx = Fwith mass M = 5kg,dumper value

C = 10N s/m, springconstant k = 1000N/m,

F = sin(20t)N withforcing frequency ω = 20

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Mass-springer dumper system Matlab

To solve 5¨x + 10 ˙x + 100x = sin(20t), with x(0) = 0, ˙x(0) = 0

Mass-springer dumper system Matlab

function dxdt=diffEq(t,x)

dx1dt = x(2);

dx2dt = -2*x(2)-20*x(1)+0.2*sin(20*t);dxdt=[dx1dt;dx2dt];

end

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

• such algorithms may take enormous computation time;

• simulation of dynamic systems with several state equations can becomputationally demanding;

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Multi-step and Implicit Methods

Euler, Runge-Kutta and adaptive RK methods are single-step andexplicit methods

• Multi-step methods: use a set of earlier iterates up to the currentiterate xi to compute the next iterate xi+1

⇒ Earlier iterates xi, xi−1, , xi−m are used to compute xi+1.For example:

Multi-step and Implicit Methods

I BDF is a well known efficient multi-step method:

BDF = Backward Differentiation Formula

I While IRK is a well known implicit method:

IRK = Implicit Runge-Kutta Method

I Both BDF and IRK are widely used for the direct discretization ofcontinuous dynamic optimization problems

I Particularly, when the IVP involves a stiff-differential equations,BDF and IRK are methods of choice

Stiff differential equations: A differential equation

˙

x = f (t, x)

is stiff if the (Jacobi) matrix ∂f∂x has eigenvalues with widely spreadnegative real parts

⇒ some states change fast while others are slower

Examples: mass-spring systems with large spring constants,

(chemical) process engineering models, etc

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Multi-step and Implicit Methods BDF

I Adaptive RK methods can solve some stiff problems, but they are

in general not efficient for a system with several states

(A) BDF: Given xi, xi−1, , xi+1−m to compute xi+1 Find an m-thdegree polynomial P that interpolates the m + 1 points

(ti+1, xi+1), (ti, xi), (ti−1, xi−1), , (ti+1−m, xi+1−m),

A good choice for the interpolating polynomials P is

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Observe that

P (ti+1−j) = xi+1−j, j = 0, 1, , m

In particular P (ti+1) = xi+1

Thus, replace ˙xi+1 by ˙P (ti+1) to obtain

L0(ti+1)f (ti+1, xi+1)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

• If f is a nonlinear function, all BDF algorithms require the solution

of nonlinear equaitons

=⇒ BDF is used in conjuction with a Newton method

I Matlab function: ode15s uses a BDF algorithm

Advantages:

• are suitable for the solution of stiff differential equations

• can be easily generalized to solve DAEs

• BDF 2 up to BDF 6 are known to be stable

• widely used in circuit simulation

Disadvantages:

• may require the solution of nonlinear equations

• faces difficulties if the step-size h is constant (i.e requires

adaptation of the step size)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

Multi-step and Implicit Methods IRK

(B) IRK:

approximation xi+1of x(ti+1)?

Idea:

satisfying (the requirements):

˙

P (ti+ τlh) = f (ti+ τlh, P (ti+ τlh)), l = 1, , m, (10)where τ1, τ2, , τm ∈ (0, 1) and h = ti+1− ti the steps size

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

• This special polynomial satisfies equation (10).

• Integrating equation (9) on both sides

An m-stage IRK Algorithm:

Given the points τ1, τ2, , τm∈ [0, 1):

• If τ1, , τm are known the vector b = (b1, , bm) and the matrix

The difference in IRK methods lies:

• on the selection of the parameters τ1, , τm

• on special adjustments to reduce the size of the matrix A

• IRK methods are also known as collocation methods.Some examples:

Polynomila):

√ 15 15

5

36 −

√ 15 30

√ 15 24

2 9

5

36 −

√ 15 24

√ 15 30

2

9+

√ 15 15

5 36 5

18

4 9

5 18

(b) Radau IIA

(c) Radau 5

(d) Lobatto IIIA

(e) Diagonally implicit RK methods

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations