Analysis and improvement of the nonlinear iterative techniques for groundwater flow modelling utilising MODFLOW

The numerical solution of nonlinear problems involves a process of iteration where head values are changed such that they satisfy both the head dependent boundary conditions such as the

Analysis and improvement of the nonlinear iterative techniques for groundwater flow

modelling utilising MODFLOW

Andrew Michael Durick

B Eng (EnvEng) – Bachelor of Engineering in Environmental Engineering – Griffith University 1995

Master of Applied Science

(Research)

Centre of Statistical Science and Industrial Mathematics

Statement of Original Authorship

The work contained in this thesis has not been previously submitted for a degree or diploma

at any other higher education institution To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made

Signed:

Date:

in the discipline of groundwater modelling

The author also acknowledges the partial funding provided by the Queensland Department of Natural Resources & Mines for course fees, the use of resources, and the flexibility to take leave and work around study commitments

The author would also like to thank the School of Mathematics at QUT for a scholarship to ensure enrolment was active at the time of submission and adding the required time to ensure quality in the work undertaken

Thanks also go to Dr Kiran Bajracharya for his comments on the research

Finally, but no less importantly is the support of my family It has been the support

of my wife Leisa, who has shouldered the majority of the household chores and the caring for our children, which has allowed me to undertake and complete this work

as it was undertaken on a part-time basis For this I am extremely grateful

Abstract

As groundwater models are being used increasingly in the area of resource allocation, there has been an increase in the level of complexity in an attempt to capture heterogeneity, complex geometries and detail in interaction between the model domain and the outside hydraulic influences As models strive to represent the real world in ever increasing detail, there is a strong likelihood that the boundary conditions will become nonlinear Nonlinearities exist in the groundwater flow equation even in simple models when watertable (unconfined) conditions are simulated This thesis is concerned with how these nonlinearities are treated numerically, with particular focus on the MODFLOW groundwater flow software and the nonlinear nature of the unconfined condition simulation

One of the limitations of MODFLOW is that it employs a first order fixed point iterative scheme to linearise the nonlinear system that arises as a result of the finite difference discretisation process, which is well known to offer slow convergence rates for highly nonlinear problems However, Newton’s method can achieve quadratic convergence and is more effective at dealing with higher levels of nonlinearity Consequently, the main objective of this research is to investigate the inclusion of Newton’s method to the suite of computational tools in MODFLOW to enhance its flexibility in dealing with the increasing complexity of real world problems, as well as providing a more competitive and efficient solution methodology Furthermore, the underpinning linear iterative solvers that MODFLOW currently utilises are targeted at symmetric systems and a consequence

of using Newton’s method would be the requirement to solve non-symmetric Jacobian systems Therefore, another important aspect of this work is to investigate linear iterative solution techniques that handle such systems, including the newer Krylov style solvers GMRES and BiCGSTAB

To achieve these objectives a number of simple benchmark problems involving

new solution strategies investigated here One of the highlights of these comparisons was that Newton’s method when combined with an appropriately preconditioned Krylov solver was on average greater than 40% more CPU time efficient than the Picard based solution techniques Furthermore, a significant amount of this time saving came from the reduction in the number of nonlinear iterations due to the quadratic nature of Newton’s method It was also found that Newton’s method benefited more from improved initial conditions than Picard’s method Of all the linear iterative solvers tested, GMRES required the least amount of computational effort While the Newton method involves more complexity in its implementation, this should not be interpreted as prohibitive in its application The results here show that the extra work does result in performance increase, and thus the effort is certainly worth it

Contents

1 Introduction 1

1.1 Literature Review 1

1.1.1 The Problem 1

1.1.2 MODFLOW 4

1.1.3 Review of Previous Work 6

1.2 Objectives of the Thesis 11

1.3 Overview of the Thesis 12

2 Computational Techniques for Groundwater Flow Modelling 14

2.1 Nonlinear Techniques 14

2.1.1 Fixed Point Iteration 16

2.1.2 Newton Iteration (Newton’s Method) 18

2.1.3 Summary of Nonlinear Methods 25

2.2 Numerical Solution Techniques of the Linearised System 26

2.2.1 Gaussian Elimination 27

2.2.2 GMRES 28

2.2.3 Bi-CGSTAB 31

2.2.4 LU Decomposition and Solve 34

2.2.5 Preconditioning 35

2.3 Overview of Mathematical Model 38

2.3.1 General Groundwater Flow Equation 39

2.4 Overview of Computational Model 43

2.4.1 Discretisation 43

2.4.2 Finite Difference Equations 46

2.7 Issues in Applying Newton Method to MODFLOW 54

3 Code Implementation 55

3.1 Approach 55

3.2 Coding Philosophy 56

3.3 Code Options 57

3.4 Details of MODTest 58

3.4.1 Model used in the development of the code 62

3.4.2 Reflections on Picard and Newton Implementation 62

4 Results and Discussion 65

4.1 Benchmark Problems 65

4.2 Benchmark Problem 1 66

4.3 Benchmark Problem 2 71

4.4 Benchmark Problem 3 76

4.5 Discussion of Picard Method Results 80

4.5.1 Solver statistics for the Picard Method 80

4.5.2 Summary of the Picard Method 89

4.6 Discussion of Newton Method Results 91

4.6.1 Solver Statistics for the Newton Method 91

4.6.2 Summary of the Newton Method 101

4.7 Comparison of Picard and Newton Results 102

5 Conclusions and Future Work 110

5.1 Conclusions 110

5.2 Future Research Directions 113

6 Bibliography 115

7 Appendix 1 119

File and Subroutine Descriptions 119

8 Appendix 2 123

Key Subroutines written for Study 123

List of Figures

Figure 2.1 – Neighbour Coordinates 45

Figure 2.2– Row, column and layer ordering 45

Figure 2.3– Face at location (i,j+1/2,k) 47

Figure 3.1 – Flowchart for MODTest 59

Figure 3.2 – Example of Solver_ops.in 60

Figure 4.1 – Active model grid and boundary conditions in Benchmark Problem 1 68 Figure 4.2 – Head solution contours for Benchmark Problem 1 69

Figure 4.3 – Model grid for Benchmark Problem 2 72

Figure 4.4 – Head solution contours for Benchmark Problem 2 73

Figure 4.5 – Head solution surface for Benchmark Problem 2 74

Figure 4.6 – Model grid for Benchmark Problem 3 76

Figure 4.7 – Head solution contours for Benchmark Problem 3 77

Figure 4.8 – Head solution surface for Benchmark Problem 3 78

Figure 4.9 – Comparison of Subspace and Time Required for Solution of Picard-GMRES(m) 86

Figure 4.10 – Comparison of Subspace and Time Required for Solution of Newton – GMRES(m) 95

Figure 4.11 – Outer Iteration Count and Total CPU time 99

Figure 4.12 – Outer Iteration Count Compared to F 105

Figure 4.13 – Comparison of Changes in F for Successive Iterations 107

Figure 4.14 – Outer Iteration Count Compared to Maximum Head Change 108

List of Tables

Table 3.1 – Combinations of Nonlinear Techniques and Linear Solution Schemes 57

Table 4.1 – Best Performance for each solver for Benchmark Problem 1 70

Table 4.2 - Best Performance for each solver for Benchmark Problem 2 74

Table 4.3 - Best Performance for each solver for Benchmark Problem 2 using improved initial conditions 75

Table 4.4 - Best Performance for each solver for Benchmark Problem 3 78

Table 4.5 - Best Performance for each solver for Benchmark Problem 3 using improved initial conditions 79

Table 4.6 – Results of Picard and Gaussian Elimination 81

Table 4.7 – Results of Picard and Gaussian Elimination using closer Initial Conditions 81

Table 4.8 – Results of Picard and GMRES 81

Table 4.9 – Results of Picard and GMRES with the closer Initial Conditions 83

Table 4.10 – Results of Picard and GMRES Restarted 84

Table 4.11 – Results of Picard and GMRES Restarted with the closer Initial Conditions 87

Table 4.12 – Results of Picard and BiCGSTAB 88

Table 4.13 – Results of Picard and BiCGSTAB with the closer Initial Conditions 88

Table 4.14 – Results of Picard and LUDecomp 89

Table 4.15 – Results of Picard and LUDecomp with the closer Initial Conditions 89

Table 4.16 – Results of Newton and Gaussian Elimination 91

Table 4.17 – Results of Newton and Gaussian Elimination with the closer Initial Conditions 91

Table 4.18 – Results of Newton and GMRES 92

Table 4.19 – Summary of Krylov Subspace Dimension for each Outer Iteration 93

Table 4.20 – Results of Newton and GMRES with the closer Initial Conditions 93

Table 4.21 – Results of Newton and GMRES Restarted 94

Table 4.23 – Results of Newton and BiCGSTAB 97 Table 4.24 – Results of Newton and BiCGSTAB with the closer Initial Conditions 97 Table 4.25 –Results of Newton and LUDecomp 98 Table 4.26 – Results of Newton and LUDecomp with the Improved Initial

Conditions 100 Table 4.27 – Summary of Average Solver Improvement from Improved Initial

Conditions (Benchmark Problem 2) 103 Table 4.28 – Ratio of successive F between Picard and Newton methods 106

in the mathematical equations used to simulate groundwater flow

Nonlinearity in the formulation of the groundwater flow problem occurs when the conditions being simulated have watertable (unconfined) characteristics, or when the actions on the aquifer from boundary influences involve a function of the aquifer head that is nonlinear The governing equation for groundwater flow in three dimensions is:

t

h S W z

h K z y

h K y x

h K

where K xx , K yy , and K zz are values of hydraulic conductivity along the x,y, and z

coordinates respectively (units of [Lt-1]), S s is the term representing specific storage

of the porous medium measured in [L-1], h is the term representing potentiometric

head with the units of [L], t refers to time, and W is a source/sink term expressed as a

volumetric flux with the units of [t-1] Equation (1.1) is derived in full in Section

2.3.1, and the reader is directed there for more detail

When an aquifer system is modelled, the three-dimensional aquifer is conceptualised

into aquifer layers, which are further discretised into model nodes [1] Flow between

these nodes is defined by two parameters that describe the aquifer’s ability to store

and transmit water These terms are referred to as storativity and transmissivity

respectively Transmissivity is defined as the hydraulic conductivity (K) of the

aquifer material multiplied by the saturated thickness (b), such that

h T y x

h T

Note that the ‘z’ dimension is not considered within the layer approach and the

storage term has become storage coefficient (S c = S s b) When the aquifer layer is

modelled as confined, saturated thickness remains constant and thus, the

transmissivity also remains constant In unconfined conditions the saturated

thickness can vary The elevation of the bottom of the aquifer, and the head in the

aquifer are required to calculate the saturated thickness (b) The governing equation

in the unconfined layer approach can be written as:

h S W

h h K

h h

where the storage term is replaced with its unconfined counterpart - specific yield It

is realised that the following can be written:

y

h y

h h

x

h x

h h

h S y

h K y x

h K

2 2

The nonlinear nature of equation (1.6) comes from the fact that the head terms on the

left hand side are raised to the second power, while the head term on the right hand

side is only to the first power [1]

The numerical solution of nonlinear problems involves a process of iteration where

head values are changed such that they satisfy both the head dependent boundary

conditions (such as the stream interaction example above) and the unconfined head

and resulting flow in the aquifer Typically, the approach to solving the nonlinear

problems can be split into what is usually termed outer and inner iterations The

outer iterations are where the current estimate of the solution is used to set up any

model input that is dependent on the values in the solution This modifies the system

of equations and gives a linearised representation using the current heads The

solution then moves into the inner iteration, where the linearised system is solved

using linear iterative solution techniques

It is the methods undertaken to address the nonlinearity of the equations that is the

focus of this work, particularly for the outer iteration of the nonlinear solution

structure Strong nonlinearity can cause convergence of the iterative scheme to fail,

or can hinder convergence rates significantly The addition of a variety of nonlinear

solution options gives the numerical tools more flexibility to deal with the nonlinear nature of the equations and increases the likelihood that a solution can be achieved

There is a plethora of pre-written software in existence (commercial and freely distributed) specifically for the simulation of groundwater flow Some of this code is written to address concise aspects of groundwater flow, while others facilitate the simulation of a variety of conditions Of all the code available, the most popular and hence most used is MODFLOW The MODFLOW code is distributed by the United States Geological Survey (USGS) and is freely downloadable from the their website (www.usgs.h2o.com) MODFLOW has become the industry standard through its simplicity and flexibility, and is used in the Queensland Department of Natural Resources, Mines & Energy (NRM&E) for the simulation of groundwater flow NRM&E are primarily concerned with the modelling of regional scale aquifers for the purposes of resource allocation and typically, model sizes are in the order of tens

of thousands of model cells (nodes) and involve complex model geometries and watertable conditions

The MODFLOW code first appeared in the early 1980’s with development occurring between 1981 and 1983 A release with full documentation (USGS Open-File Report) followed in early 1984 Its release coincided with the increase in general availability of personal computers MODFLOW has been updated throughout the years following The first major update since its release is often referred to as MODFLOW 88 [2], which saw the code convert from FORTRAN66 to FORTRAN77, and the addition of some add-on packages The second major update occurred in 1996, and saw the inclusion of additional packages to deal with the more complex boundary conditions (such as Flow and Head Boundary Package, Horizontal Flow Barrier update, Transient Leakage Package and Reservoir Package), and also included updates to some of the numerical solvers available (such as Preconditioned Conjugate Gradient version 2 update and the new Direct Solution package) During this period there had been little change made to the core program,

In recent times MODFLOW2000 has appeared, which has involved significant change to the operation of the code, as now the groundwater flow model is encased into its own inverse problem solver for the purposes of calibration Again the core code and groundwater flow equations remain the same, and again there are new additional packages available in this release responding to a need for further complexity to the boundary packages

MODFLOW’s modular nature and its ability to accommodate new packages with little change to the core code was one of the design philosophies that the original programmers adhered to This allowed other hydrologists and programmers to code

up there own calculations for the representation of various boundary conditions, and

it is some of these external codes that have found there way into successive releases

of MODFLOW The original programmers also tailored a design philosophy around the hardware available at the time of the original release, thus they optimised the code around the limited amount of memory available on most computers at the time These constraints led to the use of the Strongly Implicit Procedure (SIP) and the Slice Successive Over Relaxation (SSOR) linear solvers in the original release These solvers were chosen as they conform to the memory usage requirements of only four times the dimension of the problem However, the advancement of computer technology (capacity and speed) and the corresponding advancement in numerical techniques have meant that the concern of memory usage has reduced considerably This was shown with the MODFLOW2000 release that came with an algebraic multi-grid method (LMG) This solver requires far more storage space as it successively creates coarser grids and needs to retain these in the solution process

In the report on this solver, the authors presented evidence that the constraints on earlier versions of MODFLOW should not be adhered to, particularly when the LMG solver can outperform the existing solvers in MODFLOW by two to twenty five

times [3]

Nonlinearity occurs in the unconfined (phreatic surface or watertable) condition as head and flow are explicitly linked through the use of a variable transmissivity within

the user supplied hydraulic conductivity multiplied by the calculated saturated thickness The saturated thickness in the unconfined situation is calculated by

subtracting the bottom elevation of the cell from the head in the cell

The solvers in the first release of MODFLOW (SIP and SSOR) use a fixed-point method to overcome nonlinearities This iterative strategy uses at every iteration the latest estimate of head values (from the previous iteration) to update the boundary flows and transmissivity values in the flow equations for the current iteration In the MODFLOW88 release, a new Preconditioned Conjugate Gradient (PCG) method that uses a modification of the fixed-point method is included The modification sees the definition of an outer and inner iteration where the update of boundary conditions and transmissivity occurs in the outer iteration and this is followed by a series of inner iterations until the solution converges, after which it returns to the outer iteration again Complete convergence is obtained if the count for the inner iteration when the solution returns to the outer iteration is equal to one [4]

Currently the official releases of MODFLOW from the USGS deal with nonlinearities through variations of the fixed-point iterative method, which is also referred to as Picard Iteration There are alternatives to the fixed-point approach, the most popular being Newton’s method In the literature the only instance where MODFLOW has been linked directly to one of the Newton variants is MODFLOW-Surfact This is a commercial software package distributed by Hydrogeologic Inc and provides the user with the option to use Newton Linearisation with a backtracking, or line searching, scheme to overcome nonlinearities tied in with an adaptive time-stepping option

The majority of the available literature focusing on numerical methods for groundwater flow modelling is on the solution of the linearised system The literature highlights in many cases that a fixed-point method is used and it is the linear solver that has been targeted for gains in speed and stability This is expanded

Kuiper [5] compared the Strongly Implicit Procedure (SIP) and the Incomplete Choleski Conjugate Gradient (ICCG) methods and used five two-dimensional problems for testing, of which two were nonlinear ‘water table’ problems The problems utilised two-dimensional versions of the same equations that are now found

in MODFLOW While Kuiper studied solving nonlinear problems, he only applied a simple fixed-point method to the equations to account for the values in the coefficient matrix being a function of head This was applied to SIP at every iteration, and to ICCG at various intervals of iterations

In 1987, Kuiper [6] progressed to evaluations of three-dimensional problems using the SIP and ICCG solvers, however this time both the Picard and Newton techniques for dealing with nonlinearities were compared The SIP solver was tested with both nonlinear techniques, however the ICCG was limited to Picard, due to the need to preserve symmetry of the coefficient matrix for the Choleski preconditioner It was found that the Picard iteration combined with the ICCG solver provided the best performance in solving the three-dimensional nonlinear problem For the two-dimensional nonlinear problem, the Newton and Picard SIP methods performed equally as well as the Picard preconditioned conjugate gradient methods In general it was also found that there was little difference between the performance of the Picard and Newton methods for line SOR or SIP [6] This was an unexpected result and probably due to the problem specifics In particular, the starting value or initial estimate used in the Newton method could have been too far away from the solution

The convergence rate of a numerical method to solve a system of equations depends

on the spectral properties of the coefficient matrix The spectral properties of a coefficient matrix used in the Picard method are usually completely different from the spectral properties of a Jacobian formed through application of Newton’s method

on a system of equations Therefore, there must be other factors affecting the performance of the Newton’s method in Kuiper’s 1987 work and while Kuiper [6] didn’t achieve the success reported by other researchers with the Newton iterative scheme for overcoming nonlinearities, it should be pointed out that the linear solvers

resulting from application of Newton’s Method Since then, significant advances have been made in the solution of non-symmetric systems such as the Generalised Minimisation RESidual (GMRES) [7] and Bi-stabilised Conjugate Gradient (BiCGSTAB) methods [8]

Work in the field of unsaturated groundwater flow solving the Richard’s equation has found Newton’s method to exhibit significant benefits over simple fixed point methods and this is expanded upon in the next section

In the literature the majority of work dealing with nonlinearities in relation to porous media involves the solution of the unsaturated Richard’s equation The methodologies and application of the nonlinear iterative procedures presented in the unsaturated work are applicable to saturated flow models such as MODFLOW

The Picard and Newton iteration schemes are discussed in detail in Section 2.1, however it should be noted that comparisons have been made of the two methods in the literature and these comparisons highlight the strengths and weakness of both methods Kuiper [6] found that Picard and Newton performed “approximately” the same on the basis of the amount of computational effort required for determining the solution

Newton’s method, under the right conditions such as an initial iterate that is sufficiently close to the solution, can be very effective at overcoming nonlinearities Paniconi et al [9] in the process of overcoming the nonlinearities in an unsaturated flow problem found that the Newton method can become quadratically convergent, while the Picard method is limited to linear convergence rates Newton’s method needs this advantage, as it is more complex when compared computationally (in terms of floating point operations) to the Picard scheme Furthermore, Paniconi et al [9] found that the Newton method, on a per-iteration basis, is more expensive than the Picard method, with the ‘per iteration CPU cost’ of the Picard method only half that of the Newton method This finding was also independent of the problem dimension

Implementation of the Full Newton method can be computationally expensive, due in most part to the requirement of generating the Jacobian matrix In evaluating the Jacobian numerically, there exist opportunities to make the computations more efficient, particularly due to local conservation of mass in the conductance of water between neighbouring nodes In fact, only the diagonal terms need evaluation, as the off diagonal terms can be filled from components of the diagonal calculation in a column-wise fashion [10]

Variants of the Newton Method exist and involve selective updating of the slope matrix and are usually employed when numerical computation of the Jacobian terms

is costly [11] Paniconi et al [11] found that successful convergence of the Newton method is reliant on good initial solution estimates This was also mentioned later by Paniconi et al [9] where it was reported that a poor initial estimate may result in divergence Li [12] presented a simplified Newton Iteration, which utilises an integrated hydraulic conductivity and the diffusion-dominated nature of the unsaturated flow process It was found that this approach converged faster than a modified Picard scheme

The Newton scheme had previously been limited by the available linear solvers, in particular the inefficiency of linear solvers when dealing with large sparse non-symmetric matrices This problem has been rectified to some extent through the use

of Krylov subspace algorithms that have become increasingly reliable and efficient [9]

While Newton’s method has faster convergence rates, the Picard method does provide some advantages of its own Paniconi et al [11] found Picard to be simpler

to implement and on a per iteration basis it was less computationally costly Picard retains the symmetry of the coefficient matrix that arises from groundwater flow and Paniconi et al [9]describes how the Newton linearisation generates a non-symmetric matrix (Jacobian), while the Picard preserves the symmetry Preservation of symmetry allows for exploitation of more efficient storage schemes by certain linear solvers Both methods preserve the precise banded nature of the coefficient matrix,

however in an efficient storage scheme the Newton method would require more storage due to loss of symmetry

The aim of this work is to examine numerical methods that converge successfully when the system becomes highly nonlinear and is difficult to solve Panday et al [13] found that the Newton scheme shows it strength when dealing with high nonlinearities and extreme heterogeneities, where often the Picard scheme performs very poorly to the point of failure Paniconi et al [9] found that convergence rates for Picard were slow, and even failed, when the problem involved complex time varying boundary conditions, and strongly nonlinear characteristic equations

Newton’s method also requires that the initial estimate be significantly close to the actual solution, and for groundwater modelling this is likely to be the case [14] During a transient simulation the initial iterate is sourced from the previous timestep, and as groundwater movement is usually slow, it is likely that the solution at the current timestep is going to be close to the preceding timestep The quadratic convergence rates associated with Newton’s Method are usually only applicable at the local scale, which is suited to the transient simulation of groundwater [14] Under steady state conditions, the initial estimate of the solution process can be significantly away from the final converged solution, so it is expected that Newton’s method may not exhibit the theoretical convergence rates initially, and may even fail

to converge

A combined Picard-Newton approach has been suggested by Paniconi et al [9] to overcome the Newton method’s sensitivity to the solution starting point, exploiting the strengths of each method They produced a method that starts with the Picard method to improve the initial solution estimate, then after certain criteria are met, the Newton method is utilised for the solution The switch is driven by a relative reduction in the convergence rates One slight disadvantage of this method is that it requires both symmetric and non-symmetric storage schemes

The work done by Hydrogeologic Inc for their code MODFLOW-Surfact involved

possibility of diverging given certain situations, however techniques can be used to stabilise the solution MODFLOW-Surfact overcomes oscillatory behaviour through

an under-relaxation formula, and backtracking provides further stability Here the

L2-norm residual for the current solution and the preceding solution are compared and if the residual is increasing, the update vector is shortened by a factor This process is repeated until the current iteration residual is an appropriate factor of the previous one The solution then proceeds to the next Newton iteration The additional computation involved in revisiting the solution is offset by the solvers robustness with highly nonlinear problems The resulting asymmetrical matrix system resulting from the application of the Newton iteration is solved in the MODFLOW-Surfact code using an Orthomin solver [15]

1.2 Objectives of the Thesis

The main objective of this thesis is to investigate the use of Newton’s method to overcome the nonlinearities formed from the equations used in MODFLOW Current official releases of MODFLOW from the USGS only provide the fixed-point / Picard iteration for overcoming these nonlinearities The addition of the Newton method would provide more flexibility in MODFLOW’s approach to solving more complex and sophisticated groundwater models where strong nonlinearities are present in the formulation It is thought that the Newton method would increase the chance of achieving adequately converged solutions to the ever-increasing nonlinear nature of complex model setups that are becoming commonplace in NRM&E

Through the use of code written specifically for this study, Newton’s method is compared to Picard’s method using a carefully chosen set of nonlinear benchmark problems The code written for the testing of the nonlinear options is controlled in that it must represent the groundwater flow equation as it appears in the original MODFLOW code, and it must allow for input of existing MODFLOW files The code developed for this thesis has been limited to two-dimensional model setups and

to maximise the difficulty in solution, the code solves steady state unconfined (water

The research sees the implementation of an inexact Newton method for the solution

of the MODFLOW equations Inexact Newton methods employ some strategy to approximate the Jacobian at each iteration and typically finite difference approximations to the derivatives provide a good balance between accuracy and efficiency Furthermore, variations of the Inexact Newton method are investigated that seek to minimise the number of times the complete Jacobian is generated and factored during the outer iterations The variants studied here include the Chord, Shamanskii and Definitive Newton methods, which differ in the decision of when to update the Jacobian matrix using the current solution

Finally, from the realisation that the success of the nonlinear iteration method could

be reliant on the numerical method employed to solve the linearised system during the inner iteration, the performance of four linear solvers has also been studied These solvers are two direct methods – Gaussian Elimination (GE) and Lower Upper Decomposition (LUdecomp), and some Krylov based solvers – Bi-Conjugate Gradient Stabilised (BiCGSTAB) and the Generalised Minimal Residual (GMRES) method The Krylov based solvers also link to some preconditioning options such as Incomplete Lower Upper factorisation with zero infill (ILU(0)) and a simple SSOR preconditioner on the coefficient matrix for the linearised system (from both Picard and Newton)

1.3 Overview of the Thesis

The thesis comprises five chapters, each of which is outlined below

undertaken The reader is introduced to the MODFLOW code and explanations of why the nonlinear options are being explored The work undertaken in the area of porous medium flow modelling and nonlinear methods is also presented A clear statement of the thesis objectives is also provided

Chapter 2 presents an overview of the mathematical and computational models and then details the techniques for overcoming nonlinearities and some of the numerical solution schemes utilised in later chapters for testing the chosen benchmark problems

nonlinear and numerical techniques

nonlinear and numerical techniques Discussions and analysis of the results of this testing are also presented

computational strategy investigated throughout the thesis The chapter also recommends future research directions that it is felt would add value to the work undertaken here

Chapter 2

Groundwater Flow Modelling

2.1 Nonlinear Techniques

As mentioned in Chapter 1, there are two popular options for solving nonlinear

systems The first method is referred to as the Fixed Point or Picard Method, and the

second is Newton’s method These methods aim to solve a system of nonlinear

equations of the form:

( )x =0

where F(x) represents a system of n nonlinear equations with n variables, and thus

can be considered in the form of:

( )

( , , , ) 0

0,

,,

0,

,,

2 1

2 1 2

2 1 1

=

=

=

n n

n n

x x x f

x x x f

x x x f

KMK

K

, (2.2)

where the f are known as coordinate functions In this work it is assumed that i

N N

R R

D

F: ⊂ → where D is some convex, simply connected region contained

within the domain of the vector valued function F Equation (2.1) defines a typical

problem that requires solution in a variety of engineering and science applications

Before the two main methods of overcoming nonlinearity are discussed, it is

important to note that iterative methods can be grouped by their “rates of

convergence” If we let the sequence { }( )n N

for n sufficiently large

Note that throughout this chapter unless otherwise stated, implies 2-norm The

above classifications are taken from [14], and are presented here for completeness

because they are mentioned in discussions concerning the nonlinear techniques

throughout this chapter The two nonlinear techniques of interest are the Fixed Point

/ Picard and Newton methods

2.1.1 Fixed Point Iteration

Picard, or Fixed Point iteration as it is sometimes known, is the simplest method used

to overcome nonlinearities Picard iteration is the most commonly used procedure

for the solution of nonlinear flow equations [6] It is simple to implement in a

computer programming language and is computationally inexpensive [9]

Many nonlinear problems are naturally posed as fixed-point problems A fixed-point

problem can be viewed as:

( )x G

where G:D⊂R N →R N maps the nonlinear nature of the equations, xis called a

fixed point The iterative solution of equation (2.6) takes the form of:

(n ) ( )( )n

x G

x +1 =

, (2.7)

and prior to discussing the convergence properties of this method, it is essential that

two definitions be made

continuous on Ω with Lipschitz constantγ if

DEFINITION 2.2 Let Ω⊂R N and let G:Ω→R n G is a contraction mapping on Ω

if G is Lipschitz continuous on Ω with Lipschitz constant γ <1

Definitions 2.1 and 2.2 are utilised in the following theorem about the convergence

rate for a fixed point:

THEOREM 2.1 Let Ω be a closed subset of N

R and G be a contraction mapping on

Ω with Lipschitz constant γ <1 such that G( )x ∈Ω ∀ x∈Ω A unique fixed point

2.1.1.1 Description of Picard Method

In descriptive terms, the Picard iteration, specifically in it’s application within MODFLOW’s PCG2 solver, involves starting with an initial estimate of the head in each model cell Using these initial heads, the head dependent boundary conditions and variable transmissivity values are calculated and updated in the linear approximation matrix system This linearised system is solved through one of a number of available numerical linear solvers, a solve that occurs on what are referred

to as inner iterations When either convergence is achieved, or the maximum inner iteration total is reached, the solution moves to the outer iteration where the new heads (from the most recent inner iterations) are again used to update the head dependent boundary conditions Convergence of the nonlinear system is deemed to have been achieved if the very first inner iteration after an update from an outer iteration is successful

These concepts are best demonstrated in the algorithm below, which identifies the steps used in MODFLOW when the Preconditioned Conjugate Gradient (PCG2) or

Algorithm 1 – PICARD METHOD

1 DO WHILE (iter<max_outer_iter) and (max_abs_residual<tol)

a Formulate the equation with the current solution estimate

b DO WHILE (iter_inner<max_inner_iter)

i Solve linearised system for h

ii If (max_abs_residual<tol) then

1 If (iter_inner = 1) exit outer iteration loop

2 Else exit inner iteration loop

c Set (h) equal to new current solution

The Picard iteration method can be varied by adjustment of the maximum amount of

inner iterations that need to take place before the head dependent variables and

transmissivity values are updated with the current estimate of head As stated in

Section 1.1.3, the Picard method is easy to implement in terms of coding, as can be

seen from the above algorithm

Newton Iteration is a numerical technique used to overcome the non-linearity of the

equation being solved In recent times it has become the method of choice for

dealing with nonlinearities in the solution of many engineering and scientific

applications It is not as prevalent in saturated groundwater flow modelling, most

likely due to the adequacy of the Picard method for the level of model complexity

applied in the past

Newton’s Method is summarised as follows:

(n ) ( )n ( )( )n ( )( )n

x F x F x

where ( )n+ 1

x is the updated iterate, ( )n

x is the current iterate, ( )( )n

x

F ′ is the Jacobian, and ( )( )n

x

F is the function evaluated using the current iterate

The Newton method in descriptive terms starts with the evaluation of the function at

terminate If the termination criterion (based on ( ) ( ) 0

x

Solution of the above system is used to update the solution for the current iteration of

the original system of equations through:

) ( ) ( ) 1 (n n n

x x

*

22

then, Kelly [14] states that if K >0 such that ( )n ∈Β( )δ

x , the Newton iterate given

in equation (2.9) satisfies

(n 1) ( )n 2

e K

, (2.17)

from which it can be concluded that the Newton iteration presented in Equation (2.9)

converges q-quadratically to x The above result is proved in [14] and again the *

interested reader is directed to this document for further detail

The algorithm for the standard application of Newton’s method to solve a nonlinear

system appears below Note the algorithm uses an LU factorisation with a forward –

backward solve, however there are a number of different methods that could be

employed to solve the linearised system of equations, by replacing steps 2b and 2c

Algorithm 2 – NEWTON METHOD

The term tol in Algorithm 2 (Step 2) is a termination tolerance that scales the original

residual vector Components 2a and 2b in Algorithm 2 is where most of the

computational effort is required Methods can be used to reduce the effect of this on

the total computational time and these are discussed in Section 2.1.2.1

With N being the dimension of the problem, if the Jacobian was dense it would

require O(N 3 ) floating point operations to factorise F ′( )x [14] Of course, the

Jacobian is rarely dense, so it could be expected that this projected cost is an over

estimate in most cases In fact, many sparse Jacobian matrices can take advantage of

the sparsity patterns and become more computationally efficient

Components of the Jacobian can be generated through finite differencing resulting in

an Inexact Newton method This method is demonstrated below in equation (2.18):

=

′

0,0

0,

x F

F

x x

x F x x F x

F

εεε

ε

, (2.18)

where ε is the perturbation factor, typically chosen as εM , εM being the

machine epsilon

The cost of the difference calculation is reduced in this case because the standard

function F has already been calculated for use on the RHS of equation (2.10) and

through the evaluation of the solution from the previous time step In a saturated

groundwater flow context, and due to the conservation of mass around the branch

conductance terms, savings in computation time can also be made for neighbouring

cells, thus reducing the computational effort for the calculation of the Jacobian

entries

2.1.2.1 Variations in Newton’s Method

Changes to the frequency of updating the Jacobian matrix with the current solution

will result in variants of the standard Newton Method These variants aim to reduce

the computational cost of evaluating the Jacobian at each step, as is shown in steps

2a and 2b of Algorithm 2

If the initial estimate is sufficiently close to the solution, the chord method usually

gives the best performance for large dimensioned problems [14]

2.1.2.1.1 Chord Method

This method varies from the standard Newton method by only generating the

Jacobian once and factoring it prior to the inner iterations It is formulated from the

initial guess; therefore the method is incredibly reliant on the quality of the initial

guess The change is shown in the iteration step for the Chord Method as follows:

(n ) ( )n ( )( ) ( )( )n

x F x F x

e e K

K Chord for convergence The implementation of the Chord method is

outlined below in Algorithm 3

Algorithm 3 CHORD METHOD

As can be seen from Algorithm 3, the movement of Steps 2 and 3 out of the loop

(Step 4) reduces the overall computation effort required Nevertheless, without

knowledge of a reasonably accurate starting point ( ) 0

x , the Chord method may struggle to converge

2.1.2.1.2 Shamanskii Method

Cycling through a Newton step followed by a number of Chord steps leads to a

method that has been named the Shamanskii method This class of method is higher

order and converges q-superlinearly with q-order larger than two [14] In

comparison with the standard Newton method, higher q-orders can be obtained

through the Shamanskii method with reduced requirement to generate (and factorise)

the Jacobian

The number of chord steps (m) becomes an input parameter to the method Variation

in this value can result in changing the method’s performance to achieve a solution

Some problems may have a specific nature that benefits from a particular range of m

To progress from the initial estimate (x(0)) to the solution from the current Newton

step (x(sol)) the following steps take place:

( ) ( ) ( ) ( )

) ( ) (

) ( 1 ) 0 ( )

( ) 1 (

) 0 ( 1 ) 0 ( )

0 ( ) 1 (

11

:

m sol

j j

j

x x

m j for x

F x F x x

x F x F x x

−

(2.21)

where m is the number of “chord” steps prior to a Jacobian update As mentioned

before, the Shamanskii Method can converge q-superlinearly to x with q-order of *

e K

The algorithm for the Shamanskii method is presented below in Algorithm 4

Algorithm 4 – SHAMANSKII METHOD

iv IF ||F(x)|| < tol * r 0 : EXIT

As can be observed in Algorithm 4, the requirement to generate and factor the

Jacobian (steps 2a and 2b.) is reduced in comparison to the full Newton’s method if

m is greater than 1 If m is equal to 1, Algorithm 4 becomes the full Newton method

as was outlined in Algorithm 2, and if m=∞the Chord method is recovered

2.1.2.1.3 Definitive Newton Method

A further development of the Newton method in relation to the formation of the

Jacobian, is to add a condition that aids with the decision of when F ′( )x should be

updated The decision is made by a comparison of the previous residual norm and

the current residual norm, and takes the form:

x F

The “update tol” in equation (2.23) should be in (0,1) and is typically set to 0.5 It

should however be noted that this value can critically affect the efficiency and

performance of the scheme When the criterion in equation (2.23) is met, the

standard Newton step is undertaken where the Jacobian matrix is updated using the

current estimate to the solution

The definitive method can be combined with the other methods, such as the Shamanskii method, to provide two options for determining when to update the Jacobian matrix The following algorithm shows the application of this strategy:

Algorithm 5 – SHAMANSKII & DEFINITIVE NEWTON METHOD

The Picard method is straightforward to implement, however it converges only linearly The only real variation in the method concerns the decision of when to update the coefficient matrix with the latest value of the dependent variable

The Newton method is more complicated to implement, however it can offer quadratic convergence rates It also offers a range of variants that revolve around the decision of when to update the Jacobian matrix, and an inexact strategy to generate the Jacobian entries

Overcoming the nonlinear iteration is only part of the overall solution because the nonlinear outer iteration strategy must be paired with an appropriate and effective solution method for solving the resulting linearised system Significant development has occurred in this area over the last two decades and some of the techniques investigated in this thesis are detailed in the next section

2.2 Numerical Solution Techniques of the

Linearised System

With respect to the Picard iteration, the most commonly used linear solvers in flow modelling are the alternating direction implicit procedure (ADI), successive over-relaxation (SOR), or the strongly implicit procedure (SIP) [6] In recent times, Krylov subspace methods have become popular for solving linear equations, in particular the conjugate gradient method (CG) for symmetric systems and the generalised minimised residual (GMRES) method for non-symmetric system

Krylov methods fall into a group of numerical solvers described as non-stationary Non-stationary iterative methods have information in the computations that changes

at each iteration, and typically involve constants that are computed from inner products of residuals or other vectors that are formed from the iterative method [16]

Various authors have compared the merits and performance of various solvers for the solution of two- and three-dimensional linear and nonlinear groundwater flow problems [4], [5], [6] These studies usually focus on comparing two, or more, iterative solvers (SIP, SOR, CG) While results varied based on the problem size and complexity being solved, there was still a common theme in the results Specifically, the preconditioned conjugate gradient methods provided the most competitive convergence rates [17]

Iterative methods are good at removing error components with short spatial persistence (short wavelength errors) Therefore it will be the long wavelength errors existing in the initial estimate that determines the rates of convergence These long wave errors are particularly hard to minimise with the initial estimate when the model domain is complex, nonlinear and heterogenous [17]

Paniconi et al [9] suggested relaxation as a methodology to enhance the nonlinear iterative schemes and to overcome oscillations in the solution process, particularly when dealing with strong variability in the head values

In this work it has been decided to examine four linear solution schemes for the solution of the linearised matrix system within the test code These four methods are Gaussian Elimination (GE), Bi-Conjugate Gradient Stabilised (Bi-CGSTAB), Generalised Minimised Residual (GMRES), and LU decomposition and solve (LUDecomp) As these are the solvers used in the test code, they are presented in relevant detail below GEP and LUDecomp are similar in how they solve the linear system, however LUDecomp was included in the list of test solvers as it splits the decomposition and solve phases of the solution process, allowing it to be utilised with the Newton variant methods of Shamanskii and Definitive Newton

Gaussian elimination is a direct solver method that constructs an equivalent upper

triangular system (Ux = y ) representation of the original system of equations and

then performs a backward substitution to solve for x This process can break down

when the diagonal element of the coefficient matrix equals zero

If we consider the formation of the upper triangular system at the (k-1) st step the

reduced matrix takes the form:

1 1

2 , 1 2

, 1 2

1 , 1

1 2 1

22

1 12

11

1

0

k nn k

nk

k kn k

kk

k n k k

k k k

k k

n n k

a a

a a

a a

a

a a

a a

a

A

L

MOMLL

MO

LL

LL

, (2.24)

which is a row equivalent form of the augmented matrix system [A : b] In this

system, changes made to the A matrix are also applied to the b vector

When a diagonal element (referred to as the pivot) is classed as weak, or found to

equal zero, some simple row permutations can be made to the linear system to

correct the problem and progress the solution This process known as pivoting

involves searching the current column below the diagonal, and finding the strongest

entry, i.e., the entry of maximum absolute magnitude The row containing the

relevant element can be swapped with the current row, thus ensuring that the

diagonal element for the current row is strongest Such a process reduces the impact

of floating point error, but does not eliminate it totally

The decision of what row to swap is called the pivoting strategy There may be a

number of non-zero elements below the diagonal and the choice of which row to

swap is important As computer arithmetic is fixed in precision, truncation errors can

occur and propagate To minimise this, it is suggested that the row with the

maximum absolute value:

k

mp a

below the current diagonal (pivot) be swapped While not strict, choosing the largest

pivot element will usually result in smaller error being propagated The particular

implementation of this method used for this research involves the use of a threshold

value (1-6) that adjusts smaller pivots to this value to reduce the chances of the

system breaking down The interested reader can find more detail of this method in

any numerical analysis textbook

Saad and Schultz in 1986 proposed an extension of the Minimal Residual (MINRES)

Residual (GMRES) [7], and in recent times it has become the method of choice for

finite element researchers In short, GMRES aims to approximate the exact solution

minimised through a least squares problem It uses the Arnoldi process to build an

orthogonal basis for the Krylov subspace, where the approximation of the solution

lies in the affined space x0+K m(A,r0), where:

1 0

2 0 0

,r Span r Ar A r A r A

A modified Gram-Schmidt orthogonalisation is adopted during the Arnoldi process

GMRES is presented in Algorithm 6 in detail, however the method can be described

in broad steps The process starts with an initial estimate (x0) that is used to form an

initial residual of the system being solved The norm of this initial residual vector

divides the initial residual vector to form the first Krylov unit vector The

Gram-Schmidt process is then applied to a vector formed by multiplication of the original

system coefficient matrix and the current Krylov vector, which produces a new

orthonormal vector and the entries in the next column of an upper Hessenberg

matrix This process continues until the algebraic grade of the matrix is reached, or

some specified maximum subspace dimension is attained The Hessenberg matrix is

then reduced through Givens rotations to produce an upper- triangular form, enabling

the resulting upper triangular system to be used in a backward substitution to solve

for a vector (y) for the least squares solution This vector is used to form the

approximate solution

Algorithm 6 – GMRES

1 r0 =b−Ax0,

2 0

is often used [7]

The restarted version of GMRES seeks to reduce the amount of storage required It

works by the user presetting the value of m, thus restricting the size of the Krylov

subspace to this value The Arnoldi process is restarted using the last estimate of the

solution (x m ) as the new initial estimate The decision made on the value of m can be crucial to the solution process with the convergence stalling if m is too small There are also examples of problems that will only converge on the nth step, and thus any

value of m less than n will fail to converge, and as such, it can be concluded that the choice of m is very dependent on the problem being solved In fact, the use of

restarting can seriously reduce the performance of the solver due to the loss of previous information through discarding the previous subspace [14]