Exploratory spatial data analysis

Phương pháp phân tích không gian mở rộng nhằm tìm kiếm mối liên kết trong không gian

Exploratory spatial data analysis in a geographic

information system environment

Robert Haining{, Stephen Wise and Jingsheng Ma

University of Shef®eld, UK

[Received June 1996 Revised January 1998]

Summary The paper describes SAGE, a software system that can undertake exploratory spatial data analysis (ESDA) held in the ARC/INFO geographical information system The aims of ESDA are described and a simple data model is de®ned associating the elements of `rough' and `smooth' with different attribute properties The distinction is drawn between global and local statistics SAGE's region building and adjacency matrix modules are described These allow the user to evaluate the sensitivity of results to the choice of areal partition and measure of interarea adjacency.

A range of ESDA techniques are described and examples given The interaction between the table, map and graph drawing windows in SAGE is illustrated together with the range of data queries that can be implemented based on attribute values and locational criteria The paper concludes with a brief assessment of the contribution of SAGE to the development of spatial data analysis.

Keywords: Adjacency matrix; Area data; Brushing; Local and global statistics; Regionalization

1 Introduction

Exploratory spatial data analysis (ESDA) is the extension of exploratory data analysis (EDA) to the problem of detecting spatial properties of data sets where, for each attribute value, there is a locational datum This locational datum references the point or the area to which the attribute refers Examples include rainfall measurements taken at a number of sample sites in a region or mortality rates for a set of wards or counties EDA is a collection of descriptive techniques for detecting patterns in data, identifying unusual or interesting features (including detecting errors), distinguishing accidental from important features and for formulating hypotheses from data EDA may also be employed after data modelling to assess aspects of model ®t The set of exploratory techniques combines techniques that are visual (including charts, graphs and ®gures) and numer-ical but statistnumer-ically robust Exploratory techniques generally stay `close' to the original data, mean-ing that they use relatively simple intuitive manipulations of the data

ESDA is an extension of EDA to detect spatial properties of data: to detect spatial patterns in data, to formulate hypotheses which are based on, or which are about, the geography of the data and to assess spatial models The class of techniques that are used is, as in EDA, visual and robust However, it is important to be able to link numerical and graphical procedures with the map and to

be able to answer questions such as `where are those cases on the map?', `where do attribute values from this part of the map lie in the data summary?' or `which areas on the map lie in this subregion of the map and meet speci®ed attribute criteria?' The map is an essential additional tool for exploring spatial data

1998 Royal Statistical Society 0039±0526/98/47457

The Statistician (1998)

47, Part 3, pp 457±469

{Address for correspondence: Department of Geography and Shef®eld Centre for Geographic Information and Spatial Analysis, University of Shef®eld, Shef®eld, S10 2TN, UK.

E-mail: R.Haining@shef®eld.ac.uk

This paper reports on the development of a software system for carrying out ESDA linked to the ARC/INFO geographical information system (GIS) Because there are many types of spatial data we focus only on what Cressie (1991), pages 7±10, called `lattice' data, a term which includes the general case where the regions that partition the map may be irregular in shape Here the attribute values must be standardized in some way so that values in different regions are comparable The denominator is usually a measure of area in the case of a spatially continuous variable like crop yields or a count of households or individuals (for example) in the case of a spatially discrete variable like population The wish to link spatial data analysis to the GIS is because the GIS has become widely used for geographical data management and cartographic modelling and because it has functionality that facilitates the development of many spatial analysis techniques including spatial data analysis Recent papers have considered the types of analytical capabilities that are most suited to a GIS environment (Goodchild et al., 1992; Fotheringham and Charlton, 1994) The arguments and illustrations in this paper are drawn from one such project that has led to the development of the SAGE package (Haining et al., 1996) The development of SAGE has been based on the assumption that even in the typical GIS environment which is characterized by very large data sets there is still an important role for simple and familiar statistical methods For an alternative view see, for example, Openshaw (1994) SAGE has also been built using wherever possible existing, well-tested, software All the processes of data input, data management and data analysis are provided within the GIS without the need to export or import data ®les during the analysis Fig 1 shows SAGE with all the four types of window open: the table window (which has limited spreadsheet capability) displaying the current set of data and any new variables created during a session, the map window, a graph window and the text output window that returns statistical output such as model parameters Note that the linked windows facility is being used with selected data cases identi®ed in the table, map and graph windows More than one graph window can be opened and linked with the other windows

Fig 1 SAGE: displaying the four types of window and the linked windows facility

458 R Haining, S Wise and J Ma

The next section de®nes a data model for the patterns which ESDA may be used to detect, making the distinction between `whole map' and `local' statistics Section 3 considers the impor-tance of the regional partition and the de®nition of adjacency in ESDA Section 4 describes ESDA techniques in SAGE for detecting properties of a single mapped attribute Section 5 comments brie¯y on the availability of other software packages for implementing spatial data analysis

2 A data model for spatial pattern in a single attribute data set

One simple data model for EDA distinguishes between the `smooth' component of the data which derives from some summary of the data and the `rough' component which is the residual (Tukey, 1977) Thus

data ˆ smooth ‡ rough:

A spatial data set comprises for each case an attribute value and its locational identi®er If we disregard the locational identi®er initially this leads to the association of smooth and rough with just the (non-spatial) attribute values of the data set Such non-spatial smooth properties include the central tendency of the distribution measured by the median, the dispersion of the distribution measured by the interquartile range and the shape of the distribution depicted by box plots or histograms The non-spatial rough property is the difference between the data value and the smooth value and outliers are de®ned as cases with particularly high levels of rough Outliers are identi®ed

as data values that are more than a certain distance above or below the upper or lower quartile respectively With some modi®cation this decomposition can also be adapted to spatial data When the locational identi®er is included smooth and rough properties need to be de®ned in terms of where on the map the cases are found Smooth spatial properties include spatial trends, spatial autocorrelation (the propensity, in the case of positive autocorrelation, for similar values to

be found together across the whole map) and spatial concentrations (the propensity for large values to be found together and/or low values to be found together across the whole map) Again the rough component is the distance between the data value and the smooth component The residuals may show evidence of localized patterns of spatial autocorrelation and spatial concentra-tion A spatial outlier is a case where the attribute value is very different from neighbouring attribute values This suggests a modi®ed data model for ESDA of the form

data ˆ (trend ‡ spatial covariation ‡ concentration) ‡ (residuals and spatial outliers):

A model similar to this model, though differing in detail, applies to data ordered in time as well ESDA techniques fall into two broad categories: `global' or whole map statistics, which pro-cess all the cases for an attribute, and `focused' or local statistics, which propro-cess spatially de®ned subsets of the data, one subset at a time, and which may involve a sweep through all the de®ned subsets looking for evidence of localized properties of the mapped dataÐor the residuals after, say, the removal of the trend component

ESDA can be applied to de®ned subareas of the map, e.g by applying global or focused statistics only to cases falling within a de®ned window Different methods can be distinguished on the basis of how the subset of cases is de®ned The majority of systems that are currently available allow brushing, in which the selection of areas is made interactively on the map Once the selection is complete, the graphical or statistical results for this subset are displayed This is the style of interaction provided by SAGE, where the brushing can take place in any one of the cartographic, tabular or graphical views of the data, resulting in the identi®cation of the selected cases in all the other views Subsequent calculations (e.g of summary statistics) can be restricted

to the currently selected subset of cases

Exploratory Spatial Data Analysis 459

Craig et al (1989) suggested and implemented an extension to this idea in which the calcu-lation of statistical results would be done as the brushing was done If the selection of areas was made using a ®xed window (e.g a circle), then as this was moved across the map the statistics would be recalculated and graphics redisplayed, allowing the user to explore differences across the region The technique known in the geographical literature as geographically weighted regression,

in which a regression model is ®tted to data in a de®ned ®xed window and then re®tted as the

Fig 2 (a) Regionalization of Shef®eld, aggregating enumeration districts into 29 regions on the basis of deprivation scores, (b) histogram window showing the population sizes of the 29 regions (equality criterion) and (c) histogram of the interquartile ranges of the enumeration district level deprivation scores for the 29 new regions (homogeneity criterion)

460 R Haining, S Wise and J Ma

window is moved over the area, falls into this general category (Brunsdon et al 1996, 1998) Fitting models to data subsets in this way (as opposed to data summaries) raises questions, however, about the interpretation and comparability of results Results will depend, for example,

on the extent to which statistical assumptions are satis®ed across the different subsets In that respect the technique is quite unlike sensitivity analysis in regression which proceeds by deleting small subsets of cases to assess their in¯uence on parameter estimates and model predictions As far as we are aware, this general form of spatial data analysis has not yet been implemented in software linked to any GIS although it is being implemented in other computing environments (see Bivand (1998), Dykes (1998) and Unwin and Hofmann (1997))

The model for map pattern described in this section is not formal and the distinction between what is trend, spatial autocorrelation or spatial clustering is deliberately not well de®ned The techniques that will be discussed may be used to identify attribute properties but cannot be said to estimate the various components of map pattern

Any spatial analysis based on area data must recognize that results are dependent on the form of the regional partition One of the elements of SAGE is a simple region building module that is appropriate for ESDA Spatial properties may also depend on de®nitions of the pseudo-ordering of

Fig 2 (continued)

Exploratory Spatial Data Analysis 461

the regions SAGE allows for alternative de®nitions of adjacency between regions We discuss these now

3 Handling the spatial framework in SAGE

3.1 Region building

Spatial data analysis often starts from small spatial building-blocks (e.g UK census enumeration districts), aggregating these until the resulting regions constitute a satisfactory basis for statistical analysis Aggregation may be necessary to create robust rates for analysis, to reduce the effects of any suspected locational or attribute data inaccuracies, to make data analysis tractable or to facilitate visualization (Wise et al., 1997) ESDA does not necessarily require such aggregation and in some spatial data sets (e.g analysing electoral outcomes by constituency) the spatial unit is naturally de®ned and relevant both to the underlying process as well as to subsequent interpreta-tion However, if aggregation is required for any of the reasons given above then it should be possible to aggregate according to speci®ed criteria and then to construct other similar or equally plausible aggregations fairly quickly and easily to assess whether ®ndings change signi®cantly This amounts to allowing the user to examine for possible effects arising from the modi®able nature of the areal units, a matter of particular concern in analysing geographical data and one which has a long history of study (Kendall, 1939; Openshaw, 1984)

SAGE allows the user to construct aggregations according to three criteria: homogeneity (minimizing within-group variance of one or more attributes), equality (minimizing the difference between the total value of an attribute, such as population size, across regions) and geographical compactness The importance to be attached to each of these criteria in forming the regionaliza-tion can be adjusted through the use of weights within an objective funcregionaliza-tion The regionalizaregionaliza-tion

is a k-means-based classi®cation that allows the user to start from one of many initial allocations

of zones to regions and then allows swaps at the boundaries Swaps may be allowed even if one or two of the individual criteria become worse, provided that the overall function improves and provided that those that do become worse do not exceed a user-de®ned threshold There is further description of this module in Wise et al (1997)

Fig 2(a) shows a regionalization based on one of the SAGE algorithms building up from enumeration district scores for the Townsend index of material deprivation (Townsend et al., 1988) Fig 2(a) shows the construction of 29 `deprivation' regions from the 1159 enumeration districts in the Shef®eld region The histograms provide evidence of the extent to which the algorithm has been able to meet the equality criterion (measured by regional population countsÐ Fig 2(b)) and the homogeneity criterion (measured by the intra-region interquartile range for the enumeration district level Townsend scoresÐFig 2(c)) It appears that the equality criterion is quite well satis®ed except for two areas that are far too large and will need to be split The homogeneity criterion shows that there is intra-regional variation in deprivation However, the new regionalization is still a considerable improvement over other partitions at the same scale such as wards (there are 29 in Shef®eld) in terms of demarcating areas of similar deprivation (For a discussion of this see Haining et al (1994).)

3.2 Adjacency measures

Many spatial analysis techniques require the analyst to de®ne the set of neighbours of each region

in the map partition and to de®ne the relative weights to be attached to each paired neighbour Unlike time, geographic space has no natural order and with irregular regional units there may be

a need to explore the sensitivity of results to many alternative de®nitions of neighbourhood As it

462 R Haining, S Wise and J Ma

is loaded into memory SAGE automatically creates a de®nition and creates the measures needed for two other neighbourhood or adjacency matrices (W) These are derived from the stored adjacency relationships held by ARC/INFO in which each line segment or arc has a direction and

a list is maintained of the polygons (regions) that lie on the left-hand and right-hand sides of each arc (Ding and Fotheringham, 1992) The adjacency matrix automatically generated by SAGE is a simple binary adjacency matrix determined by whether regions share a common boundary (1) or not (0) Two other matrices are constructed using intercentroid distances and the length of the shared common boundary These can be converted by the user into an appropriate adjacency matrix (Haining (1993), pages 73±74) There is a further module in SAGE that allows the user to create other matrices or to modify the automatically generated matrices

4 Exploratory spatial data analysis for identifying properties of a univariate data set

As illustrated by the following, EDA summaries and graphics that do not depend on any spatial referencing have important roles to play in ESDA

(a) MedianÐESDA query: which areas have attribute values above (or below) the median?

Do they show any evidence of pattern?

(b) QuartilesÐESDA query: which areas lie in the upper (or lower) quartile? If FU and FL

denote the upper and lower quartiles then which cases have attribute values that are greater than FU‡1:5( FUÿFL) or less than FLÿ1:5( FUÿFL) and may be de®ned as outliers? (c) Box plotsÐESDA query: where do cases that lie in particular areas of the box plot occur

on the map? Where are the outlier cases located on the map? The two previous queries can

be subsumed within this query

(d) HistogramsÐESDA query: where do cases that relate to particular bars of the histogram occur on the map?

Fig 3 shows a box plot of standardized incidence rates of a form of cancer in Shef®eld displayed in the graphics window and all the cases lying above the median are `brushed' and highlighted in the map window Note that most of the areas with high rates are to be found in the eastern and central area of Shef®eld which includes many of the more deprived parts of the city ESDA techniques for identifying spatial properties of the attribute data usually require a de®nition of adjacency Here we de®ne a general n 3 n (n corresponding to the number of regions

on the map) adjacency matrix W with non-negative elements {wij} where the subscripts reference regions i and j and by de®nition we set wiiˆ0 In some cases the row sums of W are standardized

to a constant (usually 1) It is important to recognize that many of the techniques for ESDA can (and probably should) be replicated with different de®nitions of W for there is no natural ordering

In addition it is often appropriate to replicate the analysis by taking a sequence of distances or

`lag' (step) orders on the graph of regions to detect properties at different spatial distances Where the map consists of many small areas a simple smoothing method may reveal general patterns (such as a trend) that are not apparent from the mosaic of values Kernel estimation, in its simplest form, involves passing the equivalent of a moving average or `local mean' ®lter across the surface:

MAiˆ Xi‡P

j

wijXj

j

wij

where the weight w is 1 if region j shares a common boundary with region i and is 0 otherwise

Exploratory Spatial Data Analysis 463

(wiiˆ0) Other weights and constructions for kernel estimation can be used which are also implemented in SAGE A slight modi®cation to this method for smoothing and hence detecting trends in spatial data would be to replace the value in a region i with the median value from the set that includes Xiand the values in the adjacent regionsÐa moving median or `local median' ®lter (MMi) This would still further reduce the effect of extreme values on the smoothed surface The smoothed component of the map can then be extracted from the map by computing XiÿMAior

XiÿMMi Using the median smoother rather than the mean results in areas with particularly high rates standing out even more strongly as areas with a large element of rough This last stage, using

MAi, is similar to the process of smoothing by spatial differencing described by Cliff and Ord (1981), p 192, provided that the weights are de®ned in the same way The principal distinction lies

in whether the value at i is or is not included in the term MAi

Where it is thought that attribute values might decrease (or increase) away from a speci®c area such as the centre of a city then a transect of values might be helpful and can be implemented in SAGE SAGE also allows the construction of a series of `lagged' box plots in the graphics window where the ®rst box plot is the ®rst-order neighbours of the selected region, the second box plot is the second-order neighbours and so on (Haining (1993), p 224) This second method is only likely

to be useful provided that all the areas are of similar size and shape but in those cases can indicate the presence of trend and dispersal around the trend

Whole map statistical tests have been developed for testing for global spatial autocorrelation (e.g Moran's I ) and spatial concentration (e.g the Getis±Ord G- and G-statistics) (Cliff and Ord, 1981; Getis and Ord, 1992) and these techniques are available in SAGE However, these tests are not based on robust estimators (of the centre of the distribution of values); nor could they be described as exploratory Values of the statistic do not have any intuitive interpretation They are really more appropriate for con®rmatory work A simple ESDA tool in SAGE that can explore for these properties is based on the scatterplot Values of an attribute (X ) are plotted on the vertical

Fig 3 Box plot of standardized incidence rates for a cancer by regions of Shef®eld linked to a map of the regions and highlighting all cases with higher than expected rates (rates greater than 100)

464 R Haining, S Wise and J Ma

axis against the weighted values of the neighbours (ÓjwijXj) on the horizontal where the weights should be standardized to sum to 1 A scatterplot where there is a general upward sloping scatter

to the right is indicative of positive spatial autocorrelation, i.e adjacent values tend to be similar

If the scatter slopes downwards to the right this is indicative of negative spatial autocorrelation; adjacent values tend to be dissimilar (If the scatter is linear and shows little evidence of disper-sion this is indicative of spatial trend) Fig 4 illustrates the scatterplot applied to standardized incidence rates for a form of cancer for Shef®eld There is a general trend in the scatterplot, suggesting spatial autocorrelation

Points on the scatterplot in the extreme parts of the top right-hand or bottom left-hand quadrants may be ¯agging regions that show a concentration or clustering of high or low values Points on the scatterplot lying well below or well above any part of the general scatter may indicate regions with attribute values that make them spatial outliers For example, an attribute value that is close to the mean of the distribution of values, encircled by values at or close to the lower tail of the distribution, could be an outlier There are no very clear cases in Fig 4, but six points lying distant from the line have been selected to illustrate that, as the histogram shows, such spatial or geographical outliers need not be outliers in the statistical distributional sense This identi®cation of spatial outliers can be made a little more formal by running a regression line through the scatter Cases with standardized residuals that are greater than 3.0 or less than ÿ3.0 might be ¯agged as possible spatial outliers although this simple test, if based on the least squares

®t, will tend to overstate the number and size of outliers (see Haining (1993), pages 214±215) As noted earlier it is possible to brush any part of the scatterplot to identify where the regions are on the map and the corresponding values are also highlighted on the spreadsheet

Local statistics, available in SAGE, can be used to assess the presence of localized spatial autocorrelation or concentration The Getis±Ord (Gi-) statistic for detecting localized

con-Fig 4 Scatterplot of standardized incidence rates of a cancer against the average of the rates in adjacent regions: cases with low rates but surrounded by regions the average of whose rates is at or near the expected rate (100) are highlighted; these cases are also highlighted in the histogram

Exploratory Spatial Data Analysis 465

centrations (or localized clusters) in an attribute which is positive valued with a natural origin is de®ned:

Gi ˆP

j

wijXj

P

j

Xj

where wij is the entry in the weights matrix W where wii 0 and the statistic is computed for each region in turn (Getis and Ord, 1992) A large value of Gi signals a clustering of high values around region i; a small value signals a clustering of low values around i The local Moran statistic

is de®ned (Anselin, 1995):

Iiˆxi

P

j

wijxj

where xiand xjsignify deviations from the mean A large positive value of Iisignals a local set

of similar values in the neighbourhood of region i; a large negative value signals a local set of dissimilar values at i

The Gi-values are comparable (same mean and variance) if the weights matrix W is standardized so that row sums equal a constant In this case the set of n regional values for each of these statistics could be rank ordered to signal where localized clusters might exist on the map, or treated as a distribution and examined (as suggested above) for extreme values which can then be brushed to identify the cases on the map Formal tests of signi®cance are available in SAGE and the standardized form of the statistic will be required to allow for non-constant means and variances if W has not been standardized For the local Moran statistic, standardization of the statistic is always required since, although the expected values are constant if W is standardized

so that row sums equal a constant, the variances are not

There is often an advantage to simultaneously using spatial and non-spatial statistical methods

to tease out and then to demonstrate the presence of interesting data properties Unwin (1996) gave an example of the use of the standardized form of the Gi-statistic together with a graphical approach using the histogram of the original data to illustrate the way that each can complement the other in an exploratory analysis to locate clusters of extreme values of a variable:

`Having applied both approaches it is then easier to understand what is going on and the graphical approach can be used to present and explain the results to others'

(Unwin (1996), p 396) Fig 5 shows a map of the extreme positive values of the Gi-statistic (signi®cant at the 5% level) computed from the standardized incidence rates and indicating a cluster of cases The histogram of the standardized incidence rates is not indicative of particularly high rates simply on a region-by-region basis

In addition to the facilities described here for performing ESDA, SAGE has additional capabilities including Bayesian smoothing to adjust rates based on different base populations (Clayton and Kaldor, 1987) and graphical and numerical techniques for exploring and analysing relationships between attributes SAGE can ®t different types of regression model and provide regression diagnostics for con®rmatory spatial data analysis In addition to the standard regression model and testing for residual spatial autocorrelation, SAGE enables the user to ®t various types

of spatial regression model including a model with spatially autocorrelated errors and a model with spatially lagged terms in the set of explanatory variables The latter may include spatially lagged versions of one or more of the explanatory variables; it may also include spatially lagged values of the response variable among the set of explanatory variables All these models are described in, for example, Haining (1993), pages 339±341 The description of these facilities will

be the subject of Haining et al (1998) Some of the ESDA facilities described above can also be

466 R Haining, S Wise and J Ma