Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 7)

Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 7)

7 Three-Dimensional Multiscale Watershed Segmentation

of MR Images

Ioannis Pratikakis, Hichem Sahli, and Jan Cornelis

CONTENTS

7.1 Introduction7.2 Watershed Analysis7.2.1 The Watershed Transformation7.2.1.1 The Continuous Case7.2.1.2 The Discrete Case7.2.1.3 The 3-D Case7.2.1.4 Algorithms about Watersheds7.2.2 The Gradient Watersheds

7.2.3 Oversegmentation: A Pitfall to Solve in Watershed Analysis7.3 Scale-Space and Segmentation

7.3.1 The Notion of Scale7.3.2 Linear (Gaussian) Scale-Space7.3.3 Scale-Space Sampling

7.3.4 Multiscale Image-Segmentation Schemes7.3.4.1 Design Issues

7.3.4.2 The State of the Art7.4 The Hierarchical Segmentation Scheme7.4.1 Gradient Magnitude Evolution7.4.2 Watershed Lines during Gradient Magnitude Evolution7.4.3 Linking across Scales

7.4.4 Gradient Watersheds and Hierarchical Segmentation inScale-Space

7.4.5 The Salient-Measure Module7.4.5.1 Watershed Valuation in the Superficial

Structure-Dynamics of Contours7.4.5.2 Dynamics of Gradient Watersheds in Scale-Space7.4.6 The Stopping-Criterion Stage

7.4.7 The Intelligent Interactive Tool2089_book.fm Page 271 Tuesday, May 10, 2005 3:38 PM

272 Medical Image Analysis

7.5 Experimental Results7.5.1 Artificial Images7.5.2 Medical Images7.6 Conclusions

References

7.1 INTRODUCTION

The goal of image segmentation is to produce primitive regions that exhibit geneity and then to impose a hierarchy on those regions so that they can be groupedinto larger-scale objects The first requirement concerning homogeneity can be verywell fulfilled by using the principles of watershed analysis [1] Specifically, ourprimitive regions are selected by applying the watershed transform on the modulus

homo-of the gradient image We argue that facing an absence homo-of contextual knowledge,the only alternative that can enrich our knowledge concerning the significance ofour segmented pixel groups is the creation of a hierarchy, guided by the knowledgethat emerges from the superficial and deep image structure The current trends aboutthe creation of hierarchies among primitive regions that have been created by thewatershed transformation consider either the superficial structure [1–4] or the deepimage structure [5, 6] alone In this chapter, we present the novel concept of dynamics

of contours in scale-space, which integrates the dual-image structure type into asingle one Along with the incorporation of a stopping criterion, the proposed integrationembodies three different features, namely homogeneity, contrast, and scale Appli-cation will be demonstrated in a medical-image analysis framework The output ofthe proposed algorithm can simplify scenarios used in an interactive environmentfor the precise definition of nontrivial anatomical objects Specifically, we present

an objective and quantitative comparison of the quality of the proposed schemecompared with schemes that construct hierarchies using information either from thesuperficial structure or the deep image structure alone Results are demonstrated for

a neuroanatomical structure (white matter of the brain) for which manual tation is a tedious task Our evaluation considers both phantom and real images

segmen-7.2 WATERSHED ANALYSIS 7.2.1 T HE W ATERSHED T RANSFORMATION

In the field of image processing, and more particularly in mathematical morphology,gray-scale images are considered as topographic reliefs, where the numerical value

of a pixel stands for the elevation at this point Taking this representation into account

we can provide an intuitive description of the watershed transformation as in raphy, where watersheds are defined in terms of the drainage patterns of rainfall If

geog-a rgeog-aindrop fgeog-alls on geog-a certgeog-ain point of the topogrgeog-aphic surfgeog-ace, it flows down thesurface, following a line of steepest descent toward some local surface minima Theset of all points that have been attracted to a particular minimum defines the catch- ment basin for that minimum Adjacent catchment basins are separated by divide lines or watershed lines A watershed line is a ridge, a raised line where two sloping

Three-Dimensional Multiscale Watershed Segmentation of MR Images 273

surfaces meet Raindrops falling on opposite sides of a divide line flow into differentcatchment basins (Figure 7.1)

Another definition describes the watershed line as the connected points that liealong the singularities (i.e., creases or curvature discontinuities) in the distancetransform It can also be considered as the crest line, which consequently can beinterpreted by two descriptions: firstly, as the line that consists of the local maxima

of the modulus of the gradient, and secondly, as the line that consists of the zeros

of the Laplacian These intuitive descriptions for the watershed-line constructionhave been formalized in both the continuous and discrete domain

7.2.1.1 The Continuous Case

In the continuous domain, formal definitions of the watershed have been workedout by Najman [7] and Meyer [8] The former definition is based on a partial orderingrelation among the critical points that are above several minima

Definition 1: A critical point b is above a if there exists a maximal descendingline of the gradient linking b to a

Definition 2: A path γ: ] −∞, +∞ [ → R2 is called a maximal line of thegradient if

and

Definition 3: A maximal line is descending if

Definition 4: Let P(ƒ) be the subset of the critical points a of ƒ that are aboveseveral minima of ƒ Then the watershed of ƒ is the set of the maximallines of the gradient linking two points of P(ƒ) This definition of Meyer[8] is based on a distance function that is called topographical distance.Let us consider a function ƒ: Rn→R and let supp(ƒ) be its support The

FIGURE 7.1 ( Color figure follows p 274 ) Watershed construction during flooding in two dimensions (2-D).

274 Medical Image Analysis

topographical distance between two points p and q can be easily defined

by considering the set Γ(p,q) of all paths between p and q that belong to

supp(ƒ)

Definition 5: If p and q belong to a line of steepest slope between p and q

(ƒ(q) > ƒ(p)), then the topographical distance is equal to

TD(p,q) = ƒ(q) −ƒ(p)

Definition 6: We define a catchment basin of a regional minimum m i, CB(m i),

as the set of points x ∈ supp(ƒ) that are closer to m i than to any other

regional minimum with respect to the topographical distance

j≠i⇒ TD(x, m i) < TD(x, m j)

Definition 7: The watershed line of a function ƒ is the set of points of the

support of ƒ that do not belong to any catchment basin

7.2.1.2 The Discrete Case

Meyer’s definition [8] can also be applied for the discrete case if we replace the

continuous topographical distance TF by its discrete counterpart Another definition

is given by Beucher [1] and Vincent [9] The basic idea of the watershed construction

is to create an influence zone to each of the regional minima of the image In that

respect, we attribute a one-to-one mapping between the regional minima and the

catchment basin

Definition 8: The geodesic influence zone IZA(B i) of a connected component

B i of B in A is the set of points of A for which the geodesic distance to B i

is smaller than the geodesic distance to any other component of B

IZA(B i) = {p ∈A, ∀j∈ [1,k]\{i}, d A (p,B i ) < d A (p,B j)}

Definition 9: The skeleton by influence zones of B in A, denoted as SKIZ A B,

is the set of points of A that do not belong to any IZ A (B i)

SKIZA (B) = A/IZ A (B)

with

IZA (B) = ∪i ∈[1,k] IZA (B i)

Definition 10: The set of catchment basins of the gray-scale image I is equal

to the set obtained after the following recursion (Figure 7.2)

where

hmin, hmax are the minimum and maximum gray level of the image,

respec-tively

T h (I) is the threshold of the image I at height h

minh is the set of the regional minima at height h

Definition 11: The set of points of an image that do not belong to any

catchment basin correspond to the watersheds.

7.2.1.3 The 3-D Case

A brief but explicit discussion about watersheds in three-dimensional (3-D) spacewas initiated by Koenderink [10], who considered watersheds as a subset of the

density ridges According to his definition, “the density ridges are the surfaces

generated by the singular integral curves of the gradient, that is, those integral curvesthat separate the families of curves going to distinct extrema.” In cases where weconsider only families of curves that go to distinct minima, then the produced densityridges are the watersheds For a formal definition of the watersheds in 3-D, thereader can straightforwardly extend the definitions in Sections 7.2.1.1 and 7.2.1.2.For the definition of Najman [7] in the 3-D case, we have to consider that the points

in P(ƒ) are the maxima and that the two types of hypersaddles are connected to twodistinct minima These points have, in one of the three principal curvature directions,slope lines descending to the distinct minima; the two slope lines run in oppositedirections along the principal curvature direction These points make the anchor pointsfor a watershed surface defined by these points and the slope lines that connect them

7.2.1.4 Algorithms about Watersheds

The implementation of the watershed transformation has been done by using thefollowing methods: iterative, sequential, arrowing, flow-line oriented, and flooding

The iterative methods were initiated by Beucher and Lantuéjoul [11], who

suggested an algorithm based on the immersion paradigm The method

FIGURE 7.2 ( Color figure follows p 274 ) Illustration of the recursive immersion process.

Level 3 Level 2

Level 1

X1

expands the influence zones around local minima within the gray-scalelevels via binary thickenings until idempotence is achieved.

The sequential methods rely on scanning the pixels in a predefined order, in

which the new value of each pixel is taken into account in the processing

of subsequent pixels Friedlander and Meyer [12] have proposed a fastsequential algorithm based on horizontal scans

The arrowing method was presented by Beucher [1] and involves description

of the image with a directed graph Each pixel is a node of the graph, andthe node is connected to those neighbors with lower gray value The word

“arrowing” comes from the directed connections of the pixels

The flow-line oriented methods are those that make an explicit use of the

flow lines in the image to partition it by watersheds [5]

The flooding methods are based on immersion simulations In this category,

there are two main algorithms The algorithm of Vincent and Soille [9] andthe algorithm of Meyer [13] For an extensive analysis and comparisons ofthe algorithms that are based on flooding, the interested reader can refer tothe literature [14, 15]

7.2.2 T HE G RADIENT W ATERSHEDS

Whenever the watershed transformation is used for segmentation, it is best to apply

it only on the gradient magnitude of an image, because then the gradient-magnitudeinformation will guide the watershed lines to follow the crest lines, and the realboundaries of the objects will emerge It has no meaning to apply it on the original

image Therefore, from now on, we will refer to gradient watersheds, thus explicitly

implying that we have retrieved the watershed lines from the modulus of the gradientimage Examples of gradient watersheds in two dimensions (2-D) and 3-D can beseen in Figure 7.3 and Figure 7.4–7.5, respectively The singularities of the gradientsquared in 2-D occur in the critical points of the image and in the points where thesecond-order structure vanishes in one direction This can be formulated as:

(7.1)(7.2)

FIGURE 7.3 Gradient watersheds in 2-D.

where x, y denote Cartesian coordinates and w, v denote gauge coordinates [16].

The gradient can be estimated in different ways It can be computed as (a) theabsolute maximum difference in a neighborhood, (b) a pixelwise difference between

a unit-size morphological dilation of the original image and a unit-size ical erosion of the original image, and (c) a computation of horizontal and verticaldifferences of local sums guided by operators such as the Roberts, Prewitt, Sobel,

morpholog-or isotropic operatmorpholog-ors The application of gradient operatmorpholog-ors as in case c reduces theeffect of noise in the data [17] In the current study, the computation of the gradientmagnitude is done by applying the Sobel operator Accordingly, in the case of 3-D,the singularities of the gradient squared occur due to the following conditions

(7.3)

(7.4)

where x, y, z denote Cartesian coordinates and w, v, u denote gauge coordinates with

w in the gradient direction and (u, v) in the perpendicular plane to w (the tangent

plane to the isophote)

Similar to the 2-D case, the gradient magnitude in 3-D can be estimated indifferent ways All of the existing approaches issue from a generalization of 2-D

FIGURE 7.4 (a) The cross-sections of the 3-D object and (b) their 3-D gradient watersheds.

(a)

(b)

L2x+L2y+L2z= ,0(L x = ∧0 L y= ∧0 L z=0)

L2x+L2y+L2z ≠ ∧0 L ww = ∧0 L wv = ∧0 L wu =0

edge detectors Lui [18] has proposed to generalize the Roberts operator in 3-D byusing a symmetric gradient operator Zucker and Hummel [19] have extended to 3-

D the Hueckel operator [20] They propose an optimal operator that turns out to be

a generalization of the 2-D Sobel operator The morphological gradient in 2-D hasbeen extended to 3-D by Gratin [21] Finally, Monga [22] extends to 3-D the optimal2-D Deriche edge detector [23] For the implementation of the gradient watersheds

in 3-D, the current study has adopted the 3-D Zucker operator for the 3-D magnitude computation

gradient-7.2.3 O VERSEGMENTATION : A P ITFALL TO S OLVE IN W ATERSHED A NALYSIS

The use of the watershed transformation for segmentation purposes is advantageousdue to the fact that

• Watersheds form closed curves, providing a full partitioning of the imagedomain; thus, it is a pure region-based segmentation that does not requireany closing or connection of the edges

• Gradient watersheds can play the role of a multiple-point detector, thustreating any case of multiple-region coincidence [7]

• There is a one-to-one relationship between the minima and the catchmentbasins Therefore, we can represent a whole region by its minima

Those advantages can be useful provided that oversegmentation, which is

inher-ent to the watershed transformation, can be eliminated An example of tation is shown in Figure 7.6 This problem can be treated by following two differentstrategies The first strategy considers the selection of markers on the image andtheir introduction to the watershed transformation, and the second considers theconstruction of hierarchies among the regions that will guide a merging process.The next sections of this chapter are dedicated to the study of methods followingthe second strategy

oversegmen-FIGURE 7.5 A rendered view of the 3-D gradient watershed surface and the orthogonal

sections.

7.3 SCALE-SPACE AND SEGMENTATION

7.3.1 T HE N OTION OF S CALE

As Koenderink mentions [24], in every imaging situation one has to face the problem

of scale The extent of any real-world object is determined by two scales: the innerand the outer scale The outer scale of an object corresponds to the minimum size

of a window that completely contains the object and is consequently limited by thefield of view The inner scale corresponds to the resolution that expresses the pixelsize and is determined by the resolution of the sampling device

If no a priori knowledge for the image being measured is available, then we

cannot decide about the right scale In this case, it makes sense to interpret the image

at different scales simultaneously The same principle has been followed by thehuman visual front-end system Our retina typically has 108 rods and cones, and aweighted sum of local groups of them make up a receptive field (RF) The profile

of such an RF takes care of the perception of the details in an image by scaling up

to a larger inner scale in a very specific way Numerous physiological and physical results support the theory that the cortical RF profiles can be modeled byGaussian filters (or their derivatives) of various widths [25]

psycho-7.3.2 L INEAR (G AUSSIAN ) S CALE -S PACE

Several authors [24, 26–35] have postulated that the blurring process must essentiallysatisfy a set of hypotheses like linearity and translation invariance, regularity, locality,causality, symmetry, homogeneity and isotropy, separability, and scale invariance.These postulates lead to the family of Gaussian functions as the unique filter for

scale-space blurring It has been shown that the normalized Gaussian Gσ(x) is the

only filter kernel that satisfies the conditions listed above:

Here x⋅x is the scalar product of two vectors, and d denotes the dimension of

the domain The extent of blurring or spatial averaging is defined by the standarddeviation σ of the Gaussian, which represents the scale parameter An example ofthis spatial blurring can be seen in Figure 7.7 From this example, it can clearly beseen how the level of detail in the image decreases as the level of blurring increasesand how the major structures are retained

The scale-space representation of an image is denoted by the family of derived

images L(x, σ) and can be obtained as follows: let L(x) be an image acquired by

some acquisition method Because this image has a fixed resolution determined bythe acquisition method, it is convenient to fix the inner scale as zero The linear

scale-space L(x,σ) of the image is defined as

where ⊗ denotes spatial convolution Note that the family of derived images L(x,σ)

depends only on the original image and the scale parameter σ

Lindeberg [29] has pointed out that the scale-space properties of the Gaussiankernel hold only for continuous signals For discrete signals, it is necessary to blurwith a modified Bessel function, which, for an infinitesimal pixel size, approachesthe Gaussian function

FIGURE 7.7 An MR brain image blurred at different scales (a) σ = 1, (b) σ = 4, (c) σ = 8, (d) σ = 16.

Koenderink [24] has also shown that the generation of the scale-space as defined

in Equation 7.6 can be viewed as solving the heat equation or diffusion equation

(7.7)

The conductance term c controls the rate of blurring at each time step If c is a

constant, the diffusion process is called linear diffusion, and the Gaussian kernel isthe Green’s function of Equation 7.7 In this case, the time parameter replaces the

scale parameter in Equation 7.6 with t = σ2/2c, given the initial condition L(x,0) = L(x) The diffusion flow is a local process, and its speed depends only on the intensity

difference between neighboring pixels and the conductance c The diffusion process reaches a state of equilibrium at t → ∞ when all pixels approach the same intensity value

7.3.3 S CALE -S PACE S AMPLING

The scale-space representation is a continuous representation In practice, however,

it is necessary to sample the scale-space at some discrete values of scale Anequidistant sampling of scale-space would violate the important property of scaleinvariance [30] The basic argument for scale invariance has been taken from physicsexpressing the independence of physical laws from the choice of fundamental param-eters This corresponds to what is known as dimensional analysis, which defines that

a function that relates physical observables must be independent of the choice ofdimensional units The only way to introduce a dimensionless parameter is by intro-ducing a logarithmic measure [30] Thus, the sampling should follow a linear anddimensionless scale parameter δτ, which is related to σ according to the following:

(7.8)

where n denotes the quantization level A convenient choice for τ0 is zero, whichimplies that the inner scale σ0 of the initial image is taken to be equal to the lineargrid measure ε At coarse scales, the ratio between successive scales will be aboutconstant, while at fine scales the differences between successive scales will beapproximately equal

7.3.4 M ULTISCALE I MAGE -S EGMENTATION S CHEMES

The concept of scale-space has numerous applications in image analysis For aconcise overview, the interested reader can refer to the literature [16] In this paper,scale-space theory concepts are used for image-segmentation purposes

7.3.4.1 Design Issues

For the implementation of a multiscale image-segmentation scheme, a number ofconsiderations must be kept in mind A general recipe for any multiscale segmen-tation algorithm consists of the following tasks:

1 Select a scale-space generator that will build the deep structure and

govern the simplification process for the image structure

2 Determine the linking scheme that will connect the components (features)

in the deep image structure Naturally, an immediate question arises aboutwhich features will be the ones that will be linked The answer is one ofthe components that are apparent for the linking-scheme description Theother components are the rules that will steer the linking and the optionsthat will be examined for the linkages (bottom-up, top-down, doublylinked lists)

3 Attribute a significance measure of the scale-space segment This implies

that a valuation has to be introduced at the localization level, for eitherthe region or the segmentation contours, by retrieving information fromtheir scale-space hierarchical tree

All the above considerations have been combined in different ways and leddifferent authors to advocate their own multiscale segmentation scheme In thefollowing section, the state of the art is presented

7.3.4.2 The State of the Art

In Lifshitz and Pizer’s work [36], a multiscale “stack” representation was constructedconsidering isointensity paths in scale-space The gray level at which an extremumdisappears is used to define a region in the original image by local thresholding onthat gray level The same authors observed that this approach can be used to meet

the serious problem of noncontainment This problem refers to the case that a point,

which at one scale has been classified as belonging to a certain region (associated

to a local maximum), can escape from that region when the scale parameter increases.Moreover, the followed isointensity paths can be intertwined in a rather complicatedway

Lindeberg [37] has based his approach on formal scale-space theory to construct

his scale-space primal sketch This representation is achieved by applying a linking

among light or dark blobs Because a blob corresponds to an extremum, he usedcatastrophe theory to describe the proposed linking as interactions between saddlesand extremum To attribute a significance measure for the scale-space blob, heconsidered three features: spatial extent, contrast, and lifetime in the scale-space.Correspondence between two blobs of consecutive scale is attributed by measuringthe degree of overlap

Multiscale segmentation of unsharp blobs has also been reported by Gerig et al.[38] They applied Euclidean shortening flow, which progressively smoothes thelevel curves and lets them converge to circles before they disappear at singularitypoints Object detection is interleaved with shape computation by analyzing thecontinuous extremum paths of singularities in scale-space Assuming radially sym-metric structures, the singularity traces are attributed to the evolution time.Eberly [39] constructed a hierarchy based on annihilation of ridges in scale-space He segmented each level of the scale-space by decomposing the ridges into

curvilinear segments, followed by labeling Using a ridge flow model, he made aone-to-one correspondence of each ridge segment to a region At each pixel in theimage, the flow line is followed until the flow line intersects a ridge Every pixelalong the path is assigned the label of the terminal ridge point The links at thehierarchical tree are inserted, based on how primitive regions at one scale becomeblurred into single regions at the next scale The latter single primitive region isconsidered to be the parent of the original two regions because it overlaps those tworegions more than any other region at the current scale.

The segmentation scheme of Vincken [40, 41] and Koster [42] is based on thehyperstack, which is a generalization to 3-D of the stack proposed by Koenderink[24] Between voxels at adjacent scale levels, child-parent linkages are establishedaccording to a measure of affection [42] This measure is a weighted sum of differentmeasures such as intensity difference, ground volume size, and ground volume meanintensity A ground volume is the finest-scale slice of a 4-D scale-space segment.This linking-model-based segmentation scheme has been applied not only for thelinear scale-space, but experiments have also been reported [43] for gradient-depen-dent diffusion and Euclidean shortening flow Vincken et al [40, 41] used thehyperstack in combination with a probabilistic linking, wherein a child voxel can

be linked to more than one parent voxel The multiparent linkage structure istranslated into a list of probabilities that also indicate the partial-volume voxels and

to which extent these voxels belong to the partial-volume class of voxels Thus, anexplicit solution for the treatment of partial-volume effects caused by the limitedresolution, due to the acquisition method and leading to multiple object voxels, isproposed

Using linear-scale evolution of gray-scale images, Kalitzin et al [44] proposed

a hierarchical segmentation scheme where, for each scale, segments are generated

as Voronoi diagrams, with a distance measure defined on the image landscape Theset of centers of the Voronoi cells is the set of the local extrema of the image Thisset is localized by using the winding number distribution of the gradient-vector field.The winding number represents the number of times that the normalized gradientturns around its origin, as a test point circles around a given contour The process

is naturally described in terms of singularity catastrophes within the smooth scaleevolution In short, this approach is a purely topological segmentation procedure,based on singular isophotes

Griffin et al [45] proposed a multiscale n-ary hierarchy The basic idea is to

create a hierarchical description for each scale and then link these hierarchies acrossscale In a hierarchical description of the structure, the segments are ordered in atree structure A segment is either the sum of its subobjects or a single pixel Thishierarchy is built by iteratively merging adjacent objects The order of merging isbased on an edge-strength measure that combines pure edge strength along withperceptual significance of the edge, determined by the angle of the edge trajectory Thelinking of the hierarchies proceeds from coarse to fine scales and from the top of thehierarchies to the bottom First, the roots in the hierarchies are linked, then the subobjects

of the roots are matched, etc This results in a multiscale n-ary hierarchy.

The multiscale segmentation framework presented in this chapter deals with regionsproduced after the application of the watershed transformation and its subsequent

tracking in scale-space In a similar spirit, other authors have produced works inthis field:

Jackway [46] applied morphological scale-space theory to control the number

of extrema in the image, and by subsequent homotopy-linking of the dient extrema to the image extrema, he obtained a scale-space segmentationvia the gradient-watershed transformation [46] In this case, the watershedarcs that are created at different scales move spatially and are not a subset

gra-of those at zero scale

Gauch and Pizer [5] presented an association of scale with watershed aries after a gradual blurring with a series of Gaussians When an intensityminimum annihilates into a saddle point, the water that drains towards theannihilated minimum now drains to some other intensity minimum in theimage This defines the parent-child relationship between these two water-shed regions By continuing this process for all intensity minima in theimage, a hierarchy of watershed regions is defined

bound-Olsen [6] analyzed the deep structure of segmentation using catastrophetheory In this way, he advocated a correspondence between regions pro-duced by the gradient watersheds at different scales

7.4 THE HIERARCHICAL SEGMENTATION SCHEME

The relationship between watershed analysis and scale-space can be attributed tothe simplification process that is offered by the scale-space On the one hand, adecreasing number of singularities occur during an increasing smoothing of theimage On the other hand, the duality of the watershed segments increases with theirrespective minima in the gradient image Both contribute to an attractive frameworkfor the examination of a merging process in a segmentation task A detailed expla-nation of this relationship, along with the produced results, will be given in thefollowing

7.4.1 G RADIENT M AGNITUDE E VOLUTION

As discussed in Section 7.3.4.1, when we think about the implementation of amultiscale segmentation scheme, certain considerations have to draw our attention.The very first consideration is the selection of the image-evolution scheme In thiswork, we have studied the gradient-magnitude evolution The basic motivation isthat treating a problem of an uncommitted front-end, contrast and scale are the onlyuseful information Gradient magnitude provides the contrast information, and scale

is inherent to the evolution itself

During the image evolution according to the heat equation L t = L n, the squared image follows an evolution according to the following:

Using a Cartesian coordinate system

7.4.2 W ATERSHED L INES DURING G RADIENT M AGNITUDE E VOLUTION

The second consideration (see Section 7.3.4.1) for building up a multiscale tation scheme is the determination of the linking scheme for the selected features

segmen-in the deep image structure In a watershed-analysis framework, the selected featuresare the regions that are produced by the gradient watersheds, each of them corre-sponding to a singularity (regional minimum) of the gradient-magnitude image.Because the proposed segmentation scheme relies on the behavior of singularities

in time, we have used catastrophe theory to study an explicit classification of thetopological changes that occur during evolution and to explain their linking in scale-space In this study, we have drawn the conclusion that two types of catastrophes(fold and cusp) occur during the gradient-magnitude evolution The detailed alge-braic analysis can be found in a work by Pratikakis [47]

Using the duality between the regional minima and the regions produced by thegradient watersheds, we can observe how the watershed lines behave during thisevolution Figure 7.8,Figure 7.9, and Figure 7.10 give a clear insight of this behavior.Looking at Figure 7.9, we can observe both catastrophe types The fold catastrophe

is perceived as an annihilation of the regional minimum, and the cusp catastrophe

is perceived as merging between two regional minima to give one minimum Thisbehavior is reflected on the watershed-line construction by an annihilation of water-shed-line segments Obviously, this demonstrates why the placement of watershed

analysis into a scale-space framework makes it an attractive merging process ertheless, there is a major pitfall In Figure 7.10, it is clearly evident that, duringthe evolution of the gradient magnitude, the watershed lines become increasinglydelocalized This situation does not permit us to have a merging process by onlyconsidering the critical-point events and retrieving the produced segments at thedesired scale This also explains why the deep image structure has to be viewed asone body and not as a collection of separated scaled versions of the image understudy To achieve a single-body description of the deep image structure, we need tolink (connect) all the components or features of this structure For segmentationpurposes, this linking is useful because it guides us to achieve a segmentation at thelocalization scale This is feasible by describing all the spatial changes and interac-tions of the singularities that also influence the saliency measure of the localizedwatershed segments The next section of this chapter provides a detailed description

Nev-of the proposed linking scheme

FIGURE 7.8 Successive blurring of the original image.

7.4.3 L INKING ACROSS S CALES

We have already explained that interaction between singularities during the tude-gradient evolution is attached to behaviors of either a fold or a cusp catastrophe.The critical points disappear with increasing scale, and this event is the generic way

magni-in which it happens The term generic means that if the image is changed slightly,

the event may change position in scale-space, but it will still be present Apart fromthe disappearance, another event is also generic This is the appearance of two criticalpoints [36, 48, 49] In a more detailed way, the generic events of the minima in thegradient magnitude are as follows:

No interaction with other singularities (Figure 7.11a)

Creation in a pair with a saddle (Figure 7.11b)

Splitting into saddle and two minima (Figure 7.11c)

FIGURE 7.9 Behavior of the regional minima during the gradient-magnitude evolution.

Annihilation with a saddle (Figure 7.11b)

Merging with a saddle and another minimum into one minimum (Figure7.11c)

In Figure 7.11, all the generic events are schematically described, using brokenlines to indicate linking between the minima of two adjacent regions in scale-space

As scale increases from bottom to top, one can observe how interactions betweencritical points can lead to merging of two adjacent regions due to the underlyingone-to-one correspondence between a minimum and a region

Linking of the minima (parent–child relationship) for successive levels is applied

by using the proximity criterion [24] This criterion checks the relative distance forall the minima at scale σi that have been projected on the same influence zone

IZA (B j)i+1 at scale σi+1 with respect to the original minimum of this influence zone

An example can be seen in Figure 7.12, which represents the linking for two

FIGURE 7.10 Watershed segment merging and delocalization of the watershed lines during

the gradient-magnitude evolution.

FIGURE 7.11 (Color figure follows p 274.) Generic events for gradient-magnitude

successive levels of the evolution example that is depicted in Figure 7.8,Figure 7.9,and Figure 7.10.Figure 7.12a shows the regional minima at scale σi that have beenspatially projected onto level σi+1 The watershed lines at level σi+1 are also shown,and these delimit the influence zones at this level The regional minima at scale σi+1

can be seen in Figure 7.12b

For the sake of clarity, for each regional minimum in Figure 7.12a and Figure7.12b, there is a marker of different shape and gray value that makes them distinct

A linking for the minima (m j)σi at scale σi and the minima at scale σi+1

appears in Figure 7.12c After the linking stage, we have for each quantization scalelevel a labeling for the minima with respect to their linking ability These labels are

of two types Either the minimum is annihilated/merged and will not be considered

in the next levels, or the minimum does not interact with other singularities andtakes up the role of the father label for all the minima that were annihilated ormerged and were situated at the same influence zone This labeling is guided by theproximity criterion The projected minimum (∈ IZA (B p)i+1), which is closest

to the minimum , is considered the father, and the rest of the projected minimaonto the same influence zone are considered annihilated Closeness is defined withrespect to the topographic distance (see Section 7.2.1.1), which is a natural distancemeasure following the steepest gradient path inside the catchment basin From theimplementation point of view, we have to mention that we use ordered queues to guide toward In that way, we avoid problems caused by the presence of plateaus.Being consistent with the theory, we have to keep in mind that a generic event

in gradient-magnitude evolution is also the creation/splitting of minima In practice,this event can be understood as an increasing of minima in successive levels in theevolution Due to the quantization of scale, such an increase in the amount of minimararely occurs, and even if it happens, its scale-space lifetime is very short Thismotivated us to keep the same linking scheme for this event, too In the case that acreation contributes to an increasing amount of minima, then linking is done withthe closest minimum of the two new ones, while the other is ignored

The proposed linking scheme is a single-parent scheme that links the regionalminima and their respective dual watershed regions in successive evolution levels.This region-based approach is chosen to avoid problems of a pixel-based linkingcaused by the noncontainment problem (see Section 7.3.4) An additional advantage

of a region-based approach, specifically when a watershed-analysis framework isused, is the inherent definition of a search area for the linking process, namely theinfluence zones, that otherwise, in a pixel-based approach, has to be defined in an

ad hoc manner.

The aim of the proposed linking is to determine which contours (common

borders) can be regarded as significant, without any a priori information about scale,

spatial location, or the shape of primitives

7.4.4 G RADIENT W ATERSHEDS AND H IERARCHICAL S EGMENTATION

mσi m′σi+1

hierarchy is only based on scale At this point, we go toward the description of how

to enrich this hierarchy and make it more consistent Consistency will be obtainedbecause the hierarchy is based on more features than only scale A hierarchicalsegmentation of an image is a tree structure by inclusion of connected image regions.The construction of the tree structure follows a model consisting of two modules.The first module is dedicated to evaluate each contour arc with a salient measure,and the second module identifies the different hierarchical levels by using a stoppingcriterion A schematic diagram can be seen in Figure 7.13 As mentioned in Section7.3.4.1, the third consideration for constructing a multiscale segmentation scheme

is the significance measure The following subsection explains this measure and how

we attribute it to the watershed segments at the localized scale

7.4.5 T HE S ALIENT -M EASURE M ODULE

7.4.5.1 Watershed Valuation in the Superficial

Structure-Dynamics of Contours

The principle of dynamics of contours [3] uses the principle of dynamics of minima[2] as an initial information for the common contour valuation of adjacent regions(see Figure 7.14) The additional information that is used is based on the tracking

of the flooding history In such a way, a contour valuation can be found by thecomparison of dynamics between the regions that have reached the contour of interest

during a flooding The dynamics of a minimum m1 is easily defined with a flooding

scenario Let h be the altitude of the flood when, for the first time, a catchment basin with a deeper minimum m2 (m2 < m1) is reached The dynamics of m1 is then simply

equal to h-altitude (m1) Each catchment basin is attributed the value of the dynamics

of its minimum The contour valuation that is attributed to each common border ofthe regions at a certain scale σ = σ0 (superficial structure) is denoted as

FIGURE 7.13 Dynamics of contours in scale-space algorithm.

Gaussian

blurring

Child-Parent Linking Dynamics of minima

Adjacent regions

Down Projection

&

Contour valuation No

Yes

Null Hypoth- esis i

END

No

Hierarchical Segmented Levels HLi

STOPPING CRITERION MODULE

SALIENT MEASURE MODULE

i+1

of noise sensitivity Experimental results have been reported in the literature [50].Motivated by the shortcoming in noise sensitivity, it came as a natural consequence

to obtain a contour valuation by considering the behavior of the catchment basins

in scale-space

7.4.5.2 Dynamics of Gradient Watersheds in Scale-Space

Once the parent–child linkages have been completed, the next step is to valuate thegradient watersheds at the localization scale σ0 Let us assume that we want to

valuate the gradient watershed that separates regions A and B (see Figure 7.15).From the linking step, we have created a linkage list, Λ(m,n) (see Figure 7.15a), where m and n denote the regions at the localization scale.

In our example, m = A and n = B This list provides the following information: region F is attributed to a root for regions A and B at scale S4 This has occurred

because (a) region A has been linked to region D at scale S2, and region D has been

FIGURE 7.14 Flooding list for point P, the basis for computing dynamics of contours that

3 2