Analysis of 3 d maxillofacial image data 2

It is particularly important in 3-D data analysis to detect image features such as step or jump edges and crease or fold edges because they often correspond to boundaries of objects.. Fi

Chapter 2

Feature Detection

2.1 Introduction

Feature detection is an essential early step in image processing and analysis algorithms

It is particularly important in 3-D data analysis to detect image features such as step (or jump) edges and crease (or fold) edges because they often correspond to boundaries of objects 3-D data is commonly provided as a range image (or a depth map) Step edges correspond to pixels where the depth values are discontinuous, while crease edges are where the surface normals are discontinuous It should be noted that crease edges include both ridge and valley edges It is straightforward to detect step edges in a range image because a standard gradient-based operator may be applied, as

in the case of intensity images However, a special operator needs to be designed for detecting crease edges since the depth discontinuities in their vicinity are slight In this chapter, we study methods for detecting crease edges that can be employed for analyzing maxillofacial image data We conduct comparative studies of existing methods and propose a novel technique for crease edge detection Subsequently, we apply the technique to the extraction of tubular structures in 3-D space and compare with eigenvalue analysis

Various approaches have been proposed for the detection of crease edges, such as (a) residual analysis [29]−[31], (b) contour map analysis [32], [33], (c) morphological methods [34]−[37], (d) surface-fitting methods [38], [39], (e) first-derivative-based

methods [40], [41], (f) second-derivative-based methods [42], [43], and (g) surface normal analysis (SNA) [44]−[47] Residual analysis uses the difference between an input image and its smoothed image to detect edge points [29] This is essentially equivalent to the use of the high-frequency component of an input image Hence, the method will respond strongly to step edges, but not to crease edges In fact, this technique was originally devised for detecting step edges [31], and may not be well suited to crease edge detection The next three methods (b)−(d) are designed specifically for detecting crease edges However, since they are dependent on the shape and direction of a crease edge, their applications are limited First-derivative-based methods detect crease edges by finding a pixel whose first derivative is zero or nearly zero We examine this approach using synthetic and real range images and demonstrate its inherent problem in Section 2.2.1 Second-derivative-based methods detect crease edges by searching for the local maximum of the second derivative of an input image We study this approach using the same test images and summarize its properties in Section 2.2.2 SNA is a direct method for detecting crease edges by finding discontinuities in the surface normal We compute the discontinuities in the surface normal using a simple linear operation and describe its performance in Section 2.2.3

In addition to studying existing methods, we propose a novel method for detecting crease edges in an image in Section 2.2.4 The method, which we call GOA, detects crease edges by finding discontinuities in gradient orientation and is closely related to SNA The difference between the two methods is that GOA is based on gradient orientation whereas SNA is based on the surface normal We compare these two

methods in Section 2.2.5 Both GOA and SNA are successfully used for detecting crease edges in Chapter 3

Up to now, we have dealt with range images that contain 3-D information of an object, but in fact, they are only 2-D matrices from a mathematical point of view In medical image data analysis, however, 3-D volumetric data sets are recently becoming more common due to the increasing use of CT and magnetic resonance imaging (MRI) Feature detection in 3-D space can be approached from two aspects: the detection of the variations in voxel intensity and orientation The former can be achieved rather straightforwardly by extending an existing gradient-based operator to 3-D, for example,

as described in [48] Then, we may detect 3-D edges that are essential to extract features in 3-D space On the other hand, a well-known technique that uses gradient orientation is eigenvalue analysis [49]−[51] The first step of eigenvalue analysis is the construction of the local gradient structure tensor (GST), and the second step is principal component analysis (PCA) of the GST By analyzing the three eigenvalues obtained by PCA, the technique can extract a 3-D object of a particular structure, such

as, a blob, a line, and a plane We study the performance of eigenvalue analysis using synthetic volume data in Section 2.3.1

In Section 2.3.2, we apply GOA to a volumetric data set by extending the algorithm to 3-D Since GOA is capable of detecting the points where gradient orientations are discontinuous, it may be effective for detecting a 3-D line that is highly discontinuous

in orientation We compare the performance of 3-D GOA with that of eigenvalue analysis using synthetic volume data corrupted by Gaussian noise in Section 2.3.3 In

Chapter 4, 3-D GOA is effectively used for enhancing tubular structures in the jawbone

2.2 Crease Edge Detection

2.2.1 First-derivative-based Methods

The methods described here attempt to locate a crease point by searching for a pixel whose first derivative is zero or nearly zero The first derivative is computed in a direction that maximizes the second directional derivative The following is a brief description of Haralick’s method that is also known as the height condition [40]

The Hessian matrix H at a point in an image g may be expressed with the second

xy xx

g g

g g

and , be the corresponding eigenvectors A crease point is a pixel whose first derivative in the direction of is zero

1

v v2

1

v

Instead of the Hessian matrix, the use of the GST has been suggested [41] The GST

at a point in an image g is defined by the dyadic product of the gradient vector with

y

y x x

y x y

x

g g

g

g g g

g g g

at the same time

Figure 2.1: (a) A synthetic range image (b) Step edges (red) (c) Crease edges (blue)

Figure 2.2: (a) A real range image (b) Step edges (red) (c) Crease edges (blue)

For simplicity, the magnitudes of the first derivative of the images, 2 2

y

g

M = + , are computed using the Sobel operator We observe the points whose first derivatives are very small by changing the threshold from 4 to 7 (Figures 2.3(a)-(d)) and also from

10 to 40 (Figures 2.4(a)-(d)) White pixels indicate points where the magnitude of the first derivative is lower than the threshold values, and they are supposed to correspond

to crease edges In Figures 2.3(a) and 2.4(a), parts of the crease edges in the images are barely detected, while scattered false edges are also detected in the smooth surface areas The number of the false edges increases rapidly when the threshold values are further relaxed For instance, the occlusal (biting) surfaces of the posterior (back) teeth are almost all white and the dental features such as the cusps (high points or peaks) and fissures are not detected selectively (Figures 2.4(c), (d)) The backgrounds are also selected as crease points because they are completely flat and the magnitudes of the first derivative are zero The exclusion of the background, however, may be easily achieved by ignoring very dark regions in the range image

The local integration of gradient vectors when the Hessian matrix or GST is constructed is intended to reduce the over-detection of false edges at the cost of image details However, as long as searching for a pixel whose first derivative is zero or nearly zero is performed, many false edges will be inevitably detected in flat or very smooth areas in an image The selection of suitable threshold values for edge detection

is frequently problematic because of the conflicting requirements of minimizing the number of false edges detected while simultaneously minimizing the number of true edges missed [27] Therefore, the first-derivative approach is not suitable when the detection of smooth crease edges is concerned

Figure 2.3: Crease edges detected in a synthetic range image by a based method (a) M <4 (b) M <5 (c) M <6 (d) M <7

first-derivative-Figure 2.4: Crease edges detected in a real range image by a first-derivative-based method (a) M <10 (b) M <20 (c) M <30 (d) M <40

2.2.2 Second-derivative-based Methods

These methods attempt to detect crease edges using the second derivative of an image The second derivative can be obtained by applying the Laplacian operator to an image

or applying a first-derivative operator twice [42]

We summarize Khalifa’s approach, described in [42], here Given an input range image g ( y x, ), the gradient of the image ∇ is computed using the Sobel operator: g

T y x

T

g g y

g x

A first-derivative operator is again applied to each of the derivative images and

to obtain , , , and The magnitude of the second derivative and its direction are then given by

+

yx xx

yy xy

g g

g g

2

M

derivative has large values The second derivative of the image is computed using the 3×3 Laplacian operator L [78]:

2102

5.025.0

L (2.9)

Both step and crease edges can be seen in Figure 2.5(a) As the threshold value is raised from 2 to 8, however, the crease edges start to vanish, leaving only step edges (Figures 2.5(b)−(d)) This clearly shows that the magnitudes of step edges are greater than those of crease edges For the same reason, the method completely fails to detect crease edges in the real range images and the strong responses are seen only at the pixels where step edges are present (Figure 2.6) Therefore, in using the second derivative, we require an extra step for separating crease edges from step edges It is difficult to determine an appropriate threshold value because the magnitude of the second derivative of crease edges is not necessarily well separated from that of step edges Another potential problem is that computation of the second derivative is generally sensitive to noise It may be necessary to apply a low-pass filter to a noisy input image at the cost of image details The Laplacian of Gaussian (LoG) is a well-known solution to this problem [52]

2

M

The computation of the second-derivative-based methods is highly efficient especially when the Laplacian operator is used because it requires only one convolution mask In addition, it is easy to identify roof and valley edges with the sign of the second derivatives of the image, given by

ridge

if )sgn(

valleyif

)sgn(

(2.10)

The second derivative was successfully used for extracting the global structure of decorated characters in a recent work [43] In the proposed algorithm, input images were blurred to smoothen characters and make the ridges of the characters more conspicuous

To summarize, a second-derivative-based method may be an appropriate choice only if

it is not necessary to separate crease edges from step edges, or if there are no step edges in the given image

Figure 2.5: Crease edges detected in a synthetic range image by a second-derivative-

based method (a) M2 >2 (b) M2 >4 (c) M2 >6 (d) M2 >8

Figure 2.6: Crease edges detected in a real range image by a second-derivative-based method (a) M2 >4 (b) M2 >8 (c) M2 >12 (d) M2 >16

2.2.3 Surface Normal Analysis

A crease edge can be detected by finding a point where surface normals vary abruptly, which is the concept underlying surface normal analysis (SNA) We first derive the unit surface normal Denoting the unit surface normal at a point on an object by n , we express the unit surface normal using surface gradient components If

we move a small distance in the x direction, the change of height is

g ]

,0 ,1

y

g ] ,1 ,0

,(10)

,(0

1

y x g

y x g y

x g y x g

y x

y x

),()

,(),(1

1)

,(

),(

),()

,(

2

y x g y x g y x g y

x n

y x n

y x n y

x

x x

z y

Discontinuities in will indicate the presence of crease edges in an image Yokoya et

al define the crease edge magnitude

n

R by the maximum angular difference between

adjacent unit surface normals within a 3×3 mask [44], [46]:

max),(x y = −1 x y ⋅ x+k y+l − ≤k l≤

Morphological gradient operators are also used to evaluate the discontinuities in n

[45], while the wavelet transform is used for the same purpose [47] We evaluate the crease edge magnitude more straightforwardly by computing the first derivative of

x y x n y x d

y y x n y x d

x y x n y x d

y y x n y x d

x y x n y x d

z zy

z zx

y yy

y yx

x xy

x xx

),()

,(

),()

,(

),()

,(

),()

,(

),()

,(

),()

,(

(2.14)

The crease edge magnitude is then expressed by combining these components:

2 2 2 2 2 2

),(x y d xx d xy d yx d yy d zx d zy

The Sobel operator is again used as the first derivative operator because of its computational efficiency and smoothing effect The experimental results show that the crease edges of the test images are clearly highlighted; see Figures 2.7 and 2.8 that display the values of For instance, as shown in Figure 2.8, the important dental features such as the cusps and fissures are effectively extracted by the method The result shows high contrast, like a binary image, which indicates the ease in selecting a proper threshold value for extracting edge points Parts of the step edges are also detected because they also exhibit discontinuities in surface normals However, unlike the second-derivative-based method that responds more strongly to step edges, the step edges detected by SNA are less dominant SNA is fairly sensitive to crease edges and detects few false edges in smooth surfaces This technique is currently one of the most frequently used methods for detecting crease edges [44]−[47]

2

S

Figure 2.7: Crease edges detected in a synthetic range image by SNA

Figure 2.8: Crease edges detected in a real range image by SNA

2.2.4 Gradient Orientation Analysis (2-D)

The gradient magnitude M = g2x +g2y (Eq (2.5)) serves as a direct measure for detecting step edges in an image Therefore, most edge detection methods found in the literature make use of it In contrast, it is not so common to use the gradient direction

an image [28] Both methods are based on the detection of symmetry in gradient orientations It is possible to extend the idea to detect crease edges in an image Here

we describe a method that can detect crease edges in a range image by finding

discontinuities in gradient orientation We name this approach gradient orientation analysis (GOA)

Figure 2.9 illustrates the concept of GOA using the range image in Figure 2.1(a) The portion of the image enclosed by the square box (Figure 2.9(a)) is shown enlarged in Figure 2.9(b) Please note that the subimage contains both step and roof edges Figure 2.9(c) shows the gradient vectors of the subimage Prominent gradient vectors are seen where step edges are present It is clear that step edges can be detected by thresholding the magnitude of the gradient vectors Figure 2.9(d) depicts the orientations of the gradient vectors Since we are only interested in gradient orientation, all the vectors in (d) are assigned the same magnitude The roof edges that are parallel to the step edges are now clearly visible Thus, we will be able to detect crease edges by finding the discontinuities in gradient orientation

Figure 2.9: Concept of gradient orientation analysis (a) A synthetic range image (b)

An enlarged portion of the image (subimage) (c) Gradient vectors in the subimage (d) Gradient orientations in the subimage

Let us describe the procedure of GOA Following the previous notations, the gradient orientation at a point ( y x, ) in an image g can be rewritten as

0,02

0)

(tan)

,(

1

y x

y x

x x y

g g

g g

g g g y

x

π

π

The range of θ is (−π,π], and pixels for which both and are zero are ignored

in the subsequent steps because there is no gradient information Since crease edges are points in an image where gradient orientations change sharply, we obtain the discontinuities in gradient orientation by taking its first derivatives For this, we compute

x

g g y

)(sinθ

c

y

x s

s

y x y x

)(cos

)(cos)

(cos

)(sin

)(sin)

(sin

θ

θθ

θ

θθ

(2.17)

The magnitude of the discontinuities in gradient orientations may be evaluated by

2 2 2 2

),(x y s x s y c x c y

D = + + + (2.18) The first derivative of the gradient orientation is approximated using the Sobel operator Table 2.1 shows the values for various patterns of unit gradient vectors illustrated in Figure 2.10 If the gradient vectors in a small neighborhood point

in the same direction, becomes zero, regardless of its direction (Figures 2.10(a), (b)) When a crease edge is observed, has a value of 8 for the pattern

of Figure 2.10(c) and 8.49 for Figure 2.10(d) The latter response is slightly greater because the Sobel operator is slightly more sensitive to diagonal discontinuities than to horizontal and vertical discontinuities [59] When a peak (or pit) is observed,

),(x y D

),(x y D

),(x y D

),(x y D

takes the largest value, 9.66 (Figure 2.10(e)) It is interesting to note that takes

a smaller value for a random pattern as in Figure 2.10(f) because of the smoothing effect of the derivative operator It should be stressed that GOA responds strongly to discontinuous but highly structured patterns This feature is especially useful for the detection of crease edges

),(x y D

Table 2.1: Values of D(x,y) for the vector patterns of Figure 2.10

Vector Patterns (a) (b) (c) (d) (e) (f) Values of D(x,y) 0 0 8 8.49 9.66 1.23

Figures 2.11 and 2.12 show the crease edges detected in synthetic and real range images by GOA The results appear very similar to those of SNA except that the crease edges are slightly clearer than the step edges We will discuss the differences between SNA and GOA in the next section