Analysis of 3 d maxillofacial image data 3

Dental Cast Generation of Plan-view Range Image 3.2.3 Generation of Panoramic Range Image 3.3.3 First Tooth-interstice Detection 3.4 Dental Arch Detection 3.3.2 Ridge Detection 3.3.1 Sec

3 The outer surface of the teeth, i.e., tooth surfaces that are next to the cheeks (opp lingual)

from the gums in the panoramic range image (Section 3.7) A validation test confirms that the method works well with dental casts representing a variety of malocclusions

Dental Cast

Generation of Plan-view Range Image (3.2.3)

Generation of Panoramic Range Image (3.3.3)

First Tooth-interstice Detection (3.4)

Dental Arch Detection

(3.3.2) Ridge Detection (3.3.1)

Second Tooth-interstice Detection (3.5)

Integration (3.6)

Segmented Teeth

Alignment (3.2.3)

Tooth&Gum Separation

(3.7) Digitization (3.2.1)

Figure 3.1: Flow chart giving an overview of the segmentation procedure

3.2 Data Acquisition and Processing

3.2.1 Laser Scanner

A variety of methods for acquiring digital data of the dentition have been reported in the literature The 3-D profile of wax imprints can be obtained using a depth-from-absorption technique [4]−[6] Range scanners have been used to scan dental study models for display and storage purposes [60]−[62] A different approach that reconstructs the 3-D model of the patient’s dental occlusion by integrating a sequence

of images captured by an intraoral video camera is reported by S.M Yamany et al in [63]

We digitize the dental study models with a commercially available laser scanner (Cyberware 3030R/HIREZ/MM [1]) that is particularly suited to digitizing small, highly detailed objects where resolution and accuracy are important factors (Figure 3.2) A vertical stripe of laser light is projected onto the plaster cast that is placed on the motion platform As the object moves laterally, it is scanned by the stationary laser beam, and this is repeated for different rotated positions Range information is computed from each scan based on triangulation The bundled software produces the complete 3-D representation of the model by merging the range data from different scans Spatial resolution is 0.15~1.00 mm horizontally and 0.30 mm vertically Depth resolution depends on the surface quality of the object, but is typically 0.05~0.20 mm

Figure 3.2: Laser scanner (Cyberware 3030R/HIREZ/MM Model)

3.2.2 Data Format

Figure 3.3 shows a tooth digitized by the laser scanner The surface is represented with numerous small triangular patches We save the 3-D image data as a VRML4 file, in which an object is described as a list of vertices and a list of face indices (Figure 3.4) The list of vertices contains the 3-D coordinates of each vertex, while the list of face indices defines which three vertices form a single triangular patch

4 VRML stands for “Virtual Reality Modeling Language” and is a file format for describing 3-D objects and worlds interactively

Figure 3.3: Tooth surface digitized by the Cyberware scanner

.

The first face is drawn

by connecting vertices

1, 2, and 3.

Figure 3.4: VRML format

While the VRML format is adequate for visualization and data storage, it is not suitable for analysis because of its irregular 3-D structure There is also the inherent added complexity of working in the 3-D domain compared to 2-D Thus, instead of

processing the VRML data directly, we devise algorithms that are applied to the range

images (or depth maps) generated from the 3-D data Since the range image is a 2-D matrix (though it has 3-D information encoded in the intensity values), the complex 3-

D tooth segmentation problem is significantly simplified For instance, the regular structure of the range image makes mathematical operations such as convolution readily applicable We employ both plan-view and panoramic range images to provide the 3-D information necessary for tooth segmentation It should also be noted that the image size is about 100 KB, which contrasts with 10 MB for the original mesh data

3.2.3 Plan-view Range Image

In preparation for computing the plan-view range image, the digitized dental model has to be aligned to a standard orientation We first specify the occlusal plane (the imaginary surface at which the upper and lower teeth touch) by manually identifying four reference points (Figure 3.5): the disto5-buccal cusps of the first molars (labeled 1 and 4) and the buccal cusps of the first premolars (2 and 3) A reference point need not

be precisely located as the program searches for the highest point in the vicinity of the manually selected point The cast is aligned such that the occlusal plane is parallel to

the x-y plane and the two disto-buccal cusps have the same y-coordinates

5 Posterior, i.e towards the back of the mouth (opp mesio-)

Figure 3.5: Digitized dental cast after alignment and four reference points (1, 4: buccal cusps of the first molars, 2, 3: buccal cusps of the first premolars)

disto-A plan-view range image is obtained by mapping the vertices of the 3-D data onto a

regularly spaced 2-D array that is parallel to the x-y plane (Figure 3.6) The x and y

coordinates of each vertex determine the location of the corresponding pixel in the 2-D array, whereas the z coordinate of the vertex is used for computing the height of the vertex from the 2-D array, i.e., the reference surface When two or more vertices are mapped onto one pixel in the 2-D array, the highest vertex is selected because it is the only point visible in the plan view The computed heights are saved as the pixel values (gray levels) of the 2-D array

Figure 3.7 shows a generated plan-view range image, comprising 300 by 300 pixels with each pixel corresponding to a spatial separation of 1/3 mm The highest 15-mm of the 3-D data is quantized into 256 gray levels (i.e., 8-bit unsigned char), giving a height resolution of 0.059 mm per gray level It is pertinent to note that the spatial and

height resolution of the range image are comparable to those of the scanner, thus ensuring the preservation of fine details in the conversion process

3-D model

Regularly spaced 2-D array

Surface of a 3-D model

x y

Figure 3.6: Mapping of the vertices onto a 2-D array

Figure 3.7: Plan-view range image of the digitized dental cast

3.3 Determination of the Dental Arch

In orthodontics, a complete description of the form of the dental arch is essential whenever changes due to growth or orthodontic therapies are under investigation [19] Orthodontists need to know the maxillary (upper jawbone) and mandibular (lower jawbone) arches for assessing the degree of malocclusion [6] Apart from its clinical use, the dental arch plays an important role in our tooth-segmentation algorithm Current computer-assisted methods of determining the arch form require the user to manually select several feature points in the cast image [2] or interactively define the arch [65], [66] We propose to improve on this with an automated method to determine the tooth-based dental arch

3.3.1 Ridge Detection

From the plan-view range image, we extract orthodontic features such as the incisal edges and cusps These features appear as local peaks that form ridges (roof edges) Existing detection methods [38], [67] are greatly dependent on the gradient of the ridge and may fail if the slope is gentle, which is often the case with the cusps of the posterior (back) teeth We use a new approach, GOA, to detect both gentle and sharp ridges Discontinuities in gradient orientation will indicate the presences of local peaks

Let h ( y x, ) be a plan-view range image The gradient of h ( y x, ) is written as

x y x h y

x q

y x p y x h

/),(

/),()

,(

),(),( (3.1)

The gradient orientation at a point ( y x, ) is defined as

0),(),(/),(tan)

,(

1

y x q y

x p

y x q y

x p

y x p y x p y x q y

x y x y

x s

y x s y x

y

x

/),(sin

/),(sin)

,(

),()

,(sin

x y x y

x c

y x c y x

y

x

/),(cos

/),(cos)

,(

),()

,(cos

,(x y s2 x y s2 x y c2 x y c2 x y

0),(sgn),(),(

2 1

y x h y

x D y x

),(128),()

,

2

y x h y

x D y x

where 128 is an arbitrary threshold on a scale of 0 to 255 We obtain a binary feature map by thresholding M ( y x, ) D2(x,y) with a threshold T given by

σ

µ+

=

T , (3.8) where µ and σ denote the mean and the standard deviation of the entire area of

, respectively Finally, we remove isolated points in as

M′( , )= ( , )o , (3.9) where denotes the opening operation (erosion followed by dilation) and the

structuring element B is a 3 by 3 matrix of ones Figure 3.8(a) shows the ridges

extracted from the plan-view range image of Figure 3.7

o

It should be noted that, unlike standard methods of detecting ridges, we use neither the magnitude of the gradient nor the surface normal directly GOA is capable of detecting ridges irrespective of their gradient magnitude because it focuses solely on the detection of discontinuities in gradient orientation

3.3.2 Two-step Curve Fitting

The extracted ridge pixels may be used to provide only a crude form of the dental arch

It is necessary to extract and use the orthodontic features that clinically define the dental arch; these are the incisal edges, tips of the canines, and buccal cusps of the posterior teeth Our procedure employs a two-step curve fitting technique that ensures both stability and flexibility

In the first step, we apply curve fitting using a third-order polynomial that is robust to over-fitting and yet is sufficiently flexible to describe the crude form of the arch

(Figure 3.8(a)) The best-fitting curve is obtained from weighted least-squares fitting that minimizes the error term, ε :

w , (3.11)

where corresponds to the x coordinate of the central pixel in the series of local

peaks along the dental arch We next proceed to set up one-pixel-width inspection spokes that are oriented perpendicular to the detected arch and search for the pixel with the largest intensity along each spoke (Figure 3.8(b)) The spoke interval is currently set at 1.5 pixels (equivalent to 0.5 mm), which is small enough so as not to miss the orthodontic features The magenta dots in Figure 3.8(c) show the detected local peaks that include the orthodontic features necessary for determining the final dental arch

m

x

In the second step, we find the curve that fits these local peaks by using the weighted least-squares fitting described above with a fourth-order polynomial This curve is flexible enough to describe the dental arch (Figure 3.8(d)) and is in fact commonly used by orthodontists [68], [69] Unlike symmetrical curves such as the parabola, ellipse or catenary (hyperbolic cosine), higher order polynomials can describe asymmetric curves and do not introduce a bias in the assessment of the dental arches Figure 3.9 shows various forms of dental arches determined by the proposed method

(a) (b)

Figure 3.8: Two-step curve fitting technique for determining the dental arch (a) First curve fitting to the extracted ridge pixels (b) Inspection spokes (c) Detected local peaks (d) Second curve fitting to the local peaks (the dental arch)

(a) (b)

Figure 3.9: Various forms of dental arches (a) Malaligned upper teeth with outwardly inclined incisors (b) Severely malaligned upper teeth (c) Malaligned lower teeth (d) Asymmetrically aligned lower teeth

3.3.3 Panoramic Range Image

We generate a panoramic range image by computing the distance between the buccal surface of the teeth and the reference surface that is defined by extending the detected dental arch (described as a fourth-order polynomial) in the direction perpendicular to

the x-y plane Hence, the reference surface is written as

e dx cx bx ax

y = 4 + 3+ 2 + + , (3.12)

where a, b, c, d, and e are the coefficients of the polynomial The plane that is normal

to the reference surface at any point (x0,y0) on the arch is then given by

−

=+

0

2 0

3 0

0 0

2 0

ax

x x

d cx bx

ax B

Ax

The points of intersection of this plane with the buccal surface of a tooth are obtained by taking those vertices (sample points) that are located within a short distance from the plane This is necessary since the vertices may not lie exactly on the plane

),,(x y z

The distance from the selected points to the reference surface are subsequently computed and quantized to 256 gray levels to produce the range image Figure 3.10 shows a panoramic range image of size 51 by 245 pixels The pixel size is 0.3 mm vertically and 0.5 mm horizontally, with one gray level corresponding approximately

to 0.06 mm It is clear that the process of generating the range image does not cause any significant information loss

Figure 3.10: Panoramic range image of the digitized dental cast

3.4 Tooth-interstice Detection in the Plan-view Range

Image

Tooth segmentation begins with the detection of the interstices between the teeth in the

plan-view and panoramic range images The two results are subsequently combined to obtain the positions and the orientations of the interstices A complete segmentation also requires the teeth to be separated from the gums

3.4.1 Detection of Tooth-interstice Orientations

Tooth interstices, which appear as straight grooves or valleys in the plan-view range image, intersect the dental arch at approximately 90 degrees In earlier work [4], [5], the assumed dental arch is the curved axis of the dental wax imprint, which is not directly related to the arrangement of the teeth This does not matter if the teeth are well aligned However, since we aim to handle dental casts with a variety of malocclusions, it would be more appropriate to use the tooth-based dental arch (previous section) We set up inspection spokes along the detected arch and perpendicular to it, and rotate them about their intersections with the arch (Figure 3.11(a)) Two height profiles along the spokes before and after rotation are shown in Figure 3.11(b) When an inspection spoke is correctly positioned and oriented with the tooth interstice, the largest height value of the spoke will be smaller than those of other positions and orientations

Lingual side

Buccal side

Lingual side (before rotation) (after rotation)

(b)

Figure 3.11: Detection of tooth-interstice orientations (a) An inspection spoke and its rotation about the intersection with the dental arch (b) Cross-sectional profiles along the inspection spoke at a right orientation (before rotation) and at a wrong orientation (after rotation)

The detailed procedure for detecting the tooth-interstice orientations is as follows:

1 Set up inspection spokes perpendicular to the dental arch with the spoke interval equal to the spatial resolution of the scanner

2 Find the depth value of the highest pixel of the ith inspection spoke, i.e., h0 ,i

3 Rotate the ith inspection spoke by ± 10°, ±20°, ±30° about its intersection with the arch

4 Find the depth value of the highest pixel of each orientation as well, i.e., ,

i

h10 ,i

h−10 , h20 ,i h−20 ,i h30 ,i h−30 ,i

5 Find the best orientation of the ith spoke by selecting the smallest depth value

from the seven values and store both the depth value and the selected orientation:

.30 ,20 ,10 ,0fromselectedangle

The)(

, ,

, , , , ,min)(

2

, 30 , 30 , 20 , 20 , 10 , 10 , 0 1

o o o

=

i g

h h h h h h h i

6 Repeat steps 2 to 5 for all the inspection spokes

In this way, the best tooth-interstice orientation at each sample point is stored in g2(i)

3.4.2 Detection of Tooth-interstice Positions

The values show the depth profile along the dental arch (Figure 3.12) We would like to detect valleys that appear periodically in the graph since they correspond to the tooth interstices Therefore, the detection of tooth interstice positions is reduced to the task of finding significant valleys in a plot of these height values against position We refer to this plot as the depth graph

1 i g

Figure 3.12: Depth profile along the dental arch (the depth graph), g1(i)