volumetric reconstruction and real-time deformation modeling of biomedical images

Black color indicates attenuation projections with 200k incident photon count.. Images from the firstrow down are a simulated phantom, b backprojection, c F-MAP initialized to uniform im

VOLUMETRIC RECONSTRUCTION AND REAL-TIME DEFORMATION MODELING OF BIOMEDICAL

IMAGES

by Pei Chen

A dissertation submitted to the Faculty of the University of Delaware inpartial fulfillment of the requirements for the degree of Doctor of Philosophy inElectrical Engineering

Summer 2006

c

° 2006 Pei ChenAll Rights Reserved

UMI Number: 3220796

3220796 2006

Copyright 2006 by Chen, Pei

UMI Microform Copyright

All rights reserved This microform edition is protected against unauthorized copying under Title 17, United States Code.

ProQuest Information and Learning Company

300 North Zeeb Road P.O Box 1346 Ann Arbor, MI 48106-1346 All rights reserved.

by ProQuest Information and Learning Company

VOLUMETRIC RECONSTRUCTION AND REAL-TIME DEFORMATION MODELING OF BIOMEDICAL

IMAGES

by Pei Chen

Conrado M Gempesaw II, Ph.D

Vice Provost for Academic and International Programs

I certify that I have read this dissertation and that in my opinion it meetsthe academic and professional standard required by the University as adissertation for the degree of Doctor of Philosophy.

Signed:

Kenneth E Barner, Ph.D

Professor in charge of dissertation

I certify that I have read this dissertation and that in my opinion it meetsthe academic and professional standard required by the University as adissertation for the degree of Doctor of Philosophy

Signed:

Karl V Steiner, Ph.D

Professor in charge of dissertation

I certify that I have read this dissertation and that in my opinion it meetsthe academic and professional standard required by the University as adissertation for the degree of Doctor of Philosophy

Signed:

Charles Boncelet, Ph.D

Member of dissertation committee

I certify that I have read this dissertation and that in my opinion it meetsthe academic and professional standard required by the University as adissertation for the degree of Doctor of Philosophy

Signed:

Chandra Kambhamettu, Ph.D

Member of dissertation committee

Mr Simon Ellwanger, Mr Tuncer C Aysal, Mr Steve Krufka, and Dr PraveenThiagarajan; and the staff at the Department of Electical and Computer Engineeringand at the Delaware Biotechnology Institute.

I must also thank my wife, Yan Wen, without whom none of this would everhave been possible

The research presented in this dissertation is funded in part by the NationalScience Foundation under Grant HRDC9800175 and 9875658 and NIH-NCRR Grant

2 P20 RR 016472-04

TABLE OF CONTENTS

LIST OF FIGURES viii

LIST OF TABLES xvi

ABSTRACT xvii

Chapter 1 INTRODUCTION 1

1.1 Medical Image Acquisition Techniques 3

1.2 Medical Modeling and Simulation using Virtual Reality 7

1.3 Dissertation Organization 10

2 STATISTICAL RECONSTRUCTION FOR DIGITAL TOMOSYNTHESIS 12

2.1 Introduction 13

2.2 Statistical Tomosynthetic Reconstructions 16

2.2.1 Maximum A Posteriori (MAP) Reconstruction 18

2.2.2 Statistical Models 19

2.2.3 Iterative Optimization of 3D MAP Reconstruction 21

2.3 Multi-Resolution Statistical Tomosynthesis 23

2.4 Simulation Methods 28

2.5 Results and Discussions 31

2.5.1 Comparison of Reconstruction Images 32

2.5.2 Convergence of Log-Posteriori and NMSE 33

2.5.3 Experiments on Interpolation and Down-sampling Algorithms 41

2.6 Reconstructions Using Real X-Ray Image Data 45

2.6.1 The X-Ray Imaging System 45

2.6.2 The Reconstruction Process and Results 48

2.7 Conclusion 48

3 REAL-TIME DEFORMATION MODELING FOR VIRTUAL SURGERY SIMULATION 51

3.1 Introduction 52

3.1.1 Non-physical modeling 53

3.1.2 Physical modeling 53

3.1.2.1 Finite element method 54

3.1.2.2 Mass-spring model 55

3.2 Mass-Spring Deformable Model 59

3.3 Adaptive Spring Constants 67

3.4 Model Convergence And Stability 82

3.5 A Surgery Simulation System with PHANToM Haptic Feedback 85

3.6 Implementation and Results 90

3.7 Conclusion 93

4 FUTURE WORK 95

4.1 Tomosynthetic Reconstruction 96

4.1.1 Fast Re-projection 96

4.1.2 Local Tomosynthesis 96

4.2 Deformation Modeling 97

4.2.1 Dynamic Simulations 97

4.2.2 Collision Detection for Multi-Point Interactions 97

4.2.3 Simulation of Virtual Cutting 98

4.2.4 Validation of Haptic Simulations 98

4.2.5 Evaluation of Surgical Simulations 99

4.2.6 The Distributed Surgical Simulator 99BIBLIOGRAPHY 101

LIST OF FIGURES

Computed Tomography MRI = Magnetic Resonance Imaging PET

= Positron Emission Tomography SPECT = Single Photon

of the MRI image of the same human head at the same anatomical

located between a planar detector and an X-ray source The

projections are taken from discrete source/detector locations Eight(K = 8) projections are captured in this example 17

corresponding neighbor voxel 20

reconstruction is performed from the coarsest scale up to the finestscale As the MAP reconstruction converges at a coarser scale k, thefinal reconstruction is interpolated up and used to initialize the

MAP reconstruction at the next finer scale 24

2.4 Three x − y image slices at different depth z of the simulated

phantom The linear attenuation coefficient of each region is

higher attenuation coefficient Black color indicates attenuation

projections with 200k incident photon count Images from the firstrow down are (a) simulated phantom, (b) backprojection, (c)

F-MAP initialized to uniform image, (d) M-MAP-2 initialized to

uniform image, (e) M-MAP-3 initialized to uniform image 34

projections with 200k incident photon count Images from the firstrow down are (a) simulated phantom, (b) backprojection, (c)

F-MAP initialized to backprojection, (d) M-MAP-2 initialized to

backprojection, and (e) M-MAP-3 initialized to backprojection 35

projections with 20k incident photon count Images from the firstrow down are (a) simulated phantom, (b) backprojection, (c)

F-MAP initialized to uniform image, (d) M-MAP-2 initialized to

uniform image, and (e) M-MAP-3 initialized to uniform image 36

projections with 20k incident photon count Images from the firstrow down are (a) simulated phantom, (b) backprojection, (c)

F-MAP initialized to backprojection, (d) M-MAP-2 initialized to

backprojection, and (e) M-MAP-3 initialized to backprojection 37

projections with 2k incident photon count Images from the first rowdown are (a) simulated phantom, (b) backprojection, (c) F-MAP

initialized to uniform image, (d) M-MAP-2 initialized to uniform

image, and (e) M-MAP-3 initialized to uniform image 38

2.11 Image slices of simulated phantom and reconstructions from eight

projections with 2k incident photon count Images from the first rowdown are (a) simulated phantom, (b) backprojection, (c) F-MAP

initialized to backprojection, (d) M-MAP-2 initialized to

backprojection, and (e) M-MAP-3 initialized to backprojection 39

z = 4.8cm slice image in Figure 2.6, 2.8, and 2.10: (a) uniform

initialization, 200k incident photon count projections, (b)

backprojection initialization, 200k incident photon count projections,(c) uniform initialization, 20k incident photon count projections, (d)backprojection initialization, 20k incident photon count projections,(e) uniform initialization, 2k incident photon count, (f)

uniform initialization, 200k incident photon count projections, (b)backprojection initialization, 200k incident photon count projections,(c) uniform initialization, 20k incident photon count projections, (d)backprojection initialization, 20k incident photon count projections,(e) uniform initialization, 2k incident photon count, (f)

the backprojection reconstructions are also plotted in the left

column for comparison: (a) uniform initialization, 200k incident

photon count projections, (b) backprojection initialization, 200k

incident photon count projections, (c) uniform initialization, 20k

incident photon count projections, (d) backprojection initialization,20k incident photon count projections, (e) uniform initialization, 2kincident photon count, (f) backprojection initialization, 2k incidentphoton count projections 43

nearest-neighbor, and median interpolators: (a) log posteriori

functions versus computation time in seconds, (b) NMSE functionsversus computation time in seconds All reconstructions are

initialized to uniform images All projections use 2k incident photon

2.16 Performance of the M-MAP-3 reconstructions using bilinear,

nearest-neighbor, and median down-samplers: (a) log posteriori

functions versus computation time in seconds, (b) NMSE functionsversus computation time in seconds All reconstructions use uniforminitialization images and bilinear interpolator Incident photon

count of simulated projections is 2k 44

DirectRadiography flat panel detector (b) The dosimeter and theimaging workstation 46

detector 46

above 47

(a) and (c) are the backprojection reconstruction images (b) and(d) are the M-MAP-3 reconstruction images The depth of each

generated by the deformable model presented in [30] 56

generated by the chainmail model in [48] 56

deformation of a blood vessel generated by the deformable model

presented in [16] 56

and to its adjacent nodes by edge springs For graphical simplicity,all springs are represented by straight lines in this figure (a) No

deformation occurs and all nodes are at their home positions (b)

Node i is displaced to a deformed position by an external force,

while its adjacent nodes are still at their home positions 60

3.5 Example of deformed node index table (DNIT) when an external

force is applied at an surface contact point (SCP) located in a

triangle face The cross in this figure indicates the current SCP

where external force is exerted Nodes indexed by the first, the

second, and the third rows of DNIT are denoted by black round

dots, black triangles, and gray squares, respectively Deformation ispropagated from the first row down until a stopping criteria is

satisfied 62

proposed mass-spring model (a) No deformation occurs (b) Whenthe CPD is small, the deformation is only propagated to a few nodesclose to the SCP (c) As the CPD is increased, deformation is

propagated to more nodes further from the SCP (d) The CPD

simple mesh system given the same deformation input For

simplicity, edge springs are represented by solid lines, while home

springs are not shown From top to bottom in (a): The mesh is inits rest shape; the deformation with a high ratio of spring constants

function of node 0’s x-coordinate for different RSC values 70

model generated by the proposed mass-spring model (a) The meshstructure is in its rest shape (b) The deformation with a high kh/ke

model (FEM) and the mass-spring models Images on the top rowshow the FEM model, while images on the bottom row show the

mass-spring model From left to right, deformations of the surfacecontact point (SCP) are assumed to be 5 mm, 10 mm, and 15 mm,which are equivalent to deformations of 10%, 20%, and 30% at theSCP 72

3.10 Comparison of deformations generated by the FEM model and

mass-spring model with uniformly distributed RSCs For simplicity,only three sets of results are shown here From the first row down,the Poisson’s ratios of the FEM model are 0.10, 0.30, and 0.49 Theleft column shows the nodal displacement as a function of the

spatial distance to the SCP The middle column shows the absolutedifferences of nodal displacement The right column shows the

normalized difference of nodal displacement To illustrate the effect

of the RSC, each FEM model is compared with the mass-spring

model with a low and a high RSC values 74

mass-spring model with adaptive RSCs For simplicity, only threesets of results are shown here From the first row down, the RSC0values in equation (3.24) and (3.25) are −0.10, −0.14, and −0.18,and the Poisson’s ratios of the FEM model are 0.10, 0.30, and 0.49.The left column shows the nodal displacement as a function of thespatial distance to the SCP The middle column shows the absolutedifferences of nodal displacement The right column shows the

models vs RSC0 The tested values of the Poisson’s ratio of the

FEM model are 0.10, 0.20, 0.30, 0.40, and 0.49 (b) Optimal values

Poisson’s ratios 78

exponentially increasing RSC for a higher resolution mesh The leftcolumn shows the nodal displacement as a function of the spatial

distance to SCP The middle column shows the absolute differences

of nodal displacement The right column shows the normalized

difference of nodal displacement (a)–(c) For a small CPD

(|d| = 5.0), the proposed adaptive mass-spring model can closely

approximate the FEM model, yielding absolute difference less than 1

mm (d)–(f) For a more significant CPD (|d| = 10.0), although theabsolute difference between the proposed model and the FEM modelincreases, the deformation generated by the proposed model can stillclosely follow the one generated by the FEM model and the relativedifference remains at the same level 80

3.14 Comparisons of the FEM model and the mass-spring model with

exponentially distributed RSC utilizing an irregular mesh grid (a)Nodal displacement vs spatial distance to the SCP (b) The

absolute difference of nodal displacement (c) The normalized

difference of nodal displacement 81

for real-time simulations The grid resolutions utilized to generatethe tetrahedral meshes are (a) 5 × 5 × 5, (b) 11 × 11 × 11, and (c)

21 × 21 × 21 84

resolution with different deformations 85

device for haptic feedback (b) The SimSurgery simulator developedbased on the proposed mass-spring model utilizes an SGI Prism

Visualization Server, a PHANToM haptic device, a DELL PC, and acomputer aided virtual environment (CAVE) studio 87

(b) A tetrahedral mesh of a kidney surface generated from the CTdata in (a) by Amira The kidney model is rendered in wireframemode for demonstration of the mesh structure (c) The same kidneymodel rendered in Gouraud shading mode 88

the tetrahedral mesh generation software (b) The resulting

tetrahedral mesh In order to illustrate the interior mesh topology,the top portion of the tetrahedral mesh is shown in (b) 89

(a-c) show the deformations of a kidney model in wireframe mode,when a user pushes and pulls on a surface node through the

PHANToM device From left to right the first row: (a) No

deformation occurs; (b) The kidney is being pushed by a user; (c)The kidney surface is being pulled by a user For simplicity, only

surface nodes of the kidney model are shown Images in the bottomrow (d and e) are showing the complete scene in Gouraud shadingmode for optimal visual quality 92

3.21 Frame rates of the kidney simulation at different rendering modes

and resolution levels For a particular drawing model (wireframe orGouraud shading mode), the model resolution has a negligible

impact on the graphic frame rate as the number of the mesh nodesincreases by four times from the low resolution to the high

resolution For a particular resolution level, the graphic frame ratedrops slightly due to the computational cost of the normal update.Nevertheless, the impact of this decrease to the visual feedback is

hardly noticeable 94

LIST OF TABLES

M-MAP-3 algorithms R.R.=reconstruction resolution,

P.R.=projection resolution, N.I.=number of iterations 29

DNIT 67

deformation algorithm 90

= Median Resolution, H.R = High Resolution 91

Medical image reconstruction and medical simulation are active research eas in the field of medical image processing and have received a great deal of attentionover the past ten years Medical image reconstruction allows a physician to visualizethe interior organs and tissues of patient’s body in a nondestructive or minimallyinvasive way for improved diagnosis and better treatment selection Medical simu-lation can model the biochemical nature, metabolic characteristics, and geometricarrangement of human organs such that the change in these properties can be stud-ied and predicted when the environmental parameters are altered This dissertationcovers two distinct but related areas of work: 1) a maximum a posterior (MAP)tomosynthetic reconstruction for X-ray imaging, and 2) real-time deformation mod-eling of soft organs and tissues utilizing an adaptive mass-spring model, which isthe basis of a prototype for a virtual surgery simulation system The proposedtomosynthetic reconstruction algorithm described in Chapter 2 is based on Bayes’theorem and reconstructs a scanned object by optimizing an objective function in amulti-resolution framework such that both image quality and algorithmic efficiencyare improved In Chapter 3, a virtual surgery simulation system is developed based

ar-on an adaptive mass-spring deformable graphical model proposed for soft tissueand organ modeling The presented deformable model is able to generate visuallyrealistic real-time deformations for visual feedback Coupled with the haptic feed-back provided by a PHANToM device, the presented simulation system allows users

to interactively manipulate virtual organs and tissues to simulate routine clinicalprocedures such as bronchoscopies or kidney biopsies

Chapter 1

INTRODUCTION

Improved understanding of both the normal and pathophysiological processes

of life in humans can help to improve the quality and reduce the cost of healthcare, which have become increasingly important across the world in the twenty-firstcentury Such advancement in life science depends on the development of advancedmethods to accurately visualize, measure, model, and simulate anatomic structuresand functional variables of human bodies

The past two decades have witnessed significant advances in biomedical ing, analysis, visualization, modeling, and simulation For instance, the advances

imag-in diagnostic radiology have led to imag-innovative two-, three-, and higher-dimensionalimage acquisition modalities that have become important clinical tools Anotherexample is related to advances in interactive visualization and modeling of medicaldata and surgical procedures, such as medical virtual reality (VR), opening newrealms into the practice of medicine Three-dimensional visualization techniquesallow images obtained from modern medical imaging systems to be interactivelydisplayed and manipulated so as to evoke sensorial experience similar to that of areal experience Such 3-D virtual environments allow physicians to be immersed into

a virtual environment, to assume any viewpoint, to interactively explore dynamicfunctional processes and detailed anatomy, to make accurate image-based measure-ments, and to manipulate or control interventional medical procedures Generalsteps of digital medical imaging procedures are summarized in Figure 1.1 The

RadiographyUltrasound Imaging

Postprocessing

Figure 1.1: General steps of digital medical imaging procedures CT = Computed

Tomography MRI = Magnetic Resonance Imaging PET = PositronEmission Tomography SPECT = Single Photon Emission ComputedTomography

diagram illustrates an overview of all the operations that can be applied to ical image data for different purposes and possibly with different objectives Forinstance, in an X-ray radiographic procedure, a two-dimensional radiograph is gen-erated in the image acquisition procedure and is directly viewed by a physician tostudy the patient’s body On the other hand, in a computed tomographic (CT)imaging procedure, a three-dimensional CT image data set usually goes through

med-each step shown in Figure 1.1 from image acquisition, preprocessing, image struction, to postprocessing In addition, a CT image set is often utilized to studyanatomical structures and pathophysiological processes by applying image analysisand image modeling techniques such as volumetric rendering, surface reconstructionand visualization, and feature identification.

Since the major contributions of this work, statistical tomosynthetic struction and interactive modeling of soft tissues, reside in the domains of medicalimage acquisition, medical modeling, and medical simulation, the rest of this chap-ter is focused on the introduction to the fundamental concepts of these three steps.For comprehensive discussions on all processes illustrated in Figure 1.1, please refer

recon-to [36, 94] and the references therein

Since the first X-ray image (Figure 1.2) was generated by Wilhelm Conrad

medical imaging has significantly evolved due to the collective contributions by searchers and scientists in many disciplines related to medicine, engineering, andthe basic sciences The goal of a medical image acquisition system is to captureand record information about the physical and functional properties of tissues ortissue components Accuracy and efficiency in performing this task are fundamentalrequirements of medical imaging systems The common use of three-dimensionalvolume image acquisition via standard clinical scanning technologies has proven theimportance of three-dimensional medical image visualization and analysis techniques

re-in routre-ine clre-inical practice Multi-modality imagre-ing techniques such as ComputedTomography (CT) and Magnetic Resonance Imaging (MRI) produce images that re-veal different characteristics of the structure and function of the human body Eachimaging modality produces potentially complementary information about the bodyfor the diagnosis and treatment of disease For instance, X-ray CT imaging measure

Figure 1.2: First radiograph of a hand made by W C R¨ontgen on December 22,

1895 Original plate is in the Deutsche Museum, Munich, Germany

(a) (b)

Figure 1.3: (a) An example of the CT image of a human head (b) An example of

the MRI image of the same human head at the same anatomical level.spatial distribution of X-ray attenuation coefficients that are based on the density

of the tissue or part of the body being imaged For Single Photon Emission puted Tomography (SPECT) imaging, a radioactive pharmaceutical is injected intothe body to interact with selected organs or tissues to form an internal source of ra-dioactive energy that is used for imaging Such an image acquisition process providesuseful metabolic information about the physiological functions of the organs Mag-netic Resonance Imaging (MRI) is a good example of a medical imaging techniqueutilizing non-ionized radiation MRI uses external magnetic energy to stimulateselected atomic nuclei such as hydrogen protons The excited nuclei become the in-ternal source of energy, providing electromagnetic signals that yield high-resolutionimages of the human body with excellent soft-tissue characterization capabilities.The type of medical image acquisition technique selected for a particular diagnosisprocedure depends on the physiological information needed For instance, MRI iscommonly used to study soft tissues such as the brain, while transmission CT and

Com-(a) (b)

Figure 1.4: (a) A CT scanner in operation (b) An MRI scanner in operation.tomosynthesis are more suitable to visualize hard tissues such as bones This dif-ference between CT and MRI images is illustrated in Figure 1.3, which shows a CTand an MRI slice images at the same anatomical level of a human head In the CTimage (Figure 1.3 (a)), skull tissues are better visualized, while in the MRI image(Figure 1.3 (b)), the brain tissue and other soft tissues of the human head becomeclearer

One essential procedure shared by those three-dimensional volume imagingtechniques such at CT, MRI, and SPECT is reconstruction of the final image of ascanned body For instance, a CT scanner uses X-rays to generate cross-sectional,two-dimensional images of the body Images are acquired by rapid rotation of the

by a ring of sensitive radiation detectors located on the gantry around the patient(Figure 1.4 (a)) The final image is generated from these measurements utilizing thebasic principle that the internal structure of the body can be reconstructed frommultiple X-ray projections Figure 1.3 (a) shows a typical CT image of a crosssection of a human head Similarly, an MRI scanner is also designed to producecross-sectional images of internal structures of a human body Unlike a CT scan,

however, a MRI scanner does not use x-rays Instead, it uses a strong magnetic fieldand radio waves to scan a patient’s body as the patient slides through the gantry(Figure 1.4 (b)) A similar reconstruction technique to that of CT is utilized toproduce very clear and detailed computerized images of the body MRI is commonlyused to examine the brain, spine, joints, abdomen, and pelvis Figure 1.3 (b) shows

a typical MRI image of the same cross section as the one shown in Figure 1.3 (a)

The major contribution of the first part of the dissertation to the field ofmedical X-ray imaging is a novel multi-resolution algorithm that solves the prob-lem of tomosynthetic reconstruction based on Bayesian models, yielding significantimprovements for both reconstruction image quality and algorithmic efficiency

Computer-based reconstruction and rendering of three-dimensional medicalimage data sets is beginning to replace the mental reconstruction from two dimen-sional image slices However, although volumetric visualization serves a number ofimportant application in basic research, clinical diagnosis, and treatment or surgeryplanning, it is limited by relatively long rendering times, high computational costs,large memory storage requirements, and reduced interaction between user and therendered image Interactive manipulation and real-time medical procedure simula-tion are still difficult for volumetric medical data One approach to circumvent theseconstraints is to faithfully transform the volumetric image data into surface modelsthat can be rapidly displayed, manipulated, quantitatively analyzed, and utilized tomodel specific medical procedures Figure 1.5 shows examples of a volume-renderedand a surface-rendered torso of a patient

Producing realistic data models and providing real-time interaction with ical data resides in the domain of virtual reality (VR) The use of virtual realitytechnology opens new realms in the teaching, practice, and investigation of medical

med-(a) (b)

Figure 1.5: (a) Volume rendering of a CT data set provided by Christiana Care

(b) The surface data extracted from the volume data in (a)

and surgery procedures, allowing the user to be “immersed” in the virtual ment to enrich the realities The application of VR in medical education, trainingand simulation has become prevalent, including basic anatomy instruction, clinicaldiagnosis, medical treatment planning, and surgical rehearsal A successful virtualenvironment must meet two generic requirements: it must accurately represent itsreal-world equivalent, and it must be highly interactive, supporting multi-sensoryfeedback, such as visual, haptic, and acoustic feedbacks Specifically, the simulationsystem must be able to achieve a visual update rate of at least 30 frames per second,and must have system latency (the response between user input and system output)

environ-of less than 100 milliseconds

While it is possible to generate photo-realistic representations of volumetricdata such as CT and MRI data sets, it is, however, very difficult to achieve adequatevisual update rates to generate real-time visual feedback at such highly detailedvolumetric representations Thus, the complexity of the medical image data setsmust be reduced to meet the real-time rendering requirement One approach todata reduction is to visualize only the required object surface data That is, to

generate accurate geometric models comprised of surface graphic elements, such asnodes, lines, and polygons, to represent objects in the volume This form of surfacerepresentation of medical data is usually referred to as a polygonal surface mesh Acommonly used surface mesh is a triangular surface mesh, where surface elementsconsist of nodes, lines, and triangle faces.

Although computational costs and memory requirements are significantly duced by utilizing surface data, several critical features, such as modeling of defor-mation or cutting of soft tissues, are sacrificed due to the missing interior structuresdata A trade-off between computational performance and accuracy of modeling isusually made by combining surface data with volumetric data

re-In the latest medical virtual reality applications, the modeling of soft sue mechanics and deformations for realistic visual and haptic feedback has beenidentified as a key technique for the successful development of surgery simulationsystems Since the human body is mainly comprised of soft tissue, organ shapeschange significantly when manipulated with surgical instruments or devices, andthrough contact with adjacent organs during a surgical procedure Consequently,the simulation of accurate deformation is a critical feature that can be visuallyenhanced by introducing a deformable graphical model

tis-The contribution of the second part of this dissertation focuses on new ods for deformation modeling of human tissue and organs based on an adaptivemass-spring model These models are optimized and evaluated through a series

meth-of experiments designed to compare the mass-spring model with a finite elementmethod (FEM) model In addition, a prototype of a surgery simulation system thatutilizes the proposed deformable model is developed to generate realistic real-timevisual feedback A PHANToM haptic device is integrated in the system for enhancedrealism

1.3 Dissertation Organization

The organization of this dissertation is as follows Chapter 2 presents a novelmulti-resolution algorithm to solve the problem of tomosynthetic reconstructionbased on Bayesian models Tomosynthesis is briefly reviewed and the limitations ofprevious Bayesian methods are studied To overcome these limitations, we propose

a multi-resolution model that reduces the overall computational complexity andimproves convergence by incorporating models of coarser imaged object data andprojections into the reconstruction An iterative optimization algorithm based on asurrogate function and coordinate decent is also presented to compute the Bayesianreconstruction The proposed method reconstructs an imaged object in a coarse-to-fine order utilizing projections at multiple resolutions Iterations at finer resolutionsare initialized to interpolations of coarser reconstruction data

Chapter 3 of the dissertation presents a real-time adaptive deformable ical model, based on a volumetric tetrahedral mesh, for interactive modeling of suchobjects as human organs and tissues in a virtual surgery simulation The presentedmodel is a mass-spring model to simulate deformations of a nonrigid object by it-eratively solving a nodal force equation at each mesh node An adaptive approach

graph-to adjust the spring constants for accurate deformation modeling is proposed byminimizing the difference of nodal displacement between the presented model and

a ground-true model, selected here to be the output of the finite element method(FEM) The convergence of the mass-spring model is also studied for optimal real-time performance of deformation computations A prototype of a surgery simulationsystem is developed based on the proposed adaptive mass-spring model To enhanceinteraction and realism, a PHANToM haptic device is integrated into the simulationsystem to provide high-fidelity force feedback The proposed simulator allows users

to interactively manipulate virtual organs through the PHANToM device tion results show that the proposed model can simulate elastic deformations of soft

Simula-organs and tissues with both high visual realism and real-time performance utilizingtetrahedral meshes of varied resolutions.

In Chapter 4 of the dissertation, conclusions are drawn based on the tations in Chapter 2 and 3 Finally, future directions are discussed for the researchwork in tomosynthetic reconstructions, surgical simulations, and deformation mod-eling

presen-Chapter 2

STATISTICAL RECONSTRUCTION FOR DIGITAL

TOMOSYNTHESIS

This chapter presents a novel multi-resolution algorithm to solve the problem

of tomosynthetic reconstruction based on Bayesian models Tomosynthesis is brieflyreviewed and the limitations of previous Bayesian methods are studied Bayesiantype reconstruction algorithms are widely utilized in conventional tomographic imag-ing due to their image modeling and noise equabilities that yield improved recon-structions compared to the alternative methods, such as backprojection and alge-braic reconstruction techniques (ART) For tomosynthetic three-dimensional imagereconstruction, however, Bayesian type algorithms become computationally inten-sive and impractical for clinical application, due to the high demand on memorystorage and slow convergence of the optimization process To overcome these limi-tations, we propose a multi-resolution model that reduces the overall computationalcomplexity and improves convergence by incorporating models of coarser imagedobject data and projections into the reconstruction An iterative optimization al-gorithm based on a surrogate function and coordinate decent is also presented tocompute the Bayesian reconstruction The proposed method reconstructs an imagedobject in a coarse-to-fine order utilizing projections at multiple resolutions Itera-tions at finer resolutions are initialized to interpolations of coarser reconstructiondata Simulations and results are presented comparing the proposed algorithm with

alternative methods, such as the fixed-resolution Bayesian algorithm and the projection method These results show that the presented multi-resolution methodcan improve reconstruction quality, while reducing computational cost.

Tomosynthesis is a three-dimensional X-ray imaging technique introduced inthe 1970s [50] The invention of tomosynthesis constituted a substantial improve-ment over X-ray tomography and radiography in that it allows the reconstruction

of an arbitrary number of image slices from a single scanning process Compared totomography, a typical tomosynthesis system can significantly reduce total imagingtime, cost, and radiation dosage applied to the imaged object such as the patient’sbody The projections used in tomosynthetic reconstruction can be acquired using aslightly modified radiographic geometry with a flat panel detector, which minimizesthe production cost and complexity of the imaging system However, tomosynthe-sis is only now enjoying widespread interest due to recent advances in large-areaflat panel digital x-ray detectors, high speed computers, and low cost storage de-vices For additional background on tomosynthesis, please refer to [37, 50] and thereferences therein

Sharing a similar mathematical foundation with limited-angle tomography [106],tomosynthesis can use any reconstruction algorithms available to limited-angle to-mography Since the 1980s, many reconstruction techniques have been developedthat attempt to solve the limited-angle problem [34, 35, 38, 39, 42, 53, 56, 67, 69, 73,

79, 84, 92, 96, 106, 108, 115] Among these techniques, Bayesian methods for thetic reconstruction are widely used due to their ability to significantly improveimage quality by modelling noise and incorporating regularization models into thereconstruction process [34, 35, 53, 69, 115] The fundamental concept in Bayesianreconstruction is to consider the linear attenuation coefficient map of the imagedobject and its projection as two discrete random processes An objective function

tomosyn-(likelihood or a posteriori function) is established to model the dependency of theunknown attenuation data on the observed projection data The reconstruction ofthe attenuation map is then equivalent to the optimization of the Bayesian objectivefunction.

There are, however, two major challenges in designing Bayesian type rithms for tomosynthesis Bayesian reconstructions are iterative methods based

algo-on optimizatialgo-on of the objective functialgo-on The optimizatialgo-on process involves voxelvalue updates at each iteration, and computational cost becomes prohibitive forthree-dimensional data Corresponding to the computational cost is the fact thatconvergence of the iteration process is very slow In addition, a fine tuned prior

is necessary to regularize the maximum likelihood (ML) estimate and improve theimage quality A Markov Random Field (MRF) is a commonly used prior thathas been shown to yield improved image quality [44, 47, 71] However, only localinteractions in an imaged object are efficiently modelled by MRF models [44] andstructures at different scales are treated equally To overcome these problems, it

is desirable to design a reconstruction algorithm that takes into account differentstructure scales and yields a reduction of overall computational cost

Instead, ordered subsets (OS) algorithms for digital transmission tomographyhave enjoyed considerable success in Bayesian type reconstruction [40, 59, 63, 77] It

is reported that OS methods can improve the convergence and reduce the overallcomplexity of maximum likelihood expectation maximization (ML-EM) and otherBayesian type reconstruction algorithms Reconstructions using ordered subset algo-rithms require the subdivision of projection data into an ordered sequence of subsets.The number of the subsets is usually chosen as 4, 8, 16, or 32 [40, 59] For optimalperformance, the subsets are usually ordered such that the projections corresponding

to angles with maximum angular distance from previously used angles are chosen at

each step This type of algorithm, however, is problematic in tomosynthetic struction since tomosynthesis is typically designed to work with a very limited num-ber of projections The total number of projections acquired for tomosynthetic re-construction is usually tens or even fewer [37, 50, 51, 75, 76, 78, 96, 108, 110, 113, 115].Ordering and subdividing such small number of projections does not yield muchimprovement of the reconstruction algorithm.

recon-Multi-resolution techniques are alternative approaches to improve Bayesianreconstructions It has been shown that multi-resolution techniques improve ob-ject modeling and algorithmic efficiency Instead, many algorithms utilizing multi-resolution modeling have been developed for tomographic reconstruction [7, 8, 10,

33, 44, 74, 86, 88, 90, 93, 97, 98, 117] Most of these efforts, however, focus on the lization of multi-resolution modeling in direct inversion of the radon transform andregion of interest reconstruction [7, 8, 33, 74, 88, 90, 93, 98] In [44], Frese, Bouman,and Sauer introduced an adaptive wavelet graph image model for Bayesian typetomographic reconstructions with spatially non-local measurements Their graphmodel computes the Bayesian MAP estimates of the scale sequence for the recon-struction data Each scale level is reconstructed by the fixed resolution MAP algo-rithm followed by the re-adaption of the model parameters Their model, however,requires storage and re-computation of scaling and wavelet coefficients after the re-construction on each scale is performed For three-dimensional reconstruction which

uti-is encountered in tomosynthesuti-is, thuti-is wavelet graph model can cause high tional cost and extensive memory consumption In addition, the re-adaption of themodel parameters uses a nonlinear operator that requires a training procedure

computa-The motivation of this work is to accelerate the convergence and reduce theoverall computational cost of Bayesian algorithms for tomosynthetic reconstruc-tion In this paper, we introduce a new multi-resolution prior for tomosyntheticreconstruction based on the Bayesian estimate The basic concept of the proposed

model is to consider the imaged object and projection data on multiple resolutionscales The dependencies of data across scales are represented by interpolations ofthe reconstruction data and down-samplings for the projections A multi-resolutionmaximum a posteriori (M-MAP) reconstruction is presented along with this model.The proposed algorithm performs the reconstruction across all scales in a coarse-to-fine order The reconstruction of each coarser scale is interpolated to initialize theiteration at the next finer scale At each scale, a fixed-resolution MAP (F-MAP)estimate is computed utilizing an optimization algorithm described in [39].

The remainder of this chapter is organized as follows In Section 2.2, the thor develops the fixed-resolution statistical tomosynthetic reconstruction Section2.3 extends the statistical reconstruction to a multi-resolution framework Simula-tion methods used to test the proposed algorithm are described in 2.4 Simulatedresults are presented in Section 2.5, comparing the proposed method and alternativereconstruction methods, such as F-MAP and Backprojection Finally, conclusionsare drawn in Section 2.7

A tomosynthesis system consists of three parts: an X-ray source, a flatpanel digital X-ray detector and a storage and processing unit, as shown in Fig-ure 2.1 Generally, the X-ray source can be positioned at arbitrary locations inthree-dimensional space with the detector on the opposite side of the imaged ob-ject This is distinct from conventional computed tomography (CT) that only allowsthe X-ray source to move on a circle, or arc, coplanar with the focal plane Thetomosynthesis setup offers extra degrees of freedom for the source/detector config-uration Unlike CT, which requires expensive dedicated equipment, tomosynthesiscan use a conventional radiography geometry with simple modification for data ac-quisition

Figure 2.1: Example of a circular tomosynthesis geometry Imaged object is

lo-cated between a planar detector and an X-ray source The projectionsare taken from discrete source/detector locations Eight (K = 8) pro-jections are captured in this example

Although, in theory, there is no restriction on geometry utilized by thesis, to simplify the pre-processing and the projection acquisition procedure, thetypical geometries used in the practical tomosynthesis system are circular trajecto-ries and linear trajectories [50, 67, 79, 108] A simple linear motion blurs structures

tomosyn-in one dimension only, leadtomosyn-ing to ltomosyn-inear streak artifacts which can still obscure usefulinformation in the direction perpendicular to the blurring direction in final recon-structions On the other hand, a circular motion of the source and detector pairresults in out-of-plane structures being blurred into rings, two dimensional blurringpatterns, which have been shown to yield better image quality ?? Bearing the samereason, the discussion in this paper is restricted to circular tomosynthesis as shown inFigure 2.1 Note that the presented model and algorithm can be straight forwardlyextended to more complex source/detector motions, such as hypocycloidal

In the scanning geometry described in Figure 2.1, X-ray photons are uated by an imaged object and collected by a two-dimensional flat panel digital

atten-detector that is parallel to the reconstructed image plane These X-ray photonsform projection images that are used by the tomosynthesis algorithm to reconstructthe attenuation map of the imaged object In the following discussion, it is assumedthat a total number of K projection images are acquired at discrete detector lo-cations In the presented model, the projection data and the attenuation map ofthe imaged object are treated as discrete data sets The projection data are de-

the detector, where M is the total number of pixels in all projection images Areconstruction region containing the imaged object is discretized into N voxels Let

three-dimensional discrete attenuation map of the reconstruction region is denoted

The presented method is based on Bayes theory More specifically, we utilize

a maximum a posteriori (MAP) estimate to recover the discrete attenuation map.Let ˆf denote the reconstruction of this attenuation map The MAP reconstruction

where Bayes equation has been used to obtain the expression in (2.1) The

P (r|f), and the satisfaction of the prior model P (f) In the case where the priormodel, P (f ), is defined as an uniform distribution, the solution to the MAP problem

in (2.1) reduces to the maximum likelihood (ML) reconstruction

2.2.2 Statistical Models

The prior model, P (f ), in MAP estimate is a regularization factor to thefinal reconstruction The desired priors in tomosynthesis are those distributions thatincrease the probability of the realizations which feature slow change and decreasethe probability of the noisy data Such prior models should consider the interactions

of the local voxels, while incorporating the abrupt change across the edges or featureboundaries Markov Random Field (MRF) has been shown to be efficient in thisapplication [47, 55, 71], and thus we adopt a similar approach here The generalform of the MRF model is given by the Gibbs distribution

j The neighbors of j are defined as those voxels on the three-dimensional grid thathave at least one vertex in common with voxel j The key to this model is the

our current application, ψ(·) is drawn from a class of potential functions that havethe following form,

Figure 2.2, where the weights are inversely proportional to distance between voxel

s and j Further information on MRF priors is available in [11, 47, 55] and thereferences therein

For the likelihood distribution, since the measurement is assumed photoncounts, it is reasonable to model the projection data as an independent Poisson

Figure 2.2: Definition of weights ωsj for the neighbor voxels of fj Each weight is

contributed by all voxels This model describes the collected photon counts as the

The Poisson distribution in (2.6) is assumed here for conciseness withoutlosing generality of the proposed model This assumption for the projection data,however, is not strictly true, since two important phenomena have been neglected inthis transmission model First, as the X-ray penetrates the imaged object, a portion

of photons are scattered instead of being attenuated According to previous studies,the scattered photons may be a significant contribution to the detected signal Thusthe scattering effect is usually modeled as extra background count measured by the

detector and added to the mean value of the Poisson distribution [39, 42] Secondly,the x-ray beam utilized in a practical tomosynthesis scan is polyenergetic, thus theattenuation coefficient of the imaged object depends on the energy of the individualphoton In this case, the Poisson model in (2.6) is no longer accurate and will causebeam hardening in the final reconstruction [39] In addition, most current availableflat panel detectors are of the integrating type instead of the photon counting type,thus the photon counting process assumed for the projection data is an approxima-tion The investigation of these effects is beyond the scope of this dissertation, thus

we do not take them into account in the proposed model

Substituting (2.4), (2.5) and (2.6) into the log-posteriori function in (3) yieldsthe objective function for the MAP estimate,

L(f , r) =

MX

j=1X

j = 1, 2, , N

There are many algorithms for MAP reconstructions based on the tive optimization [11, 12, 35, 38, 39, 42, 55, 72, 73, 99] In the proposed reconstruc-tion method, we have adopted a precomputed curvature version of the paraboloidalsurrogate coordinate decent (PS,P,CD) algorithm introduced in [39] and designedfor X-ray CT The proposed algorithm is based on coordinate decent (CD) andparaboloidal surrogate functions for the log likelihood and log prior presented in (2.7).This algorithm has been shown to be effective in that it yields simpler objective func-tions and faster convergence than other paraboloidal surrogate algorithms that useoptimum curvature which in turn is simpler than other algorithms that directly

itera-minimize the objective.

The proposed algorithm is formulated using the quadratic approximations tothe Poisson likelihood in (2.7) Applying the second-order Taylor’s expansion to thelikelihood function in (2.7), yields a quadratic approximation that is easier to mini-

for the quadratic approximation at current iteration is given as follows [38],

MX

i=1

NX

j=1

1

Ãlij

NX

A similar approach is used to generate a surrogate function for the log prior given

in the following equation,

NX

j=1





12X

2X