Automatic cell model generation from integrated cell image database

In particular cell image segmentation and cell tracking through a series of images has the potential to increase the throughput of cell experiments... In this paper, local contrast enhan

SEGMENTATION OF DIFFERENTIAL INTERFERENCE CONTRAST CELL IMAGE

TU YAJING

(B.S (Hons.), Beihang University)

A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF

SCIENCE

DEPARTMENT OF BIOLOGICAL SCIENCE NATIONAL UNIVERSITY OF SINGPAORE

2012

II

Acknowledgements

I would like to thank my supervisor, Prof Paul Matsudaira, for his mentorship and continued support during my time as a student in NUS, my co-supervisors, Prof Peter So and Lisa Tucker-Kellogg, for their guidance through the time Thanks to the people in the lab and it was a great time to

be working with them and special thanks to Yip Aikia for providing the DIC cell images that are used in this paper Thanks to my brother and all my friends who gave me the encouragement during the hard times Finally I would like to dedicate this thesis to my parents, for their unwavering support and love

III

Contents

CHAPTER 1 INTRODUCTION V

1.1OVERVIEW OF CELL IMAGE ANALYSIS 2

1.2INTRODUCTION TO MICROSCOPY 5

Bright Field Microscopy 5

Differential Interference Contrast Microscopy 5

Fluorescent microscopy 7

1.3CELL IMAGE SEGMENTATION OVERVIEW 9

1.3 MOTIVATION 12

1.4 REMAINDER OF PAPER 13

CHAPTER 2 BACKGROUND OF IMAGE SEGMENTATION 14

2.1IMAGE ENHANCEMENT 14

2.1.1 Histogram equalization 15

2.2IMAGE DENOISING 16

2.3WATERSHED 20

2.3.1 Basic tools for watershed 20

The Morphological Gradient[45] is defined as: 20

2.3.2 Watershed segmentation 23

2.4ACTIVE CONTOUR 24

2.4.1SNAKES 25

2.4.2LEVEL SET APPROACH FOR ACTIVE CONTOURS 26

2.4.3ACTIVE CONTOURS WITHOUT EDGES 28

Mumford-Shah model 28

Chan-Vese active contour 29

IV

CHAPTER 3 ALGORITHMS 32

3.1EXPERIMENT DATA 32

3.2IMAGE PRE-PROCESSING 34

3.3SEEDED WATERSHED 36

3.3ACTIVE CONTOURS 38

3.4CELL TRACKING 41

3.5CELL OVERLAPPING 42

CHAPTER 4 EXPERIMENT RESULTS 43

4.1SEEDED WATERSHED 43

4.2ACTIVE CONTOURS 44

4.3EXTENSION TO IMAGE SEQUENCE 46

4.4DISCCUSSION 49

CHAPTER 5 CONCLUSION 50

REFERENCES 52

V

Abstract

Image segmentation is a complex problem with many practical applications In particular cell image segmentation and cell tracking through a series of images has the potential to increase the throughput of cell experiments This paper addresses the problem with DIC cell images

In this paper, local contrast enhancement and N-L means image denoising are proposed for image pre-processing which improves the quality of the image to a great extend After that several image segmentation methods are applied The first solution is based on a seeded watershed segmentation technique, and the second one is based on active contours using level set function The algorithm is further extended to cell tracking problems The active contours produces good results for images with single cell, and for cell clustering the combination of active contours and seeded watershed produced good results

VI

List of Figures

Figure 2.1 Illustration of Gaussian function 18

Figure 2.2 Illustration of N-L means 19

Figure 2.5 The level set evolution 26

Figure 2.6 Active contour example 1 31

Figure 2.7 Active contour example 2 31

Figure 3.1 Three datasets for experiment 33

Figure 3.3 Results for three image preprocessing 35

Figure 3.4 The procedure of marker-based watershed 37

Figure 3.5 The effect of initial contour to the final contour 39

Figure3.6 The effect of choosing different scales for rough outline calculation, 40

Figure 3.7 The procedure of active contour for DIC image segmentation 41

Figure 3.8 The procedure of active contour for DIC cell tracking 42

Figure 4.1The procedure for cell segmentation based on marker based watershed 44

Figure 4.2 The procedure for cell segmentation based on active contour 45

Figure 4.3 Additional results using Active Contours 46

Figure 4.4 Single cell tracking result using set A 47

Figure 4.5 Multi-cell tracking result using set B 48

Chapter 1

Introduction

Humans receive substantial information from the surroundings everyday and most of the information is obtained by vision The image, whether it 2D or 3D, gray or color, is a way of recording such kind of information Especially with the advent of computers and the development of relevant mathematic techniques, digital image analysis and pattern recognition has drawn a lot of attention from researchers and scientists in identity recognition, space exploration, remote sensor and many other industry fields The image analysis has also become popular in cell biology study and there is no doubt the application of such quantitative analysis will prompt new development

Image segmentation is usually the first step in computer vision tasks and sometimes it is the most challenging part It is playing a great role in image analysis since an accurate segmentation will separate the most desirable units, which contain certain features, from the background so that those units can provide more meaningful and easier way to be interpreted by computer Many methods of image segmentation have been brought up since 1970’s, however most of them target at some certain problems and no generic approach is found to solve all segmentation problems

2

In this chapter, a few things will be covered First, we will review the application of image analysis in cell biology study; then types of microscopy used in biology research and different segmentation methods regarding that specific microscopy image will be briefly introduced; in the end, the motivation for DIC image segmentation will be given and discussed

1.1 Overview of Cell Image Analysis

Since the invention of the first microscopy in 17th century, biologists have revealed a micro world that is built up with cells which cannot be observed by naked eyes By examining these cells under microscopy, thousands of hundreds of biological questions have been answered Even today it is still a standard and primary way to study cellular function However, these images are all inspected and processed by hand, for example, what is the size of a cell, how fast the cell is moving, or even what the collective migration pattern of cells is and etc is The consequences are that the whole process will not be only laborious but also more error-prone and the results will be subjective to the person who interprets it With the advent of the computer and the development of computer vision theory such kind of image analysis can be used to supplement and replace human visual inspection thereby yielding in a more efficient and automatic way Hence it is no surprise that this technique is widely used in the fields of studies such as cell shape changing, collective cell migration and even the recognition of biological objects

3

Cancer cells are the cells that have abnormal growth, division and they may even evade other neighboring tissues Such cells usually have different morphological characteristics from normal cells For example, the cells will look more round due to the abnormal cytoskeleton structure, the cells lose the contact inhibition with the substrate and this gives them the ability to move faster and be more invasive, moreover the cancer cells usually have larger nuclei with irregular shape[1] Therefore the final diagnosis or grading for cancer can be made based on cell morphology and tissue structures, and these decisions will provide information to further cancer treatment The computational visual interpretation was applied for the detection of precursors

of cancer[2, 3] and this has greatly reduced the incidence of those cells developing into more dangerous disease; in [4] the image analysis was used

to grade transitional cell carcinoma of the bladder Features of those cells were extracted and were used for further cell classification, and it achieved a grading result that is similar to the pathologist while more objective and reproducible

The high content screening technology (HCS) is now widely used, which allows cheap and fast collection of large image data For example, the functional genomics is a study that attempts to describe the functions and interactions of genes and proteins, revealing the relationship between the genome and its phenotype, to be more specific, how the expression of a certain gene affects the signaling inside and outside the cells thus lead to different cell function or behavior By using HCS thousands of images of cells

4

can be collected Those cells are stained either by some chemicals, such as fluorophore, or RNA interference (RNAi), so that the cell shapes could be screened systematically which indicated how genes controls a specific cell-biological process [5-7] The collected data can later be used for image analysis The HCS is also important in drug discovery [8-11] In order to study the effects of drugs on the desired target cell statistically, large quantities of cell images will be needed These images will be passed to image analysis pipeline that automatically extracts cell features, which may not be even detected by human eyes Then these features will be trained to build a classifier that can distinguish those normal and abnormal cells

Moreover, there is a tendency to study cell functions in a dynamic way, which captures and tracks the cells using a time-lapsed microscopy This is especially helpful in cell migration study, the understanding of which will give insights into many aspects such as embryonic development, wound healing as well as tumor cell formation and etc[12, 13] The image analysis, such as cell tracking and cell circularity, will again help to provide more robust and quantitative data and aid in the building of a cell migration model[14, 15] Down to a more detailed scale, cell migration is always associated with signaling pathway and protein sub-cellular interaction and transportation Tagged with green fluorescent protein (GFP) in living cell, the desired proteins can be imaged and traced by fluorescence microscopy [16-18], providing more information in protein sub-cellular distribution and

5

compartmental transportation and after all how do they determine the cell migration

1.2 Introduction to microscopy

Bright Field Microscopy

The bright field microscopy is the simplest and the elementary form of all optical microscopy techniques A typical bright field image is a dark sample with bright background When a specimen is placed on the stage, light from the light source passes through a condenser and is focused on the specimen The stains, pigmentation, or dense areas of the specimen will absorb some of the transmitted light so that this contrast allows the user to see the specimen

The simplicity of bright field microscopy makes it a popular technique

to some extend and However, this technique still has limitations For example, the contrast of most samples is low which makes most details undetectable, although by sample staining may help to reveal more structures it will also introduce some other details in the specimen that are not supposed to be present Moreover, the technique requires strong light source for high magnification applications and such intense light may damage the sample cells by producing lots of heat

Differential Interference Contrast Microscopy

DIC (differential interference contrast) is a very powerful tool for visualizing unstained specimens, providing the ability to observe living

6

organisms, tissues or cells A typical image of DIC gives a 3D look of the specimen, creating bright light and dark shadows on the respective faces The interferometry theory is used to get information about the optical path length of the sample [19] The whole process starts with the light passing through a polarizing filter and this makes the light wave oscillate in only in direction After that the light will pass through a two-layered modified Wollaston prism The prism will split the light into two beams which are orthogonally polarized and spatially separated When the light reaches the sample plane, the two beams go through different paths One goes through the sample while the other one just passes through the background The two beams will be combined again by another Wollaston prism located between objective lens and sample plane Different segments of the sample have different refractive indices and thickness “When the beams are compiled by the second prism and a second polarizing filter they reconstitute the vibrational planes of the beams, which causes amplitude variations that are seen as differences in brightness”[19] In general, steep gradient in path length gives high contrast with lines and edges emphasized while regions having shallow optical path slopes produce insignificant contrast and often appear as the same intensity level as the background

It is easy to tell the DIC microscopy has many advantages over the bright field microscopy, including the capacity to view living and unstained biological samples in a natural state and providing high-contrast and high-

of the fluorescence microscopy shares the same principle to generate a light microscope image, which is fundamentally different from transmitted or reflected white light techniques such as bright field microscopy and differential interference contrast microscopy

We are all familiar with phosphorescent, which shows a delay in brightness after the material absorbs energy from an external source Similar but a little different from phosphorescent, fluorescent is the process that gives out the emission light after absorption of energy in a short time, usually on the scale of nanosecond Fluorescent microscopy is the equipment that can visualize materials that are either nature fluorescent or stained by some chemicals such as fluorephores or gene transition such as GFP Because of

Internal conversion and vibration relaxation (10-12 s)

Intersystem crossing

9

Figure 1.1 (A) Jablonski diagram; (B) Stroke’s shift

Although the fluorescent microscopy is now a standard tool for biology researches, there are still some limitations this technique The most prominent one is with photo-bleaching When a fluorephore is excited by external energy the structure of this fluorephore is prone to be instable and degradation This would bring problems to quantitative image intensity measurement Another limitation is that the agents used to make the cell fluorescent will also have the potential to change the behavior of cells

1.3 Cell Image Segmentation Overview

The automatic microscopy image analysis is now drawing more and more attention from the biologists With the efforts of scientists coming from both computer science and biology, many image analysis problems are addressed and there is also some bio-image software ready for use In

400 450 500 550 600 650 700 750 0

20 40 60 80 100

Stoke’s

shift

of microscopy or image acquisition the images generated will have uneven illumination or when in 2D images it is often the case that cells touching each other All these scenarios will make the simple algorithms fail and urge researchers to find a more complex solution At the nuclear level, the cell nuclei are more regulated and they are always quite distinct from the background In cytometry or HCS applications, numerous algorithms are suggested, such as watershed and region-grow methods for the clustered nuclei in cytometry applications[25, 26], in[27] the author addressed the problems of weak edge information and uneven illumination by combining several methods in a multi-scale manner However, at the cell level the segmentation is more challenging and it is even more difficult if no nuclei information is available In[25] the author developed an automatic algorithm for cell segmentation by combing a number of processing such as watershed, double thresholds, region merging and quality control Jones et al.[28] also suggested using voronoi method to find the boundary between the adjacent cells Despite the fact of the variety of fluorescent imaging segmentation, considerable effort has gone into the quantitative measurement of object

a special image acquisition that the images from different focal plane should all be taken, which makes this approach not so practical Wu et al [35] presented an early solution to the segmentation of unstained living cells in their paper The method is a two stage segmentation in which an approximate region that the cell resides in is first found by image variance map threshold

In the second step the cell from the remaining background in the approximate region is further segmented [36] also presented a robust segmentation

1.3 Motivation

Despite the fact that there are substantial papers published regarding the problem of cell image segmentation and many tools available to do standard processing, they are not able to address all the problems and meanwhile most of these studies and software are more towards the application of fluorescent imaging While fluorescent imaging is essential, other microscopy imaging approaches, particularly DIC, due to its intrinsic

13

features, supplements the usage of fluorescent microscopy and provides enhanced information for living cell study

However, those segmentation methods on DIC images are either based

on model reconstruction or gradient phase removal, and all of them require the prior-knowledge of DIC microscopy and relevant parameters which make them not so practical

Because of the importance of DIC imaging technique and poor research in its object segmentation currently, we took the effort to explore the possible way to solve the problem of DIC cell image segmentation

1.4 Remainder of paper

The remainder of the paper is organized as follows: Chapter 2 contains the background information on a variety of common segmentation and clustering algorithms; Chapter 3 contains a description of the algorithms; Chapter 4 has the results of all algorithms presented and finally Chapter 5 is the conclusion of this paper

14

Chapter 2

Background of Image Segmentation

There are many approaches to solve segmentation problems: the threshold method based on the difference in intensities of background and object, level set methods rely on PDE’s, while the watershed method treats the image as topography This chapter will discuss some background algorithms that are useful for DIC image segmentation, such as image pre-processing, namely local contrast image enhancement and N-L means image denoising, as well as active contours and watershed methods

2.1 Image Enhancement

Image enhancement is an important part in image processing It aims

to improve the image quality, give more interpretability for human viewers, remove or reduce redundant information and facilitate with further image analysis There are several approaches for Image enhancement techniques and they can be categorized into spatial domain and frequency domain methods [40] The frequency domain methods adopt flourier transform, either blurring the image by a low pass filter or sharpening the image by a high pass filter On the contrast, the spatial domain methods operated directly on pixels, and these methods will be covered in more detail

15

2.1.1 Histogram equalization

Histogram equalization is a common technique for image enhancement [40, 41] By applying statistic theory it transforms the image by stretching the original histogram to uniformly distribution Such adjustment will change the pixel value and therefore improves the image contrast

Consider an image whose intensity levels is in the range of [0, L-1], the total number of pixels is n, and n is the total number of pixels of intensity level k

k

r The proportion of pixels with value r kis given by:

k

k r

n P

n

 (2.1) Where k 0,1 L1, and the function for histogram equalization is given by:

output value; m i j is the mean value of the neighboring regions with x( , )center ( , )i j ; k is some co-efficiency When k 1, if x i j( , )m i j x( , ) then ( , ) ( , )

f i j x i j so that the value is increased; whereas ifx i j( , )m i j x( , ), then ( , ) ( , )

f i j x i j the value is decreased Overall, the local enhancement can improve the local details

2.2 Image Denoising

The image could always be corrupted by noise, either during acquisition or transmission Image denoising is the process to remove or reduce noise in the image So that the image noise model could be presented as:

17

For the spatial domain method, the median filter is the most common one It is a non-linear filter, which assigns a pixel with the median number of pixels in a mask To illustrate, suppose there is an image that is of size m*n, a mask of k*l is defined and moves across the whole image Let (i,j) be the center of the mask when it moves to a certain region of the image, and also let T be the median value of all the pixels that are covered by the mask So that (i,j) is assigned to the value T

In frequency domain methods, a typical way to smooth an image is Gaussian filter It is a convolution in the spatial domain and a low pass filter which attenuating high frequency in frequency domain The 2D Gaussian function has the equation as:

2 2 2

2 2

1 ( , )

Where x and y are the distances from the center point respectively,

 is the standard deviation of the Gaussian distribution which determines the shape of Gaussian function (figure 2.2 (a)) Theoretically, the Gaussian distribution is non-zero everywhere However, in practical only the points within three times standard deviation are used and the rest points are truncated This distribution is presented as Gaussian Convolution Kernel in discrete space Figure 2.2 (b) is an example when  is 1;

18

Figure 2.1 Gaussian function with  =1.0 (a) Gaussian curve in continuous space; (b)

Discrete Gaussian approximation or Gaussian Convolution Kernel

Both median and Gaussian filters have the advantage of simple calculation; however they will also bring the problem of blurring to the resulted images For example, they can’t preserve the fine structures, details and textures since this information all behave like noise in frequency domain The N-L means method is a way to address this problem

Given a noisy image v{ ( ) |v i iI} , and define Nk as a square

neighborhood of a certain size with center point at pixel k i and j are two

pixels so the Euclidean distance between i and j in the noisy image model has the following equation:

|| ( i) ( j) || a || ( i) ( j) || a 2

Where  is the standard deviation of Gaussian white noise distribution

So the similarities between pixel i and j is defined as:

19

2 2, 2

h

N v N v i

z( ) exp( || ( ) 2( )|| )

2 , 2

is a normalizing constant

h controls the smoothness, determining the decay of the exponential function,

and here it refers to the change of weights as a function of Euclidean

distances For example, when h is small, the decay of exponential is more

distinctive and the resulted image will preserve more detailed information Meanwhile, the weights w i j also satisfy the conditions 0( , ) ( , ) 1i j  and

Therefore the final value of each pixel is give by the weighted average

of all the pixels in the image:

Figure 2.2 illustration of N-L means When calculate the value in region 4, since region

3 more resembles to 4 more weight is give to 3, whereas 1 and 2 are given smaller

weights

20

2.3 Watershed

The first watershed was brought up in geography, but now it is widely used in image segmentation This method is a regional segmentation based

on mathematical morphology In[43] and Lantu_ejoul gave the first definition

to watershed: suppose that the landscape is flooded by falling rain The water will come to the lowest point first and starts to go up the surface When water comes from different regions about to meet, a dam is built along the ridge In [44] Vincent and Soille gave an alternative way to descried the process: treat each pixel value in the image as the height on that point, drill holes in every local minima and immerse the region in a lake Therefore the water will come from these holes and will fill up the catchment basins Similarly, a dam is built

at the points where water coming from different basins would meet The whole process will stop when the water immersed the whole landscape

2.3.1 Basic tools for watershed

22

The minima, in watershed context, are one of the primary important features that are extracted from an image The topographic surface S can be defined as set of all the points { , ( )}x f x belonging to X The altitude of surface point{ , ( )}x f x can be corresponded to the gray value at point x

The minima of an image, also called regional minima, are defined as[47]:

Consider two points s1and s2 of S, the path between s x f x1( , ( ))1 1 and

Figure 2.4, A topographic minima

23

2.3.2 Watershed segmentation

The concept of watershed derives from geography In image segmentation, it takes the value of a pixel as the height of that point in a 3D landscape There are mainly two ways to achieve watershed, one is based on immersion simulation and the other is by the rain falling model

Similarly, the water dropping simulation gives another way of the whole process The rain will drop on the landscape and due to the gravity it will fall along a path which leads to the local minima most quickly So if drops eventually come to the same region then the points where they landed on the landscape belong to the same region Only the water drops on the ridge has the equal potential to fall into any adjacent regions

The immersion simulation approach describes a scenario that in a uneven landscape the water begins to immerse the region from each local minima; As the water rises up, the water comes from different basins will meet So a dam will be built on that point and in the end the landscape will be divided into different regions by these dams The dam on the ridge is celled watershed

However, the traditional watershed is very sensitive to the noise which leads to a serious image over segmentation Therefore, many improved watershed have been proposed and have achieved good experimental result These methods aim to minimize the influence of noise and fine textures, preserve essential contours, reduce the number of regions and avoid region merging In the following, methods based on markers will be discussed

24

The watershed segmentation is quite sensitive to image quality An image with noise and other factors would always lead to over segmentation, resulting in the desired contour be overlaid by many other irrelevant contours

An effective way to solve this problem is to use marker-based watershed The seeds are selected either manually or automatically They will be assigned as the lowest points in the image, as in the gradient image, and then watershed method will be applied to this image

2.4 Active Contour

The active contours model is now widely used in image segmentation and object tracking The main idea of active contours is to evolve a curve to fit the boundary and the curve moves under internal and external forces and in the end stops when energy function is minimized Active contour models have the advantage that the final curve will always be a closed and smooth area regardless of the image quality, to illustrate, blurred images, spurious edges

or broken edges

There are two main types of active contour models: parametric active contours[48, 49] and geometric active contours[50-54] Parametric explicitly defines the curve is by curves points, so that it only needs to move the points according to some energy function The geometric active contours, on the other hand, implicitly define the curve by transforming the curve to a higher dimension function, for example, 2d curve line to 3d curve surface