Improved Dijkstra''s algorithm on the image segmentation problem

This paper presents the improved multiple-source Dijkstra algorithm as an efficient solution to the multiple objects, interactive image segmentation problem, which can be applied for precomputing process in many tasks of image processing. Given an input image, we can build a sparse undirected graph of pixels and pick some pixels with predefined labels as sources.

IMPROVED DIJKSTRA'S ALGORITHM ON THE IMAGE

SEGMENTATION PROBLEM

Nguyen Nam Phuc1*, Le Tien Hung2, Nguyen Quoc Trung3, Ha Huu Huy4

Abstract: This paper presents the improved multiple-source Dijkstra algorithm

as an efficient solution to the multiple objects, interactive image segmentation

problem, which can be applied for precomputing process in many tasks of image

processing Given an input image, we can build a sparse undirected graph of pixels

and pick some pixels with predefined labels as sources By running the Dijkstra’s

algorithm on the whole graph, the closest source with the corresponding label from

all pixels can be determined efficiently in terms of both time performance and

accuracy The main contributions include the idea of parallel precomputing the

local shortest distances in a neighborhood of each pixel to build an undirected

graph and combining label-correcting methods with the Dijkstra’s algorithm to

reduce the running time Our experimental results on real-world images

demonstrate significant improvement to the existing image segmentation methods in

terms of both time performance and accuracy

Keywords: Shortest distance; Dijkstra’s algorithm; Graph; Parallel; Image segmentation; Label correcting

1 INTRODUCTION

In recent years, the task of image segmentation has attracted a much attention in

a variety of interactive applications [1] Roughly speaking, the ultimate goal of the task is to extract objects from images with as few user interactions as possible Many image segmentation algorithms have been proposed in the literature and could be classified into different groups as edge detection [2], thresholding [3] and clustering based [4] segmentations However, designing an automated image segmentation technique which can be applied fully in real-world applications is still a challenging problem The main reason is about the large requirement of input information from user to define the ground truth of objects, especially when the input images contain multiple objects with different colors or when users need

to change the input information frequently to archive satisfactory results And for the iris/vessel detection problems [5,6], it is difficult to enhance multiscale vessel-like structures, especially on the images with low contrast or high homogeneity of illumination conditions

In contrast, the graph based approaches [7] have been broadly used in large-scale image database applications and they have become a major trend in image segmentation Greig et al [8] evaluated two-colour or binary images by using Ford-Fulkerson algorithm for finding the maximum flow in a capacitated network Felzenszwalb determined the similarity of image regions by measuring the internal variation between neighboring pixels [9] Recently, Wei et al [10] presented a multi-source variant of the Dijkstra's algorithm which allows us to perform interactive segmentation User can predefine labels for a small number of pixels (seeds) and then the closest seed from each unlabeled pixel is determined in an accurately way (see Fig.1) Their computational approach is promising since it requires the initiating settings from user only once

(a) (b)

Figure 1 Seeds 1 and 2 represent two different labels 1 and 2 (a) By finding

the closest seeds, the whole pixels in the image can be properly marked (b) Figure

is borrowed from [10]

All of the existing graph based algorithms use only single thread to check the local connectivity of the embedded graph before each propagation of the Dijkstra’s algorithm And because the Dijkstra’s algorithm performs at most one iteration per node, its required extra overhead (e.g., to extract the node with minimum label) depends on the total number

of nodes in the whole graph However, with the rapid development of scanning technology, the input images for segmentation could have very large size nowadays It means the computational cost is expensive for the classical Dijkstra's algorithm and it may not be executed interactively The speed time of the Dijkstra's algorithm is more concerned

in the cases when segmentation must be performed multi time on sets of image databases for analyzing purposes

In this paper, we focus on improving the Dijkstra’s algorithm in term of time performance Our framework is based on the idea of partitioning an image into regions by using multiple-source Dijkstra's algorithm for label checking, such that all pixels in each region have the same label The complete framework consists of two stages In the first stage, the input image is mapped into a closed undirected graph where its local connectivity is precomputed parallelly It allows us to break down one of the most expensive part of the Dijkstra’s algorithm into sub-problems which can be easily solved in multi-threading mode

In the second stage, there are some pixel with individual label mapped to will be set as seeds for the Dijkstra's algorithm In each iteration, nodes with new distance information are inserted into or extracted from a set/list using incremental sorting operations Once the algorithm ruminates, the label of all unprocessed pixels is changed to the label of their closest seed by computing their shortest distances from the seeds To avoid using expensive internal sorting in linked set/list as common and reduce the extra overhead, we adapted the label correcting strategies on priority queue Thanks for that the propagating speed is boosted up significantly and our framework can be applied in interactive applications We have evaluated our framework on real-world images and observed promising results

The rest of the paper is organized as follows Section 2 reviews related work on image segmentation and the Dijkstra's algorithm Section 3 explains about the graph generation and how to efficiently perform the Dijkstra's algorithm on it with the label correcting strategies applied Section 4 contains our experimental results and discusses the merits and limitations of our work Finally, Section 5 concludes the paper

2 RELATED WORKS 2.1 Graph-based segmentation

Image segmentation is a wide range topic Therefore, in this section we will briefly discuss only some of the classical interactive and graph-based methods in recent years

For the graph-based algorithms, the image segmentation is treated as a graph partitioning problem In such approaches, a graph G=(V, E) needs to be prebuilt where each node vi  V of the graph corresponds to a pixel pi of the input image  Certain nodes vi, vj  V are connected by edges eij  E whose weights measure the distance between the corresponding pixels pi, pj   in some feature space There are several weighting functions can be found in [1,11] In our work, an exponential weighting function is introduced for gray images as from [11]:

2

1

i j

I I

ij



where Ii indicates the value of intensity at pixel pi   and  is the sensitivity coefficient When the input image is given in RGB mode, Eq (1) is modified as below with Ci is the color vector at pi:

1

T

i j i j

C C C C

ij

In the graph-based approaches, each component (or region) in a segment S corresponds to a connected subgraph G' = (V; E'), where E'  E This means any segmentation is induced by a subset E'  E The quality of a segmentation can be measured in different ways, but in general we prefer all the elements to be similar

in the same component to be dissimilar in different components In other words, the edges between two vertices should have relatively low weights in the same component and higher weights in different components

The earliest graph-based algorithms use fixed thresholds plus local measures for segmenting Zahn [13] proposed an algorithm based on the minimum spanning tree

of the graph This algorithm has been extended for point clustering with the weights are computed based on distances between points Another algorithm proposed in [14] uses a measure based only on the nearest neighbors of each node

in graph to decide which edges to break in partitioning However, the local variability around nodes alone may not be enough to get a reasonable measure of image variability over arbitrarily large regions of the input image In the graph cuts based algorithms [1], the chosen foreground/background seeds are treated as sources for max-flow/min-cut operations to returns a set of edges with minimum total weight as boundary One of the main issues in the graph cuts is that the boundary may take a shortcut over a protruded section of the object in an attempt

to minimize boundary length Moreover, the segmentation results are strongly sensitive to noise and becomes fragmented on images with low quality In the random walks based algorithms [15], the weight of an edge eij was treated as the probability of a particle at node vi traveling to its neighboring node vj Given the

seeds, one may compare the probability that a particle initiating at any pixel travels first to the foreground or background seeds and then assign that pixel to the corresponding label A common problem of random walks based algorithms is the tendency to produce segmentations that are overly likely to provide balance between the foreground and background, which may make images hard to be properly segmented

2.2 Multi-source Dijkstra's algorithm

The graph based approaches naturally link the segmentation problem on an image and the shortest path problem on a graph, since the label of each pixel pi is set depends on the shortest distances between sources and the corresponding node

vi Although the numerical techniques [16,17] have advantage in fast computing the shortest distances, their accuracy and stability are still the common issues we need to concerns about for segmentation of small or non-trivial shapes In contrast, the computational solutions, includes the Dijkstra's algorithm [18], are more reliable and they are widely used for especially in iris/vessels detecting applications Recently, Wei et al [10] adopted a multi-source variant of the Dijkstra's algorithm to change the label on pixels by running once Their idea is to find the closest seed for each pixel but not the shortest distance from all of the seeds Suppose the certain weighted graph G = (V, E) is given and there exists two kinds of seeds S1 = {f1, f2,…, fm} and S2 = {b1, b2,…, bn}, where fi denotes the foreground seeds and bi is the marked background seeds Firstly, all of the seeds are initialized with their distance to zero, and labeled with 1 and 2 respectively before putting them into a list Q During the Dijkstra’s expansion, the pixel’s label

is marked as same as its closest seed’s label Thus, the foreground can be finally segmented from the background when all pixels in the image are marked with their certain labels Nowadays, with the rapid development of modern image capture/scanning technology, the input images for segmentation are often very large, so the embedded graphs could have millions of nodes or more It means the Dijkstra's algorithm could be the bottleneck of the partitioning process and hard to

be used in real-time applications In addition, for the analysis purposes, the partitioning process needs to be carried out multiple times until the results match the user's requirements, so designing graphs with simple data structures for storing

is a good way to reduce computing resource and running time

3 THE PROPOSED METHOD 3.1 Efficient graph construction

The graph construction is required only once before running the Dijkstra's algorithm, then all information about the local connectivity is stored in conventional incidence lists Each node stores its incident edges and each edge stores two incident nodes plus its length

struct Node

{

}

struct Edge

{

unsigned int id_v1, id_v2; //index of the related start and end vertices

}

In graph based algorithms, the 4-connected and 8-connected neighborhood structures are usually used to transfer the local connectivity from the input image into its corresponding graph Thus, only weights of the edges that connect two adjacent nodes can be computed using Eq (1) or (2) Additionally, we can increase area of the local neighborhood to speed up the convergence of the Dijkstra's algorithm, the trade-off is that some disconnected similar objects could be linked because of the wrong labels marked on the pixels between these objects

Our construction algorithm is shown in Algorithm 1 For each vertex vi  V, we compute distances from vi in a local neighborhood (vi) Then all these distances are taken to form the edges from vi Computing the distances is the bottleneck of the entire graph construction algorithm due to the expensive math functions In our implementation, the distances, i.e the edges' length, can be computed in parallel on the CPU/GPU: the graph data structure is accessible to all threads in the read-only manner, and each thread maintains its own data To avoid the written conflicts, the thread for node vi produces directed edges originated from vi The gain of the constructing time strongly depends on the number of threads on CPU as it is shown

in Table 1

Table 1 The total time for edges’ construction on images with different size

T single_thread is the time required in the classical Dijkstra’s algorithm which runs on

single thread Ttwo_threads and Tfour_threads are our constructing time for edges on two

threads and four threads respectively

Image size T single_thread (ms) T two_threads (ms) T four_threads (ms)

Algorithm 1: Parallel Graph Constructing Algorithm

Input: Image 

Output: Connected graph G = (V, E)

1 initialize all nodes vi in V and edges ei in E

2 parallel for each pixel pi in  //each loop runs on on separate thread

3 for each pixel pj in the neighborhood (pi) of pi

4 Compute distance dij between pi and pj using Equation (1)

or (2)

5 Update dij to edge eij

7 end for

8 return graph G

3.2 Improving performance of the Dijkstra's algorithm

To compute the shortest distances from a source, say v1, to all of the other nodes in graph G, the Dijkstra’s algorithm maintains a label vector (d1, , d|V |) and a set of nodes C, called the "candidate list", starting with d1 = 0, di = ∞ for all i

≠ 1 and C = {v1} It iteratively processes the nodes from C and terminates only when C is empty Upon the termination, the shortest distance from v1 to node vi is equal to label di In each iteration, it takes the front node v* with the smallest label from C to update distances to v*'s neighborhood Different data structures of C (for example, binary or Fibonacci heaps) can be used to determine the node with smallest label in C Since each node enters and exits C exactly once, the Dijkstra’s algorithm needs |V| iterations to complete

In our work, the label correcting methods are adopted to maintain C as a priority queue with multiple entrances, from which nodes can enter or exit in constant time O(1) Node insertion follows the small label to the front (SLF) scheme [19] and node extraction follows the large label last (LLL) scheme [20]: when a node vj enters the priority queue C, its label dj is compared with the label di

of the top node vi of C Node vj is placed at the top of C if dj ≤ di; otherwise vj is placed at the bottom of C Node vi remains at the top of C while its label is less than a threshold; otherwise it is sent to the bottom of C The threshold is usually set

as the mean label value of all nodes in C The Dijkstra’s algorithm performs |V| iterations at most and requires some extra overhead to extract the node with minimum label per iteration In contrast, the SLF-LL methods requires more than

|V| iterations but the overhead per iteration is smaller The performance of the Dijkstra's algorithm and the SLF-LLL methods is evaluated on several images for comparison As Table 2 shows, the SLF-LLL methods can almost double the runtime performance achieved by the Dijkstra’s algorithm for the same image

Table 2 Performance of the Dijkstra's algorithm (T Dijkstra ), and our

SLF/LLL-enhanced method (T SLF/LLL ) on same graphs

200K 0.20 s 0.08 s 400K 0.37 s 0.18 s 548K 0.48 s 0.26 s 800K 0.70 s 0.33 s 1.28M 1.09 s 0.51 s

Algorithm 2: SLF/LLL-enhanced Dijkstra's algorithm

Output: The shortest distance vector D = {d1, d2, ., d|V|} and the label vector L = {l1, l2, , l|V|}

1 initialize D to ∞;

2 initialize L to 0;

3 initialize status vector P = {p1, p2, , p|V|} to false

4 initialize priority queue C as empty

5 for each source si in S

6 initialize di to 0

7 initialize li to i

8 initialize pi to true

9 add si to C

10 end for

11 initialize  to 0

12 while C is not empty

14 extract minimum element vi from C

15 if di greater than

size of C



then

16 add node vi to bottom of C

17 else

18 set  equal to ( - di)

19 set pi equal to false

21 end if

22 end while

23 for each node vj in the neighborhood (vi) of vi

24 if dj greater than (di+ eij) then

25 initialize dtemp equal to dj

26 set dj equal to (di+ eij)

27 set lj equal to li

28 if pj is false then //add element to C using SLF policy

37 end for

38 end while

39 return vectors D and L

4 EXPERIMENTAL RESULTS

We implemented our framework on a 64-bit PC with an Intel Xeon 4 cores CPU and 32GB memory The parallelized parts are implemented to run on CPU using

multi-threading OpenMP interface Our results are compared with [10] to illustrate the time performance We observed that our parallel implementation on a quad-core CPU was up to 3 times faster than when using a single-quad-core, thus it allows interactive manipulation on image with million pixels (see Table 3)

Table 3 Comparing results of graph based algorithms T W is the running time of Wei’s algorithm [10] Our required memory is shown in the third column The last

column reports executing time of our framework (T ours )

y

T ours

2560*1080 2.60 s 132.7 MB 1.10 s

As we adopted the multi-source Dijkstra's algorithm, our segmentation results should have similar quality in comparison with results of the method proposed in [10] The main advantage obtained is about the time efficiency which allows our work be more interactive and suitable for real-time applications As we divided the construction of embedded graph from the main Dijkstra’s algorithm, this process can be effectively solved on multi-cores CPUs The precomputing process costs less than 20% of total running time of the whole algorithm on our computer with 8 threads CPU This ratio can be smaller if we have more thread available on CPUs, and it means the expensive computational part is not the bottleneck of the Dijkstra’s algorithm anymore Thus, the main performance boost comes from the SLF/LLL-enhanced strategy as Table 3 shows

Figure 2 The procedure of our framework (a) and (c) are the original RGB

and gray images from BSDS300 database The corresponding segmentation results

are in (b, (d)

Our experiment is completed on different image datasets Some of our segmentation results are shown in Figures 2, 3 and 4 In the Fig 2, there are 3 groups of pixels picked at the background, red and yellow areas as the seeds for multiple objects segmentation It is noted that the user’s labeled seeds can be freely settled and the precomputing graph needs

to be processed only once, so the gain is more noticed in cases we want to modify the previous seeds and rerun the algorithm until the results are satisfied This is the main advantage of our work, especially for interactive image segmentation problems.

Our proposed framework is also tested on DRIVE database images for vessels detection problem Fig 3 shows the results in segmenting small vessels from the retinal

images Each pixel finds the nearest seed from itself, so the quality of segmentation strongly depends on the connectivity of similar regions in the input images It can be seen that the small vessel pixels are segmented properly with high speed, but as part of the trade-off, there could be some tiny areas appeared in background For some complex images with some physically unconnected pixels or noises, the segmentation results are drastically influenced Fortunately, we can change the seeds setting (see Fig 3) or applied some preprocessing technics to improve the quality of segmentation In the Fig 4, the input image was preprocessed using noise filters from [21], and thanks to it, our framework can produce satisfactory segmentation results.

(a) (b) (c) (d)

Figure 3 The segmentation results for vessels detection problem (a) is the

original image with 2 seeds given The corresponding segmentation result is in (b) With one more seed added in (c), the vessels’ background can be properly detected

(b)

(a) (b)

Figure 4 Fast 3-labels Iris Segmentation result (b) The original image (a) is from the

present CASIA V1 iris database.

5 CONCLUSION

In this paper, we introduce an improved multi-source Dijkstra's algorithm for image segmentation with high performance in both time and user's convenience aspects The first improvement is about designing an efficient way for graph constructing, which can be carried out in a parallel fashion on CPUs Instead of solving the shortest distance problem multiple times, we describe this process through a multi-source variant of the Dijkstra's algorithm and replace its inserting/extracting policies by the SLF/LLL rules to reduce the overhead in its propagation Plus a comfortable seed selection for users, we show that these improvements make our algorithm performs remarkably well on a variate range of large images with multiple objects contained inside

Although our work can out-perform other graph based algorithms running on CPUs with only a small additional memory usage, it is still challenging to detect the tiny objects in noise images In our algorithm, picking correct seeds is an important task to guarantee the optimal performance, hence it is highly desirable to pre-process the input images if they are strongly noised For the common images from modern scanning/capturing devices, our algorithm can efficiently produce satisfactory segmentation results

REFERENCES

[1] Y Y Boykov, M Jolly, “Interactive graph cuts for optimal boundary region segmentation of objects in n-d images”, in: Proceedings Eighth IEEE International

Conference on Computer Vision ICCV 2001, volume 1, 2001, pp 105-112 vol.1

[2] I Sobel, “An isotropic 3x3 image gradient operator”, Presentation at Stanford A.I

Project 1968 2014)

[3] R Kohler, “A segmentation system based on thresholding”, Computer Graphics and

Image Processing 15 (1981) 319-338

[4] M E Celebi, H A Kingravi, P A Vela, “A comparative study of e_cient initialization methods for the k-means clustering algorithm”, Expert Systems with

Applications 40 (2013) 200-210

[5] R Alshehhi, P Marpu, W Woon, “An automatic cognitive graph-based segmentation for detection of blood vessels in retinal images”, Mathematical Problems in

Engineering 2016 (2016) 1-15

[6] F Alkoot, “A review on advances in iris recognition methods”, International Journal

of computer engineering research 3 (2012) 1-9

[7] P F Felzenszwalb, D P Huttenlocher, “Effcient graph-based image segmentation”,

International Journal of Computer Vision 59 (2004) 167-181

[8] D Greig, B Porteous, A Seheult, “Exact maximum a posteriori estimation for binary images”, Journal of the Royal Statistical Society, Series B 51 (1989) 271-279 [9] P F Felzenszwalb, D P Huttenlocher, “Image segmentation using local variation”,

in: Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat No.98CB36231), 1998, pp 98-104

[10] X Wei, W Lu, W Xing, “A rapid multi-source shortest path algorithm for interactive image segmentation”, Multimedia Tools and Applications 76 (2017)

21547-21563

[11] L Grady, “Random walks for image segmentation”, IEEE Trans Pattern Anal

Mach Intell 28 (2006) 1768-1783

[12] C T Zahn, “Graph-theoretical methods for detecting and describing gestalt clusters”, IEEE Transactions on Computers C-20 (1971) 68-86

[13] P F Felzenszwalb, D P Huttenlocher, “Image segmentation using local variation”,

in: Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat No.98CB36231), 1998, pp 98-104

[14] X Yang, Y Su, R Duan, H Fan, S Y Yeo, C Lim, L Zhong, R S Tan, “Cardiac image segmentation by random walks with dynamic shape constraint”, IET

Computer Vision 10 (2016) 79-86

[15] N Forcadel, C Le Guyader, C Gout, “Generalized fast marching method: applications to image segmentation”, Numerical Algorithms 48 (2008) 189-211