topology optimization for dht based application layer multicast f

DHT has many advantages: Decentraliza-tion, scalability, fault tolerance, load balancing, data integrity, and performance,...Those properties make DHTs are very suitable for deploying mu

Abstract 1

1.1 Motivation 2

1.2 Objectives 4

1.3 Contributions 5

1.4 Thesis structure 5

2 Background 6 2.1 Multicast 6

2.1.1 Introduction 6

2.1.1.1 IP Multicast 7

2.1.1.2 Application Layer Multicast 8

2.1.2 Application layer multicast protocols 9

2.1.2.1 Application Domain 10

2.1.2.2 Deployment Level 10

2.1.2.3 Group Management 10

2.1.2.4 Routing Mechanism 11

2.2 DHT-based multicast 12

2.2.1 Introduction of P2P Networks 12

2.2.1.1 Unstructured P2P Network model 13

2.2.1.2 Hybrid P2P Network model 14

2.2.1.3 Structured P2P Network model 15

2.2.2 DHT-based structure P2P networks 16

iv

2.2.2.1 Chord Network 18

2.2.2.2 Content Adressable Network 20

2.2.3 DHT-based multicast 22

2.2.3.1 CAN-based multicast 22

2.2.3.2 Chord-based multicast 24

2.2.3.3 Scribe 25

2.2.4 Topology optimization issues for DHT-based multicast 26

2.3 Related works on topology optimization for DHT-based multicast 26

2.3.1 SplitStream 26

2.3.2 Capacity Aware Multicast based on Overlay Network - CAM-Chord 27

2.3.3 DHT-based lightweight broadcast algorithms in large-scale com-puting infrastructures 28

3 Bandwidth Adaptive Multicast over Chord : BAM Chord 31 3.1 Overview 31

3.1.1 Node identifier 32

3.1.2 Finger table 34

3.2 Network Construction 35

3.3 Multicast method 36

4 Simulations and Evaluations 38 4.1 Simulation description 38

4.2 Evaluation 38

4.2.1 The depth of multicast tree 38

4.2.2 Control Overhead 40

2.1 Type of transmissions 6

2.2 IP Multicast 8

2.3 A Comparison between IP multicast and application-layer multicast 9 2.4 P2P Network 12

2.5 A unstructured P2P Network with a server 13

2.6 Gnutella-like P2P 14

2.7 An example of Hybrid P2P Model 15

2.8 Scheme of a classic Hash Table 16

2.9 Scheme of a Distributed Hash Table (DHT) 17

2.10 A Chord ring with 6 bit identifier Notice how the finger table is organized and how K54 is looked up following Chord’s algorithm 18

2.11 Example of Chord Finger table 20

2.12 Example of Content Addressable Network 21

2.13 Smart multicast in CAN The dot represents the starting node 23

2.14 An example of Chord-based multicast method In this example, Node 1 sends messages to all nodes in the Chord ring 24

2.15 A simple example illustrating the basic approach of SplitStream Original content is split into two stripes and correlatively, an inde-pendent multicast tree is built for each stripes A node is a leaf in one tree and a interior in the other 27

2.16 An example of CAM- Chord, neighbors of x N = [0 31], cx = 3 28

2.17 Balanced Distributed broadcast tree construction using token in 16-node Chord 29

2.18 Balanced Distributed broadcast tree construction using partition in an 11-node Chord 29

3.1 An example of CAM- Chord, neighbors of x N = [0 31], cx = 3 33

vi

3.2 An example of CAM- Chord, neighbors of x N = [0 31], cx= 3 353.3 An example of CAM- Chord, neighbors of x N = [0 31], cx= 3 37

4.1 Average path length 394.2 Maximum path length 404.3 Average path length in Chord, CAM-Chord and BAM-Chord after

32 time steps 404.4 Average link number per node in Chord 414.5 Average link in Chord, CAM-Chord and BAM-Chord after 32 timesteps 41

2.1 Definition of variables for node n, using m-bit identifiers 19

viii

In recent years, Distributed Hash Table (DHT) becomes active and ongoing area ofresearch at a lot of universities and labs DHT has many advantages: Decentraliza-tion, scalability, fault tolerance, load balancing, data integrity, and performance, Those properties make DHTs are very suitable for deploying multicast services atapplication layer and in fact, DHT-based network such as CAN, Chord, Pastry,Tapestry, etc can be used to implement Internet-scale application layer multicast.However, early DHT-based multicast systems are insufficient in addressing all

of these issues: Heterogeneous node capacity, large- scale multicast and dynamicmembership Moreover, in those system, when one node joins into system through anarbitrary way, some factors are not considered: nodes bandwidth, nodes positon onDHT network (i.e node identifiers), thus, the multicast tree can be built inefficientlyand not balance in structure The solution for assigning an appropriate number ofchild nodes to each node is far from optimal in term of bandwidth: If the number

of child nodes is too high, low capacity node will be overloaded, therefore slows theentire session multicast down If the number of child nodes is too low, high capacitynodes will be used inefficiently

In this thesis, we study the method to optimize topology for DHT-based cast We propose a DHT- based bandwidth adaptive multicast system that forcus

multi-on host heterogeneity, scalibility, fault tolerate In our system, nodes bandwidth isfirstly considered, result of this process is the basis for determining the level of thenode and correlatively caculating nodes identify Level of a node is used to definemaximum number of its child nodes As a result, in our model, each node is assigned

an optimal numbers of child nodes to forward multicast data Thus, our method canmake tradeoff between depth of the multicast tree and bandwidth of every node andtake advandtages of DHTs in maintaining multicast tree in churn overlay Systemchosen for implementation and avaluation is Chord This model is called BandwidthAdaptive Multicast over Chord: BAM-Chord

1

1.1 Motivation

Today, Internet continues to grow rapidly, many communication requirements that

we must deal with Cause of the explosion in data communications, demand ontransferring large volumes of data to many destination quickly increases To solvebandwidth problems, multicast is a effective solution As the most cost-effectiveway of delivering the same data to a multiple receivers at the same time, multicast

is suited to a large number of applications as mobile Internet, e-commerce, contentrich applications, and multi-media services: television broadcast, video conferencing.The word ”multicast” is typically used to refer to network layer multicast - IPmulticast and was proposed about some decades ago However, IP multicast stillhas not been widely deployed because of various technical and administrationalreasons The disadvantages of implementing multicast at the IP level has led to

an interesting application-level multicast Application layer multicast refers to theimplementation of multicast capability at the application layer instead of networklayer A number of application-level multicast systems have been proposed that arebuilt using structured peer-to-peer (p2p) overlays

Originally, distribute hash table (DHTs) were developed with applications likepeer-to-peer file sharing Recent years, there has been considerable interest in ap-plying DHTs to overlay multicast applications since it has many advantages: Decen-tralization, Scalability, Fault tolerance, Load balancing, and performance, Thoseproperties make DHTs are very suitable for deploying multicast services at applica-tion layer and in fact, DHT-based network such as CAN, Chord, Pastry, Tapestry,

2

etc has been already used to implement Internet -scale applicastion layer muticast.However, early DHT-based multicast systems are insufficient in addressing a number

of technical issues:

• Heterogeneous output bandwidth of Node: Since the number of child nodes

of a node in a multicast tree is decided based on the DHT algorithm withoutconsideration of nodes bandwidth capacity, a node with low bandwidth maybecome a bottleneck if it is an internal node of a multicast tree

• Effective bandwidth utilization: If a node with high bandwidth is a leaf node,the system can not utilize the bandwidth capacity of the node to deliver mul-ticast messages

• Dynamic membership: The optimization of multicast trees must be achievedeven when member nodes join or leave at anytime

To limit the multicasting load on a node, M Castro, el al propose a solution

in SplitStream (M.Castro, ), where one node informs its out-degree When it ceives a futher join requests, it pushdown this request to its children Nevertheless,Bharambe, et al explored and found heterogeneity issue is not address with Split-Stream (Feb, February 24 25 2005) With CAM CAM-Chord and CAM-Koordedesigns in (6-1, 6 10 June 2005), heterogeneity is tackled by allowing node to al-ter out-degree according to its capacity However, they did not describe how tomaintain topology when the out-degree limits Therefore, heterogeneity issue re-mains open, and should be addressed before deploying DHTs for high-bandwidthmulticasting applications S Ratnasamy, M Handley, R Karp, and S Shenkerproposed CAN-based multicast (S Ratnasamy & Shenker, Nov 7 9 2001) Thatwas the first application-level multicast scheme scales to groups of several thou-sands of nodes Solution in (S.Q.Zhuang & J.D.Kubiatowicz., 2001) can scales tolarge groups but has a single root node for each multicast group It supports theany-source model only by having the root node operate as a reflector for multi-ple senders There have been several ways to deal with dynamic membership: In(S.Q.Zhuang & J.D.Kubiatowicz., 2001), they ensure that the root node of the mul-ticasting tree does not handle all requests to join or leave the multicast group; In(J Li & Lim, 2005), use multiple interior-node- disjoint trees to avoid single points

re-of failure in tree structures; In (S El-Ansary & Haridi, February 2003) The earliest

DHT-based broadcasting work by El-Ansary, et al did not address the issue ofdynamic membership.

Moreover, in those system, one node join into system through an arbitrary way,thus, the multicast tree can be built inefficiently and not balance in structure Thesolution for assigning an appropriate number of children to each node is far fromoptimal in term of bandwidth: If the number of children is too high, low capacitynode will be overload, therefore slows the entire session multicast down If thenumber of children is too low, high capacity nodes will be used inefficiently

In this thesis, we study the method to optimize topology for DHT-based cast We propose a DHT- based bandwidth adaptive multicast system that forcus onhost heterogeneity, scalibility, fault tolerate System chosen for implementation andavaluation is Chord, a DHTs representative We extend Chord to be more flexibleand it is called BAM-Chord: Bandwidth Adaptive Multicast Chord Our methodcan make tradeoff between depth of the multicast tree and out-degree of every nodeand retain the advantages of a DHT network

• Study topology optimization problems for DHT-based multicast;

• Study models have been proposed on topology optimazation for DHT-basedmulticast, advantages and disadvantages of those model

• Propose a improvement model for deploying multicast over DHT network:BAM-Chord

• Evaluate the simulation results and compare with some existing model

1.3 Contributions

In the scope of this thesis,we propose a model for deploying multicast service overDHT-based network This thesis not only introduces the background of P2P net-work, DHT-based network, overall view of DHT-based multicast and related tech-nical issues, but also makes the following contributions:

• Proposes an improvement that can optimize network topology This makesmulticast tree more balanced and node’s bandwidth adaptation This

• Proposes an algorithm to find an optimal position when node joins into ticast session

mul-• Proposes a improvement model that can address technical issues in earlierexisting systems

on topology optimization for DHT-based multicast

• Chapter 3 descirbes proposed model for deploying multicast over Chord, aDHT representative: Banwidth adaptive multicast over Chord: BAM-Chord

• Chapter 4 shows the simulation and evaluate the simulation results in parison with Chord, CAM-Chord

com-• Chapter 5 concludes our work and points out some possible improvements ofour system in the future

IP, TCP, UDP, HTTP, FTP, However, when one sender sends the same data tomany receivers, it is not effective to use unicast.

Broadcast is a type of transmission in which information is sent from just onesource but is received by all destinations connected to the network This wouldmean that every time a node use broadcast transmission, all the other node willreceive that information

An example of broadcast is ARP (Address Resolution Protocol) which will cast the address resolution request to all other computers on the network Anotherexample is DHCP (Dynamic Host Configuration Protocol)

broad-Figure 2.1: Type of transmissions

6

Multicast is a very much different from Unicast and Broadcast in definition andapplication as well It is a type of transmission or communication in which theinformation is sent by one source to a set of destination.

Many people think is an excellent transmission method for multimedia: Internetradio broadcast, television broadcast, video conferencing, stock market tickers, slidepresentations, etc However, multicast is also suited to a large number of otherapplications: file transfers to multiple locations, dynamic web page updates, onlinegaming, news feeds, chatrooms,

Multicast is the most efficient method of delivering the same data to multiplereceivers at the same time Servers send only one data stream to reach any number

of end-users Using unicast, a server would need to send out as many streams asthere are receivers This increases the CPU load, bandwidth required to reach thataudience Using multicast, there will only be a single copy of the data streamedacross any links, therefore, can decrease bandwidth without upgrading links androuters

2.1.1.1 IP Multicast

The type of communication between one sender and one receiver (a connection tween a Web browser and a server) can be addressed by basic TCP/TP network.However, there is an emerging class of applications require multi-point communi-cations between one or more senders and several receivers in order to work well

be-In this case, data is sent from a sender to the receivers asynchronously as needed

IP Multicast is designed to solve this problem First introduced by S Deering in

1988 [1], IP multicast is a method of sending Internet Protocol (IP) datagrams to agroup of interested receivers in a single transmission In IP multicast, router has to

be equipped with multicasting capabilities Router actively participate in multicastand has a very important role Routers make copies of packets as need and forwardtowards multicast receivers

In this model, all interested host form a multicast group for communicating witheach other through sending a message to multicast router using multicast groupmembership discovery protocol as: IGMP (IPv4) or MLD (IPv6) IMGP standsfor Internet Group Management Protocol and MLD stands for Multicast ListenerDiscovery After joining to a multicast group, a host can receive all data that aresent for that belonging multicast group without knowing the source address Each

Due to the increasing number of application such as video-conferencing, party games, private chat rooms, web cache replication and database/directory repli-cation, and a lack of widely deployment of IP multicast in all IP-based networks,there has been interest in multicast protocols that can be supported without relying

multi-on the IP multicast infrastructure, and do not depend multi-on multicast-capable routers.This is the so-called application-layer multicast (ALM)

2.1.1.2 Application Layer Multicast

The concept of application level multicast is implementation of multicasting tionality at application layer instead of network layer As a result, the multicast-related functionalities are moved from routers to end-hosts The basic idea ofapplication-layer multicast is shown in Figure 2.3 : S establishes unicast connec-tions and send data to D1 and D2; In turn, D1, D2 delivers data to D3, D4 viaunicast connection respectively

func-In IP multicast, data packets are replicated at routers inside the network func-In

Figure 2.3: A Comparison between IP multicast and application-layer multicast

ALM, data packets are replicated at end hosts Logically, the end-hosts form anoverlay network, and the goal of ALM is to construct and maintain an overlay fordata transmission Since IP Multicast is implemented by network routers and avoidsmultiple copies of the same multicast packet on the same link ALM is implemented

by application end-host and can not avoid multiple copies of the same packet onthe same link Thus, ALM is less efficient than network layer multicast However,compared to IP multicast, application level multicast has some advantages:

• First, most proposed ALM models do not require any special support fromnetwork routers, therefore, ALM can be widely deployed

• Second, ALM is more easily deploy than IP multicast because ALM avoidsissues related to iner-domain multicast

• Third, in recent years, numbers of improvements in ALM technology come its weakness and ALM become a up-and coming solution, DHT-basedmulticast is an example

over-2.1.2 Application layer multicast protocols

There have been a lot of ALM protocols with a different approaches and tics In this thesis, we only review and give brief introduction about some importantcategories and general approaches of different protocols as presented in (M Hosseini,2007)

characteris-2.1.2.1 Application Domain

The application domain determines the number of users that a protocol must port, the data types a protocol supply users, and the metrics that multicast treeattempts to optimize Thus, ALMs deploy as some categorization:

sup-• Audio/video streaming: Protocols in this application domain usually involve

a single source distributing media to a large number of receivers The metric

is bandwidth and latency

• Audio/video conferencing: Protocols in this application domain usually involve

a multi-party conferencing session, interacting groups in this session have small

to medium size Important metrics are bandwidth and latency

• Generic multicast service Protocols in this application domain usually try tocreate a generic multicast service based on specific metrics that can affect avariety of applications

• Realiable data broadcast and file transfer: Reliable transfer and distribution

of files Bandwidth is the only metric

2.1.2.2 Deployment Level

Deployment level determines what level the protocol is expected to be deployed:

at the infrastructure level or end system level Infrastructure-level, also known asproxy-based ALM protocols, requires the deployment of dedicated servers/proxies

on the Internet where they self-organized into an overlay network and provides atransparent multicast service to the end-user End system level ALM protocolsassume only a unicast service from the infrastructure and expect end-system hosts

to participate in providing the multicasting functionality by taking on some of theforwarding responsibility

2.1.2.3 Group Management

After deciding what is application domain and deployment level of ALM protocol,designer must make some key decisions to determine how to manage a group ofnodes in a multicast session Some key factors are considered: How users find outthe multicast session, how users join a session, how users leave, can the users stillcontribute to existing multicast session even if they are not a part of it; Group

management is centralized or distributed and how this affects the design and serviceprovided; Mesh-first approach or tree-first approach is taken? The advantages andthe disadvantages of each approach?

Mesh-first versus Tree-first In the mesh-first approach, the mest topology iscreated at the begining, member nodes keep a connected in this topology Thenmulticast tree is built based on the mesh relative of nodes to the root In the tree-first approach, the multicast tree is built directly without any mesh The membernodes explicitly select their parent from the known members in the tree Parents

is chosen based on running algorithm to detect and avoid loops The advantages oftree-first approach is that this approach gives direct control over the tree, mer nodescan select a best parent neighbor that has enough resources Another advantage ofthe tree-first approach is that it has a lower communication overhead However, thedisadvantage is that when a member changes a parent, it drags all of it descendentswith it The advantage of the mesh-first approach is fault tolerance, it is possible

to manipulate the tree topology to a significant extent by selecting mesh neighborsand changing the metrics This approach is suitable for multi-source applications.The disadvantages of this approach is that it is difficult to do load balancing and ithas higher control overhead

Source Specific Tree versus Shared Tree Source-specific tree is built by minimizingthe length of the path to a specific individual destination Shared tree is built byminimizing the total number of hops or the cummulative end-to-end delay to forwardthe packet to all the destination The choice between a source-specific tree andshared tree usually depends on whether the multiple sources use the same overlayfor data distribution or not When there is a multiparty communication, shared treesare preferred because it is better than source specific tree in term of the maintenancecost On the other hand, source specific tree allows for optimization of the tree for

a given source, but cannot support efficiently multiple sources on that tree

2.1.2.4 Routing Mechanism

Group 1: Shortest Path This group of ALMs use RTT measurement to determinethe shortest path tree from the source to the end hosts and minimize the time delayfor each applicastion while considering the degree constraint and QoS A ShortestPath Tree constructs a minimum cost path from source node to all its receivers Forexample: Yoid, SpreadIt, TAG

Group 2: Minimum Spanning Tree This group tries to construct a low costtree A Minimum Spanning Tree is a tree with minimum total cost spanning all themembers For example: ALMI, HBM

Group 3: Clustering Structure This group constructs a cluster of nodes that can

be use to construct trees For example: NICE, ZIGZAG

Group 4: Peer-to-Peer Structure In P2P structure, Reverse-path forwarding orForward-path forwarding is used for routing

Figure 2.4: P2P Network

When Shawn Fanning created Napster in 1999, he did not imagine that today,P2P networks have been receiving increasing demand from users P2P networks aredecentralized, self-organized, and dynamic, they offer an alternative to the tradi-tional client-server model of computing In contrast to Client-server architecture,P2P networks are almost unlimited in their scalability

In recent years, due to the advantages of P2P networks, they have been widelyused in many ares: Instant Messaging, File Sharing, Collaborative Community, IP

Telephony, High Performance Computing - Grid Computing the P2P networks havebecome a focus of research at almost every major university.

2.2.1.1 Unstructured P2P Network model

Unstructured P2P network is form when each participating node connects to othersarbitrarily, and such networks can be easily constructed When a new peer wants

to join the network, it can copy links of another existing node then forms its ownlinks

Figure 2.5: A unstructured P2P Network with a server

In an unstructured P2P network, if a peer wants to find desired data in thenetwork, the query has to be flooded through the network to find as many peers aspossible However, there is no way to guarantee that flooding will find a peer hasthe desired data Thus, the main disadvantage with unstructured P2P networks isthat the queries may not always be resolved When a peer looks for data shared byonly a few other peers, its search will be likely unsuccessful and hence such networkstypically have very poor search efficiency Also, the bandwidth cost of searching onunstructured P2P network would grow exponentially to the number of connectednodes

The typical unstructured P2P networks is Gnutella Peer nodes of the systemare organized randomly in unstructured overlay and described as servent A newnode, who wants to join in the system, would perform an operation ”Join” Firstly,

it would contact with a bootstrap node Then bootstrap node would send back its

own contact list of existed node, which is chosen randomly The new node wouldstore this list of the existed node as its neighbors The new node may reach othernodes in the communication network base on its neighbor Following figure shows

an example of locating resources in a Gnutella-like P2P environment

Figure 2.6: Gnutella-like P2P

2.2.1.2 Hybrid P2P Network model

In Unstructured P2P Network model, there is no correlation between a peer and itsdata, queries might not always be resolved In hybrid P2P network model, problems

of routing and lookup in unstructured P2P Network are solved There peer nodesare divided into two types, data nodes and hub nodes Data node stores real dataand information of a hub node Hub node is the same as a ”server”, it do not storereal data It store only indexes to files in the network To join in the system, eachnode has to contact with a server Since, the server would create indexes for thesefiles, and then it stores these indexes in its database

As unstructured P2P Network model, Hybrid P2P network is easily constructedbecause it does not need to implement distribution and routing algorithms More-over, Hybrid model does not use query flooding or random searching, since it ismore efficient than unstructured model However, this model is essentially based

on ”servers” to guarantee successful routing and searching, system can not exist ifthose servers get error

In Hybrid P2P Model, eDonkey network is a typical example There is no centralhub for the network eDonkey client programs connect to the network to share files

Figure 2.7: An example of Hybrid P2P Model

eDonkey servers act as communication hubs for the clients, allowing users to locatefiles within the network

2.2.1.3 Structured P2P Network model

Unlike Hybrid P2P network where some hub nodes acting as ”servers”, the tured P2P Network is completely decentralized and self-organizing Structured P2Pmodel is constructed to achieve efficiently data searching, routing and distributionamong peers Each node has capacities of storing data and routing to other nodesbased on a common routing algorithm Nodes are arranged in a structured fashion.When a new node joins into the network, a network identify (ID) is assigned to

Struc-it, this ID is created by using a global consistent hash function Node ID representits position on the network and correlatively, a portion of data corresponding to.When a node wishes to look for a resource, it must be redirected to the node which

is supposed to hold it by using a efficient look up function

Structured P2P network is a scalable, efficient, completely decentralized andself-organizing, and load balanced model Each node stores information of O(logN)neighbors Routing algorithm allows it looking up any key with O(logN) hop time.Because of the routing algorithm based on globally consistent hash function, a set ofkeys are distributed equally to the key space However, this P2P network model faces

to some challengers such as: avoiding bottlenecks in particular nodes, adaptation to

nodes joining or leaving

Distributed Hash Table (DHT) system -a structured P2P network representativewith many advantages, is a perfectly solution to solve these chalenges, and efficientlytackles all disadvantages of unstructured P2P networks In this network model,nodes are organized in a structure of key space: Ring, d-dimensional coordinatespace, and link with other nodes in a pre-determined way: finger table, neighbortable Routing and searching process is based on key These features is very suitablefor multicast Some well known DHTs are Chord, Pastry, Tapestry, and ContentAddressable Network were introduced about the same time in 2001 Since thenthis area of research has been quite active We will discuss about DHT in the nextsection

2.2.2 DHT-based structure P2P networks

In the world of decentralization, DHTs recently have had a revolutionary effect.Knowledge of DHTs algorithms is going to be a key ingredient in future developments

of distributed applications Achieving good performance with DHTs is an activeand ongoing area of research DHTs stand for Distributed Hash Table, a typicaldefinition is: ”a class of decentralized distributed systems that provide a lookupservice similar to a hash table: (name, value) pairs are stored in the DHT” Incomputer science, a hash table is a data structure that uses a hash function to mapkeys to their associated values

Figure 2.8: Scheme of a classic Hash Table

As we can see in Figure 2.8, A hash table associates data with keys so that thehash of each key returns the position of the bucket associated with this key

Similarly,in DHT, nodes have roles as buckets in the classic hash tables Thehash of the key is used to obtain the identifier of the node responsible of a certaindata This allows each participating node can retrieve the value associated with

a given key efficiently Beside that, DHTs nodes are responsible for storing andmaintaining the mapping from keys to values, therefore, when nodes join, leave orget fail, it causes a minimal amount of disruption.

Figure 2.9: Scheme of a Distributed Hash Table (DHT)

Figure 2.9 shows how to distribute data in DHT-based P2P network Each data

is hashed into a key by using a consistent hash function The key is distributed to

a peer node that is responsible for the key

In recent years, Distributed Hash Tables (DHTs) become active and ongoingarea of research at almost every major university and lab Some characteristicalproperties of DHTs:

• Decentralization: there is no central node as a server in the system

• Scalability: the system can ensure efficiently when scales to thousands ormillions of nodes

• Fault tolerance: the system should be reliable even with nodes continuouslyjoining, leaving, and failing

• Load balancing, data integrity, and performance (in particular, ensuring thatoperations such as routing and data storage or retrieval complete quickly.)DHTs form a reliable, scalable, fault tolerant and efficient infrastructure that can beused to build more complex services, internet-scale system such as distributed filesystems, peer-to-peer file sharing and content distribution systems, cooperative webcaching, multicast, anycast, domain name services, and instant messaging Thereare a lot of research has been published on the subject to analyze various differentDHT algorithms, properties and performance In this thesis, we study the method

to optimize topology for DHT-based multicast, system chosen for implementation

and evaluation is Chord, one of the first DHTs representatives But we believe thatour method can adapt to other DHT-based multicast algorithms.

2.2.2.1 Chord Network

Chord was introduced in 2001 by Ion Stoica, Robert Morris, David Karger, FransKaashoek, and Hari Balakrishnan (Stoica et al., 2001) Chord protocol is one of thefirst typical DHT-based protocols, an infrastructure that map file identifiers ontonode identifiers The Chord protocol specifies how new nodes join the system, howexisting nodes leave the system, how to find the locations of keys, and how to recoverfrom the failure of existing nodes, provide effcient routing and searching process

Figure 2.10: A Chord ring with 6 bit identifier Notice how the finger table isorganized and how K54 is looked up following Chord’s algorithm

Chord use consistant hashing function to assigns each node and key an m-bitidentifier An hash function such as SHA-1 A nodes identifier is chosen by hashingthe nodes IP address, while a key identifier is produced by hashing the key Theterm key refers to both the original key and its identifiers Similarly, the term nodewill refer to both the node and its identifier under the hash function The identifierlength must be large enough to make the probability of two nodes or keys hashing

to the same identifier negligible

In Chord, all node identifiers are arranged in an identifier circle and the circlecannot have more than 2m nodes, where m is the number of bits in the key/nodeidentifiers Pairs (id, key) ranges from 0 to 2m−1 Figure 2.10 shows an identifiercircle with m = 6 The use of consistent hashing tends to load balancing purpose:

Table 2.1: Definition of variables for node n, using m-bit identifiers

Notation DefinitionFinger[k].start (n+2k-1) mode 2m, 1= k =m

.interval [finger[k].start, finger[k+1].start ).node First node = n.finger[k].startSuccessor The next node on the identifier circle; or

finger[1].nodePredecessor The previous node on the identifier circle

each node receives the same number of keys

Routing on Chord: Each node maintains additional information about othernodes, , in the form of a finger table, this is used to suppord routing and lookupprocess Routing process can be ablsolutely completed through linear searching:messages are step by step sent to node on Chord ring to destination However,when number of node increases, it is completely inefficient and bandwidth costlyusing Finger table using in Chord is solution for efficient routing and lookup.Structure of finger table is showed on Table 1.1

Finger talbe has at most m entries The ith entry in the table at node n containsidentify of the first node, s that succeeds n by at least 2i−1 on the identifier circle,i.e., s=successor(n + 2i−1) where 1 <= i <= m Note that all arithmetic is modulo

2m Node s is the ith finger of node n, and denote by n.finger[i] The first finger

of n is the immediate successor of n on the circle successor(n); it is refered as thesuccessor

When a node wants to locate a certain key on the Chord ring, it uses its fingertable It searches the finger table for an entry that is as far as possible but doesnot exceed the key, and forwards the request to that node The node that receivesthe request repeats this process, until a node can not find an appropriate next step.Then, the successor of that node is the node that stores the desired key An example

of routing is showed on Figure 2.11 With routing that use finger table, the lookupprocess takes at most O(logN) steps,where N is the number of nodes in the network.Stabilization: Nodes can join or leave at any time To guarantee correctness

of lookups, a ”Stabilization” procedure is used to keep nodes’ successor pointers

up to date and this procedure runs periodically at every node When node n runStabilization, it would asks n’s successor for the successor’s predecessor p If there

is no node joining between n and n’s successor, n’s successor would return a pointer,