Development and research of models of organization distributed cloud computing based on the software defined infrastructure

Development and Research of Models of Organization Distributed Cloud Computing Based on the Software defined Infrastructure Procedia Computer Science 103 ( 2017 ) 569 – 576 Available online at www sci[.]

Procedia Computer Science 103 ( 2017 ) 569 – 576

1877-0509 © 2017 The Authors Published by Elsevier B.V This is an open access article under the CC BY-NC-ND license

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems”

doi: 10.1016/j.procs.2017.01.064

ScienceDirect

XIIth International Symposium «Intelligent Systems», INTELS’16, 5-7 October 2016, Moscow,

Russia Development and research of models of organization distributed cloud computing based on the software-defined infrastructure

I Bolodurina, D Parfenov*

Orenburg State University, 13, prospekt Pobedy Orenburg460018, Russia

Abstract

Data centers are widely used for the placement of highly loaded applications and solutions used to process large amounts of data (Big Data) The study developed a model of cloud application and services The novelty of this model is the use of methods of intellectual analysis and prediction of dynamic characteristics in the study of multicomponent systems In order t o provide flexibility built models used agent-oriented programming Plurality objects of data center modeled as agents are discussed Each node in this case is a platform of agents, which controls other agents Agents that act as data resources for cloud applications and cloud services A distinctive feature of the proposed model is a set of individual parameters needed to perform tasks It formalizing reflected in the quality of service compliance requirements as part of resource characteristics

© 2017 The Authors Published by Elsevier B.V

Peer-review under responsibility ofthe scientific committee of the XIIth International Symposium «Intelligent Systems»

Keywords: cloud computing; computing resources; defined networks; virtual data center; defined infrastructure;

software-defined storage

1 Introduction

In recent years, cloud computing has become a popular approach, used to provide an access to the services and

number of limitations, caused both by consumers and by providers of cloud services For the consumer, cloud resources are endless in terms of scalability However, if we consider the economic aspect of their consumption,

* Corresponding author

E-mail address: parfenovdi@mail.ru

© 2017 The Authors Published by Elsevier B.V This is an open access article under the CC BY-NC-ND license

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems”

then their ability to scale significantly narrows From the side of cloud service provider the set of services and computing power are limited In order to maximize the economic aspect of cloud services by increasing the number

of users, providers have to apply policies for the flexible usage of allocated resources, while minimizing operation costs Thereby in today’s virtual and physical data centers the problem of resources and cloud applications management is an important issue, because it has a direct impact on the operation costs

In the past few years the large IT corporations (such as Amazon, Google, IBM, Microsoft, Oracle et c.) develop

a renewed approach to the management of the resources and objects in data centers, used for the cloud applications The main trend in this sphere is the optimization of the data centers resource consumption In recent our researches

we have developed the approaches to the storage optimization of cloud applications’ data and to improvement in

application instances in cloud environment In addition, the review of researches in this field has shown that the problem of optimizing resource selection for the specific types of cloud applications is insufficiently investigated5,6,7

Using of virtual infrastructure becoming the primary model of consumption today, the ability to create infrastructures on-the-fly is an absolute necessity for providers In addition, overlapping of these virtual infrastructures on the same physical infrastructure to satisfy diverse service level agreements (SLA) requirements (at same time) is a challenging task On one hand, this requires intelligent decisions and placement while provisioning (i.e., virtual to physical mapping) On the other hand, continuous transformation of workload and physical infrastructure mandates the infrastructure to be continuously mutating, adapting constantly to honor the SLA guarantees Software-defined infrastructure (SDI) has completely changed the way of organization and management

of infrastructure, offering more simplicity, flexibility, and monetary benefits compared to a traditional view of infrastructure cloud computing

The traditional problem of modern data centers is to ensure QoS In recent years in real data centers to ensure the service level agreement used two ways have gained best result in enterprise data centers – virtualization of physical servers and virtualization of network But this methods is not always work good

We propose the approach to solve the all of this problems Our approach is based on the dynamical management

of resources cloud applications This paradigm supported in software-defined infrastructure of data centers But this way has other problems The main bottleneck between the infrastructure levels is storage systems With the rapid growth of data centers and the unprecedented increase in storage demands, the traditional storage control techniques are considered unsuitable to deal with this large volume of data in an efficient manner Existing approaches of

solution for this issue by abstracting the storage control operations from the storage devices and get it inside a

But the inadequate resource allocation, lack of I/O performance prediction and insufficient isolation are affecting the storage performance in the cloud storage environment In order to guarantee the quality of service, SDS is an effective approach in data centers However, the lack of intelligence, robustness and self adjustment are blocking the applications and promotions of SDS heavily This paper focuses on the QoS resource scheduling problem in virtual data center (VDC) For the understanding of all the problems we have built a model of VDC and generalized model

of cloud application (Section 2) Section 3 describes algorithms of scheduling queue in cloud storages We present the efficiency experiment results in Section 4 and the conclusion in Section 5

2 Modeling

For understanding the operation principles of a cloud application, we need to define its place in the infrastructure

of the virtual data center Virtual data center is a dynamic object, changing in time t, its state can be formalized as a

directed graph of the following form:

VirtDC Node t Connect t CloudAppl t , (1)

where vertices Node t( ) {Node t i( )}i 1,N are active elements of the VDC infrastructure (computing nodes, storage systems and others);

i V

Connect t Connect t are active user connections to the cloud applications;

1,

( ) { i( )}

i C

CloudAppl t CloudAppl t are active instances of the cloud applications, launched on the virtual resources The major feature of cloud applications is the approach, in which users have access to them and to their services, and they do not know anything about their actual location Most often, users know only the address of aggregation node and application name The cloud system automatically selects the optimal virtual machine for the request, which will have been processed on it

2.1 Generalized model of cloud application

Before we talk about the resource allocation for the cloud applications, it is necessary to determine their structure, basic parameters and key characteristics of their operation, affecting the efficiency of their usage For this purpose,

we have developed a generalized model of cloud application

The generalized model of cloud application is a multilayer structure formalized in a form of graphs, describing the connections of individual elements The model can be represented in the form of three basic slices, detailing the connections of the specific objects of VDC infrastructure: applications, related services and allocated resources The cloud application is a weighted directed acyclic graph of data dependencies:

 , 

Its vertices G are tasks, which get information from the sources or process it in accordance with the users’ requests; directed edges V are dependencies of the tasks on the data sources between corresponding vertices

where Res are the resource requirements; NAppl is the number of application instances; Utime is the estimation of the users request execution time; SchemeTask are communication schemes of data transmission between sources and

computing nodes

following tuple:

v u v Tdata Mdata Fdata Vdata Qdata (4)

where u and v are linked vertices; Tdata is the type of transmitted data; Mdata – the access method to the information source (REST, JSON and others); Fdata – the physical type of accessed object (file in the storage system, local file, distributed database, data services and so on); Vdata – the traffic volume estimated on the

accessed data (in Mb), the requirements for the QoS (quality of services)

The originality of the model is in the fact that for each application the consolidated assessment of its work with data sources is calculated It allows to predict the performance of the whole cloud system

As mentioned earlier, cloud service in one of the key slices in the generalized model of cloud application Cloud services as an autonomous data source for the application, for which it acts as a consolidated data handler Generally, cloud service is highly specialized and designed to perform a limited set of functions The advantage of

connecting cloud application to the service is in an isolated data processing, in contrast to the direct access to the raw data, when cloud application does not use a service The usage of services reduces the execution times of user requests Cloud service is formalized as a directed graph of data dependencies The difference lies in the fact that from the user point of view cloud service is a closed system Cloud service can be formalized as a tuple:

where AgrIP is the address of aggregation computing node; NameServ – the service name; Format – the required

format of output data

The aggregator of a service selects optimal virtual machines, on which it is executed In addition, all its applications are distributed between predefined virtual machines or physical servers Their new instances are scaled dynamically, depending on the number of incoming requests from cloud applications, users or other services

To describe the assignment of cloud applications and services in the virtual data center infrastructure, we have also implemented the model of cloud resource Cloud resource represents an object of DC, which describes the behavior and the characteristics of the individual infrastructure elements, depending on its current state and parameters The objects of virtual data center are disk arrays, including detached storage devices, virtual machines, software-defined storages, databases of various kinds (SQL/NoSQL) In addition, each cloud service or application imposes requirements on the number of computing cores, the RAM and disk sizes, the presence of special libraries

on physical or virtual nodes, used to launch their executing environments Each cloud resource can be formalized as follows:

Cloud s T s Param State Core Rmem Hmem Lib (6)

where TRes is the type of resource; Param – the set of parameters; State – the state of resource; Core – the number

of computing cores; Rmem – the size of RAM; Hmem – the size of disk; Lib – the libraries requirements

The distinctive feature of the proposed model is the resource universality, which allows to explore it from the user’s points of view (as a closed system) and from the point of view of virtual DC software-defined infrastructure (as an open system)

The novelty of the model is the simultaneous description of the data placement, the associated cloud applications and the state of virtual environment, considering network topology

2.4 Model of software-defined storage

We developed the model of the software-defined storage, which details the resource model of the virtual data center It is represented in the form of a directed multigraph, its vertices are the VDC elements, which are responsible for applications’ data placement (e.g virtual disk arrays, DB and so on) Thus as arcs perform data communication with the cloud applications and services The weights will be used in the consolidated assessment, consisting of characteristics of the data demand, as well as utilization of specific devices, both the physical and the virtual level

where StgNode – elements of the virtual data center, responsible for the placement of cloud application data; Data –

data communication with cloud applications and services

Each element of the software-defined storage has individual characteristics:

 , stg, ( ), ( ), ( ), stg( ) ,

ki kij j

ki

Vol t  ‰N – the available storage capacity in Mb; R ki( )t and W ki( )t are read and write speeds;

( ) {" "," "}

stg

ki

s t  online offline – its state

The generalized model of cloud-based applications as the basis for planning the development of algorithms of cloud services and applications on top of virtual machines and physical servers to virtual data center infrastructure,

as well as scheduling algorithm and data access cloud services and applications, which is an original approach

3 Algorithmic implementation

Computing and networking resources provide the bulk of the support for cloud applications But current resource control systems are not capable of controlling heterogeneous resources that include computing and networking resources in combination with other resources In addition, current control systems are not capable of realizing the flexibility, scalability and economic advantages that would be inherent in the integrated control of converged heterogeneous resources in virtual data center infrastructure

To correct this disadvantage, we have implemented an algorithm to optimize workload and performance of cloud applications and services in virtual data center The algorithm allows providing QoS, and is based on methods

of ant colony and heuristic data mining

The initial problem of optimization can be divided into four subproblems of preparation of plans of access to resources:

x Mapping cloud applications and services to virtual machines

x Mapping virtual machines to physical computational nodes

x Mapping virtual storages to data storages

x Mapping virtual channels to physical network channels

We can solve the subproblems 1-3 by an algorithm based on the ant colony optimization approach For 4 subproblem we use greedy algorithm

The algorithm’s scheme is:

x Build the graph R The graph form is chosen so that path in the graph determines mapping of cloud applications and services to virtual machines;

x Build the graph G in the graph R The graph form is chosen so that path in the graph determines mapping of virtual machines and virtual storages;

performance and maximum data storage memory volume

according to path B i

x Update the pheromone values on the arcs of the graph A depending on the target function values Fi

x If the stop condition isn’t satisfied, go to stage 3

As a result, the algorithm generated table of pheromones It is used for routing data flows The values of the

pheromones:

( ) ,

k

k

nd nd

k

k nd

id

i N

R

R R

R

H W



¦

(9)

We describe the algorithm ants in more detail

select next to go to the top among the neighbor, not visited vertices according to the probabilities calculated using the formula:

1 (| | -1)

nd

k

l p

N

D



ant - the value characterizing the level of utilization of the arc

If selection is possible to produce, then the total time an ant, add the time spent on the transition, designated as the selected top and add it to the list of visited vertices;

If the transition is not possible or an ant lifetime exceeds TTL, then exit the algorithm

For each sub-paths from this vertex to update:

a) the statistics table according;

b) the routing table;

c) a table of pheromones

Let us explain some of the details of the algorithm and method call it At regular intervals from each node is sent

to the ant in the selected destination node in order to find the optimal path For each node the ant s selection takes place in accordance with the statistics of the traffic that passes through s, with probability proportional to the volume

of a previously transmitted traffic from s to d

In the process of direct travel ant (steps 1-4) to the end point, it records the history of visited vertices, which contains a pair - number visited vertex and the transition to this the top of the previous one Next vertex selected in substep 1) in step 4, in accordance with the current level of pheromones of all vertices adjacent to the current node

the destination node, it is destroyed The best value for α QoS ranges [0.2, 0.5] the middle segment was taken for

long time In this case, if the journey time exceeds the TTL threshold, an ant destroyed After a "warm-up" the network routing table is filled with the best ways Selection of the optimal route from s to occur based on the finding

of the greatest value in the j column of the routing table for each node, starting from s

4 Experimental

For the experiment we have used the cloud system It includes 4 OpenFlow switches (2 HP 3500yl, 2 Netgear GSM7200), 8 computing nodes (32Gb RAM, 4 cores), 1 server (32Gb RAM, 8 cores) with OpenFlow controller and

1 server (32Gb RAM, 4 cores) for monitoring function As a selected fat tree topology with three levels Routers connected compounds having the speed 1000 Mbit / s, and the computers are connected to a third level router via the second level network connections at 100 Mbit/s

We prepare two research scenarios that differ in the number of physical hosts and number types of cloud applications First scenario used 10 hosts connected to the 4 types of cloud applications; second scenario used 35 hosts connected to 8 types of cloud applications

[1, 10]

i

cloud The probability of selection of each type of task is equal to 0.5 It should be noted that if more than 10 hosts, which employ the processes of the first type, and 5 for the second, QoS requirements are guaranteed to be broken During the experiment, the percentage of QoS violation, depending on the intensity of the revenues applications Graphs of the first and second scenarios are shown in Fig 1,a and Fig 1,b, respectively Each point corresponds to the index, averaged over 20 randomly generated flow problems

Fig 1 Dependence percent QoS violations of intensity

In both cases the growth disorders bandwidth requirements and latency problems with increasing flow rate, and the second scenario data values more Designed algorithms are effective, it requires a comparison with existing analogues, which is planned to be done in terms of the emulated network

5 Conclusions

Our research has resulted in the generalized model of cloud application in virtual data center The model has been developed considering the requirements for computational resources These models reflect all the basic characteristics of modern cloud application and services as well as the use of SDN For data flow routing in SDNs

we have developed the reactive and proactive methods

Acknowledgements

The research work was funded by Russian Foundation for Basic Research, according to the research projects No 16-37-60086 mol_а_dk and 16-07-01004

References

1 Bein D, Bein W, Venigella S Cloud Storage and Online Bin Packing Proc of the 5th Intern Symp on Intelligent Distributed Computing

2011; p 63-68

2 Nagendram S, Lakshmi JV, Rao DV et al Efficient Resource Scheduling in Data Centers using MRIS Indian J of Computer Science and

Engineering 2011; 2(5), p 764-769

3 Arzuaga E, Kaeli DR Quantifying load imbalance on virtualized enterprise servers Proc of the first joint WOSP/SIPEW international

conference on Performance engineering 2010; p.235-242

4 Mishra M, Sahoo A On theory of VM placement: Anomalies in existing methodologies and their mitigation using a novel vector based

approach Cloud Computing (CLOUD), IEEE International Conference 2011; p.275-282

5 Korupolu M, Singh A, Bamba B Coupled placement in modern Data Centers IEEE Intern Symp on Parallel & Distributed Processing 2009;

p.1-12

6 Singh A, Korupolu M, Mohapatra D Server-storage virtualization: integration and load balancing in Data Centers Proc of the 2008

ACM/IEEE Conf on Supercomputing 2008; p.1-12

7 Plakunov A, Kostenko V Data center resource mapping algorithm based on the ant colony optimization Proc of Science and Technology Conference (Modern Networking Technologies) (MoNeTeC) 2014; p.1-6

8 Darabseh A, Al-Ayyoub M, Jararweh Y, Benkhelifa E, Vouk M, Rindos A SDStorage: A Software Defined Storage Experimental Framework

Proc of Cloud Engineering (IC2E) 2015; p.341- 346

9 Parfenov D, Bolodurina I Approaches to the effective use of limited computing resources in multimedia applications in the educational

institutions WCSE 2015-IPCE; 2015

The novelty of the model is the simultaneous description of the data placement, the associated cloud applications and the state of. .. online offline – its state

The generalized model of cloud- based applications as the basis for planning the development of algorithms of cloud services and applications on top of. .. communication with the cloud applications and services The weights will be used in the consolidated assessment, consisting of characteristics of the data demand, as well as utilization of specific