Decision making for the software architecture structure based on the criteria importance theory

Decision Making for the Software Architecture Structure Based on the Criteria Importance Theory Procedia Computer Science 104 ( 2017 ) 27 – 34 Available online at www sciencedirect com 1877 0509 Crown[.]

Procedia Computer Science 104 ( 2017 ) 27 – 34

1877-0509 Crown Copyright © 2017 Published by Elsevier B.V This is an open access article under the CC BY-NC-ND license

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Peer-review under responsibility of organizing committee of the scientific committee of the international conference; ICTE 2016

doi: 10.1016/j.procs.2017.01.050

ScienceDirect

ICTE 2016, December 2016, Riga, Latvia Decision Making for the Software Architecture Structure Based on

the Criteria Importance Theory Sergey Orlova,*, Andrei Vishnyakovb

a

Transport and Telecommunication Institute, Lomonosova str.1, Riga, LV-1019, Latvia

b

BNP Paribas, 10 Harewood Ave, Marylebone, London, NW1 6AA, United Kingdom

Abstract

Software architectural decisions have a significant impact on the software development process as well as on the quality of developed software systems In this paper, the technique that allows selecting the optimal software architecture among several alternatives is proposed This selection technique is reduced to the criteria importance theory for decision-making problems with

a hierarchical criterion structure For applying it, we need to pick up a set of metrics that assess the characteristics of the software architecture Next, we need to determine metrics scale and create the hierarchical criterion structure with all the relations between software metric groups The results allow us making conclusions about usefulness of the proposed technique during architecture design phase for software systems

© 2016 The Authors Published by Elsevier B.V

Peer-review under responsibility of organizing committee of the scientific committee of the international conference; ICTE 2016

Keywords: Multicriteria decision analysis; Hierarchical criterion; Criteria importance theory; Software architecture; Architecture metric

1 Introduction

The formation of architecture is the first and fundamental step in the software design process and provides the framework of a software system that can perform the full range of detailed requirements1,2

Most of the existing techniques for constructing a software architecture are not well formalized and are usually not based on any mathematical theory1 Therefore, the problem of software architecture selection and analysis based

* Corresponding author Tel.: +371 26563068

E-mail address: orlovs.s@tsi.lv

Crown Copyright © 2017 Published by Elsevier B.V This is an open access article under the CC BY-NC-ND license

( http://creativecommons.org/licenses/by-nc-nd/4.0/ ).

Peer-review under responsibility of organizing committee of the scientific committee of the international conference; ICTE 2016

on quantitative evaluation is very important In other words, it would be desirable to have a formalized technique that is based on mathematical theory, and which allows the user to analyze and make decisions when choosing a software architecture or its components

Several techniques have been proposed to assist software architects in making architecture decisions3 There are several groups of such techniques, where some of them focused on architecture trade-off analysis, quality evaluation model analysis, performance optimization and some others well-known techniques3

Some other studies propose the usage of the criteria of efficiency and the architecture efficiency metrics for quantitative evaluation of a software architecture structure4 The disadvantage of this method is that the components

of the architecture efficiency metrics are explicitly defined, and we cannot easily extend them to reflect the required software architecture features

In this paper, we consider exactly the structural organization of the architecture In the initial stages of software development there were no standardized approaches for the development of the structural organization of software systems Those that existed were too much generalized and not very accurate In consequence of that the development of such systems it is required much more time and financial resources Usually, highly skilled professionals were required for such development To reduce the development costs the more formalized techniques began to emerge This work resulted in the appearance of software design patterns The patterns later became more complex and some evolved to architectural patterns2 Nowadays there are plenty of different design patterns With the help of these patterns you can build a variety of different architecture alternative, and the question then is how to make a choice among these alternatives To answer this question there have been proposed different approaches that allows to make such decisions Unfortunately, most of existing techniques does not well examine architectures’ structural organization Usually they based solely on expert evaluations3

We propose a technique that allows us to make architectural decisions when creating structure of a software architecture using set of architectural patterns In other words, this technique allows us to choose the best structural organization of the software architecture that is build using the architectural patterns

The proposed technique is based on the so-called criteria importance theory for decision-making problems5 It allows decisions to be made when choosing a software architecture system from among several alternatives and lacks the disadvantages that exist with other methods

2 Selection of optimal architecture

The software architecture is the structure of the system, which comprise software components, the externally visible properties of those components, and the relationships among them1 The software architecture is a complex design artefact

It's a complicated task to make a validation of software architecture at early design stage and it's much easier to rely on tried and tested approaches for solving certain classes of problems One of the approaches is to use architectural patterns1,2 The architectural patterns improve partitioning and promote design reuse by providing solutions to frequently recurring problems They allow reducing a risk by reusing successful designs with known engineering attributes

Creating a software architecture based on architectural patterns we need to bear in mind that different architectural patterns are focused on different areas and solve different problems That is why they might be categorized in several groups There are several approaches of architectural patterns classification1,2 Usually the software isn't limited to a single architectural pattern It is often a combination of architectural patterns that make up

a complete system For instance, there might be SOA based architecture where some services are designed using

layered or N-tier architecture approach1,2

During the software architecture design stage, there could be obtained several different reference architectures, built using architectural patterns In this case, there is a problem of comparison and choose the best architecture that

is best satisfying the software requirements

In this paper, we propose the technique that allows selection of the optimal software architecture among several alternatives This selection technique could be reduced to the criteria importance theory for decision-making problems with a “flat” model of criteria5 or a hierarchical criterion structure6 The use of “flat” model for selecting

the optimal software architecture could be found in our another paper4 In this paper, we focus on criteria importance theory for decision making problems with a hierarchical criterion structure6

To apply this technique, we need to perform several steps represented on Fig 1

Fig 1 Selection of optimal architecture

3 Model definition for software architecture selection

The mathematical model of individual decision making for multiple criteria includes the following components5:

set of alternative software architectures X; vector criterion f; preference and indifference relations of the decision

maker (DM), which are denoted as Ɋ (preference) and I (indifference)

Each alternative x from the set of alternatives X is characterized by a number of criteria f i , i = 1, , m, which are called particular criteria The ordered set of such criteria forms a vector criterion f = (f1, …, f ɬ ) The criterion f i is

the function defined on X and taking its values from z i which is called a common scale (or number of assessments of such criterion) According to the standard software engineering terminology, we call particular criteria metrics

Assume that we have a number different of software architecture options For example, the architecture of some software system can be implemented based on service-oriented pattern; alternative architecture could be based on a Multitier pattern; another with the use of a micro services architectural pattern and so on We define a set of such

alternative software architectures as X = {X i ¨i = 1, …, n} Let us assume that all alternative metrics are homogeneous, i.e they are all measured using the same scale and have the same range defined as Z0 Suppose that

the number of scale gradations is finite, then: Z0 = {1, …, q}, where q > 1

In other words, each metric f i from the set of alternative software architectures X can take the values from the set

of scale gradations Z0 Assume that all estimates are expressed in numerical form and higher values are preferable to

smaller ones Thus, each software architecture alternative X i characterized by values f i (X i) of every metric and forms

its vector estimate y = f(X i ) = (f1(X i ), , f ɬ (X i)) Alternative software architectures are compared by comparing their vector estimates The set of all possible vector estimates is defined as ܼ ൌ ܼ଴௠

The obtained individual metrics are used as part of a hierarchical structure, an example of which is shown on Fig

2 We build our hierarchical structure using a “bottom-up” approach, where on the bottom level (level 3) we have

eleven individual metrics, which are denoted as f1,…f11

In the next step, we need to combine the individual metrics into groups so that their logical grouping allows aggregated metrics to be obtained Therefore, at level two the individual metrics are grouped into disjoint sets and form new vector criteria In our example, the metrics related to this level are denoted as ݂ଵ, ଶଷଶ, ସହ଺଻ଶ , ଼ଽଵ଴ଵଵଶ

Similar operations are performed at level 1, so we get two vector criteria: ݂ଵଶଷଵ and ݂ସହ଺଻଼ଽଵ଴ଵଵଵ

Finally, at the top level (level 0) we get the single vector criterion ݂ ଴

It is worth noting that each intermediate vector criterion should have a clear meaning For example, if criterion

݂ସହ଺଻ଶ determines the domain-orientation, then metrics ݂ସ, ହ, ଺ and ݂଻ must represent some components of domain-orientation

In addition, we need to determine the coefficients of importance for each vector criterion (apart from the top-level vector criterion) To calculate the coefficients of importance of the lower lever vector criterion, we should consider the coefficient of importance of corresponded higher-level vector criterion For example, value ܽଶଷଶ depends

on ܽଵଶଷଵ

To obtain the coefficients of importance, we first need to get qualitative and quantitative information about the importance of the metrics

Fig 2 Five-level hierarchy and the corresponding coefficients of importance Į

The fact that the metric f i is equally important to metric f j is denoted as f i § f j A vector estimate that includes such metrics has indifferent preference: ݕܫݕ௜௝ where y ij — vector estimate, which is obtained from vector y by replacing its components y i and y j

The fact that metric f i from one group is more important than metric f j from another group is denoted as f i ظf j In

that case, for the pair of vector estimates y and y ij, the DM prefers the first one to the second one, and it is denoted as

y P 0 y ij In addition to this, it is necessary to be able to quantitatively indicate how many times the metrics from one group are more important than the metrics from the other group To do so, we define the matrix of degrees of importance superiority ܪ ൌ ฮ݄௜௝ฮ, where i, j = 1, , m When using it the preference relation is written as follows:

f i ظ௛ ೔ೕ

f j , which means that metric f i is h ij times as important as metric f j

The set of relations f i § f j and f i ظf j for the metrics used forms the qualitative information about the metrics importance Ÿ On the contrary, the set of relations f i ظ௛೔ೕf j forms the quantitative information about the metrics importance Ĭ

For our problems with the hierarchical structure, we need to use the degree of importance superiority applicable

to groups of criteria One of the approaches is to create a relevant NT-model based on our quantitative information6

To create the NT-model, we should get the coefficients of group importance, and based on these we get the

coefficients of importance of the individual scalar criteria and the final importance coefficients Since all the criteria

in such model are of equal importance, we can reduce the problem to a general criteria importance theory problem

We can also consider vector criteria of different levels as groups of component scalar criteria6 Thus, we can consider the level 0 criterion as a metric for selection of the optimal software architecture, i.e getting the value of this metric for all the alternatives to determine the optimal solution To apply that approach, we should (using quantitative information on the criteria importance) get the final coefficients of importance and then use a suitable decision rule

4 Metric selection

For the formation of the vector criteria, it is necessary to define a set of metrics that will be used for software architecture evaluation For our case study, we use the minimum number of required metrics At the same time, we rely on the classical approach, which is adopted in software engineering theory for assessing the quality of project design, including size (complexity), coupling and cohesion2 Moreover, we consider the functional complexity that

is measured using a modification of the well-known Function Point metric by Albrecht7 Also, we have a bit extended the number of metrics by the well-known quality characteristics from ISO/IEC 25023:20167 The resulting set of metrics can be slightly expanded, since it doesn’t change the essence of the technique in general

For a comprehensive evaluation of the architecture, we group the above-mentioned metrics into the hierarchical structure and define new aggregated metrics, including: Structural Quality (Architecture Size, Architecture Links), System Characteristics (Domain Characteristics, Agility)

Metrics that evaluate the structural quality of the architecture could be found in our another paper4, which contains the detailed description and definition of the following used metrics: Inverted Architecture Functional Points, Architecture Stability, Architecture Relational Cohesion

For the next group of metrics, we’ve chosen the metrics, that evaluate the architecture’s system characteristics Metrics from this group are based on international standard ISO/IEC 25023:20167 We split the metrics from this group into two subgroups, the architecture’s domain characteristics and architecture’s agility

We define the scale for evaluating the quality characteristics in the range from 0 to 1, where a low value means that this characteristic is poorly supported by the considered architecture and a high value of the characteristic indicates the opposite

Metrics from Domain Characteristics group evaluate the architecture’s domain-orientation Metrics from Agility group evaluate the architecture’s flexibility Metrics from these groups are listed in Table 1

Table 1 Domain characteristics and Agility metrics

Suitability Proportion of components in the software

architecture that is well fitted for the

required domain

X = A / B, where: A — Number of components in the software architecture that is

well fitted for the required domain; B — Total number of components in the

software architecture Simplicity Proportion of components in the software

architecture that have simple structure

X = A / B, where: A — Number of simple components in the software

architecture; B — Total number of components in the software architecture

Responsibility Proportion of components in the software

architecture have a single responsibility

X = A / B, where: A — Number of components in the software architecture with

single responsibility; B —Total number of components

architecture that is implemented with the

reuse of known patterns

X = A / B, where: A — Number of components in the software architecture that

reuse any software patterns; B — Total number of components

Extensibility Proportion of components in the software

architecture that could be potentially be

extended

X = A / B, where: A — Number of components in the software architecture that

could be extended; B — Total number of components in the software

architecture Scalability Proportion of components in the software

architecture that could be easily scaled

X = A / B, where: A — Number of components in the software architecture that

could be scaled; B — Total number of components in the software architecture

Interoperability Proportion of components in the software

architecture that is easy to integrate with

other systems

X = A / B, where: A — Number of components in the software architecture that

is easy to integrate with other systems; B — Total number of components in

the software architecture Availability Proportion of components in the software

architecture that is well suitable to handle

large amount of data

X = A / B, where: A — Number of components in the software architecture that

is well suitable to handle large amount of data; B — Total number of

components in the software architecture

To apply the criteria importance theory model, we need to convert the linear scale for all metrics to an ordinal scale, where values are distributed from 1 to 10 Thus, all the metrics will have a common scale and higher values are preferable to lower ones

The complete hierarchy of the obtained software metrics is represented in Fig 3 The selected metrics allow us to obtain a vector criterion for software architecture evaluation

Fig 3 Software architecture metrics hierarchy

5 Preference relations for hierarchical model

Let us determine the preference relations for all metrics in the hierarchical structure for each level For level 1,

we define the following relation: {Structural Quality ظଷȀଶ System Characteristics} Level 2 contains the following relations: Structural quality: {Architecture Links ظଷȀଶ Architecture Size}; System Characteristics: {Domain characteristics ظଷȀଶ Agility} The bottom level is composed of the following relations: Architecture Links: {Architecture Stability ظଷȀଶ Architecture Relational Cohesion; Domain Characteristics: Suitability ظଷȀଶ Simplicity; Simplicity § Responsibility; Responsibility ظଷȀଶ Maturity}; Agility: {Extensibility ظଷȀଶ Scalability; Scalability § Interoperability; Interoperability ظଷȀଶ Availability} These relations form the quantitative information (Ĭ) about the criteria importance; and they are used to calculate the importance coefficients of the groups for our NT-model

In our case, there are two groups at the 1st level, and four subgroups on the 2nd level At the first level, we have determined the following relationship: ݂ଵଶଷଵ ظ ଷ ଶ Τ ݂ସହ଺଻଼଻ଽଵ଴ଵଵଵ

As long as sum of the importance coefficients of one group must be equal to one, we obtain the group importance coefficients of the first level:

ܽሼଵଶଷሽଵ ൌଷ

ହǡ ܽሼସହ଺଻଼ଽଵ଴ଵଵሽଵ ൌଶ

The second level is defined by the following relationships: ݂ଶଷଶ ظ ଷ ଶ Τ ݂ଵǢ ݂ସହ଺଻ଶ ظ ଷ ଶ Τ ଼݂ଽଵ଴ଵଵଶ Ǥ

This allows us to get the group’s importance coefficients for the second level:

Next, we need to consider the impact of the first-level importance coefficients on the coefficients of the second level To do that, the group’s importance coefficient of the second level should be multiplied by the group’s coefficient from level 1, which includes that second-level subgroup:

ܽሼଵሽଵିଶൌ ܽሼଵଶଷሽଵ ൈ ܽሼଵሽଶ ൌ ଺

ଶହǢ ܽሼଶଷሽଵିଶൌ ܽሼଵଶଷሽଵ ൈ ܽሼଶଷሽଶ ൌ ଽ

ଶହǢ

ܽሼସହ଺଻ሽଵିଶ ൌ ܽሼସହ଺଻଼ଽଵ଴ଵଵሽଵ ൈ ܽሼସହ଺଻ሽଶ ൌ ଺Ǣ ܽሼ଼ଽଵ଴ଵଵሽଵିଶ ൌ ܽሼସହ଺଻଼ଽଵ଴ଵଵሽଵ ൈ ܽሼ଼ଽଵ଴ଵଵሽଶ ൌ ସǤ (3)

The influence coefficients of the higher levels in the definitions above are marked by the superscript "1-2"

The third level is defined by the following relationships:

݂ଶ ظ ଷ ଶ Τ ݂ଷǢ ݂ସ ظ ଷ ଶ Τ ݂ହǢ ݂ହ ൎ ݂଺Ǣ ݂଺ ظ ଷ ଶ Τ ݂଻Ǣ ଼݂ ظ ଷ ଶ Τ ݂ଽǢ ݂ଽ ൎ ݂ଵ଴ଷ Ǣ ݂ଵ଴ଷ ظ ଷ ଶ Τ ݂ଵଵଷ Ǥ

Finally, there are the following initial importance coefficients of partial criteria for the third level:

ܽଵ ൌ ͳǡ ܽଶൌଷହǡ ܽଷൌଶହǡ ܽସൌଶହଽǡ ܽହൌଶହ଺ ǡ ܽ଺ൌଶହ଺ǡ ܽ଻ൌଶହସ ǡ ଼ܽൌଶହଽ ǡ ܽଽൌଶହ଺ǡ ܽଵ଴ଷ ൌଶହ଺ ǡ ܽଵଵଷ ൌଶହସǤ (4)

Having this information allows us to obtain the final coefficients of importance

To do this, the original importance coefficients of the particular criteria are multiplied by the influence coefficients of the first- and second-level groups that include these particular criteria:

ܽଵൌ ܽሼଵሽଵିଶൈ ܽଵൌଶହ଺ Ǣ ܽଶൌ ܽሼଶଷሽଵିଶൈ ܽଶൌଵଶହଶ଻Ǣ ܽଷൌ ܽሼଶଷሽଵିଶൈ ܽଷൌଵଶହଵ଼Ǣ ܽସൌ ܽሼସହ଺଻ሽଵିଶ ൈ ܽସൌ଺ଶହହସǢ

ܽହൌ ܽሼସହ଺଻ሽଵିଶ ൈ ܽହൌ ଷ଺

଺ଶହǢ ܽ଺ൌ ܽሼସହ଺଻ሽଵିଶ ൈ ܽ଺ൌ ଷ଺

଺ଶହǢ ܽ଻ൌ ܽሼସହ଺଻ሽଵିଶ ൈ ܽ଻ൌ ଶସ

଺ଶହǢ ଼ܽൌ ܽሼ଼ଽଵ଴ଵଵሽଵିଶ ൈ ଼ܽൌ ଷ଺

଺ଶହǢ

ܽଽൌ ܽሼ଼ଽଵ଴ଵଵሽଵିଶ ൈ ܽଽൌ଺ଶହଶସǢ ܽଵ଴ൌ ܽሼ଼ଽଵ଴ଵଵሽଵିଶ ൈ ܽଵ଴ଷ ൌ଺ଶହଶସǢ ܽଵଵൌ ܽሼ଼ଽଵ଴ଵଵሽଵିଶ ൈ ܽଵଵଷ ൌ଺ଶହଵ଺Ǥ (5)

After reducing to a common denominator, it looks like: a1=150/625; a2=135/625; a3=90/625; a4=54/625;

a5=36/625; a6=36/625; a7=24/625; a8=36/625; a9=24/625; a10=24/625; a11=16/625

Therefore, N for the NT-model is the vector (150, 135, 90, 54, 36, 36, 24, 36, 24, 24, 16), and the dimension of the NT-estimates is equal to 625

By applying the final coefficients of importance for the NT-model we get NT-estimates for each alternative Since all criteria in the NT-model are of equal importance, for the comparison of two vector estimates we can use

any method of Criteria Importance Theory

6 Case study

To illustrate the example of usage of the proposed model, we conducted a case study for which we built several software architectures using different architectural patterns We considered three different architectures for airline reservation system

From all possible solutions, we have considered three different approaches for the creation of the architecture for such system All these options are based on different architectural patterns that provide various capabilities and

introduce specific requirements The considered solutions include: A1 — Architecture that is based on a Service

Oriented Architecture (SOA) architectural pattern; A2 — Architecture that is based on a Multitier architectural

pattern; A3 — Architecture that is based on a micro services architectural pattern

For the considered architectures with the given requirements we have obtained the metric’s values Metrics f1, f2

and f3 are evaluated based on the created software architectural models; and metrics f4 – f11 are obtained using expert evaluation

After metric conversion to a common ordinal scale, we have obtained the three following vector estimates: y1 = (1, 6, 7, 8, 7, 8, 7, 8, 7, 8, 7); y2 = (2, 6, 6, 7, 5, 6, 8, 4, 6, 7, 6); y3 = (4, 5, 5, 6, 4, 7, 5, 7, 7, 8, 5)

By applying the final coefficients of importance for the NT-model we get the following NT-estimates represented

in compact form: y1NT = (1150, 6135, 7190, 8150); y2 NT = (2150, 436, 536, 6301, 778, 824); y3 NT = (4186, 5265, 654, 796, 824) The superscripts indicate how many times each value is included in the NT-estimate

By substituting the above-listed data into the proposed decision-making model, we get the result that A1 is

non-dominated; A2 and A3 are dominated by A1; and A2 is dominated by A3

7 Conclusion

The paper proposes a technique that allows selection of the optimal software architecture among several alternatives This selection technique is reduced to the criteria importance theory for decision making problems with

a hierarchical criterion structure

We defined a model based on criteria importance theory with a hierarchical criterion structure that could be used

to choose the optimal software architecture To apply the model, we picked a set of metrics that assess the characteristics of a software architecture; determined the scale for the metrics; created the hierarchical criterion structure that is required to determine the coefficients of importance

For the formation of the hierarchical criterion structure, we defined a set of metrics that includes metrics from different areas, some of which assess the size, links, domain and agility characteristics of the software architecture

To validate the proposed model, we conducted a case study for which we built three software architectures using different architectural patterns The results obtained indicate that the proposed technique is applicable for solving problems of the selection of optimal software architecture

References

1 Bass L, Clements P, Kazman R Software Architecture in Practice 3rd ed Addison Wesley; 2013 608

2 Orlov S Software Engineering Technologies of Software Development: A Textbook for Universities 5th ed St Petersburg: Piter;

2016 641

3 Falessi D, Cantone G, Kazman R, Kruchten P Decision-making techniques for software architecture design: A comparative survey

ACM Computing Surveys (CSUR) Vol.43, (4); 2011 p.1-28

4 Orlov S, Vishnyakov A Multiple Criteria Decision Making for the Structural Organization of Software Architecture Computer Science and Information Technology Horizon Research Publishing 4(4); 2016 p 147-156

5. Podinovski V Criteria importance theory Mathematical Social Sciences Vol 27; 1994 p 237–252

6 Podinovski V Podinovskaya O Criteria importance theory for decision making problems with a hierarchical criterion structure: Working paper WP7/2014/04 Moscow: Publishing House of the Higher School of Economics; 2014 28

7 Fenton N, Bieman J Software Metrics: A Rigorous and Practical Approach Third Edition CRC Press; 2014 617

Sergey Orlov, Transport and telecommunication Institute, Latvia Professor Sergey Orlov was born in Ukraina He graduated in 1970, received his Doctor of engineering degree in 1992

at Riga Aviation University He has 9 years of practical experience working in IT industry His studies are focused on Software Engineering, Object Oriented Programming and Theory of Programming Languages He has 10 monographs and textbooks, more than 110 articles published in national and international scientific journals Ex-Head of the Software Engineering department, Ex-Director of Bachelor Program in Computer Science, Professor at Software Engineering department Contact him at orlovs.s@tsi.lv or sorlov@tsi.lv

Andrei Vishnyakov, BNP Paribas, United Kingdom Andrei Vishnyakov was born in Latvia He graduated from Transport and Telecommunication Institute (TTI) in 2004, received his Master of science in Computer Science in 2006 at TTI He studied at TTI Doctorate (2009-2016) His studies are focused on Software Engineering, Software Quality and Software Architecture Design Decisions He has more than 10 articles published in national and international scientific journals He has more than 10 year of experience in IT industry Contact him at andrei.vishnyakov@gmail.com