confidence measures analysis of software security evaluation

doi:10.1016/j.proeng.2011.08.656 Procedia Engineering Procedia Engineering 00 2011 000–000 www.elsevier.com/locate/procedia Advanced in Control Engineeringand Information Science Conf

Procedia Engineering 15 (2011) 3505 – 3510

1877-7058 © 2011 Published by Elsevier Ltd.

doi:10.1016/j.proeng.2011.08.656

Procedia Engineering

Procedia Engineering 00 (2011) 000–000

www.elsevier.com/locate/procedia

Advanced in Control Engineeringand Information Science

Confidence Measures Analysis of Software Security

Evaluation

Zhengping Rena,ba*, Song Huangb, Yi Yaob, Yu Hongb

a Institute of Communication Engineering, PLAUST, BiaoYing 2 # , Yu Dao Street, Nanjing, 210007, China

b Institute of Command Automation, PLAUST, Hai Fu Xiang 1 # , Nanjing, 210007, China

Abstract

Security evaluation technologies are important to software developers and users, especially in security-critical systems However, there is very lack of effective method to acquire confidence measures of software security evaluation supported by mathematical theories In this paper, the dependability in security evaluation conclusion was studied, and confidence measures of software security evaluation were modeled from parametric estimation angle based on practical experiences and statistics theory Furthermore, a confidence measures analysis method towards evaluation conclusion of specific values was presented based on Bayes analysis to solve confidence measure acquiring problem

© 2011 Published by Elsevier Ltd Selection and/or peer-review under responsibility of [CEIS 2011]

Keywords: Software security evaluation; Evaluation system; Confidence measures; Parametric estimation; Bayes analysis

1 Introduction

With the rapid development of computer technologies, people depend on software increasingly Meanwhile, all kinds of attacks bring serious security problem to software users In security-critical systems such as military applications, security is one of the most important quality characteristics of software As the result, security evaluation technologies became research hotspots of software

* Corresponding author Tel.: +86-025-80824569

E-mail address: zhengpingren@gmail.com

engineering Software security evaluation can give the conclusion if the software product meets security requirements, using security measurement information from development and testing process It shows the security quality and builds the confidence of development, and gives important reference to user if the software product is acceptable

In this paper, the confidence measures of software security evaluation were studied by means of mathematical statistics The status quo of studies in software security evaluation was summarized first After giving the definition of confidence measures of software security evaluation from parametric estimation angle, a confidence measures analysis method towards evaluation conclusion of explicit values was presented based on Bayes analysis

2 Software Security Evaluation Methods

According to literature [1], software security is an ability of software product to protect information and data, which makes the information and data can not be read or modified by unauthorized people, and can not reject the access of authorized ones In fact, security is historically been defined more often in terms of its most popular subfactors: availability, integrity, and privacy However, security is a relatively complex concept and cannot be adequately addressed merely in terms of three parts above Typical decomposition of security can be find in [2]。Unfortunately，there is no widely accepted, industry-standard decomposition of security into a taxonomy of its component quality subfactors, and these quality subfactors do not have industry definitions As the result, there is no uniform software security evaluation method in industry People have to establish different evaluation systems according to software types (such as embed system and network application) and select different evaluation methods

Existing software evaluation methods mainly include three classes: Security Evaluation based on Security Level (SESL), Security Evaluation based on Formalism (SEF), and Quantitative Security Evaluation (QSE) SESL uses corresponding standards as evaluating criteria, such as [3][4] Literature [5] proposed SSE-CMM (System Security Engineering Capability Maturity Model) for security evaluation based on existing standards Literature [6] combines influence of availability, integrity, and privacy, using fuzzy synthetical decision-making method to evaluate system SEF proves specific security characteristic under given security hypothesis by mathematic analysis and model checking, and it mainly used in protocol analyzing and proving [7] Literature [8] established a formal evaluation model based on combined independent security factors, combined complementary security factors, and combined correlative security factors Literature [9] proposed a security evaluation method based on Bayesian function networks QSE analyzed and evaluated software system using quantitative indices, and used mathematical statistic to process the measurement data The final evaluation conclusion generally is explicit value

No matter what method, the security process is similar First, software engineering expert established

an evaluation system based on measurement theory and software type Second, trained evaluators evaluate the software security using the evaluation system on specifically steps, and obtained the evaluation conclusion Data needed in establishing evaluation system can obtain from software testing, system applying process, similar systems and simulation Since the establishing process of evaluation system must depend on the knowledge and experience of experts, they bring subjectivity to the evaluation system inevitably This is the main reason why some users of evaluation system are suspicious of evaluation conclusion and even the evaluation system itself in some situations, especially the evaluation conclusion is not according with intuitions From this angle, confidence measures are as important as the evaluation conclusion to software security evaluation Hence, it is necessary to introduce confidence measures acquiring method in software security evaluation to help the decision-maker judge the real security level of the software product

3 Confidence Measures Defining of Software Security Evaluation

Software security evaluation can be described with a function E=Me(S), where Me is an evaluating

method for some kind of specified system and S is referred to as the object software entity to be

evaluated Here we only consider the evaluation methods whose results are explicit values, and three

hypotheses are made for security evaluation process: (1) Expert knowledge introduced during the

establishing and applying of the evaluation system is independent; (2) Knowledge from every single

expert would introduce errors to the evaluation conclusion; and (3) Software security evaluation is small

samples situation

It is obvious that the expert knowledge of the establishing phase and applying phase are independent,

and even in the same phase, knowledge from different experts is also independent from each other

Therefore the impacts on the final evaluation conclusion from different expert knowledge are

uncorrelated As for the subjectivity of expert knowledge, different experts hold different opinions for the

same thing, so the evaluation deviation induced by different expert knowledge would be a stochastic

variable Hence, we can find that E is a stochastic variable form hypothesis (1) and (2) According to the

Linderberg central-limit theorem, if a stochastic variable is the sum of a set of stochastic numbers which

are independent and uniform, its probability distribution approximates to the normal distribution [10], and

so E conforms to the normal distribution, that is E~N (μ, σ 2 ) where μ is the objective security attribute

value of the software to be evaluated For software security evaluation, it is impossible to collect great

amount of evaluation data of similar software systems, thus the evaluation can be made only upon small

amount of data Under this situation, the unbiasedness and effectiveness can not be assured by the

traditional methods Instead, according to the Bayesian theory, we make full use of the prior information

to realize the calculation of the confidence measures with particular small samples

Confidence measures describe the dependability level of software security evaluation conclusion

according with the objective security attribute value, and they measure the match degree between the

evaluation conclusion and the actual security quality In this paper, we can define the confidence

measures of software security evaluation as the follows, using parametric estimation methods of

mathematical statistic theory:

Suppose S is the object to be evaluated, (e 1 , e 2 , …, e n ) is the final result obtained after the n th evaluation

of S using Me, and μ is the objective efficiency value which is an unknown parameter of the population

distribution For the given γ(0<γ<1), if statistic μˆ ( , , , )1 e e1 2 L e n and μˆ ( , , , )2 e e1 2 L e n satisfy formula (1):

{ ( , , , )n ( , , , )} 1

Then ( , )μ μ is considered as the confidence interval of μ with 1-γ as the confidence measures, here ˆ ˆ1 2

1

ˆ

μ and μ are called the lower bound and upper bound respectively ˆ2

4 Computing Method of Confidence Measures

The basic viewpoint of Bayes analysis is, in any statistics reasoning problem, a prior distribution must

be prescribed besides the samples as a basic factor to enable the reasoning process Another key point of

Bayes analysis is any resoning conclusion can only rely on the posterior distribution, and to do nothing

with the distribution of samples Now we can give the computing method of confidence measurements

based on Bayes analysis

While evaluating an objective system S, a data sample Y= (y 1 , y 2 , …, y n ) which is obtained from similar

systems can be used to acquire prior distribution of the parameters As for S, the population of the

efficiency evaluation conclusion data E conforms to the uniform distribution, that is E~N (μ, σ 2 ) with μ

and σ unknown Suppose E = (e 1 , e 2 , …, e m ) is the result conclusion after the m th evaluation of S using

evaluating system Me, we make a substitution of τ=1/σ 2 for simplicity, then we get E~N (μ, τ -1 )

4.1 Prior distribution function and posterior distribution function

According to literature [11][12], the conjugate prior distribution of (μ, τ) is :

2

0 0

( )

1 0

0

( , ) ( ) ( )

λ τ μ μ α

α β τ β

λτ

−

where μ 0 , λ 0 , α 0 and β 0 are unknown super parameters, and -∞<μ 0 <+∞, λ 0 >0, α 0 >0, β 0 >0

Suppose the evaluation conclusion sample from similar systems is Y = (y 1 , y 2 , …, y n ), these super

parameters can be calculated using maximum likelihood method with Y The expectation of (μ, τ) by

likelihood function is E[L(Y)], which is calculated by the following formula:

( )

0

2

( 2 )

0

1 0

1

1

n i

i

n

E L Y

n

n

α

α

λ

λ

+

=

(3)

To make E[L(Y)] maximum, and thus to get the super parameters in formula (2) We can get μ0 =Y ,

and β 0 =S 2 ·α 0 , where Y is the mean value of sample Y, and S 2 is the variance of sample Y

The likelihood function can not satisfy the extremum condition of λ 0 , and the variance of Y is σ 2 /n

(that is 1/(nτ)), so we use it to estimate the conditional variance to μ of the prior distribution, which is

(λ 0 τ) -1 So we have λ 0 =n

The extremum condition to α 0 of likelihood function is:

/ 2

n

In formula (4) Γ(α) is function Γ, and Γ’(α) is the derivative of Γ To calculate α 0 for different n, the

results are shown in Table 1

Table 1 Values of α 0

Parameters Values of α 0 for corresponding n

n 5 6 7 8 9 10 11 12

α 0 12.264 11.763 1.496 1.131 10.7 10.2 9.701 9.2

By acquiring the evaluation conclusion of the object software after the mth evaluating, the posterior

distribution of (μ, τ) denoted as π*(μ, τ|E) is:

1) With a known E, the conditional distribution of any μ and τ by posterior distribution is uniform

distribution, marked as P 1 *(μ|τ, E), its expectation is μ 1 and the precision (the reciprocal of the variance)

is λ 1 τ, that is:

2

1 ( 1 )

1 ( , )

2

λ τ μ μ λτ

μ τ

π

−

−

In formula (5), μ1=(λ μ0 0+ ⋅m Y) /(λ0+ )m and λ 1 =λ 0 +m

2) The marginal distribution of τ conforms to Г distribution, marked as P 2 *(τ), then we have:

( )

( )

1

1 1 1

2

1

P

α

α e β τ1

β

α

− −

∗

where α1=α0+0.5m, ( )2 ( ) (2

1

i

=

= + ⋅∑ − + ⋅ − + ), m is the amount of subsample of E, and E is the mean value of E Thus π*(μ,τ|E)=P 1 *(μ|τ,E) P 2 *(τ), then we can carry on

with the point estimation of parameters and confidence estimation according to the posterior distribution

π*(μ,τ|E)

4.2 Point estimation of parameters and conclusion confidence estimation

The point estimation of μ and σ can be calculate by μ μˆ= 1 and σˆ = β α1/( 1−1)

If it is required that the confidence measure is γ, the upper bound of e is y U, and the lower bound is y L,

then the double-side confedence point estimation is:

dou

⎞

⎟

If only y L is required, then the unilateral confedence point estimation is:

( 1 1 ) ( 1) 1

2

+

In formula (7) and (8), t(z, 2α 1 ) represents the integral quantity of t distributed density function on

interval (-∞, z) with 2α 1 as its degree of freedom, that is the result of the cumulative probability

distribution function at z, the function is t distribution with degree of freedom 2α 1

While the confidence measure is γ, the double-side confidence lower bound estimation of the

evaluation conclusion is ˆR =dou r, and it can be calculated reversely by formula (9):

( )2

1

2 1

2

1

U L

U L r

y y

τ

−

+∞

−

⋅Φ

−

1

U L

τ

5 Conclusion

In the past evaluation of software security, people usually focus on scientificalness and authority of the

evaluation system, and left the confidence level of the conclusion less concerned While in strictly

speaking, the final evaluation result should be a concrete one connected with a particular confidence measures, others without such metrics are lack of strictness and objectivity

This paper proposed a method to acquire confidence measures of software security with definite values

as the evaluation results Such method can be applied with small samples of software security evaluation data and is able to obtain both of the evaluation conclusion and its corresponding confidence measures using the Bayesian theory It solves the problem of evaluation conclusion dependability and helps all the stake-holders of the software product obtain more scientific decisions

6 Copyright

All authors must sign the Transfer of Copyright agreement before the article can be published This transfer agreement enables Elsevier to protect the copyrighted material for the authors, but does not relinquish the authors' proprietary rights The copyright transfer covers the exclusive rights to reproduce and distribute the article, including reprints, photographic reproductions, microfilm or any other reproductions of similar nature and translations Authors are responsible for obtaining from the copyright holder permission to reproduce any figures for which copyright exists

Acknowledgements

This work is supported by the National High Technology Research and Development Program of China (No.2009AA01Z402)

References

[1] GJB5236-2004 Military software quality metrics 2005

[2] Donald G Firesmith Common concepts underlying safety, security, and survivability engineering CMU/SEI-2003-TN-033

2003

[3] National Security Institute-5200-28-STD Trusted computer system evaluation criteria

http://www.radium.ncsc.mil/tpep/library/rainbow/5200.28-STD.html

[4] ISO/TEC 15408-1-2001 Common criteria for information technology security evaluation CCMB-2009-07-001

http://www.common criteriaportal.org/cc/

[5] Rushun Song Information system security risk evaluation based on SSE-CMM Application research of computers

2000.9(28):98-100

[6] Qingyi Tian Research and system implementation on information security level protection model Chong Qing: Chong Qing

University Press 2006

[7] David M, William H, Kishor S Model-based evaluation: from dependability to security IEEE Transactions on dependable

and secure computing 2004, 1 (1):48-65

[8] Qiang Yan, Zhong Chen, Yunsuo Duan, Lifu Wang Information system security measurement and evaluation model

Chinese of journal electronics Vol 31 No.9 2003.9

[9] Zhiguo Su, Qingkai Zeng Security management oriented quantitative security evaluation model Computer engineering

2008.2(34): 105-107

[10] Yu Shi, Yaowu Li Applications of probability and mathematical statistics Xi’An: Xi'An Jiao Tong University Press 2005 [11] Thomas Leonard; John S J HSO Bayes methods Beijing: China Machine Press 2005

[12] Shisong Mao Bayes statistic Beijing: China Statistical Publishing House 2005.