An efficient method for automated regression test data generation for cc++ projects

An Efficient Method for Automated Regression Test Data Generation for C/C++ Projects An Efficient Method for Automated Regression Test Data Generation for C/C++ Projects Hoang Viet Tran Center of Mult[.]

An Efficient Method for Automated Regression Test

Data Generation for C/C++ Projects

Hoang-Viet Tran Center of Multidisciplinary Integrated

Technologies for Field Monitoring (FIMO)

University of Engineering and Technology,

Vietnam National University, Hanoi, E3 building, 144 Xuan Thuy, Cau Giay, Hanoi, Vietnam thv@vnu.edu.vn

Pham Ngoc Hung Faculty of Information Technology University of Engineering and Technology, Vietnam National University, Hanoi, E3 building, 144 Xuan Thuy, Cau Giay, Hanoi, Vietnam hungpn@vnu.edu.vn

AbstractÐRegression test is a well-known method to ensure

that both unchanged and evolved functions of the evolving

software are in good quality This is known to be an expensive

task even with automated test data generation methods For this

reason, this paper proposes an effective method to maintain and

reuse the test data generation results from previous versions

for regression test of the evolved version The key idea of the

proposed method is that for unchanged units, we can reuse the

whole previous test data For evolved units, we reuse as many

as possible the solutions of the unchanged test path constraints

and generate new test data for only the new or updated test

path constraints The analysis shows that the proposed method

has a high potential of applicability in regression test of C/C++

projects in practice We give discussions about several evolving

scenarios of a given unit and how the proposed method comes

to effective in such scenarios

Index TermsÐtest data generation, C/C++ projects, concolic

testing, dynamic symbolic execution, regression test

I INTRODUCTION Software testing is an important phase in a software life

cycle which accounts for up to 50% of software development

cost [2] This is especially correct when the software is

continuously evolved during the maintenance phase where

bugs are fixed and new features are regularly added The test

which is performed when a software is evolved to ensure that

all its functions work correctly is called regression test As

an obvious result, regression test plays a key role among the

software quality assurance methods However, this is a time

costly and boring task for a manual regression test

Conse-quently, many efforts have been spent on making regression

test an automatic process Although we can use such tool as

Selenium [12] to automatically perform the test, this is not

a fully automated process since the tool does not help us

generate test data automatically For this reason, we need fully

automated methods which can generate test data, run the test

data automatically, and give us the coverage results

There have been many researches on the automated test data

generation method such as CREST [5], KLEE [6], DART [11],

SDART [16], CUTE [17], CAUT [18], and PathCrawler [21],

etc However, all those researches do not mention the

re-gression test in the context of software evolution In case

we simply apply those methods to automatically generate test data when doing regression test, the test data generation cost is still high for real projects in practice In addition, regarding the regression test, there have been many researches published such as the researches of BiczÂo et al [3] and Yoo and Harman [22], etc Although these researches are about the regression test, they focus on another aspect of the regression test problem That is to find a method for effectively select the test cases for doing regression test, not about how to efficiently generate test data and reuse the generated test data throughout the software life cycle Moreover, there are some other methods about automatically generating test data [9], [10], [23] Nevertheless, they focus on other aspects of the problem, not on a simple but effective method of maintaining and reusing the test data set during regression test Duy

et al introduced a method to automatically generate test data for regression test Lustre/SCADE programs using model checking techniques [9], [10] Yoshida et al introduced a novel technique for automated and fine-grained incremental generation of unit tests through minimal augmentation of an existing test suite [23] The techniques of Yoshida et al use the iterative, incremental refinement of test-drivers and symbolic execution, guided by a diagnostics engine In our opinion, these methods have not used of a simple but effective method during regression test That is to maintain and reuse both the test data set and the test path constraints solving results of the SMT-solver when generating test data for evolving projects This paper proposes a method for regression test data generation of evolving C/C++ projects They key idea of the method comes from an observation that in regression test, most units of the projects are unchanged We can reuse the generated test data from previous versions for doing regression test For evolved units, we can reuse the generated test data from previ-ous versions for unchanged test paths By reusing the test data generated from the previous test path constraints solutions, we can save a lot of solving time of the SMT-solver The set of solved test path constraints and their corresponding generated test data is maintained and reused during the software life cycle This will effectively reduce the cost for regression test

data generation of evolving projects A logical analysis shows

that the proposed method can significantly reduce the test data

generation cost when doing regression test in the context of

software evolution Several aspects of the software evolution

and regression test data generation are discussed in the paper

The paper is organized as follows At first, some basic

concepts about the test data generation are briefly described

in Section II Then, the overview of the test data generation

method for C/C++ projects using concolic testing method

is presented in Section III Later, the proposed method is

described in Section IV In Section IV, we give the analysis

about the proposed method and discussions about several

evolving scenarios of a given project Some researches related

to the proposed method are discussed in Section V Finally,

we conclude the paper in Section VI

II BACKGROUND This section presents some key concepts which will be used

in this paper

Definition 1 (Control Follow Graph - CFG ): Consider a

unit function, its corresponding CFG is a directed graph G=

(V, E), where V = {v0, v1, , vn} is a set of vertices and

E = {(vi, vj)|(vi, vj) ∈ V } is the set of its edges The set

of vertices V represents all statements of the unit The set of

edges E is a set of directed edges in which each edge(vi, vj)

represents the corresponding state from vi to vj

A Test path is one of the key concepts in test data generation

methods A test path is a sequence of vertices from the start

to the end vertex of a CFG A test path is formally defined as

follows

Definition 2 (Test path): For a given CFG G = (V, E), a

test path is defined as a path {v0, v1, , vn|(vi, vi+1) ∈ E},

where0 ≤ i ≤ n − 1, v0 and vn are the start and end vertex

of G, respectively

Test path constraintsare the results of a symbolic execution

for a given test path These constraints are converted into an

Lib expression as an input to be solved by an

SMT-solver From the solution of this SMT-solver, the

correspond-ing test data is generated

Definition 3 (Test path constraints): For a given test path

T P of the function f n under test, its corresponding test

path constraints are defined as the following logic expression:

P C = pc0 ∧ pc1 ∧ ∧ pcn−1, where n is the number of

conditions on T P ; pci is a constraint (0 ≤ i ≤ n − 1); pc0

and pcn−1are the path constraints corresponding to the start

and end conditions in T P , respectively

III TESTDATAGENERATION FORTHEFIRSTVERSION

A Concolic Testing

Concolic testing is the combination of Concrete and

Symbolic execution methods The method was first mentioned

by Larson and Austin in 2003 [13] The name ªConcolicº was

proposed by Sen et al in 2005 [17] The overview of concolic

testing method is shown in Figure 1 which contains three

phases marked with their corresponding numbers as follows:

• Phase 1: Initial test data set is randomly generated

Start

Generate test data at random

Execute test data & analyze covered test paths

Uncovered test path exists?

Generate

a new test data

End Initially

generated test data

Generated

Yes

(3)

Fig 1 The overview of concolic testing method.

• Phase 2: Test data generated at Phase 1 and 3 are executed and analyzed to find covered test paths

• Phase 3: If an uncovered test path exists, the method generates a new test data to cover the uncovered test path Otherwise, the test data generation process stops Although many researches have been published using con-colic testing to automatically generate test data for C/C++ projects such as CREST [5], DART [11], CAUT [18], KLEE [6], PathCrawler [21], etc., there exist certain problems These problems are addressed and improved in researches of Nguyen et al [15], [16] In this paper, we do not discuss in details those problems Hereafter, when mentioning concolic testing, we refer to the method proposed by Nguyen et al

to generate test data when testing the first version of C/C++ projects [15], [16] Details of the concolic testing phases are described in the sections below

B Random Test Data Generation

Phase 1 of concolic testing method is to generate an initial test data set at random Then, the initial test data set is passed

to the Phase 2 for execution and further analysis Details of Phase 1 is shown in Algorithm 1

Algorithm 1: Generate the initial test data set Input: f : unit under test

Output: T : the random test data set

1 begin

2 T ← ∅

3 while code coverage <100% do

4 foreach parameter in f do

6 param value← generate a random value for

parameter

10 if code coverage is not increased and

T HRESHOLD is reached then

13 end

14 return T

15 end

In the beginning, Algorithm 1 initializes T with an empty set (line 2) While the code coverage of the generated test data set T is less than100%, the algorithm does the following steps The first step is to generate a new test data at random

(line 4 - 8) Then, the newly generated test data is added to

the test data set T (line 9) Later, the algorithm checks if the

code coverage of T is increased in comparison with that of the

previous step If the coverage is not increased for a continuous

T HRESHOLD times, the algorithm stops the random test

data generation process (line 10 - 12) and returns T (line 14)

In this algorithm, T HRESHOLD is a predefined number

of continuous times where random test data generation cannot

increase the coverage If this T HRESHOLD is reached, test

data generation process needs to move to Phase 2 and Phase 3

to generate test data based on the CFG of the unit under test

C Test Data Execution and Test Path Analysis

In Phase 2, the newly generated test data from Phase 1 and

Phase 3 are executed Then, the execution results are analyzed

to find covered test paths From this analysis, the algorithm

can find an uncovered test path The uncovered test path will

be passed to Phase 3 for a new test data generation Details

of this phase is presented in Algorithm 2

Algorithm 2: Execute newly generated test data and

analyze covered test paths

Input: f : the unit under test; T : test data set

Output: uncovered test path: a new uncovered test path

1 begin

2 Execute newly created test data in T

3 test paths← get all covered test paths

4 uncovered node← find an uncovered node from

test paths

5 uncovered test path← find a test path from root to

uncovered node

6 return uncovered test path

7 end

At first, the algorithm executes all newly generated test data

in T (line 2) The execution results are analyzed to find all

covered test paths test paths (line 3) From test paths, the

algorithm finds an uncovered node uncovered node (line 4)

Then, an uncovered test path uncovered test path from the

root of the unit’s CFG to uncovered node can be found

(line 5) The algorithm returns uncovered test path and

stops (line 6)

D Test Data Generation

The most important phase in concolic testing method is

Phase 3 in which a new test data is generated to cover an

uncovered test path from the previous phase Details of this

phase is described in Algorithm 3

In Algorithm 3, given an uncovered test path,

uncovered test path, the target of the algorithm is to

generate a test data which can cover uncovered test path

The algorithm starts by initializing test data t as N U LL

(line 2) Then, uncovered test path is transformed into

logical constraints, path constraints, which is in forms of

logical expressions (line 3) using symbolic execution Later,

path constraints is solved by an SMT-solver (e.g., Z3

SMT-solver [8]) to get the corresponding solution (line 4) If

solution exists (i.e., is not N U LL), the algorithm generates

a test data t from solution (line 5 - 7) Then, the algorithm returns t and stops (line 8)

Algorithm 3: Generate a new test data Input: uncovered test path: an uncovered test path Output: t: next test data

1 begin

3 path constraints← Symbolic execution for uncovered test path

4 solution← Solve path constraints by an SMT-solver

5 if solution is not N U LL then

6 t← Generate a new test data from solution

7 end

8 return t

9 end

E Discussions

When performing experiments of concolic testing, we have the following observations:

• The speed of the whole test data generation process depends heavily on Phase 3 in which a new test data

is generated

• The speed of the test data generation in Phase 3 depends

on the constraints solving speed of the selected SMT-solver (i.e., Z3 SMT-SMT-solver) This solving speed cannot

be improved in the scope of our research

From the above observations, we can see that the test data generation speed can be improved if we can reduce the number

of times in which test path constraints are solved by an SMT-solver This result can be seen through the experiments of Nguyen et al [16] Sharing the same idea of reducing the number of times in which the SMT-solver is called, if we can store the solving result for later use when the software is evolved, we can save a lot of computational cost This is the key idea of the proposed method in this paper which will be presented in sections below

IV TESTDATAGENERATION FOREVOLVEDVERSIONS

A Changes in Test Data Generation

To reuse the SMT-solver solving results, we store the results

in a dictionary so we can easily retrieve them as needed For this reason, Algorithm 3 needs to be updated to reflect the change Details of the updated algorithm is shown in Algorithm 4

Algorithm 4 accepts an uncovered test path uncovered test path and a constraints dictionary consDic containing all test path constraints and their corresponding test data as inputs The algorithm starts by initializing the test data t with N U LL (line 2) Then, the algorithm does symbolic execution for uncovered test path to generate its corresponding test path constraints path constraints (line 3) If consDic dictionary does not contain the key path constraints (line 4), the algorithm solves path constraints by using an SMT-solver to get the

corresponding solution (line 5) If solution exists (line 6), the

algorithm generates a new test data t from solution (line 7)

Then, the algorithm adds the new pair (path constraints, t)

to consDic (line 8) and returns t (line 13) If consDic

dictionary contains the key path constraints (line 10), the

algorithm gets the corresponding test data t from consDic

(line 11) and returns t (line 13)

Algorithm 4: A test data generation variant

Input: uncovered test path: an uncovered test path

consDic: a dictionary storing test path constraints and their

corresponding test data

Output: t: next test data

1 begin

3 path constraints← Symbolic execution for

uncovered test path

4 if consDic does not contain path constraints key

then

5 solution← Solve path constraints by an

SMT-solver

6 if solution is not N U LL then

7 t← Generate a new test data from solution

8 consDic← consDic ∪ (path constraints, t)

10 else

11 t← get the test data corresponding to

path constraints from consDic

12 end

13 return t

14 end

B The Updated Concolic Testing Method

With the changes presented in Algorithm 4, concolic testing

method is updated as shown in Figure 2

Start

Generate test

data at random

Execute test data & analyze covered test paths

Uncovered test path exists?

Generate a new test data (updated)

End Initially

generated

test data

Generated

Yes

(3) Updated concolic testing

consDic

Fig 2 The updated concolic testing method.

In the updated concolic testing method, for a given version

i of the project under test, for each of its corresponding units,

if the test data of the unit under consideration need to be

generated or updated, its source code is passed to the process

as input The main difference between the updated concolic

testing method and the original method is that the process uses

Algorithm 4 in Phase 3 to generate test data Algorithm 4 uses

consDic when generating the required test data Algorithm 4

either returns an existing test data or generates a new test

data as described in Section IV-A In the end of the test

data generation process, the updated concolic testing method returns the corresponding test data vi

C Automated Regression Test Data Generation Method

The updated concolic testing is integrated in the proposed method to generate test data when doing regression test In this method, the updated concolic testing method is employed

to generate test data for all versions During regression test process, consDic is used to store test path constraints and their corresponding generated test data In addition to reusing the results stored in consDic, the method also updates consDic whenever there are new test path constraints solved by the SMT-solver and the new corresponding test data generated The method is described in Figure 3

End

Generate test data for version 1

Start Code version 1

consDic

Code version 2

Generate test data for version 2

update update

reuse

Code version n

Generate test data for version n

reuse update

Test data v 1

Test data v 2

Test data v n

Fig 3 The method for automated regression test data generation.

In the method shown in Figure 3, at the first time, code version 1 is passed to the method to generate test data version

1 (Test data v1) At the same time, consDic is updated with newly generated pairs of test path constraints and their corre-sponding test data Then, when the project code is evolved to the second version Code version 2 is passed to the method to generate test data version 2 (Test data v2) During this process, consDic is also reused and updated with new pairs of test path constraints and their corresponding test data The process is repeated throughout the life cycle of the software Because this

is a long-lasting process, both Test data vi (where1 ≤ i ≤ n) and consDic must be stored in a type of external memory like

a hard drive

D Discussions

With the method presented in this section, we aim to have

an efficient regression testing method for C/C++ projects The key idea of the proposed method is to reuse both the generated test data and test path constraints solving results from the previous versions as much as possible when testing a new version The more test data we can reuse, the more regression test cost we can save For this purpose, the generated test data corresponding to test path constraints are stored in consDic dictionary Then, consDic will be updated and reused during the regression test process

In regards to the efficiency of the proposed method, let’s

consider possible software evolving scenarios To the best of

our knowledge, there are the following evolving scenarios of

a unit from a version before evolving (version 1) to a new

version after evolving (version 2)

1) Version 1 does not have the corresponding version 2:

This is the case where the unit is removed from the

project We do not need to generate test data for it

2) Version 1 and version 2 are the same Using the proposed

method, we do not need to regenerate test data for it

3) Version 1 and version 2 are different but both of them do

not have any branch or loop statements In this case, we

have only one test path for both versions We can reuse

the test data from version 1 as the test path is already

covered by the existing test data

4) Version 2 does not have the corresponding version 1:

This is the case where the unit is added to the version 2

of the project We need to generate test data for it from

the beginning

5) Version 1 and version 2 are different and both of them

have branch or loop statements In this case, the

pro-posed method will prove its effectiveness where many

test paths can be found Because the two versions are

different, there exists new or updated test paths and

unchanged test paths With new or updated test paths,

we need to generate test data for them However, with

unchanged test paths, we can reuse the path constraints

solving results (i.e., reuse the corresponding test data)

With a real project where there are thousands of units,

the path constraints solving time can be greatly reduced

From the above discussions, we can see that with the

proposed method, among the five evolving scenarios, there

are three scenarios (scenarios 1, 2, and 3) in which we do

not need to regenerate test data for regression test In our

experience, scenario 2 is the most common one in evolving

projects in practice Reusing existing test data can save us a

lot of computational cost In scenario 4, we need to generate

test data no matter what testing method being used as this

is a newly added function The most complex scenario is

scenario 5 where the function has branch or loop statements In

compare with the case where we do not reuse the test data and

test path constraints solving results from previous versions, we

can save much cost and time by reusing them

V RELATEDWORKS

In regards to the test data generation for C/C++ projects,

many researches have been published using the static

test-ing [4], [7], [19], [20], dynamic testtest-ing [1], [14], and concolic

testing [5], [6], [11], [16]±[18], [21]

Because our proposed method is based on concolic testing,

we do not give discussion about other static and dynamic

testing methods Concolic testing, whose name is proposed in

CUTE [17], is the combination of static testing and dynamic

testing In concolic testing, the preconditions of the functions

under test are considered one of the key factors for a good

test data generation However, the researches in CREST [5],

DART [11], and CAUT [18] do not require strict precondi-tions As a result, they cannot generate the minimum number

of test data whilst maximizing the code coverage Later, several researches require rigorous preconditions to increase the quality of the generated test data such as KLEE [6], PathCrawler [21] Although the quality of the generated test data set can be improved, the generation speed is still slow which is a hard problem for the application of concolic testing

to large projects Nguyen et al proposed a method named SDART [16] which can reduce the number of times SMT-solver is called This significantly improve the speed of the test data generation process

The above test data generation methods are different from our proposed method in the following aspects Firstly, those methods are for the test data generation of a given unit, not for the regression test Our proposed method inherits the result of the most recent method (i.e., SDART) proposed by Nguyen

et al [16] Our proposed method is effective for test data generation in regression test by maintaining and reusing both the generated test data set and the test path constraints solving results from previous versions

In addition to test data generation methods, regression test attracts a lot of attentions in the research and industrial communities Considering the most closed researches to our proposed method, there are the researches of BiczÂo et al [3], Duy et al [10], and Yoshida et al [23], etc

BiczÂo et al proposed an automatic framework to manage the regression test suite [3] The method of BiczÂo et al contains two main contributions The first one is a new interpretation

of reliable test cases whilst the second one is a dynamic forward impact analyzer method This analyzer method eases the transformation of existing tests to meet the definition of reliability

From 2013 to 2015, Duy et al proposed a method to gen-erate test data for regression test using model checking tech-niques [9], [10] The method was applied to Lustre/SCADE programs The key idea of the method is to determine the minimum number of test cases, which can detect all possible errors Duy et al.’s method also defined the correlation between requirement specification and test cases For this reason, the method can find which test cases should be removed, reused, or created based on the changes of the requirement specification

In 2016, Yoshida et al introduced an automatic technique for fine-grained incremental generation of unit tests through minimal augmentation of an existing test suite [23] This techniques employs the iterative, incremental refinement of the testing process related information such as test-drivers and symbolic execution The process is guided by a diagnostics engine The method attempts to keep the number of lines of newly generated test code minimum Then, the method creates tests for modified code by copying and minimally modifying,

as many as possible, existing tests instead of creating new ones

The above methods presented in researches of BiczÂo et

al [3], Duy et al [9], [10], and Yoshida et al [23] are

different from our proposed method in several key points.

Firstly, the method proposed by BiczÂo et al [3] is about test

case selection for doing regression test effectively whilst our

proposed method is about test data generation for regression

test Secondly, the method proposed by Duy et al [9], [10]

uses model checking to generate test cases and employs the

correlation between requirement specification and test cases

to which test cases are removed, updated, or created In the

meantime, our proposed method bases on concolic testing

and directly uses the source code to decide which test data

should be removed, reused, or created Last but not least,

the method proposed by Yoshida et al [23] focuses on the

changed lines of code to modify the corresponding test cases

This method does not base on test path constraints to decide

if a test case can be reused In our opinion, test paths

and their corresponding constraints are better bases for test

data generation which ensure the required coverage of every

unit functions In addition, our proposed method focuses on

maintaining and reusing both the generated test data and the

test path constraints solutions from previous versions

VI CONCLUSION

We have proposed an efficient method for generating test

data when doing regression test of evolving C/C++ projects

The key idea of the method is to store the test path constraints

and their corresponding test data in a dictionary The test data

stored in this dictionary can be reused when doing regression

test of the evolving projects This dictionary is also updated

during the regression test to keep it up to date The logical

analysis of the possible evolving scenarios shows that the

proposed method has a great potential applicability when

testing real projects

We are in a progress of implementing this proposed method

so it can be applied in practice To show the effectiveness of

the method, we will perform experiments with several projects

and get real experimental data for analysis such as the required

time, memory, the number of generated test data, and code

coverage when doing regressing test In addition, we need to

provide a tool implementing the proposed method for both

industrial and research community so it can be widely applied

in practice

ACKNOWLEDGMENTS This work has been supported by VNU University of

Engineering and Technology under project number CN21.18

REFERENCES [1] S Anand, E K Burke, T Y Chen, J Clark, M B Cohen,

W Grieskamp, M Harman, M J Harrold, and P Mcminn An

orchestrated survey of methodologies for automated software test case

generation J Syst Softw., 86(8):1978±2001, Aug 2013.

[2] B Beizer Software Testing Techniques John Wiley amp; Sons, Inc.,

USA, 1990.

[3] M BiczÂo, K PÂocza, I ForgÂacs, and Z PorkolÂab A new concept of

effective regression test generation in a c++ specific environment Acta

Cybern., 18(3):481±501, Jan 2008.

[4] G Buckle Static analysis of safety critical software (techniques, tools,

and experiences) In F Redmill and T Anderson, editors, Industrial

Perspectives of Safety-critical Systems, pages 150±168, London, 1998.

Springer London.

[5] J Burnim and K Sen Heuristics for scalable dynamic test generation In

2008 23rd IEEE/ACM International Conference on Automated Software Engineering, pages 443±446, 2008.

[6] C Cadar, D Dunbar, and D Engler Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs In

Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, page 209±224, USA, 2008 USENIX Association.

[7] B Chess and J West. Secure Programming with Static Analysis Addison-Wesley Professional, first edition, 2007.

[8] L de Moura and N Bjùrner Z3: An efficient smt solver In C R.

Ramakrishnan and J Rehof, editors, Tools and Algorithms for the Con-struction and Analysis of Systems, pages 337±340, Berlin, Heidelberg,

2008 Springer Berlin Heidelberg.

[9] T C Duy, N T Binh, and I Parissis Automatic generation of test

cases in regression testing for lustre/scade programs Journal of Software Engineering and Applications, 6(10A):27±35, 2013.

[10] T C Duy, N T Binh, and I Parissis A regression testing approach for lustre/scade programs SoICT 2015, page 288±295, New York, NY, USA, 2015 Association for Computing Machinery.

[11] P Godefroid, N Klarlund, and K Sen Dart: Directed automated

random testing In Proceedings of the 2005 ACM SIGPLAN Conference

on Programming Language Design and Implementation, PLDI ’05, page 213±223, New York, NY, USA, 2005 Association for Computing Machinery.

[12] https://www.selenium.dev/ Selenium https://www.selenium.dev/, 2021 Online; accessed 13 November 2021.

[13] E Larson and T Austin High Coverage Detection of Input-Related Security Facults, page 9 USENIX Association, USA, 2003.

[14] G J Myers and C Sandler The Art of Software Testing John Wiley

& Sons, Inc., Hoboken, NJ, USA, 2004.

[15] D.-A Nguyen and P N Hung A test data generation method for

c/c++ projects In Proceedings of the Eighth International Symposium

on Information and Communication Technology, SoICT 2017, page 431±438, New York, NY, USA, 2017 Association for Computing Machinery.

[16] D.-A Nguyen, T N Huong, H V Dinh, and P N Hung Improvements

of directed automated random testing in test data generation for c++

projects International Journal of Software Engineering and Knowledge Engineering, 29(09):1279±1312, 2019.

[17] K Sen, D Marinov, and G Agha Cute: A concolic unit testing engine

for c In Proceedings of the 10th European Software Engineering Con-ference Held Jointly with 13th ACM SIGSOFT International Symposium

on Foundations of Software Engineering, ESEC/FSE-13, page 263±272, New York, NY, USA, 2005 Association for Computing Machinery [18] Z Wang, X Yu, T Sun, G Pu, Z Ding, and J Hu Test data generation for derived types in c program. In 2009 Third IEEE International Symposium on Theoretical Aspects of Software Engineering, pages 155±

162, 2009.

[19] N Ward The static analysis of safety critical software using malpas.

IFAC Proceedings Volumes, 22(19):91±96, 1989 IFAC/IFIP Workshop

on Safety of Computer Control Systems 1989 (SAFECOMP ’89), Vienna, Austria, 5-7 December.

[20] B Wichmann, A Canning, D Clutterbuck, L Winsborrow, N Ward, and W Marsh Industrial perspective on static analysis. Software Engineering Journal, 10:69 ± 75, 04 1995.

[21] N Williams, B Marre, P Mouy, and M Roger Pathcrawler: Automatic generation of path tests by combining static and dynamic analysis In

M Dal Cin, M Kaˆaniche, and A Pataricza, editors, Dependable Com-puting - EDCC 5, pages 281±292, Berlin, Heidelberg, 2005 Springer Berlin Heidelberg.

[22] S Yoo and M Harman Regression testing minimization, selection and

prioritization: A survey Softw Test Verif Reliab., 22(2):67±120, Mar.

2012.

[23] H Yoshida, S Tokumoto, M R Prasad, I Ghosh, and T Uehara Fsx: Fine-grained incremental unit test generation for c/c++ programs In

Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, page 106±117, New York, NY, USA, 2016 Association for Computing Machinery.