automatic test generation system for efsm based protocols

doi:10.1016/j.proeng.2012.01.284 Procedia Engineering 00 2011 000–000 Procedia Engineering www.elsevier.com/locate/procedia 2012 International Workshop on Information and Electronics E

Procedia Engineering 29 (2012) 2182 – 2186

1877-7058 © 2011 Published by Elsevier Ltd.

doi:10.1016/j.proeng.2012.01.284

Procedia Engineering 00 (2011) 000–000

Procedia Engineering

www.elsevier.com/locate/procedia

2012 International Workshop on Information and Electronics Engineering (IWIEE)

Automatic Test Generation System for EFSM-Based

Protocols

Ting Shu a,*, Qiangxin Cai b, Lianggui Liua, Yubo Jiaa

a College of Informatics and Electronics, Zhejiang SCI-TECH University, China

b CECEP Industry Development CO.,LTD, China

Abstract

Protocol conformance testing based on Extend Finite State Machine (EFSM) is an open research topic How to implement an automatic test generation system for EFSM-specified protocols is a challenging problem This paper focuses on test generation system implementation techniques Specifically, a feasible loosely-coupled system architecture and design idea is presented Firstly, the four main functional modules of this system architecture and the relationships between them are introduced respectively in detail Then, the entire test generation process of the system, divided into three key phases, is described according to the execution order of the system modules Next, ATSGEN, an integrated automatic test sequence generation system specified in EFSM, is developed Finally, a typical test example is given to demonstrate system features

© 2011 Published by Elsevier Ltd Selection and/or peer-review under responsibility of Harbin University

of Science and Technology

Keywords: Protocol conformance testing; Test sequences generation; Extend finite state machine; Abstract test suites;

1 Introduction

Protocol conformance test is an experimental activity to judge whether or not a protocol implement under test (IUT) conforms to its specification standard To avoid semantic ambiguities, the Extended Finite State Machine (EFSM) is introduced to describe specifications formally Hence, the conformance test method based on EFSM has attracted much attention [1-3] In the conformance test process, a tester can only stimulate an IUT using a set of inputs and observe the external outputs from the IUT to draw a

* Corresponding author Tel.: +0-086-0571-86843323; fax: +0-086-0571-86843323

E-mail address: shuting@zstu.edu.cn

test conclusion The sequence of the input/output pairs is called as protocol conformance test sequences

Unfortunately, this can be expensive and difficult to generate test sequences manually Thus, how to

develop an automatic test sequences generation system for EFSM-specified protocols has become an

urgently problem to be solved

Several typical methods and test generation tools, such as UPPAAL Tron [4], Phact [5], Conformance

Kit [6], have been proposed for generating EFSM-based protocol conformance test sequences However,

these research results are seldom applied in the network communications industries [1] Therefore, there is

a strong need to make more effort by researchers to narrow the big gap between academic results and

testing practice In this paper, we discuss the test generation system implementation techniques Then, a

loosely-coupled system architecture and design idea based on a data center is presented Based on these

methods, an Automatic Test Sequences Generation system (ATSGEN) is developed The feasibility of the

proposed system structure and design idea is shown through a simple test generation experiment

2 ATSGEN system architecture and design

According to the protocol conformance testing standard ISO9646, we design the general architecture

of ATSGEN (shown in Figure 1) The system framework consists of four main functional modules,

Protocol Model Construction and Information Extraction (PMCIE), Automatic Protocol Conformance

Test Sequence Generation (APCTSG), Protocol Execution Simulation (PES) and Automated TTCN-3[7]

Test Case Generation (ATTCG) These modules interact with each other and contribute to accomplish

system function The data interaction center of these modules is a local relational database system In this

way, ATSGEN can be constructed based on the loosely coupled architecture, which contributed to the

good extension feature of the system The following is a detail description of the design idea and the key

function of each module

Fig 1 ATSGEN system architecture

2.1 PMCIE module

This module is to fulfil two tasks: building the protocol formal model and making the information

extraction from an EFSM model For the former, protocol specification under testing is modelled using

the Rational Rose tool with any one of two modes: indirect mode and direct mode In indirect mode,

testers can make ATSGEN run the Rose to build the formal model In direct mode, testers can also run

independently Rose to develop a model As a result, a protocol model is generated, which is saved as a

file with the mdl extension Then, the file can be imported into ATSGEN for testing preparation

After the establishment of the formal model, for the latter, we need extract the necessary information

from it for the next test sequences generation This information includes states, variables, transitions and

the relations between them Meanwhile, this module also need support the transition condition analysis

and variables computation in the process of the test generation algorithm execution

2.2 APCTSG module

APCTSG is the core module of ATSGEN Conformance abstract test sequences can be automatically

generated in this module It implements three types of test generation algorithms [8]: control flow test

criteria (CFT), data flow test criteria (DFT), control-flow and data-flow coverage criteria (CDC) For each

test criteria, it also supports three testing generation modes respectively: test sequence generation, test

generation and intermediate results storage, test generation and simulation A variety of testing generation

modes is intended to provide convenience for testing simulation In the first operation mode, APCTSG

only generate the conformance testing sequence, but not save the intermediate results related with testing

simulation into the data center By reducing the database reading and writing operations, the testing

generation efficiency can significantly be improved in this mode In the other two modes, all the

intermediate data are stored into the data center, which is needed for the subsequent testing simulation

process But only in the third mode, ATSGEN will automatically load the testing simulation

sub-application immediately after the test sequence generation

2.3 PES module

In this sub-application, the dynamic characteristics of the protocol EFSM model under testing will be

presented in the form of executable directed spanning tree (EDST) [8] Based on the EDST technique,

PES provides two types of testing simulation: Protocol EFSM Execution Simulation (PEES) and Testing

Sequences Generation Simulation (TSGS) In PEES, tester can subjectively set the initial state

configuration of the EFSM as the root of EDST And then, an EDST is built step by step with each time a

mouse click on a node By observing the generated EDST, we can trace the execution path of the EFSM

and validate the established EFSM In TSGS, a partial EDST is automatically rebuilt based on the

intermediate data of the testing sequence generation process stored in the data center

2.4 ATTCG module

This module’s main task is that translate the test sequence generated in APCTSG into the abstract test

suite (ATS) described by TTCN-3 language In the test suite execution phase, we need a TTCN-3

compiler to compile the ATS into the executable test suite (ETS) However, TTCG is only responsible for

generating the ATS in TTCN-3, not for compiling the ATS into ETS The compilation of the ATS is to be

done by a third party compiler, such as the TTCN-3 compiler of TTworkbench

3 Test generation process of ATSGEN

As shown in the Figure 2, the entire test generation procedure consists of three main steps In the

testing preparation phase, first of all, a protocol UML model is needed to be built using the Rose And

then, the information related with test generation is extracted from the state machine model in the

generated model file Next, the PMCIE reformulates a new protocol EFSM model in memory Following

with the preparation phase, ATSGEN enters the testing generation phase In this phase, ATSGEN will

automatically generate the conformance testing sequences according to the optional test generation modes

and algorithms After the testing generation, multiple sets of the result test sequences may be generated

In the last phase, testing suites TTCN-3 description phase, these generated test sequences will be

translated into the corresponding ATS in TTCN-3 core language

Fig 2 Test generation process of ATSGEN

In the process of the testing sequence generation, APCTSG plays an important role in supporting for

the model validation and testing sequence generation process supervision Just after the model built, we

can validate the model using the PEES And then, before the testing sequences converted into TTCN-3,

we also can supervise the process of the test sequence generation through the EDST technique in the

TSGS Ether in PEES or TSGS, if an error is found in the process of the simulation, we can continue to

modify the corresponding EFSM model or correct the testing algorithm This process will be repeated

until the results meet our requirements

4 A simple testing generation example

The initiator module of INRES protocol is a widely discussed example in the protocol testing field So

we also choose the INRES as an example to show ATSGEN’s some features

In testing preparation phase, ATSGEN runs the Rose tool to build a initiator module of the modified

INRES protocol model as demonstrated in Fig.3(a), which is saved as the model file “INRES.mdl” Then,

all necessary information will be extracted from this model file to rebuild an EFSM model in the system

memory In testing generation phase, we select the CDC criteria as the conformance test generation

algorithm Then, the state “Disconnect” is selected as the initial state and all context variables in the

EFSM model are assigned the corresponding value of zero or false according to the type of each one

Finally, the APCTSG module is stimulated to generate the test results as shown in Fig.3 (b) Due to space

limitation, we do not present the process of test suites TTCN-3 description and testing simulation here

5 Conclusion

This paper introduces the definition of protocol conformance test and test automatic generation

system design and implementation techniques A conformance test generation system, ATSGEN, based

on EFSM with a loosely-coupled structure is developed The structure of ATSGEN is divided into four

main modules to illustrate respectively These modules interact with each other through a data center The

entire workflow of testing generation in ATSGEN is also described step by step Finally, based on the

modified Initiator module in INRES protocol, a test generation example is given to show the feasibility of

our system to generate abstract conformance test sequences

Fig 3 (a) Initiator module of the modified INRES protocol model in Rose; (b) Results of the testing sequences generation

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No 61101111 and No

61002016), Science Foundation of Zhejiang Sci-Tech University（ZSTU）under Grant No.1004839-Y

References

[1] Lai, R., A survey of communication protocol testing Journal of Systems and Software, 2002 62(1): p 21-46

[2] Kalaji, A.S., R.M Hierons and S Swift, An integrated search-based approach for automatic testing from extended finite state

machine (EFSM) models Information and Software Technology, 2011 53(12): p 1297-1318

[3] Petrenko, A., S Boroday and R Groz, Confirming configurations in EFSM testing IEEE Transactions on Software

Engineering, 2004 30(1): p 29-42

[4] Larsen K G, Mikucionis M, Nielsen B, et al Testing real-time embedded software using UPPAAL-TRON: an industrial case

study EMSOFT 2005 , Jersey City, NJ, USA : ACM, 2005: 299-306

[5] Feijs L, Meijs F, Moonen J R, et al Conformance testing of a multimedia system using PHACT Proceedings of the IFIP

TC6 11th International Workshop on Testing Communicating Systems, Tomsk, Russia: Kluwer, B.V., 1998: 193-210

[6] Van De Burgt S P, Kroon J, Kwast E, et al The RNL Conformance Kit Proceedings of the 2nd International Workshop on

Protocol Test Systems, North Holland: 1989: 279-294

[7] Etsi ES 201 873-1: The Testing and Test Control Notation version 3, Part1: TTCN-3 Core notation V3.2.1 2007

[8] T Shu, SQ Sun, HN Wang, WQ Xu, Test sequence generation of a communication protocol by an heuristic state

configuration exploration Journal of Beijing University of Posts and Telecommunications, 2009 32(6): p 120-124