Generating test case from user interface and mining requirements

Generating test case from user interface and mining requirements

University of Engineering and Technology, VNU

Faculty of Computer Science

Generating test case from user

interface and mining

requirements

Student: Nguyen Le Hoang

ID: 08020165 Supervisor: Truong Ninh Thuan

Chapter 1 4

Overview of the approach 4

Chapter 2 5

Algorithm and Implementation 5

2.1 Generating test case from mining requirements 5

2.2 Generating test case from GUI 6

2.2.1 Parameter and level 7

2.2.2 Combination parameters 7

2.2.3 Process to design test case 8

2.3.4 Pairwise testing 8

Chapter 3 20

Applying the method for specific examples 20

3.1 Testing Login form 20

3.2 Testing Find Student form 21

Chapter 4 24

Support tool 24

4.1 Design features 24

4.1.1 Design Overview 24

4.1.2 Class graph 25

4.2 Implementation 26

4.2.1 Interface form 26

4.2.2 Parameter form 27

4.2.3 Enter the result 28

4.2.4 Condition form 29

4.2.5 Test case generated form 30

4.3 Advantages and disadvantages of the tool 32

4.3.1 Advantages 32

4.3.2 Disadvantages 33

Chapter 5 34

Conclusion 34

References 35

The special thank goes to my helpful supervisor, PhD Trương Ninh Thuận, lecturer at Faculty of Information Technology, University of Engineering and Technology The supervision that he gave truly helps the progression during my research work Also, the support that he gave about progression of report and information which faculty provided about submitting report assist me in planning the report completion

My grateful thanks also go to lectures that taught enthusiastically during my research and study at University of Engineering and Technology and brought me the basic knowledge

in Computer Science Thus I could easily study in the research without hesitation

Last but not least I would like to thank my friends and my family who gave a huge

contribution during my whole progress of research

In recent years, the development of computer hardware and software has more importantsteps The development of producing software has now become a real industry As otherindustries, the function of the software has been produced increasingly complex, and therequirement of managing the quality of software has been concerned increasingly.Therefore, software testing must be done regularly throughout the life cycle of softwaredevelopment to ensure the quality of the software

As we know Graphic User Interface (GUI) is one of the most popular ways for users tointeract with software system At the end of the system testing phase, the tester continued

to test the functions of the software provided by GUI and the requirement of software.GUI makes testing the software system more difficult because of the nature of GUIcontrol events, the undesirable events or the infinite domain input/output whichprogrammer unforeseen Through the GUI, the tester can perform black box testing tofind the errors of the software One of the most important reasons for software testing is

to design and create the effective test cases The cost of software testing often accountsfor 40% of the development of a software project Therefore, reducing costs for creatingTest Case (time, effort) is one of the most important problems in softwaretesting Generating test cases automatically do not only help reducing costs in creatingTest Case, but also consistent the quality of test case Therefore, we research the methods

of creating test case from the graphic user interface and the requirement of software tosupport tester for testing software

This research report includes 4 parts: generating test case from software requirements,generating test case from GUI, applying the method of generating test case from GUI andsoftware requirements for specific examples, and support tool

After researching and testing, in Conclusion, we give some comments and summarize onautomatically test case generation tool, and set out directions for further research

Chapter 1 Overview of the approach

The figure below demonstrates how to generate test case from Graphic User Interface(GUI) and requirement of a software program

As you see, from GUI of the program, we will identify output parameters of the program(such as for a dialog box of a Log in modules, we will have two parameters: usernameand password of user) Besides, with the requirements of the program, we will alsoidentify output value of each parameter which we have when analyzing GUI (examplewith Log in modules, the requirement of the system is that the length of the string whichuser type is less than 20 characters)

After that, we will model parameters and values to parameter domain and value domain.Since then, we will generate test cases How to generate test case will be represented inthe detail method bellows

Value Domain Modelling

Chapter 2

Algorithm and Implementation

2.1 Generating test case from mining requirements

Software requirements play an important role in testing software First, the requirementsprovide information to check the input / output of the program correct or not Second, therequirements provide information to choose test case and assess the relevance of testing.Testing based requirements take the information of testing from the requirements ofsoftware In software testing, the requirements-based testing has many advantages: it issimpler than other testing; besides it also increases the quality of software because thedevelopment of testing phase along with the implement and design allows the use oftesting which standardize the quality of the requirements

General progress is shown in the figure below, or simply here are the steps of theprocess test case generated from software requirements

Process test case generated from requirements includes the following steps:Step 1: Develop the transition conditions With a specification language, the transitionconditions are encoded directly in the specification (functional specification).With otherlanguages, the conditions may have a metabolic manipulation

Function requirements

Requirement Chart Requirement Testing Specification Testing

Script Testing

Step 2: Develop the specification graph You can get directly from the graph specificationsheet specification.

Step 3: Develop the testing requirements The testing requirements derived from thetransfer characteristics and are generated from the transition conditions in step 1

Step 4: Develop the testing requirements of completeness Build the truth tables for allproperties in the specification graph to build the partial expression trees We can rely onthe partial expression trees or based directly on the test specification to edit theproperties

Step 5: Develop testing state-transition pair Identify all pairs of transition by: Listing allthe transitions in (M), all the output transitions (N), then create pairs of M * Ntransition The forward pair will be replaced by the graph properties from thespecification

Step 6: Develop complete testing requirements Determine the complete list of states by:from specification chart, select the order of the state (this work performed by thetester) Then, build the sequence of attributes

Step 7: Build the testing specification For each testing requirements generated prefixvalue, the value of Test Case, confirm the conditions, release conditions, and the desiredresult

Step 8: Develop test scripts Each test specification used to build a script

2.2 Generating test case from GUI

Graphic User Interface (GUI) is one of the most popular ways for users to interact withsoftware system At the end of the system testing phase, the tester continued to test thefunctions of the software provided by GUI and the requirement of software GUI makestesting the software system more difficult because of the nature of GUI control events,the undesirable events or the infinite domain input/output which programmer unforeseen.Through the GUI, the tester can perform black box testing to find the errors of thesoftware The problems is that how to create test case to test, find the error with thehighest performance and the minimal cost

Nowadays, there are many methods of generating test case automatically, in whichcombination testing methods can reduce the costs and increase the efficiency of testingvarious software applications We researched this method and apply to generating testcase automatically from GUI.

Combination testing

Combination testing is a combination of the used level parameter to generate one or moretest case With each test case, the order of execution of the input should be identified bythe tester

In this method, the tester first determines which parameters through the specification.Since then, they designed using combination testing method to generate test case

Combination testing method will be described in detail below

2.2.1 Parameter and level

P has n inputs with the corresponding values are X1, X2, …Xn Each input is called aparameter Suppose that each parameter has any value as ci, 1 ≤ i ≤ n For each value ofthe parameter is called the level | F | is the number of levels of the parameter F.Set of values, with each parameter, called the set of parameters

2.2.2 Combination parameters

Example 2.1:

Suppose that program P has two input variables are X and Y In implementing P,suppose X and Y get the corresponding value set: {a, b, c} and {d, e, f} So we have 2parameters and each parameter has three levels So we have 3 * 3 = 9 combinations ofparameters: (a, d), (a, e), (a, f), (b, d), (b, e), (b, f), (c, d), (c, e), and (c, f)

In general, k parameter and each parameter can take the n value from the set value;the number of possible combinations is n*k

Assuming each parameter combination is a Test Case, with many programs, the testcase creation is huge For example, if a program has 15 parameters, each parameter canget 4 values, the total number of Test Case is 4 ^ 15 ~ 10 ^ 9 To perform a billion times ttesting is not practical for many software applications

Example 2.2: User interface

GUI of a program has 3 menus: File, Edit and View as the figure

We have 4.3.4 = 48 combination

2.2.3 Process to design test case

The steps to design combination testing:

Step 1: Model domain of input values The model includes a set of parameters andcorresponding values (levels) of the parameter

Step 2: This model is input to the design of combinatorial procedures Combinatorialdesign procedure allows creating combinatorial objects Combinatorial object here issimply an array parameter and the corresponding value of the parameter

Step 3: Using the combinatorial object has been created to set up testing A combinatorialobject is an array of parameter combinations Each parameter can be combined to createone or more Test Case; Test Case includes each value of the inputs and expected outputs.However, all combinations are created is not feasible

Here, step 2 and step 3 can be automated

The combination of the level parameter is used to generate one or more Test Case Foreach Test Case, the order of execution of tester input should be identified

2.3.4 Pairwise testing

The purpose of the Pairwise algorithm is to reduce the amount of test case generatedbased on the input parameters that can still detect the errors of the program or system.The details of this algorithm will be presented in detail in the examples below

Example 4.1:

Let's assume a simple program which we have only two checkboxes A checkbox can be

in one of two states—checked or unchecked Imagine that our defect depends uponcheckbox A being unchecked and checkbox B being checked If we try all of the possiblesettings in combination with one another, we'll find the defect We'll set up a table, with arow for each variable Within each cell, we'll put one of the variable's possible states—set

or cleared In the leftmost column, we'll put an index number so that we can refer to eachcombination by number

Examples 4.2: Check airplane

A little more complex example: a black-box test of a dialog with three sets of radiobuttons on it, representing travel needs in North America The first set of radio buttonsspecifies a destination, by country; the second notes the choice of an aisle or windowseat; and the third affords the choice of coach, business, or economy class When the usermakes her selections and presses "OK" to close the dialog, the application will take on adifferent path of execution for each combination of given options in given states

As usual, the simplest kind of problem to detect is one that is triggered by a single variable in a single state This is called a single-mode fault (Phadke1997) For example,

consider an application that contains a defect causes it to behave bizarrely when the

"destination" option is set to "Canada" Naturally, the way to find such a defect would be

to ensure that we run at least one test with "destination" set to "Canada"

Assume now that there is a defect that depends upon two conditions—for example, thedestination field being set to "Mexico" and the service class being set to "Business" at thesame time This is known as a double-mode fault It should be clear if we run at least one

test with "destination" set to "Mexico" and the "Class" field to "Business", we'llencounter the defect.

For two radio buttons (each with three possible states) and one checkbox, there are 3 x 3

x 2, or 18 possible values to test Here's a table that shows each possible combination:

The most difficult kind of problem to find by black-box testing is one in which severalvariables are involved, and each must be in a specific, unique state to trigger the problem.For example, when the country field is set to "USA", the seat choice to "aisle", and theservice class to "First", then and only then will the bug be triggered This is known as

a triple-mode fault, or more generally as a multi-mode fault, which describes faults

associated with three or more parameters For testers, finding this kind of problemdepends on one of two approaches: testing all combinations of variables and states in theprogram; or some other technique combined with well, luck The first option is flatlyimpossible for anything but the most trivial program The key to the second option isoptimizing the tests so to be as lucky as possible

In 1997, researchers at Telcordia Technologies published a paper by Siddhartha Dalal.The content of the paper suggest that "most field faults were caused by either incorrectsingle values or by an interaction of pairs of values." If that's generally correct, we ought

to focus our testing on the risk of single-mode and double-mode faults The Telcordiapaper suggests that we can get excellent coverage by choosing tests such that

1) Each state of each variable is tested

2) Each variable in each of its states is tested in a pair with every other variable in each of

its states This is called pairwise testing or all-pairs testing.

Focusing on single- and double-mode faults reduces dramatically the number of

combinations that we have to run If all of the pairs in a given combination exist in othercombinations, we can drop that combination For example, consider combination 14: theMexico/Business pair also appears in combination 5, the Business/Window pair appears

in combination 13, and the Mexico/Window combination appears in combination 11 Wecan drop combination 14 without leaving any of its pairs untested

This table makes sure that Canada is tested at least once with each of Coach, BusinessClass, and First Class, and with the checkbox in the Aisle state and the Window state.Similarly, every option is tested with every other option We capture the double-modefault exposed in combination number 3; we need not test combination 12 Using pairwisetesting, we reduce 18 test cases to 9 test cases The easiest thing to do is to select lines,starting at the bottom of the table, and if all of the pairs for the line are duplicatedsomewhere higher up in the table, we can remove the line

Line 18 contains USA, First Class, and Window.

 the USA-First Class pair appears in line 9

 the First Class-Window pair appears in lines 16 and 17

 the USA-Window pair appears in line 12 and line 15

So we can drop line 18

Line 17 contains Mexico, First Class, and Window

 the Mexico-First Class pair appears in line 8

 the First-Class Window pair appears in line 16

 the Mexico-Window pair appears in lines 14 and 11

We can drop line 17 too

Line 16 contains Canada, First Class, and Window

 Canada-First Class appears in line 7

 Canada-Window appears in line 13

 First-Class-Window doesn't appear elsewhere in the table

We must keep line 16 to preserve the First Class-Window pair

This is an approach when the main table is very small, but a selection set with a trivialnumber of options increases complexity As we add variables or values, three thingshappen First, the size of the all-combinations table expands exponentially Second, theeffort associated with sifting out the duplicated pairs becomes enormous On the thirdpoint, there is some good news: the all-pairs selection cuts the number of requiredcombinations much more dramatically than in our example above Assume a programthat has 75 binary options; the table of all possible combinations of states would extend

to 2^75 entries or 37,778,931,862,957,161,709,568 Telcordia's paper notes that 75parameters of two states each can be pair-checked with only 28 combinations

So all-pairs testing is a very powerful technique for reducing the number of tests to berun, but there are two problems to consider.

 The first problem is that if the defect in are example requires three variables to be just

so (Destination=Canada, Class=Business, and Seat Preference=Aisle), our smallertable won't trigger the defect

 The second problem is that an all-pairs table takes a significant time to construct and

to check for anything but a small number of variables and more possible states foreach variable If you want to test for more than pairs triples, or n-tuples the problemquickly becomes intractable for a human test planner

Examples 4.3: Print

Imagine a table that contains combinations of display settings, operating systems, andprinters One column represents display resolution (800 x 600 or Low; 1024 x 768, orMedium; and 1600 x 1200, High); a second column represents operating systems(Windows XP, Windows 7, and Linux); and a third column represents printer types(PostScript, LaserJet, and Bubble Jet) We'd need a table with 3 x 3 x 3 rows—27 rows—

to represent all of the possible combinations; here are the first five rows in such a table:

Combination

Number

Display Resolution

Writing all those options out takes time; let's use shorthand All we need is a legend ormapping to associate each variable with a letter of the alphabet: Low=A, Medium=B,High=C; Win XP=A, Win 7=B, Linux=C; PostScript=A, LaserJet=B, Bubble Jet=C.Again, this is only the first five rows in the table

Combination

Number

Display Resolution

While we can replace the letters with specific values, a particular benefit of this approach

is that we can broaden the power of combination testing by replacing the letters with

specific classes of values One risk associated with free-form text fields is that they might

not be checked properly for length; the program could accept more data than expected,copy that data to a location (or "buffer") in memory, and thereby overwrite memorybeyond the expected end of the buffer Another risk is that the field might accept anempty or null value even though some data is required So instead of associating specifictext strings with the levels in column 3, we could instead map A to "no data", B to "datafrom 1 to 20 characters", and C to "data from 21 to 65,336 characters" By doing this, wecan test for risks broader than those associated with specific enumerated values

For example, suppose that we are testing an insurance application Assume that Variable

1 represents a tri-state checkbox (unchecked = A = no children, checked = B = dependentchildren, and greyed out = C = adult children) Next assume that Variable 2 represents aset of radio buttons (labeled single, married, or divorced) Finally assume that Variable 3represents a free-form text field of up to 20 characters for the spouse's first name.Suppose further the text field is properly range checked to make sure that it's 20characters or fewer However, suppose that a bug exists wherein the application produces

button is selected By using classes of data (null, valid, excessive) in Column 3, pairwisetesting can help us find the bug

With example 4.3, we can reduce 27 test cases to 9 test cases to save time and cost Here

is the table of 9 test cases with each color represented each value of parameter: print,resolution and OS

Pairwise is a balanced design because each value has the same number of times Itcan have multiple combinations to accommodate all pairs

Here's an example using pairwise algorithm with input parameters with only two values.There is the detail algorithm to reduce the number of test case

Examples 4.4

A Java applet ChemFun allows its user to create an in-memory database of chemicalelements and search for an element The applet has 5 input factors, each with exactly twopossible values

1 Operation Create, Show Two buttons

2 Name Empty, Non-Empty Data field, string expected

3 Symbol Empty, Non-Empty Data field, string expected

4 Atomic number Invalid, Valid Data field, data typed > 0

5 Properties Empty, Non-Empty Data field, string expected

Input: n=5 factors

Output: A set of factor combinations such that all pairs of input values are covered