A method for Automated User Interface Testing of Windowsbased Applications44950

The key idea of this method is to generate new test scenarios from widgets and test specification where widgets are extracted during the execution of the application and test specificati

Windows-based Applications

Duong Tran Dinh

VNU University of Engineering and

Technology

duongtd@vnu.edu.vn

Pham Ngoc Hung VNU University of Engineering and

Technology hungpn@vnu.edu.vn

Tung Nguyen Duy Toshiba Software Development

Vietnam tung.nguyenduy@toshiba-tsdv.com

ABSTRACT

This paper proposes a method for automated user interface testing

of Windows-based applications to increase the accuracy in

iden-tifying the target widgets or executing several interactions The

key idea of this method is to generate new test scenarios from

widgets and test specification where widgets are extracted during

the execution of the application and test specification is generated

by combining the interactions of widgets Furthermore, the paper

contributes some techniques to detect hidden widgets which

con-sidering as one of the most challenging problems in user interface

testing Currently, a supporting tool has been implemented and

tested with several industrial projects The details of the

experi-mental results will be presented and discussed

CCS CONCEPTS

• Software and its engineering → Software testing and

de-bugging;

KEYWORDS

Automated testing, graphical user interface testing, Windows

ap-plication

ACM Reference Format:

Duong Tran Dinh, Pham Ngoc Hung, and Tung Nguyen Duy 2018 A method

for Automated User Interface Testing of Windows-based Applications In

The Ninth International Symposium on Information and Communication

Technology (SoICT 2018), December 6–7, 2018, Danang City, Viet Nam ACM,

New York, NY, USA, 7 pages https://doi.org/10.1145/3287921.3287939

1 INTRODUCTION

User interface (UI) provides the ability to allow interactions between

humans and underlying systems Thus, checking the correctness

of UI has been considered as an important phase in the software

quality assurance However, UI is becoming more and more

com-plicated, hence manual testing has difficulties in both design and

execution steps Furthermore, it must be performed not only one

time but also every time when developers modify the source code

of applications [15] Therefore, UI testing should be performed

au-tomatically instead of manually to reduce the efforts of testers and

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full citation

on the first page Copyrights for components of this work owned by others than ACM

must be honored Abstracting with credit is permitted To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specific permission and/or a

fee Request permissions from permissions@acm.org.

SoICT 2018, December 6–7, 2018, Danang City, Viet Nam

© 2018 Association for Computing Machinery.

ACM ISBN 978-1-4503-6539-0/18/12 $15.00

https://doi.org/10.1145/3287921.3287939

increase productivity Model-based testing [12] and Record & Replay [14] are two major approaches in automated UI testing Further-more, writing test scripts directly is also known as another one which can be used with testers have programming knowledge Applying model-based testing to check the correctness of UI typically uses graph models to represent system behaviors and user interactions Each node in this model represents an interac-tion of a widget and test scenarios are created by traversing all possible paths inside it Some well-known tools applying this ap-proach can be mentioned UFT [10], PETTool [4], PATH [1] This approach requires testers represent user scenarios using the formal specification method It means that testers should have the mathe-matical knowledge, hence it is not easy to use when applying this approach in practice Record & Replay is another approach used to check the correctness of UI interactions This approach consists

of two major phases At first, all events that testers performed are recorded After that, these interactions can be replayed multiple times automatically With this approach, testers can create or run tests without programming knowledge Therefore, it is used more widely in comparison with model-based UI testing approach Be-cause this approach proved effective for using in practice, there are many tools developed using this approach such as TestComplete [9], Reran [5], Ranorex1 In order to apply this approach in practice, testers must record all test scenarios manually to create test scripts However, the number of test scenarios is usually large especially

in industrial projects Hence, recording all of them is not only a boring task but also consumes much time, cost, and efforts Instead

of covering all scenarios, typically testers only focus on the critical ones that highly cause failures based on their knowledge and ignore others Bypassing such test scenarios, testers may leave the poten-tial UI bugs in the software product With people have proficient programming, they can write test scripts directly instead of using any other methods Nevertheless, it also consumes much time and efforts to write all test scripts Moreover, they must ensure that these scripts are totally accurate (satisfy the syntax of the compiler, execute desired interaction correctly, etc)

This paper introduces a method of automated UI testing for Windows-based applications to mitigate the mentioned problems

We share motivation with replay phase in Record & Replay approach However, instead of requiring testers record all events manually, our method can generate a sequence of test scenarios coverage most cases without a lot of testers’ efforts There are three main phases of the proposed approach including widgets inspection, test scripts generation, and test execution The goal of the first phase is

to extract all widgets and their properties from the system under test (SUT) From these widgets, testers would specify the desired

1 https://www.ranorex.com/

test scenarios After that, the test scripts corresponding to each test

scenario are generated and packaged in a test project Finally, this

test project is executed on a specific compiler to obtain the results

of test scripts (e.g., pass/fail)

The rest of this paper is organized as follows At first, Section

2 introduces several outstanding related works Next, Section 3

presents the widgets inspection phase The details of the test scripts

generation and test execution are shown in Sect 4 After that,

Section 5 describes a tool that was implemented from the proposed

method and experimental results Finally, the conclusion of the

paper is presented in Sect 6

2 RELATED WORKS

Since UI is an important factor of a system, there are many

pub-lications related to automated UI testing by several authors The

methods proposed in these publications are based on two main

approaches including model-based testing and Record & Replay

Model-based testing [12] is a method which uses models to

repre-sent system behaviors and user interactions This approach consists

of three main phases Firstly, the model of the system is built from

specifications and designs Secondly, test cases are generated from

this model (a collection of UI interaction test cases is generated

based on some algorithms, conditions, etc) The number of test

cases generated depends on the algorithms used In general, this

method generates more test cases than others, this is an advantage

of model-based testing Finally, test cases are executed on the

ap-plication under test By comparing the expected outputs and actual

outputs after executing, the state success or failure of each test case

is retrieved and test reports are created The quality of generated

test cases depends on the model used Researching to find out the

good models for automated UI testing is getting more and more

attention such as UFT [10], PETTool [4], PATH [1] In these studies,

the graph model which test scenarios are created by traversing all

possible its paths is a common one [8] [3] [6]

Record and replay [14] is another technique used to check the

cor-rectness of interactive applications with graphical user interfaces

In the UI testing, this technique is used more widely than

model-based testing because it does not require mathematical knowledge

Record and replay consists of two phases corresponding to its name

Firstly, all the events that be performed by users are recorded (e.g.,

keys pressed, mouse movements, etc.) After that, these events can

be exactly replayed multiple times automatically This is a great

advantage of this technique because it assists to save a lot of testers’

efforts in executing test Testers find it easy to use record and

re-play tools for UI testing Record and rere-play technique is effective

in practice for UI testing in comparison with traditional manual

methods Thus, there was much attention in developing automated

UI testing tools based on record and replay approach for

applica-tions on various platforms such as Ranorex, Reran [5], Mobiplay

[13], VALERA [7], WaRR [2], etc

Windows API, informally WinAPI contains several functions

which programmer can use when developing applications for

Win-dows operating system By using WinAPI, an application can

dis-play output, prompt for user input, or carry out other tasks that

support interaction with users Windows API functions have been

documented in several publications, the most sufficient is written

on Microsoft Developer Network [11] Some functions are used for extracting UI information or simulating actions can be listed as GetDesktopWindows, GetWindowText, SendMessage, GetMenu, etc Microsoft also provides a library named UI Automation2on NET framework allows for programmatic access to many widgets on the Windows-based applications Developers can use UI Automa-tion library to retrieve informaAutoma-tion about the UI of the system UI Automation also allows automated test scripts to interact with the

UI This library exposes every piece of the UI to client applications

as an “Element” Elements store common properties of the widgets they represent, and they are contained in a tree structure, with the desktop as the root element Client applications can filter the Element in the tree to find the desired elements by some conditions (e.g., name, control type)

3 WIDGETS INSPECTION The proposed method in this paper can be divided into three main phases as shown in Fig 1 Firstly, all widgets are extracted from SUT during the execution of it One of the most challenges in this phase is that some hidden widgets cannot be detected, hence several techniques are introduced to overcome this problem Then,

a sequence of UI test scenarios is generated by combining the interactions of each widget which be defined by testers After that, the corresponding test scripts are generated and a test project is exported Finally, this test project is built and run to create UI test reports This section presents the details of the first phase of extracting widgets in an application

Figure 1: The overview of the proposed method

3.1 Widgets objects extraction

In this phase, all widgets and their properties are extracted from SUT These widgets are prerequisite for testers can design test sce-narios in the next phase After the test application was launched, the process identifier (PID) of the application is retrieved By using this PID, widgets are found recursively from the root widgets cor-responding to the windows of the application For each widget, the following properties need to be retrieved

• type: there are several kinds of widgets such as window, button, text box, radio button, tree view, etc

2 https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/

ui-automation-overview

•id: this is an attribute which its value was assigned by the

programmer in software developing process

•name: a string, it is usually the same as the content of this

widget

•parent and children

•size: height, width of widget

•location: relative location of widget

•screen capture of widget

• other properties: e.g., state (enable or disable), visibility, etc

For each type of widget, maybe there are some more specific

attributes need to be retrieved For example, with a text box widget,

it is necessary to determine this widget is read-only or not During

the extraction process, there are several kinds of hidden widgets

which maybe cannot be detected There are two major types of

hidden widgets including hidden widgets in the opened windows

and hidden widgets in windows which have not opened yet The

details of solutions to overcome this problem for each hidden type

are presented in the next sections

3.2 Hidden widgets in opened windows

analyzer

When a window of SUT is opened, all widgets in this window need

to be extracted However, maybe this objective cannot be achieved

because it contains some hidden widgets For example, in a combo

box at the normal state, except the current selection which is being

selected, the remain selections are invisible (Fig 2), hence they are

treated as hidden widgets In order to detect these kinds of widgets,

several appropriate actions are performed automatically at runtime

during inspection to visible widgets which were hidden before In

the above-mentioned example, the suitable action is to click the

drop-down button located on the right of this combo box (Fig 3)

Figure 2: Combo box at

normal state

Figure 3: Combo box after click drop-down button

Menu is a common design in a Window desktop application

In addition to menu items are visible (e.g., File, Edit, Format, etc

in Fig 4), there are also menu items and submenu items located

inside which only be displayed when select their parent menu

item (e.g., New, Open, Save, etc in Fig 4) Obviously, these menu

items cannot be detected, they are considered as hidden widgets

Unfortunately, the solution performing several actions on each

menu item at runtime as described above for combo box widget

type cannot be applied here because of some reasons Firstly, we

do not know how to visible children (selections) of a menu item

There are cases where users need to click on the item, other cases

only require hover instead of click Secondly, if each menu item we

perform a click action to visible its children, the structure of the

application may be unwanted changed For example, if we click on item Exit (not care this item contain children or not), the application will close, the extraction process will finish Hence, it is necessary to have another method to deal with hidden menu items In this paper,

we use some low-level functions in WinAPI to analyze and detect all menu items of SUT The details about the solution to extract each type of hidden widgets in an opened window are presented in Table 1

Figure 4: Hidden menu items in Notepad application

Table 1: Handling hidden widgets in opened windows Widget

type

Widgets are maybe hidden Solutions tree

view

children nodes when parent is not expand

expand all nodes that contain children at runtime

tab widgets in

unselected tab items

in turn select each tab item, extract all widgets in this tab item at runtime combo

box

selections that are not being selected

click drop-down button on the right at runtime menu

item sub menu items

use low-level functions

in WinAPI to extract widgets

3.3 Hidden widgets in unopened windows analyzer

Figure 5: Solution to display all windows in SUT

In fact, a Windows desktop application contains more than one window, but not all of them are opened immediately after the ap-plication is launched For example, the Notepad apap-plication only has a text editor window opening after it is started, Font selection

window will be displayed when users click Format -> Font

How-ever, there is no way to find widgets in the windows that have not

been opened yet Thus, all windows of SUT should be opened

be-fore extracting process in order to avoid ignoring widgets in these

windows This research proposes modification of SUT from source

code of the project as a solution to display all windows in it The

approach can be divided into three phases as in the Fig 5

In the first phase, all windows of the application are found by

analyzing the project’s source code For example, if the source code

is written in C#, this mission will be solved by fetching all classes,

a class inherited from one of the two following classes will be a

corresponding window

• System.Windows.Window

• System.Windows.Forms.Form

Algorithm 1: Modify source code of project

Input : Project’s source code

Output : Modified application

1 begin

2 create empty l ist

3 foreach cl ass in project do

4 if cl ass inherits W indow or F or m then

5 add cl ass to l ist

6 t emp F il e = file from path “temFile.txt”

7 foreach cl ass in l ist do

8 name = name of cl ass

9 insert statement “AppendText2File(t emp F il e ,name );”

into the first of constructor function

10 insert statement “OpenAllWindows();”

into the end of constructor function

11 function OpenAllWindows(t emp F il e ):

12 if t emp F il e ’s content not contains S econdW indow then

13 open S econdW indow

14 if t emp F il e ’s content not contains T hir dW indow then

15 open T hir dW indow

/* do the same with the remains windows */

After that, several statements which open windows are inserted

into suitable locations There are some conditions need to be

sat-isfied in the insertion process including all windows of SUT need

to be opened and each one is called open exactly one time Hence,

if a window is displayed, it does not need to insert new scripts to

open again Algorithm 1 describes the proposed method to solve

these problems The main idea of this method is to store the opened

windows by using a temporary text filetempFile.txt Each class is

corresponding to a window, in its constructor function, a statement

writing its name is inserted at the first, and another statement

call-ing toOpenAllW indows function is added at the end The content

of theOpenAllW indows function contains scripts that open all

win-dows of SUT except itself However, before opening a window, it

must be ensured that this window has not opened yet by checking

the existence of its name in the temporary text file Finally, the

modified source code is rebuilt to create a new application

Wid-gets’ properties and related information need to be saved because

they are used many times in the next phases (e.g., to find a correct

widget to execute an interaction) The easiest way for this purpose

is to write all information in files WidgetsâĂŹ screen capture im-ages are encoded into string format to write in files Because these encoded strings are often very long, so they should be stored in a separate file, independent with other attributes

4 TEST SCRIPTS GENERATION AND EXECUTION

4.1 Test scripts generation The method to generate test scripts consists of two main steps Firstly, from the widgets retrieved before, testers add a collection

of interactions for each one (e.g., click, select the first option of a combo box widget) These interactions are combined to generate a set of test scenarios A test scenario contains a sequence of widgets, each one is assigned a value from it’s set of interactions In order to avoid creating numerous scenarios when there are many widgets

or interactions, users can define several combination constraints to limit the number of scenarios generated After that, the executable test scripts represented for each scenario are exported automatically The method for specifying test scenarios is very important, it is fundamental for testers design desired scenarios Our specification method ensures that it is both cost-effective to create new test scenarios and easy for testers to use in practice By using this method, they only need to define a set of desired actions, validations for each widget They spend fewer efforts than recording events in Record & Replay technique or manual writing test scripts Test scripts are essentially a collection of statements interact with widgets of each test scenario The statement

elements.LoдinW indow.LoдinButton.Click();

is an example expresses a click action onLoдinButton widget lo-cated inLoдinW indow Declaring the parent widget LoдinW indow

in this statement is necessary because of the existence of two or more widgets have the same name but they are located in different windows For example, another window namedSecondW indow also contains aLoдinButton, so if only using LoдinButton.Click(), this action will be not clear when not know the target widget (located

inLoдinW indow or SecondW indow) Finally, test scripts and files store widgets’ information are packaged into a test project With this test project, users can run many times later independently 4.2 Test execution

Executing test project is essentially the execution of a sequence

of actions, validations with widgets There are multiple types of widgets, each one has various interaction types Basically, all inter-actions can be divided into three groups as follows

• Direct actions (hereinafter referred to as “Action”): e.g., click

a button, input text into a text box, set value for widget’s width, etc

• Validation interactions (hereinafter referred to as “Valida-tion”): e.g., check the existence of a widget, validate the value

of text attribute of a widget with some texts, etc

• Other interactions: e.g., capture the current screen, delay an amount of time, wait until a window is displayed, etc

If an interaction belongs to the Action or Validation type, there are two steps to execute it including target widget detection and

interaction execution The target widget is found from the

root-widgets using its attributes’ value achieved in the previous phase

Searching target widget often takes longer time than executing

interaction and it is considered as a challenge in UI testing UI

automated testing tools may fail when trying to find the widget

before performing an action, especially with hidden widgets (e.g.,

submenu items, selections of a combo box, etc) In this paper, in

order to overcome this problem, some low-level functions provided

by WinAPI are invoked to search destination widgets

After test project was finished running, a test report is exported

This test report needs to show the state of the execution (success

or failure) of each test scenario There are three states of the result

after performing an interaction as follows

• Success: this interaction was executed successfully

• Failure: this interaction was executed completely, but it

re-turned false For example, validation the value of text

at-tribute of a widget with some texts returns false

• Error: this interaction was not executed completely It means

that in the execution process, there was something

inter-rupted this interaction, and it could not continue For

in-stance, the target widget cannot be found

Whenever the result of executing an interaction is different from

success state, the hold current screen will be captured and written

into the test report as an additional information

5 TOOL AND EXPERIMENTS

5.1 Implemented tool

Based on the proposed approach, a Windows Desktop tool named

GTA (GUI Tesing Automation) has been developed This a

Win-dows Desktop application implemented using C# language and

WPF framework It contains four main modules including Widgets

inspection, Test scripts generation, Test execution, GUI visualizer

(Fig 6) In widgets inspection module, the widgets and their

prop-erties are extracted by using WinAPI and Microsoft UI Automation

library Users need to select the application to test by one of three

ways: select exe file, select zip file contains the source code of a

C# project, select an application instance already running

In order to show the effectiveness of GTA, it was tested with

UI of several applications In this section, a case study with

TSDV-Application1 is illustrated Figure 7 shows the interface of the tool

after extracting widgets from the application In the tree view on

the left, users can select some widgets that related to any desired

test scenarios Then, they add several interactions for each widget,

constraints for the tool combines actions to generate a sequence

of test scenarios Finally, GTA exports test scripts and packages

them into a test project which can be opened, built, and run within

Visual Studio

Figure 8 shows an example of the report which is generated

whenever running test project completely It can be seen that the

scenario Feature_1 is failed since the action checking the existence

of TaskManagerView returns failure (the image shows the screen

captured at this time) Both three remains validation actions are

succeeded

Figure 6: The architecture of GTA

Figure 7: GUI of the tool

5.2 Experiments

In the process of evaluating experiments, we used two applications named TSDV-Application1 and TSDV-Application2 (were built by TSDV) in addition to several other real well-known applications Both of these two applications include almost all types of widget and provides various kinds of interaction Hence, if the experiments are achieved from them, likely the same results will be obtained from others

There are various types of widgets, each one supports several tions Firstly, this section makes a comparison in term of the ac-curacy in simulating several common actions between GTA and commercial well-known tool named Ranorex This comparison is

Figure 8: A test report of GTA.

operated under TSDV-Application1 and Calculator (default on

Win-dows 10) The details of the experiment results are shown in Table 2

Table 2: Comparison in term of simulating several actions

between Ranorex and GTA

Widget type Actions Ranorex GTA

Button Click/ Double click/ Right

click it Success Success

Textbox Input a string in it Success Success

Combo box Set item selection of it Success Success

Window Check the existence of it Success Success

Any Send keyboard keys

combination (e.g., Ctrl + A) Success Success

Text block Validate its text equals with

a string Success Success

Tree view Expand/ Collapse/ Select

a tree view item Success Success

Tab Click a tab item (to change

item selected of it) Success Success

Combo box Click an item selection

inside it

Cannot perform action Success List box Click an invisible item of it Cannot

perform action Success

It can be seen that both Ranorex and GTA execute some simple

actions successfully (e.g., click a button, validate the text value of a

text block, expand a tree view item, etc) However, there are two

cases that Ranorex performs with errors, but GTA does not Firstly,

Ranorex APIcombobox.Items[index].Click() is invoked to click the

index-th selection insidecombobox (choose this selection), Ranorex

cannot perform this action (process stops midway and display an

error), hence the selection is not select Secondly, when using API

listbox.Items[index].Click() to click the index-th item (an invisible

item, must scroll mouse down to see it) oflistbox, Ranorex throws

an exception and displays an error in the report In contrast, GTA

simulates two actions in these cases without any mistakes These

results are achieved because of using some low-level WinAPI

func-tions in both searching target widgets and executing interacfunc-tions

processes

Test scripts can be generated if and only if the target widgets were retrieved in the previous inspection phase Hence, extracting all widgets from SUT is consider as an important task, especially with hidden widgets Therefore, we also make another comparison in term of the number of widgets extracted between GTA, Ranorex Spy, and UI Spy Ranorex Spy is a tool in commercial Ranorex tools and

UI Spy3is another provided by Microsoft Four applications are used

as the inputs including TSDV-Application1, TSDV-Application2, Calculator (default on Windows 10) and Audacity4(an open source audio software) All of the widgets in this experiment we present here are only in opened windows Although neither Ranorex Spy nor UI Spy is able to retrieve widgets in the unopened windows,

we do not put this kind of difference to this experiment because extracting widgets from unopened windows is not the objective that these tools aim to The details of the comparison are presented

in Table 3

Table 3: Comparison in terms of the number of widgets ex-tracted

Application Widget

type GTA Ranorex Spy UI Spy Calculator Combo

box Fully

Miss not selected options

Miss not selected options

TSDV Application1

Tree view Fully Fully

Miss children nodes which parent are not expanded Tab Fully

Miss widgets in not selected tab pages

Miss widgets in not selected tab pages Combo

box Fully Miss all options Fully Menu

item

Miss menu items’

screenshots

Miss menu items’

screenshots

Miss menu items’

screenshots

TSDV Application2

Tab Fully

Miss widgets’

screenshots in not selected tab pages

Miss widgets in not selected tab pages Combo

box Fully Miss all options Fully Audacity Menu

item

Miss menu items’

screenshots

Miss all submenu items

Miss all submenu items

It can be seen that both Ranorex Spy and UI Spy ignore some widgets in the extraction process Ranorex Spy always ignores some selections in a combo box widget in three applications (Calcula-tor, TSDV-Application1, and TSDV-Application2) Similar to the combo box, with a tab widget, Ranorex Spy also omits widgets in the unselected tab-pages UI Spy detects fully selections of a combo box widget in two TSDV’s application, but with Calculator app, it ignores un-selected selections Besides, UI Spy cannot detect some nodes in a tree view whose parents have not expanded yet In con-trast, GTA overcomes these limitations because it applied several

3 https://msdn.microsoft.com/en-us/library/ms727247(v=vs.110).aspx

4 https://www.audacityteam.org/

techniques at runtime inspection to deal with hidden widgets For

example, whenever the extraction process encounters a combo box

widget, GTA will automatically execute clicking the drop-down

button on the right of it to visible all item selections inside GTA

can detect all invisible menu items (e.g., New, Open inside File), but

it cannot retrieve their screenshots The reason is that it only uses

several low-level WinAPI functions to search all menu items, it

does not apply any actions at runtime to visible them Ranorex Spy

and UI Spy identify fully menu items in TSDV-Application1 the

same as GTA, but they cannot detect any invisible menu items in

Audacity application

6 CONCLUSION

This paper presented a method of automated UI testing for

Windows-based applications The proposed method includes three phases

including widgets inspection, test scripts generation, and test

ex-ecution The key idea of the proposed method is to generate new

test scenarios from the specification and widgets extracted from the

application Firstly, all widgets are extracted from SUT during the

execution of it Then, a collection of UI test scenarios are generated

by combining the interactions of each widget from the

specifica-tion After that, the corresponding test scripts are generated and

packaged into a test project Finally, this test project is built and

run on SUT to export UI test reports

The proposed method can generate a sequence of test scenarios

by combining a collection of interactions of each widget Testers

only need to define interactions of each widget, hence this method

saves testers’ efforts in creating UI interaction scenarios The

ex-ported test project can be run multiple times automatically so it

reduces the cost of test execution By using some low-level

Win-dow API functions, this method could find the correct widget and

perform interaction accurately Furthermore, this paper applied

three kinds of techniques to detect hidden widgets, which is

consid-ered as one of the most challenges in automated UI testing Firstly,

some appropriate actions are performed automatically at runtime

in extraction process whenever encounter a special widget type

to visible other widgets which were hidden before (e.g., expand

all nodes inside a tree view to visible its children) Secondly, some

low-level functions provided by WinAPI are used to search all menu

items in SUT Thirdly, the hidden widgets in unopened windows

are retrieved by modifying the source code of the input project to

display all of the windows before inspection

At the same time, the proposed method has been implemented in

the tool name GTA to demonstrate the effectiveness of this method

The experiment results have illustrated that GTA could overcome

the inaccuracies which Ranorex encountered in simulating two

actions including click an item selection inside a combo box and

click an invisible item in a list box Furthermore, GTA could detect

more hidden widgets in comparison with that of Ranorex Spy and

UI Spy Currently, GTA has been deployed for use in TSDV and received many positive feedbacks In parallel with that, TSDV are also applying the tool with several well-known applications such

as Notepad, Outlook mail, etc to obtain more experimental results Based on the assessments, GTA would be continually improving

so as to apply more effectively on various kind of projects written

on different platforms in practice Specifically, the next research would focus on solving the problem of hidden widgets detection in dynamic applications (e.g., widgets are loaded from the database)

In addition, GTA will be extended the current idea of the proposal

on other platforms such as Linux, and MacOS

REFERENCES [1] Samer Al-Zain, Derar Eleyan, and Joy Garfield 2012 Automated user inter-face testing for web applications and TestComplete In Proceedings of the CUBE International Information Technology Conference ACM, 350–354.

[2] Silviu Andrica and George Candea 2011 WaRR: A tool for high-fidelity web application record and replay In Dependable Systems & Networks (DSN), 2011 IEEE/IFIP 41st International Conference on IEEE, 403–410.

[3] Fevzi Belli 2001 Finite state testing and analysis of graphical user interfaces In Software Reliability Engineering, 2001 ISSRE 2001 Proceedings 12th International Symposium on IEEE, 34–43.

[4] Marco Cunha, Ana CR Paiva, Hugo Sereno Ferreira, and Rui Abreu 2010 PETTool:

a pattern-based GUI testing tool In Software Technology and Engineering (ICSTE),

2010 2nd International Conference on, Vol 1 IEEE, V1–202.

[5] Lorenzo Gomez, Iulian Neamtiu, Tanzirul Azim, and Todd Millstein 2013 Reran: Timing-and touch-sensitive record and replay for android In Software Engineering (ICSE), 2013 35th International Conference on IEEE, 72–81.

[6] André MP Grilo, Ana CR Paiva, and João Pascoal Faria 2010 Reverse engineering

of GUI models for testing In Information Systems and Technologies (CISTI), 2010 5th Iberian Conference on IEEE, 1–6.

[7] Yongjian Hu and Iulian Neamtiu 2016 VALERA: an effective and efficient record-and-replay tool for android In Proceedings of the International Conference on Mobile Software Engineering and Systems ACM, 285–286.

[8] Atif M Memon 2007 An event-flow model of GUI-based applications for testing Software testing, verification and reliability 17, 3 (2007), 137–157.

[9] Atif M Memon, Martha E Pollack, and Mary Lou Soffa 2001 Hierarchical GUI test case generation using automated planning IEEE transactions on software engineering 27, 2 (2001), 144–155.

[10] Ravi Narkhede, Sapana Korde, Avinash Darda, and Saurabh Sharma 2015 An industrial research on GUI testing techniques for windows based application using UFT In Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials (ICSTM), 2015 International Conference on IEEE, 466–471.

[11] Microsoft Developer Network 2010 Overview of the Windows API https: //msdn.microsoft.com/en-us/library/aa383723

[12] Jeff Offutt and Aynur Abdurazik 1999 Generating Tests from UML Specifica-tions In Proceedings of the 2Nd International Conference on The Unified Modeling Language: Beyond the Standard (UML’99) Springer-Verlag, Berlin, Heidelberg, 416–429 http://dl.acm.org/citation.cfm?id=1767297.1767341

[13] Zhengrui Qin, Yutao Tang, Ed Novak, and Qun Li 2016 Mobiplay: A remote execution based record-and-replay tool for mobile applications In Proceedings of the 38th International Conference on Software Engineering ACM, 571–582 [14] Lee J White 1996 Regression Testing of GUI Event Interactions In icsm, Vol 96 350–358.

[15] Fadi Zaraket, Wes Masri, Marc Adam, Dalal Hammoud, Raghd Hamzeh, Raja Farhat, Elie Khamissi, and Joseph Noujaim 2012 Guicop: Specification-based gui testing In Software Testing, Verification and Validation (ICST), 2012 IEEE Fifth International Conference on IEEE, 747–751.