Giáo trình SoftWare Testing

Giáo trình SoftWare Testing

Table of Contents

1 INTRODUCTION TO SOFTWARE 7

1.1 EVOLUTION OF THE SOFTWARE TESTING DISCIPLINE 7

1.2 THE TESTING PROCESS AND THE SOFTWARE TESTING LIFE CYCLE 7

1.3 BROAD CATEGORIES OF TESTING 8

1.4 WIDELY EMPLOYED TYPES OF TESTING 8

1.5 THE TESTING TECHNIQUES 9

1.6 CHAPTER SUMMARY 9

2 BLACK BOX AND WHITE BOX TESTING 11

2.1 INTRODUCTION 11

2.2 BLACK BOX TESTING 11

2.3 TESTING STRATEGIES/TECHNIQUES 13

2.4 BLACK BOX TESTING METHODS 14

2.5 BLACK BOX (VS) WHITE BOX 16

2.6 WHITE BOX TESTING 18

3 GUI TESTING 23

3.1 SECTION 1 - WINDOWS COMPLIANCE TESTING 23

3.2 SECTION 2 - SCREEN VALIDATION CHECKLIST 25

3.3 SPECIFIC FIELD TESTS 29

3.4 VALIDATION TESTING - STANDARD ACTIONS 30

4 REGRESSION TESTING 33

4.1 WHAT IS REGRESSION TESTING 33

4.2 TEST EXECUTION 34

4.3 CHANGE REQUEST 35

4.4 BUG TRACKING 35

4.5 TRACEABILITY MATRIX 36

5 PHASES OF TESTING 39

5.1 INTRODUCTION 39

5.2 TYPES AND PHASES OF TESTING 39

5.3 THE “V”MODEL 40

6 INTEGRATION TESTING 43

6.1 GENERALIZATION OF MODULE TESTING CRITERIA 44

7 ACCEPTANCE TESTING 49

7.1 INTRODUCTION – ACCEPTANCE TESTING 49

7.2 FACTORS INFLUENCING ACCEPTANCE TESTING 49

7.3 CONCLUSION 50

8 SYSTEM TESTING 51

8.1 INTRODUCTION TO SYSTEM TESTING 51

8.2 NEED FOR SYSTEM TESTING 51

8.3 SYSTEM TESTING TECHNIQUES 52

8.4 FUNCTIONAL TECHNIQUES 53

8.5 CONCLUSION: 53

9 UNIT TESTING 54

9.1 INTRODUCTION TO UNIT TESTING 54

9.2 UNIT TESTING –FLOW: 55

UNIT TESTING – BLACK BOX APPROACH 56

UNIT TESTING – WHITE BOX APPROACH 56

UNIT TESTING – FIELD LEVEL CHECKS 56

UNIT TESTING – FIELD LEVEL VALIDATIONS 56

UNIT TESTING – USER INTERFACE CHECKS 56

9.3 EXECUTION OF UNIT TESTS 57

UNIT TESTING FLOW : 57

DISADVANTAGE OF UNIT TESTING 59

METHOD FOR STATEMENT COVERAGE 59

RACE COVERAGE 60

9.4 CONCLUSION 60

10 TEST STRATEGY 62

10.1 INTRODUCTION 62

10.2 KEY ELEMENTS OF TEST MANAGEMENT: 62

10.3 TEST STRATEGY FLOW : 63

10.4 GENERAL TESTING STRATEGIES 65

10.5 NEED FOR TEST STRATEGY 65

10.6 DEVELOPING A TEST STRATEGY 66

10.7 CONCLUSION: 66

11 TEST PLAN 68

11.1 WHAT IS A TEST PLAN? 68

CONTENTS OF A TEST PLAN 68

11.2 CONTENTS (IN DETAIL) 68

12 TEST DATA PREPARATION - INTRODUCTION 71

12.1 CRITERIA FOR TEST DATA COLLECTION 72

12.2 CLASSIFICATION OF TEST DATA TYPES 79

12.3 ORGANIZING THE DATA 80

12.4 DATA LOAD AND DATA MAINTENANCE 82

12.5 TESTING THE DATA 83

12.6 CONCLUSION 84

13 TEST LOGS - INTRODUCTION 85

13.1 FACTORS DEFINING THE TEST LOG GENERATION 85

13.2 COLLECTING STATUS DATA 86

14 TEST REPORT 92

14.1 EXECUTIVE SUMMARY 92

15 DEFECT MANAGEMENT 95

15.1 DEFECT 95

15.2 DEFECT FUNDAMENTALS 95

15.3 DEFECT TRACKING 96

15.4 DEFECT CLASSIFICATION 97

15.5 DEFECT REPORTING GUIDELINES 98

16 AUTOMATION 101

16.1 WHY AUTOMATE THE TESTING PROCESS? 101

16.2 AUTOMATION LIFE CYCLE 103

16.3 PREPARING THE TEST ENVIRONMENT 105

16.4 AUTOMATION METHODS 108

17 GENERAL AUTOMATION TOOL COMPARISON 111

17.1 FUNCTIONAL TEST TOOL MATRIX 111

17.2 RECORD AND PLAYBACK 111

17.3 WEB TESTING 112

17.4 DATABASE TESTS 112

17.5 DATA FUNCTIONS 112

17.6 OBJECT MAPPING 113

17.7 IMAGE TESTING 114

17.8 TEST/ERROR RECOVERY 114

17.9 OBJECT NAME MAP 114

17.10 OBJECT IDENTITY TOOL 115

17.11 EXTENSIBLE LANGUAGE 115

17.12 ENVIRONMENT SUPPORT 116

17.13 INTEGRATION 116

17.14 COST 116

17.15 EASE OF USE 117

17.16 SUPPORT 117

17.17 OBJECT TESTS 117

17.18 MATRIX 118

17.19 MATRIX SCORE 118

18 SAMPLE TEST AUTOMATION TOOL 119

18.1 RATIONAL SUITE OF TOOLS 119

18.2 RATIONAL ADMINISTRATOR 120

18.3 RATIONAL ROBOT 124

18.4 ROBOT LOGIN WINDOW 125

18.5 RATIONAL ROBOT MAIN WINDOW-GUI SCRIPT 126

18.6 RECORD AND PLAYBACK OPTIONS 127

18.7 VERIFICATION POINTS 129

18.8 ABOUT SQABASIC HEADER FILES 131

18.9 ADDING DECLARATIONS TO THE GLOBAL HEADER FILE 131

18.10 INSERTING A COMMENT INTO A GUI SCRIPT: 131

18.11 ABOUT DATA POOLS 132

18.12 DEBUG MENU 132

18.13 COMPILING THE SCRIPT 133

18.14 COMPILATION ERRORS 134

19 RATIONAL TEST MANAGER 136

19.1 TEST MANAGER-RESULTS SCREEN 137

20 SUPPORTED ENVIRONMENTS 139

20.1 OPERATING SYSTEM 139

20.2 PROTOCOLS 139

20.3 WEB BROWSERS 139

20.4 MARKUP LANGUAGES 139

20.5 DEVELOPMENT ENVIRONMENTS 139

21 PERFORMANCE TESTING 140

21.1 WHAT IS PERFORMANCE TESTING? 140

21.2 WHY PERFORMANCE TESTING? 140

21.3 PERFORMANCE TESTING OBJECTIVES 141

21.4 PRE-REQUISITES FOR PERFORMANCE TESTING 141

21.5 PERFORMANCE REQUIREMENTS 142

22 PERFORMANCE TESTING PROCESS 143

22.1 PHASE 1 – REQUIREMENTS STUDY 144

22.2 PHASE 2 – TEST PLAN 145

22.3 PHASE 3 – TEST DESIGN 145

22.4 PHASE 4 –SCRIPTING 146

22.5 PHASE 5 – TEST EXECUTION 147

22.6 PHASE 6 – TEST ANALYSIS 147

22.7 PHASE 7 – PREPARATION OF REPORTS 148

22.8 COMMON MISTAKES IN PERFORMANCE TESTING 149

22.9 BENCHMARKING LESSONS 149

23 TOOLS 151

23.1 LOADRUNNER 6.5 151

23.2 WEBLOAD 4.5 151

23.3 ARCHITECTURE BENCHMARKING 158

23.4 GENERAL TESTS 159

24 PERFORMANCE METRICS 160

24.1 CLIENT SIDE STATISTICS 160

24.2 SERVER SIDE STATISTICS 161

24.3 NETWORK STATISTICS 161

24.4 CONCLUSION 161

25 LOAD TESTING 163

25.1 WHY IS LOAD TESTING IMPORTANT ? 163

25.2 WHEN SHOULD LOAD TESTING BE DONE? 163

26 LOAD TESTING PROCESS 164

26.1 SYSTEM ANALYSIS 164

26.2 USER SCRIPTS 164

26.3 SETTINGS 164

26.4 PERFORMANCE MONITORING 165

26.5 ANALYZING RESULTS 165

26.6 CONCLUSION 165

27 STRESS TESTING 167

27.1 INTRODUCTION TO STRESS TESTING 167

27.2 BACKGROUND TO AUTOMATED STRESS TESTING 168

27.3 AUTOMATED STRESS TESTING IMPLEMENTATION 170

27.4 PROGRAMMABLE INTERFACES 170

27.5 GRAPHICAL USER INTERFACES 171

27.6 DATA FLOW DIAGRAM 171

27.7 TECHNIQUES USED TO ISOLATE DEFECTS 172

28 TEST CASE COVERAGE 174

28.1 TEST COVERAGE 174

28.2 TEST COVERAGE MEASURES 174

28.3 PROCEDURE-LEVEL TEST COVERAGE 175

28.4 LINE-LEVEL TEST COVERAGE 175

28.5 CONDITION COVERAGE AND OTHER MEASURES 175

28.6 HOW TEST COVERAGE TOOLS WORK 175

28.7 TEST COVERAGE TOOLS AT A GLANCE 177

29 TEST CASE POINTS-TCP 178

29.1 WHAT IS A TEST CASE POINT (TCP) 178

29.2 CALCULATING THE TEST CASE POINTS: 178

29.3 CHAPTER SUMMARY 180

1 Introduction to Software

1.1 Evolution of the Software Testing discipline

The effective functioning of modern systems depends on our ability to produce software

in a cost-effective way The term software engineering was first used at a 1968 NATO workshop in West Germany It focused on the growing software crisis! Thus we see that the software crisis on quality, reliability, high costs etc started way back when most of today’s software testers were not even born!

The attitude towards Software Testing underwent a major positive change in the recent years In the 1950’s when Machine languages were used, testing is nothing but

debugging When in the 1960’s, compilers were developed, testing started to be

considered a separate activity from debugging In the 1970’s when the software

engineering concepts were introduced, software testing began to evolve as a technical discipline Over the last two decades there has been an increased focus on better, faster and cost-effective software Also there has been a growing interest in software safety, protection and security and hence an increased acceptance of testing as a technical discipline and also a career choice!

Now to answer, “What is Testing?” we can go by the famous definition of Myers, which says, “Testing is the process of executing a program with the intent of finding errors”

1.2 The Testing process and the Software Testing Life Cycle

Every testing project has to follow the waterfall model of the testing process

The waterfall model is as given below

1.Test Strategy & Planning

Software Testing has been accepted as a separate discipline to the extent that there is a separate life cycle for the testing activity Involving software testing in all phases of the

software development life cycle has become a necessity as part of the software qualityassurance process Right from the Requirements study till the implementation, there needs to betesting done on every phase The V-Model of the Software Testing Life Cyclealong with the Software Development Life cycle given below indicates the various phases

or levels of testing

1.3 Broad Categories of Testing

Based on the V-Model mentioned above, we see that there are two categories of testing activities that can be done on software, namely,

 Static Testing

 Dynamic Testing

The kind of verification we do on the software work products before the process of

compilation and creation of an executable is more of Requirement review, design review,

code review, walkthrough and audits This type of testing is called Static Testing When

we test the software by executing and comparing the actual & expected results, it is

called Dynamic Testing

1.4 Widely employed Types of Testing

From the V-model, we see that are various levels or phases of testing, namely, Unit

testing, Integration testing, System testing, User Acceptance testing etc

Let us see a brief definition on the widely employed types of testing

Unit Testing: The testing done to a unit or to a smallest piece of software Done to verify

if it satisfies its functional specification or its intended design structure

Integration Testing: Testing which takes place as sub elements are combined (i.e.,

integrated) to form higher-level elements

Regression Testing: Selective re-testing of a system to verify the modification (bug

fixes) have not caused unintended effects and that system still complies with its specified requirements

Integration Testing System Testing

User Acceptance Testing

Production Verification Testing

SDLC - STLC

System Testing: Testing the software for the required specifications on the intended hardware

Acceptance Testing: Formal testing conducted to determine whether or not a system

satisfies its acceptance criteria, which enables a customer to determine whether to

accept the system or not

Performance Testing: To evaluate the time taken or response time of the system to

perform it’s required functions in comparison

Stress Testing: To evaluate a system beyond the limits of the specified requirements or

system resources (such as disk space, memory, processor utilization) to ensure the

system do not break unexpectedly

Load Testing: Load Testing, a subset of stress testing, verifies that a web site can

handle a particular number of concurrent users while maintaining acceptable response times

Alpha Testing: Testing of a software product or system conducted at the developer’s

site by the customer

Beta Testing: Testing conducted at one or more customer sites by the end user of a

delivered software product system

1.5 The Testing Techniques

To perform these types of testing, there are two widely used testing techniques The above said testing types are performed based on the following testing techniques

Black-Box testing technique:

This technique is used for testing based solely on analysis of requirements

(specification, user documentation.) Also known as functional testing

White-Box testing technique:

This technique us used for testing based on analysis of internal logic (design, code, etc.)(But expected results still come requirements) Also known as Structural

 Evolution of Software Testing

 The Testing process and lifecycle

 Broad categories of testing

 Widely employed Types of Testing

 The Testing Techniques

2 Black Box and White Box testing

2.1 Introduction

Test Design refers to understanding the sources of test cases, test coverage, how to

develop and document test cases, and how to build and maintain test data There are 2primary methods by which tests can be designed and they are:

- BLACK BOX

- WHITE BOX

Black-box test design treats the system as a literal "black-box", so it doesn't explicitly

use knowledge of the internal structure It is usually described as focusing on testingfunctional requirements Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box

White-box test design allows one to peek inside the "box", and it focuses specifically on

using internal knowledge of the software to guide the selection of test data It is used todetect errors by means of execution-oriented test cases Synonyms for white-box include:structural, glass-box and clear-box

While black-box and white-box are terms that are still in popular use, many people preferthe terms "behavioral" and "structural" Behavioral test design is slightly different fromblack-box test design because the use of internal knowledge isn't strictly forbidden, butit's still discouraged In practice, it hasn't proven useful to use a single test designmethod One has to use a mixture of different methods so that they aren't hindered bythe limitations of a particular one Some call this "gray-box" or "translucent-box" testdesign, but others wish we'd stop talking about boxes altogether!!!

2.2 Black box testing

Black Box Testing is testing without knowledge of the internal workings of the item

being tested For example, when black box testing is applied to software engineering,the tester would only know the "legal" inputs and what the expected outputs should be,but not how the program actually arrives at those outputs It is because of this that blackbox testing can be considered testing with respect to the specifications, no otherknowledge of the program is necessary For this reason, the tester and the programmercan be independent of one another, avoiding programmer bias toward his own work Forthis testing, test groups are often used,

Though centered around the knowledge of user requirements, black box tests do notnecessarily involve the participation of users Among the most important black box teststhat do not involve users are functionality testing, volume tests, stress tests, recoverytesting, and benchmarks Additionally, there are two types of black box test that involveusers, i.e field and laboratory tests In the following the most important aspects of theseblack box tests will be described briefly

2.2.1.1 Black box testing - without user involvement

The so-called ``functionality testing'' is central to most testing exercises Its primaryobjective is to assess whether the program does what it is supposed to do, i.e what isspecified in the requirements There are different approaches to functionality testing One

is the testing of each program feature or function in sequence The other is to test module

by module, i.e each function where it is called first

The objective of volume tests is to find the limitations of the software by processing ahuge amount of data A volume test can uncover problems that are related to the

efficiency of a system, e.g incorrect buffer sizes, a consumption of too much memory

space, or only show that an error message would be needed telling the user that thesystem cannot process the given amount of data

During a stress test, the system has to process a huge amount of data or perform manyfunction calls within a short period of time A typical example could be to perform thesame function from all workstations connected in a LAN within a short period of time (e.g.sending e-mails, or, in the NLP area, to modify a term bank via different terminalssimultaneously)

The aim of recovery testing is to make sure to which extent data can be recovered after asystem breakdown Does the system provide possibilities to recover all of the data or part

of it? How much can be recovered and how? Is the recovered data still correct and

consistent? Particularly for software that needs high reliability standards, recovery testing

is very important

The notion of benchmark tests involves the testing of program efficiency The efficiency

of a piece of software strongly depends on the hardware environment and thereforebenchmark tests always consider the soft/hardware combination Whereas for mostsoftware engineers benchmark tests are concerned with the quantitative measurement ofspecific operations, some also consider user tests that compare the efficiency of differentsoftware systems as benchmark tests In the context of this document, however,benchmark tests only denote operations that are independent of personal variables

2.2.1.2 Black box testing - with user involvement

For tests involving users, methodological considerations are rare in SE literature Rather,one may find practical test reports that distinguish roughly between field and laboratorytests In the following only a rough description of field and laboratory tests will be given.E.g Scenario Tests The term ``scenario'' has entered software evaluation in the early1990s A scenario test is a test case which aims at a realistic user background for theevaluation of software as it was defined and performed It is an instance of black boxtesting where the major objective is to assess the suitability of a software product forevery-day routines In short it involves putting the system into its intended use by itsenvisaged type of user, performing a standardised task

In field tests users are observed while using the software system at their normal working

place Apart from general usability-related aspects, field tests are particularly useful for assessing the interoperability of the software system, i.e how the technical integration of

the system works Moreover, field tests are the only real means to elucidate problems ofthe organisational integration of the software system into existing procedures Particularly

in the NLP environment this problem has frequently been underestimated A typical

example of the organisational problem of implementing a translation memory is thelanguage service of a big automobile manufacturer, where the major implementationproblem is not the technical environment, but the fact that many clients still submit theirorders as print-out, that neither source texts nor target texts are properly organised andstored and, last but not least, individual translators are not too motivated to change theirworking habits

Laboratory tests are mostly performed to assess the general usability of the system Due

to the high laboratory equipment costs laboratory tests are mostly only performed at bigsoftware houses such as IBM or Microsoft Since laboratory tests provide testers withmany technical possibilities, data collection and analysis are easier than for field tests

2.3 Testing Strategies/Techniques

 Black box testing should make use of randomly generated inputs (only a testrange should be specified by the tester), to eliminate any guess work by thetester as to the methods of the function

 Data outside of the specified input range should be tested to check therobustness of the program

 Boundary cases should be tested (top and bottom of specified range) to makesure the highest and lowest allowable inputs produce proper output

 The number zero should be tested when numerical data is to be input

 Stress testing should be performed (try to overload the program with inputs tosee where it reaches its maximum capacity), especially with real time systems

 Crash testing should be performed to see what it takes to bring the system down

 Test monitoring tools should be used whenever possible to track which testshave already been performed and the outputs of these tests to avoid repetitionand to aid in the software maintenance

 Other functional testing techniques include: transaction testing, syntax testing,domain testing, logic testing, and state testing

 Finite state machine models can be used as a guide to design functional tests

 According to Beizer the following is a general order by which tests should bedesigned:

1 Clean tests against requirements

2 Additional structural tests for branch coverage,

as needed

3 Additional tests for data-flow coverage asneeded

4 Domain tests not covered by the above

5 Special techniques as appropriate syntax, loop,state, etc

6 Any dirty tests not covered by the above

2.4 Black box testing Methods

2.4.1 Graph-based Testing Methods

 Black-box methods based on the nature of the relationships (links) among theprogram objects (nodes), test cases are designed to traverse the entire graph

 Transaction flow testing (nodes represent steps in some transaction and linksrepresent logical connections between steps that need to be validated)

 Finite state modeling (nodes represent user observable states of the softwareand links represent transitions between states)

 Data flow modeling (nodes are data objects and links are transformations fromone data object to another)

 Timing modeling (nodes are program objects and links are sequentialconnections between these objects, link weights are required execution times)

 Equivalence class guidelines:

1 If input condition specifies a range, one valid and two invalid equivalenceclasses are defined

2 If an input condition requires a specific value, one valid and two invalidequivalence classes are defined

3 If an input condition specifies a member of a set, one valid and oneinvalid equivalence class is defined

4 If an input condition is Boolean, one valid and one invalid equivalenceclass is defined

2.4.3 Boundary Value Analysis

 Black-box technique that focuses on the boundaries of the input domain ratherthan its center

3 Apply guidelines 1 and 2 to output conditions, test cases should bedesigned to produce the minimum and maxim output reports

4 If internal program data structures have boundaries (e.g size limitations),

be certain to test the boundaries

2.4.4 Comparison Testing

 Black-box testing for safety critical systems in which independently developedimplementations of redundant systems are tested for conformance tospecifications

 Often equivalence class partitioning is used to develop a common set of testcases for each implementation

2.4.5 Orthogonal Array Testing

 Black-box technique that enables the design of a reasonably small set of testcases that provide maximum test coverage

 Focus is on categories of faulty logic likely to be present in the softwarecomponent (without examining the code)

 Priorities for assessing tests using an orthogonal array

1 Detect and isolate all single mode faults

2 Detect all double mode faults

1 Task testing (test each time dependent task independently)

2 Behavioral testing (simulate system response to external events)

3 Intertask testing (check communications errors among tasks)

4 System testing (check interaction of integrated system software andhardware)

2.4.7 Advantages of Black Box Testing

 More effective on larger units of code than glass box testing

 Tester needs no knowledge of implementation, including specific programminglanguages

 Tester and programmer are independent of each other

 Tests are done from a user's point of view

 Will help to expose any ambiguities or inconsistencies in the specifications

 Test cases can be designed as soon as the specifications are complete

2.4.8 Disadvantages of Black Box Testing

 Only a small number of possible inputs can actually be tested, to test everypossible input stream would take nearly forever

 Without clear and concise specifications, test cases are hard to design

 There may be unnecessary repetition of test inputs if the tester is not informed oftest cases the programmer has already tried

 May leave many program paths untested

 Cannot be directed toward specific segments of code which may be verycomplex (and therefore more error prone)

 Most testing related research has been directed toward glass box testing

2.5 Black Box (Vs) White Box

An easy way to start up a debate in a software testing forum is to ask the differencebetween black box and white box testing These terms are commonly used, yet everyoneseems to have a different idea of what they mean

Black box testing begins with a metaphor Imagine you’re testing an electronics system It’s housed in a black box with lights, switches, and dials on the outside You must test it without opening it up, and you can’t see beyond its surface You have to see if it works just by flipping switches (inputs) and seeing what happens to the lights and dials

(outputs) This is black box testing Black box software testing is doing the same thing, but with software The actual meaning of the metaphor, however, depends on how you define the boundary of the box and what kind of access the “blackness” is blocking

An opposite test approach would be to open up the electronics system, see how the

circuits are wired, apply probes internally and maybe even disassemble parts of it By analogy, this is called white box testing,

To help understand the different ways that software testing can be divided between blackbox and white box techniques, consider the Five-Fold Testing System It lays out five dimensions that can be used for examining testing:

1.People(who does the testing)

2 Coverage (what gets tested)

3 Risks (why you are testing)

4.Activities(how you are testing)

5 Evaluation (how you know you’ve found a bug)

Let’s use this system to understand and clarify the characteristics of black box and white

People: Who does the testing?

Some people know how software works (developers) and others just use it (users).Accordingly, any testing by users or other non-developers is sometimes called “blackbox” testing Developer testing is called “white box” testing The distinction here is based

on what the person knows or can understand

Coverage: What is tested?

If we draw the box around the system as a whole, “black box” testing becomes anothername for system testing And testing the units inside the box becomes white box testing

This is one way to think about coverage Another is to contrast testing that aims to coverall the requirements with testing that aims to cover all the code These are the two mostcommonly used coverage criteria Both are supported by extensive literature andcommercial tools Requirements-based testing could be called “black box” because itmakes sure that all the customer requirements have been verified Code-based testing isoften called “white box” because it makes sure that all the code (the statements, paths,

or decisions) is exercised

Risks: Why are you testing?

Sometimes testing is targeted at particular risks Boundary testing and other attack-basedtechniques are targeted at common coding errors Effective security testing also requires

a detailed understanding of the code and the system architecture Thus, thesetechniques might be classified as “white box” Another set of risks concerns whether thesoftware will actually provide value to users Usability testing focuses on this risk, andcould be termed “black box.”

Activities: How do you test?

A common distinction is made between behavioral test design, which defines tests based

on functional requirements, and structural test design, which defines tests based on thecode itself These are two design approaches Since behavioral testing is based onexternal functional definition, it is often called “black box,” while structural testing—based

on the code internals—is called “white box.” Indeed, this is probably the most commonlycited definition for black box and white box testing Another activity-based distinctioncontrasts dynamic test execution with formal code inspection In this case, the metaphormaps test execution (dynamic testing) with black box testing, and maps code inspection(static testing) with white box testing We could also focus on the tools used Some toolvendors refer to code-coverage tools as white box tools, and tools that facilitate applyinginputs and capturing inputs—most notably GUI capture replay tools—as black box tools.Testing is then categorized based on the types of tools used

Evaluation: How do you know if you’ve found a bug?

There are certain kinds of software faults that don’t always lead to obvious failures Theymay be masked by fault tolerance or simply luck Memory leaks and wild pointers areexamples Certain test techniques seek to make these kinds of problems more visible.Related techniques capture code history and stack information when faults occur, helpingwith diagnosis Assertions are another technique for helping to make problems morevisible All of these techniques could be considered white box test techniques, since theyuse code instrumentation to make the internal workings of the software more visible.These contrast with black box techniques that simply look at the official outputs of a

program

White box testing is concerned only with testing the software product, it cannot guaranteethat the complete specification has been implemented Black box testing is concernedonly with testing the specification, it cannot guarantee that all parts of the implementationhave been tested Thus black box testing is testing against the specification and willdiscover faults of omission, indicating that part of the specification has not been fulfilled.White box testing is testing against the implementation and will discoverfaults of commission, indicating that part of the implementation is faulty In order to fullytest a software product both black and white box testing are required

White box testing is much more expensive than black box testing It requires the sourcecode to be produced before the tests can be planned and is much more laborious in thedetermination of suitable input data and the determination if the software is or is notcorrect The advice given is to start test planning with a black box test approach as soon

as the specification is available White box planning should commence as soon as allblack box tests have been successfully passed, with the production of flowgraphs anddetermination of paths The paths should then be checked against the black box test planand any additional required test runs determined and applied

The consequences of test failure at this stage may be very expensive A failure of a whitebox test may result in a change which requires all black box testing to be repeated andthe re-determination of the white box paths

To conclude, apart from the above described analytical methods of both glass and blackbox testing, there are further constructive means to guarantee high quality software endproducts Among the most important constructive means are the usage of object-orientedprogramming tools, the integration of CASE tools, rapid prototyping, and last but not leastthe involvement of users in both software development and testing procedures

Summary :

Black box testing can sometimes describe user-based testing (people); system orrequirements-based testing (coverage); usability testing (risk); or behavioral testing orcapture replay automation (activities) White box testing, on the other hand, cansometimes describe developer-based testing (people); unit or code-coverage testing(coverage); boundary or security testing (risks); structural testing, inspection or code-coverage automation (activities); or testing based on probes, assertions, and logs(evaluation)

2.6 WHITE BOX TESTING

Software testing approaches that examine the program structure and derive test data from the program logic Structural testing is sometimes referred to as clear-box testing since white boxes are considered opaque and do not really permit visibility into the code

Synonyms for white box testing

 Glass Box testing

 Structural testing

 Clear Box testing

 Open Box Testing

Types of White Box testing

A typical rollout of a product is shown in figure 1 below

The purpose of white box testing

Initiate a strategic initiative to build quality throughout the life cycle of a software product

or service

Provide a complementary function to black box testing

Perform complete coverage at the component level

Improve quality by optimizing performance

Practices :

This section outlines some of the general practices comprising white-box testing process

In general, white-box testing practices have the

following considerations:

1 The allocation of resources to perform class and method analysis and to

document and review the same

2 Developing a test harness made up of stubs, drivers and test object libraries

3 Development and use of standard procedures, naming conventions and libraries

4 Establishment and maintenance of regression test suites and procedures

5 Allocation of resources to design, document and manage a test history library

6 The means to develop or acquire tool support for automation of

capture/replay/compare, test suite execution, results verification and

documentation capabilities

1 Code Coverage Analysis

1.1 Basis Path Testing

A testing mechanism proposed by McCabe whose aim is to derive a logical

complexity measure of a procedural design and use this as a guide for defining abasic set of execution paths These are test cases that exercise basic set will execute every statement at least once

1.1.1 Flow Graph Notation

A notation for representing control flow similar to flow charts and UML activity diagrams

1.1.2 Cyclomatic Complexity

The cyclomatic complexity gives a quantitative measure of 4the logical complexity This value gives the number of independent paths in the basis set, and an upper bound for the number of tests to ensure that each statement is executed at least once An independent path is any path through a program that introduces at least one new set of processing statements or a new condition (i.e., a new edge) Cyclomatic complexity provides upper bound for number of tests required to

guarantee coverage of all program statements

1.2 Control Structure testing

 Boolean expression : Condition without Relational expressions

1.2.2 Data Flow Testing

Selects test paths according to the location of definitions and use of variables

1.2.3 Loop Testing

Loops fundamental to many algorithms Can define loops as simple, concatenated, nested, and unstructured

Examples:

Note that unstructured loops are not to be tested rather, they are redesigned

2 Design by Contract (DbC)

DbC is a formal way of using comments to incorporate specification information into the code itself Basically, the code specification is expressed unambiguously using a formal language that describes the code's implicit contracts These contracts specify such

requirements as:

 Conditions that the client must meet before a method is invoked

 Conditions that a method must meet after it executes

 Assertions that a method must satisfy at specific points of its execution Tools that check DbC contracts at runtime such as JContract

[http://www.parasoft.com/products/jtract/index.htm] are used to perform this function

3 Profiling

Profiling provides a framework for analyzing Java code performance for speed and heap memory use It identifies routines that are consuming the majority of the CPU time so that problems may be tracked down to improve performance

These include the use of Microsoft Java Profiler API and Sun’s profiling tools that are bundled with the JDK Third party tools such as JaViz

[http://www.research.ibm.com/journal/sj/391/kazi.html] may also be used to perform this function

4 Error Handling

Exception and error handling is checked thoroughly are simulating partial and complete fail-over by operating on error causing test vectors Proper error recovery, notification andlogging are checked against references to validate program design

5 Transactions

Systems that employ transaction, local or distributed, may be validated to ensure that ACID (Atomicity, Consistency, Isolation, Durability) Each of the individual parameters is tested individually against a reference data set

Transactions are checked thoroughly for partial/complete commits and rollbacks

encompassing databases and other XA compliant transaction processors

Advantages of White Box Testing

 Forces test developer to reason carefully about implementation

 Approximate the partitioning done by execution equivalence

 Reveals errors in "hidden" code

3 GUI Testing

What is GUI Testing?

GUI is the abbreviation for Graphic User Interface It is absolutely essential that anyapplication has to be user-friendly The end user should be comfortable while using allthe components on screen and the components should also perform their functionalitywith utmost clarity Hence it becomes very essential to test the GUI components of anyapplication GUI Testing can refer to just ensuring that the look-and-feel of the application

is acceptable to the user, or it can refer to testing the functionality of each and everycomponent involved

The following is a set of guidelines to ensure effective GUI Testing and can be used even

as a checklist while testing a product / application

3.1 Section 1 - Windows Compliance Testing

3.1.1 Application

Start Application by Double Clicking on its ICON The Loading message should show the application name, version number, and a bigger pictorial representation of the icon No Login is necessary The main window of the application should have the same caption asthe caption of the icon in Program Manager Closing the application should result in an

"Are you Sure" message box Attempt to start application twice This should not be

allowed - you should be returned to main window Try to start the application twice as it isloading On each window, if the application is busy, then the hour glass should be

displayed If there is no hour glass, then some enquiry in progress message should be displayed All screens should have a Help button (i.e.) F1 key should work the same

If Window has a Minimize Button, click it Window should return to an icon on the bottom

of the screen This icon should correspond to the Original Icon under Program Manager Double Click the Icon to return the Window to its original size The window caption for every application should have the name of the application and the window name -

especially the error messages These should be checked for spelling, English and clarity, especially on the top of the screen Check does the title of the window make sense If thescreen has a Control menu, then use all un-grayed options

Check all text on window for Spelling/Tense and Grammar

Use TAB to move focus around the Window Use SHIFT+TAB to move focus backwards.Tab order should be left to right, and Up to Down within a group box on the screen Allcontrols should get focus - indicated by dotted box, or cursor Tabbing to an entry fieldwith text in it should highlight the entire text in the field The text in the Micro Help lineshould change - Check for spelling, clarity and non-updateable etc If a field is disabled(grayed) then it should not get focus It should not be possible to select them with eitherthe mouse or by using TAB Try this for every grayed control

Never updateable fields should be displayed with black text on a gray background with ablack label All text should be left justified, followed by a colon tight to it In a field thatmay or may not be updateable, the label text and contents changes from black to graydepending on the current status List boxes are always white background with black textwhether they are disabled or not All others are gray

In general, double-clicking is not essential In general, everything can be done using boththe mouse and the keyboard All tab buttons should have a distinct letter

3.1.2 Text Boxes

Move the Mouse Cursor over all Enterable Text Boxes Cursor should change from arrow

to Insert Bar If it doesn't then the text in the box should be gray or non-updateable Refer

to previous page Enter text into Box Try to overflow the text by typing to many characters

- should be stopped Check the field width with capitals W Enter invalid characters - ters in amount fields, try strange characters like + , - * etc in All fields SHIFT and Arrowshould Select Characters Selection should also be possible with mouse Double Clickshould select all text in box

Let-3.1.3 Option (Radio Buttons)

Left and Right arrows should move 'ON' Selection So should Up and Down Select withmouse by clicking

3.1.4 Check Boxes

Clicking with the mouse on the box, or on the text should SET/UNSET the box SPACEshould do the same

3.1.5 Command Buttons

If Command Button leads to another Screen, and if the user can enter or change details

on the other screen then the Text on the button should be followed by three dots All Buttons except for OK and Cancel should have a letter Access to them This is indicated

by a letter underlined in the button text Pressing ALT+Letter should activate the button Make sure there is no duplication Click each button once with the mouse - This should activate Tab to each button - Press SPACE - This should activate

Tab to each button - Press RETURN - This should activate The above are VERY

IMPORTANT, and should be done for EVERY command Button Tab to another type of

control (not a command button) One button on the screen should be default (indicated by

a thick black border) Pressing Return in ANY no command button control should activate

it

If there is a Cancel Button on the screen, then pressing <Esc> should activate it Ifpressing the Command button results in uncorrectable data e.g closing an action step,there should be a message phrased positively with Yes/No answers where Yes results inthe completion of the action

3.1.6 Drop Down List Boxes

Pressing the Arrow should give list of options This List may be scrollable You should not

be able to type text in the box Pressing a letter should bring you to the first item in the listwith that start with that letter Pressing ‘Ctrl - F4’ should open/drop down the list box

Spacing should be compatible with the existing windows spacing (word etc.) Itemsshould be in alphabetical order with the exception of blank/none, which is at the top or thebottom of the list box Drop down with the item selected should be display the list with theselected item on the top Make sure only one space appears, shouldn't have a blank line

3.2 Section 2 - Screen Validation Checklist

3.2.1 Aesthetic Conditions:

1 Is the general screen background the correct color?

2 Are the field prompts the correct color?

3 Are the field backgrounds the correct color?

4 In read-only mode, are the field prompts the correct color?

5 In read-only mode, are the field backgrounds the correct color?

6 Are all the screen prompts specified in the correct screen font?

7 Is the text in all fields specified in the correct screen font?

8 Are all the field prompts aligned perfectly on the screen?

9 Are all the field edit boxes aligned perfectly on the screen?

10 Are all group boxes aligned correctly on the screen?

11 Should the screen be resizable?

12 Should the screen be allowed to minimize?

13 Are all the field prompts spelt correctly?

14 Are all character or alphanumeric fields left justified? This is the default unlessotherwise specified

15 Are all numeric fields right justified? This is the default unless otherwisespecified

16 Is all the micro-help text spelt correctly on this screen?

17 Is all the error message text spelt correctly on this screen?

18 Is all user input captured in UPPER case or lowercase consistently?

19 Where the database requires a value (other than null) then this should bedefaulted into fields The user must either enter an alternative valid value orleave the default value intact

20 Assure that all windows have a consistent look and feel

21 Assure that all dialog boxes have a consistent look and feel

3.2.2 Validation Conditions:

1 Does a failure of validation on every field cause a sensible user error message?

2 Is the user required to fix entries, which have failed validation tests?

3 Have any fields got multiple validation rules and if so are all rules being applied?

4 If the user enters an invalid value and clicks on the OK button (i.e does not TABoff the field) is the invalid entry identified and highlighted correctly with an errormessage?

5 Is validation consistently applied at screen level unless specifically required atfield level?

6 For all numeric fields check whether negative numbers can and should be able to

9 Do all mandatory fields require user input?

10 If any of the database columns don't allow null values then the correspondingscreen fields must be mandatory (If any field, which initially was mandatory, hasbecome optional then check whether null values are allowed in this field.)

3.2.3 Navigation Conditions:

1 Can the screen be accessed correctly from the menu?

2 Can the screen be accessed correctly from the toolbar?

3 Can the screen be accessed correctly by double clicking on a list control on theprevious screen?

4 Can all screens accessible via buttons on this screen be accessed correctly?

5 Can all screens accessible by double clicking on a list control be accessedcorrectly?

6 Is the screen modal? (i.e.) Is the user prevented from accessing other functionswhen this screen is active and is this correct?

7 Can a number of instances of this screen be opened at the same time and is thiscorrect?

3.2.4 Usability Conditions:

1 Are all the dropdowns on this screen sorted correctly? Alphabetic sorting is thedefault unless otherwise specified

2 Is all date entry required in the correct format?

3 Have all pushbuttons on the screen been given appropriate Shortcut keys?

4 Do the Shortcut keys work correctly?

5 Have the menu options that apply to your screen got fast keys associated andshould they have?

6 Does the Tab Order specified on the screen go in sequence from Top Left tobottom right? This is the default unless otherwise specified

7 Are all read-only fields avoided in the TAB sequence?

8 Are all disabled fields avoided in the TAB sequence?

9 Can the cursor be placed in the microhelp text box by clicking on the text boxwith the mouse?

10 Can the cursor be placed in read-only fields by clicking in the field with themouse?

11 Is the cursor positioned in the first input field or control when the screen isopened?

12 Is there a default button specified on the screen?

13 Does the default button work correctly?

14 When an error message occurs does the focus return to the field in error whenthe user cancels it?

15 When the user Alt+Tab's to another application does this have any impact on thescreen upon return to the application?

16 Do all the fields edit boxes indicate the number of characters they will hold bythere length? e.g a 30 character field should be a lot longer

3.2.5 Data Integrity Conditions:

1 Is the data saved when the window is closed by double clicking on the closebox?

2 Check the maximum field lengths to ensure that there are no truncatedcharacters?

3 Where the database requires a value (other than null) then this should bedefaulted into fields The user must either enter an alternative valid value orleave the default value intact

4 Check maximum and minimum field values for numeric fields?

5 If numeric fields accept negative values can these be stored correctly on thedatabase and does it make sense for the field to accept negative numbers?

6 If a set of radio buttons represents a fixed set of values such as A, B and C thenwhat happens if a blank value is retrieved from the database? (In some situationsrows can be created on the database by other functions, which are not screenbased, and thus the required initial values can be incorrect.)

7 If a particular set of data is saved to the database check that each value getssaved fully to the database (i.e.) Beware of truncation (of strings) and rounding

of numeric values

3.2.6 Modes (Editable Read-only) Conditions:

1 Are the screen and field colors adjusted correctly for read-only mode?

2 Should a read-only mode be provided for this screen?

3 Are all fields and controls disabled in read-only mode?

4 Can the screen be accessed from the previous screen/menu/toolbar in read-onlymode?

5 Can all screens available from this screen be accessed in read-only mode?

6 Check that no validation is performed in read-only mode

3.2.7 General Conditions:

1 Assure the existence of the "Help" menu

2 Assure that the proper commands and options are in each menu

3 Assure that all buttons on all tool bars have a corresponding key commands

4 Assure that each menu command has an alternative (hot-key) key sequence,which will invoke it where appropriate

5 In drop down list boxes, ensure that the names are not abbreviations / cut short

6 In drop down list boxes, assure that the list and each entry in the list can beaccessed via appropriate key / hot key combinations

7 Ensure that duplicate hot keys do not exist on each screen

8 Ensure the proper usage of the escape key (which is to undo any changes thathave been made) and generates a caution message "Changes will be lost -Continue yes/no"

9 Assure that the cancel button functions the same as the escape key

10 Assure that the Cancel button operates, as a Close button when changes havebeen made that cannot be undone

11 Assure that only command buttons, which are used by a particular window, or in

a particular dialog box, are present – (i.e) make sure they don't work on thescreen behind the current screen

12 When a command button is used sometimes and not at other times, assures that

it is grayed out when it should not be used

13 Assure that OK and Cancel buttons are grouped separately from other commandbuttons

14 Assure that command button names are not abbreviations

15 Assure that all field labels/names are not technical labels, but rather are namesmeaningful to system users

16 Assure that command buttons are all of similar size and shape, and same font &font size

17 Assure that each command button can be accessed via a hot key combination

18 Assure that command buttons in the same window/dialog box do not haveduplicate hot keys

19 Assure that each window/dialog box has a clearly marked default value(command button, or other object) which is invoked when the Enter key ispressed - and NOT the Cancel or Close button

20 Assure that focus is set to an object/button, which makes sense according to thefunction of the window/dialog box

21 Assure that all option buttons (and radio buttons) names are not abbreviations

22 Assure that option button names are not technical labels, but rather are namesmeaningful to system users

23 If hot keys are used to access option buttons, assure that duplicate hot keys donot exist in the same window/dialog box

24 Assure that option box names are not abbreviations

25 Assure that option boxes, option buttons, and command buttons are logicallygrouped together in clearly demarcated areas "Group Box"

26 Assure that the Tab key sequence, which traverses the screens, does so in alogical way

27 Assure consistency of mouse actions across windows

28 Assure that the color red is not used to highlight active objects (many individualsare red-green color blind)

29 Assure that the user will have control of the desktop with respect to general colorand highlighting (the application should not dictate the desktop backgroundcharacteristics)

30 Assure that the screen/window does not have a cluttered appearance

31 Ctrl + F6 opens next tab within tabbed window

32 Shift + Ctrl + F6 opens previous tab within tabbed window

33 Tabbing will open next tab within tabbed window if on last field of current tab

34 Tabbing will go onto the 'Continue' button if on last field of last tab within tabbedwindow

35 Tabbing will go onto the next editable field in the window

36 Banner style & size & display exact same as existing windows

37 If 8 or less options in a list box, display all options on open of list box - should be

no need to scroll

38 Errors on continue will cause user to be returned to the tab and the focus should

be on the field causing the error (i.e the tab is opened, highlighting the field withthe error on it)

39 Pressing continue while on the first tab of a tabbed window (assuming all fieldsfilled correctly) will not open all the tabs

40 On open of tab focus will be on first editable field

41 All fonts to be the same

42 Alt+F4 will close the tabbed window and return you to main screen or previousscreen (as appropriate), generating "changes will be lost" message if necessary

43 Microhelp text for every enabled field & button

44 Ensure all fields are disabled in read-only mode

45 Progress messages on load of tabbed screens

46 Return operates continue

47 If retrieve on load of tabbed window fails window should not open

3.3 Specific Field Tests

3.3.1 Date Field Checks

1 Assure that leap years are validated correctly & do not causeerrors/miscalculations

2 Assure that month code 00 and 13 are validated correctly & do not cause errors/miscalculations

3 Assure that 00 and 13 are reported as errors

4 Assure that day values 00 and 32 are validated correctly & do not causeerrors/miscalculations

5 Assure that Feb 28, 29, 30 are validated correctly & do not cause errors/miscalculations

6 Assure that Feb 30 is reported as an error

7 Assure that century change is validated correctly & does not cause errors/miscalculations

8 Assure that out of cycle dates are validated correctly & do not causeerrors/miscalculations

3.3.2 Numeric Fields

1 Assure that lowest and highest values are handled correctly

2 Assure that invalid values are logged and reported

3 Assure that valid values are handles by the correct procedure

4 Assure that numeric fields with a blank in position 1 are processed or reported as

7 Assure that division by zero does not occur

8 Include value zero in all calculations

9 Include at least one in-range value

10 Include maximum and minimum range values

11 Include out of range values above the maximum and below the minimum

12 Assure that upper and lower values in ranges are handled correctly

3.3.3 Alpha Field Checks

1 Use blank and non-blank data

2 Include lowest and highest values

3 Include invalid characters & symbols

4 Include valid characters

5 Include data items with first position blank

6 Include data items with last position blank

3.4 Validation Testing - Standard Actions

3.4.1 Examples of Standard Actions - Substitute your specific

Cancel - (i.e abandon changes or additions)

Fill each field - Valid data

Fill each field - Invalid data

Different Check Box / Radio Box combinations

Scroll Lists / Drop Down List Boxes

Note: The following keys are used in some windows applications, and are

Mode

Document / Child window.

Close Application.

F10 Toggle menu bar

Tab Move to next

active/editable

field

previous active/editable field

Move to next open Document

window.

(Adding SHIFT reverses the

movement).

previously used application (Holding down the ALT key displays all open applications).

Alt Puts focus on

4 Regression Testing

4.1 What is regression Testing

 Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes

 Regression testing is a normal part of the program development process Test department coders develop code test scenarios and exercises that will test new units of code after they have been written

 Before a new version of a software product is released, the old test cases are runagainst the new version to make sure that all the old capabilities still work The reason they might not work because changing or adding new code to a program can easily introduce errors into code that is not intended to be changed

 The selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software Also referred to as

verification testing

 Regression testing is initiated after a programmer has attempted to fix a

recognized problem or has added source code to a program that may have

inadvertently introduced errors

 It is a quality control measure to ensure that the newly modified code still

complies with its specified requirements and that unmodified code has not been affected by the maintenance activity

4.2 Test Execution

Test Execution is the heart of the testing process Each time your application changes, you will want to execute the relevant parts of your test plan in order to locate defects and assess quality

4.2.1 Create Test Cycles

During this stage you decide the subset of tests from your test database you want to execute

Usually you do not run all the tests at once At different stages of the quality assurance process, you need to execute different tests in order to address specific goals A related group of tests is called a test cycle, and can include both manual and automated tests Example: You can create a cycle containing basic tests that run on each build of the

application throughout development You can run the cycle each time a new build is

ready, to determine the application's stability before beginning more rigorous testing Example: You can create another set of tests for a particular module in your application This test cycle includes tests that check that module in depth

To decide which test cycles to build, refer to the testing goals you defined at the

beginning of the process Also consider issues such as the current state of the

application and whether new functions have been added or modified

Following are examples of some general categories of test cycles to consider:

 sanity cycle checks the entire system at a basic level (breadth, rather than

depth) to see that it is functional and stable This cycle should include basic-level tests containing mostly positive checks

 normal cycle tests the system a little more in depth than the sanity cycle This

cycle can group medium-level tests, containing both positive and negative

checks

 advanced cycle tests both breadth and depth This cycle can be run when more

time is available for testing The tests in the cycle cover the entire application (breadth), and also test advanced options in the application (depth)

 regression cycle tests maintenance builds The goal of this type of cycle is to

verify that a change to one part of the software did not break the rest of the

application A regression cycle includes sanity-level tests for testing the entire software, as well as in-depth tests for the specific area of the application that wasmodified

4.2.2 Run Test Cycles (Automated & Manual Tests)

Once you have created cycles that cover your testing objectives, you begin executing thetests in the cycle You perform manual tests using the test steps Testing Tools executes automated tests for you A test cycle is complete only when all tests-automatic and

manual-have been run

 With Manual Test Execution you follow the instructions in the test steps of each test You use the application, enter input, compare the application output with theexpected output, and log the results For each test step you assign either pass orfail status

 During Automated Test Execution you create a batch of tests and launch the entire batch at once Testing Tools runs the tests one at a time It then imports results, providing outcome summaries for each test

4.2.3 Analyze Test Results

After every test run one analyze and validate the test results And have to identify all the failed steps in the tests and to determine whether a bug has been detected, or if the

expected result needs to be updated

4.3 Change Request

4.3.1 Initiating a Change Request

A user or developer wants to suggest a modification that would improve an existing

application, notices a problem with an application, or wants to recommend an

enhancement Any major or minor request is considered a problem with an application and will be entered as a change request

4.3.2 Type of Change Request

Bug the application works incorrectly or provides incorrect information (for example, a

letter is allowed to be entered in a number field)

Change a modification of the existing application (for example, sorting the files

alphabetically by the second field rather than numerically by the first field makes them easier to find)

Enhancement new functionality or item added to the application (for example, a new

report, a new field, or a new button)

4.3.3 Priority for the request

Low the application works but this would make the function easier or more user friendly High the application works, but this is necessary to perform a job.

Critical the application does not work, job functions are impaired and there is no work

around This also applies to any Section 508 infraction

phases of the testing process

 Information about bugs must be detailed and organized in order to schedule bug fixes and determine software release dates

Bug Tracking involves two main stages: reporting and tracking

4.4.1 Report Bugs

Once you execute the manual and automated tests in a cycle, you report the bugs (or defects) that you detected The bugs are stored in a database so that you can manage them and analyze the status of your application

When you report a bug, you record all the information necessary to reproduce and fix it You also make sure that the QA and development personnel involved in fixing the bug are notified

4.4.2 Track and Analyze Bugs

The lifecycle of a bug begins when it is reported and ends when it is fixed, verified, and closed

 First you report New bugs to the database, and provide all necessary

information to reproduce, fix, and follow up the bug

 The Quality Assurance manager or Project manager periodically reviews all New bugs and decides which should be fixed These bugs are given the

status Open and are assigned to a member of the development team

 Software developers fix the Open bugs and assign them the status Fixed

 QA personnel test a new build of the application If a bug does not reoccur, it

is Closed If a bug is detected again, it is reopened

Communication is an essential part of bug tracking; all members of the development and quality assurance team must be well informed in order to insure that bugs information is

up to date and that the most important problems are addressed

The number of open or fixed bugs is a good indicator of the quality status of your

application You can use data analysis tools such as re-ports and graphs in interpret bug data

requires unique identifiers for each requirement and product Numbers for products are established in a configuration management (CM) plan

Traceability ensures completeness, that all lower level requirements derive from higher level requirements, and that all higher level requirements are allocated to lower level requirements Traceability is also used in managing change and

provides the basis for test planning

SAMPLE TRACEABILITY MATRIX

A traceability matrix is a report from the requirements database or repository The examples below show traceability between user and system requirements User requirement identifiers begin with "U" and system requirements with "S."

Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected

In addition to traceability matrices, other reports are necessary to manage requirements What goes into each report depends on the information needs of those receiving the report(s) Determine their information needs and document the information that will be associated with the requirements when you set up your requirements database or

repository

5 Phases of Testing

5.1 Introduction

The Primary objective of testing effort is to determine the conformance to requirements specified in the contracted documents The integration of this code with the internal code

is the important objective Goal is to evaluate the system as a whole, not its parts

Techniques can be structural or functional

Techniques can be used in any stage that tests the system as a whole (System

testing ,Acceptance Testing, Unit testing, Installation, etc.)

5.2 Types and Phases of Testing

Software Requirement Specification Requirement Checklist

Functional Specification Functional Checklist

Design Document & Functional Specs Unit Test Case Documents

Design Document & Functional Specs Integration Test Case DocumentsDesign Document & Functional Specs System Test Case Documents

Unit / System / Integration Test Case Documents Regression Test Case DocumentsFunctional Specs, Performance Criteria Performance Test Case DocumentsSoftware Requirement Specification, Unit / System

/ Integration / Regression / Performance Test

Case Documents

User Acceptance Test Case Documents

Coding