Test Reports ppt

Characteristics of an effective defect report  Tells how to reproduce the problem  The report is complete and understandable  Written  Simple... Components of a defect report Descri

Test Reports

Handling Defects

Handling a defect

 Analyze the defect

 Report the defect

 Track the defect

 Retest the defect

 Close the defect

Analyzing a defect

 Is it the test, the tester, or the product?

 Reproducible vs Repeatable

 Can it be isolated?

 Find related problems

 Look for alternate paths to the problem

 Is it worth reporting?

Handling a defect

 Analyze the defect

 Report the defect

 Track the defect

 Retest the defect

 Close the defect

Reporting the defect

 Look for duplicates

 Talk with the developer

 Enter it into the system

 Make sure it will get noticed!

Characteristics of an effective

defect report

 Tells how to reproduce the problem

 The report is complete and understandable

 Written

 Simple

Components of a defect report

 Identifying information

 Description of the problem

 Various status indicators

 Comments

 Supporting information

 Miscellaneous information

 Program or product this is against

 Version or revision of the product

Components of a defect report

Description of the problem

 Title

information that the problem can be understood

acknowledged or explained in the Description

 Description

Components of a defect report

Description of the problem

Contents

 The problem itself

 What did happen

 What should have happened

 The test case used

 Any other helpful information whatsoever

 In particular, any attachments

 Keep it to 3 or 4 sentences if you can

 Avoid vague or confusing terms (“frequently”, “sometimes”)

 Avoid uncommon abbreviations

 Use any standard terminology

 Pay attention to spelling and grammar

Components of a defect report

Status indicators

 Report status

 Severity and priority

 Resolution status

Components of a defect report

Status indicators

Severity and Priority

 Severity: How “bad” is it?

 High critical defect; test cannot continue

 Medium critical defect; test can continue with a

workaround

 Low it’s a problem but we can keep testing

 Really low documentation, help files, spelling

mistakes

Components of a defect report

Components of a defect report

Miscellaneous fields (1)

 Steps to Reproduce

to reproduce the problem

lengthy procedure

repeatable or not reproducible

Components of a defect report

 The test case itself

 A diskette with data or files

 Memory dump

 Trace files, error logs, etc

Handling a defect

 Analyze the defect

 Report the defect

 Track the defect

 Retest the defect

 Close the defect

Tracking a Defect

 Must have a process in place

 Generally done by some sort of Review

Board

 The lead tester should be on that board

 The purpose is to make sure the defect

progresses from being submitted to getting closed

Retesting a Defect

 Retest can have 3 outcomes

Closing a Defect

 Add Tester Notes or Verification Notes

 Make sure that anyone else with an interest

in the problem’s resolution has a chance to test it too

 Close it, or recommend closure

Life cycle of a defect report

 Entered by a tester

 Review Board evaluates and assigns

information

 Assigned programmer fixes the problem

 A build containing the fix becomes available

 The tester retests the problem

Bug Tracking System

The need for a Defect Tracking

System

 Facilitates clear communication about defects

 Allows developers to fix bugs based on

priority

 Allows for analysis and reporting options

 Helps to manage the bugs

 Keeps a history

Items for your Defect Reports

 Identifiers: title, number, submitter, submit date

 Program name and version

 Description of the problem

 The Test Case used

Test Reports

Test Reports

 Your chance to evaluate how testing went

 List what was tested

 List what was not tested, and why

 List still-open defects (some ask for all

defects)

 Show the actual schedule (Test Plan)

 Lets management know the status of that

product, from a testing standpoint, as it went out the door

Your Test Report

 A brief description of what you did, when

 An attached list of all the Test Cases with an indication of which ones passed and which ones failed (and by default, then, which ones didn’t get run)

 A list of all the defects found (a list of Brief

Descriptions)

Examples

 Test Report template

 Test Report example