Testing Computer Software phần 5 pptx

107

• You will be asked to retract every query because these are not error reports. Don't expect to get

answers to the queries either.

• You will be asked to retract all design suggestions and most design issues. After all, if the

program's behavior matches a reviewed specification, it would hardly be fair to count it as a bug.

Our impression is that, over a few releases, perhaps 15% of the design changes suggested by testers

are implemented. In practice, this contributes strongly to the polish and usability of the program. Do

you really want to lose this information from your database?

• Plan to spend days arguing whether reports point to true bugs or just to design errors. This is

especially likely if you try to keep design issues in the database by agreeing to count only coding

errors in the employee performance monitoring statistics. If you're already sick of arguing with

people who say "but it's supposed to crash," just wait until their raise depends on whether you class

reports as coding errors or design issues.

• Expect your staff to be criticized every time they report a "bug" that turns out to be a user error.

• Expect to be asked to retract every irreproducible Problem Report It shouldn't count against the

programmer if the problem is truly irreproducible. There are lots of non-programming-errorreasons

for these problems (user error, power wobbles, hardware wobbles, etc.). If the programmer does

track down the coding error underlying an "irreproducible" problem, this report now counts against

his statistics. If he can convince you that it's irreproducible, it won't count against his statistics.

How hard should he look for coding errors underlying these reports?

• Don't expect any programmer or project manager to report any bugs they find in any product

under development.

• And someday, you 'II be sued. Many people who are fired or who quit under pressure

sue their former employer for wrongful dismissal. If you're the test manager, and

your database provided performance monitoring that contributed to the departure of

an employee who sues the company, you may be sued along with the company. This

tactic lets the lawyer ask you more questions before trial more easily than if you're

just a witness. Sound like fun? Who's going to pay your legal bills? Think before you

say, "The Company." Probably they'll be glad to let you use the company's lawyer,

but if you and the company are both defendants in the same trial, and the company's

lawyer sees a way to help the company that hurts you, what do you think will

happen? Maybe it depends on the company and the lawyer.

The objective of the database is to get bugs fixed, not to generate nice

management statistics.

LAWYERS

Everything in the problem tracking database is open to investigation in any relevant lawsuit by or against

your company (also see Chapter 14):

• Problem Reports that include tester comments raging against programmer-improfessionalism can

be very damaging evidence, even if the comments are entirely unjustified.

108

• The company might gain credibility if the database gives evidence of thorough testing and

thorough, customer-sensitive consideration of each problem.

• It is illegal to erase Problem Reports from the database in order to prevent them from being used as

evidence.

MECHANICS OF THE DATABASE

At some point you get to design your own system or to suggest extensive revisions to someone else's. From

here, we'll assume that the design is yours to change. These are our implementation suggestions for a

problem tracking system. Many other systems will satisfy your needs just as well, but variants on this one

have worked well for us.

REPORTING NEW PROBLEMS

The Problem Report (Figure 5.1) is the standard form for reporting bugs. Chapter 5 describes it in detail.

We recommend that anyone in the company can file a Problem Report. Your group allows some people

to enter problems into the computer directly. Others write reports on paper (as in Figure 1.1), which you enter

into the computer.

109

The system checks some aspects of the report as it's entered. It does not accept reports that it classifies as

incomplete or incorrect If someone doesn't know how to fill in all the required fields, ask her to report the

problem on paper. The Testing Group (you) will replicate the problem, flesh out the report, and enter it into

the computer.

On a single-user system, and in some multi-user systems, when you enter a new Problem

Report, the computer prints at least 3 copies of it. One goes to the person who reported the

problem. The second goes to the programmer, perhaps via his manager. The third copy is the

Testing Group's file copy. (If your disk ever crashes, you'll be glad you kept a copy of each

report on paper. Your paper files don't have to be elaborate, but they must include each

Problem Report.)

WEEKLY STATUS REPORTS

At the end of each week, issue status reports. Be consistent: circulate the reports to the same people, week in,

week out.

The Weekly Summary of New Problem Reports tells everyone on the project what new problems were

found this week. Figure 6.1 shows tbe new problems sorted by FUNCTIONAL AREA . Figure 6.2 shows the

same problem sorted by SEVERITY . Some project managers have strong preferences for one order over the

other. Be flexible.

The Weekly Status Report (Figure 6.3) shows the state of the project, and how this has changed since last

week. These is a popular and useful report, but don't present the numbers without careful commentary

explaining unusual jumps in the counts.