Testing Computer Software phần 4 pot

78

Our strongest recommendation for improving legibility is to use a computerized problem tracking system

(see Chapter 6). Make the computer type your reports.

NON-JUDGMENTAL

Nobody likes being told that what they did was wrong, wrong, wrong. As a tester, that's what you tell people

every day. You can ensure your unpopularity by describing problems in a way that tells the programmer you

think she is sloppy, stupid, or unprofessional. Even if you think she is, keep it out of the report. If the

programmer considers you a jerk and your reports vindictive, she will want to ignore your reports and

complain about you to her management.

Complaints about maliciously written Problem Reports can have serious consequences. First, they

reduce your chances of raises and promotions, and may cost you your job. Some testers think their

"straight" (nasty) reporting style is more courageous than foolish. But malice leads to a justifiable

movement to censor Problem Reports. Because of censorship, only some reports reach the programmers,

and censors don't just reject inappropriate wording. They also suppress reports of problems they consider

too minor or that they decide will have political repercussions they don't care to face. Once censorship

starts, some testers will stop reporting some classes of problems because they "know" that these reports

will never make it past review anyway. Under these conditions, many fixable problems are never reported

and never fixed.

Think twice, and twice again, before declaring war on programmers by expressing personal judgments in

your reports. You will almost certainly lose that war. Even if you keep your job, you will create an adversarial

relationship that will cost you reporting freedom. It will not improve product quality even if every judgment

you express is correct.

We are not saying never express a judgment. Occasionally, you may have to write a powerful, bluntly

worded report to alert management to a serious problem that a programmer will not acknowledge or fix. Fine.

Use your most effective tactics. But choose your battles carefully. Don't do this more than twice per year. If

you feel that you have to engage in more mudslinging than that, circulate your resume. Either the company

has no standards or your unhappiness in your environment is expressing itself in a very unhealthy way.

ANALYSIS OF A REPRODUCIBLE BUG

The rest of this chapter concentrates on reporting of coding errors rather than design issues. In this section,

and the next, we assume that each bug is reproducible. We explain tactics for reproducing non-reproducible

bugs shortly, in the section, "Making a Bug Reproducible."

79

Reproducibility implies the following:

• You can describe how to get the program into a known state. Anyone familiar with the program can

follow your description and get the program into that state.

• From that state, you can specify an exact series of steps that expose the problem.

To make your report more effective you should analyze it further. If the problem is complicated either

because it takes many steps to recreate or because the consequences are hard to describe, spend time with it.

Simplify the report or break it into a series of many reports. The objectives of your analysis are:

• Find the most serious consequences of the problem.

• Find the simplest, shortest, and most general conditions that will trigger the bug.

• Find alternate paths to the same problem.

• Find related problems.

FINDING THE MOST SERIOUS CONSEQUENCES

Look for the most serious consequences of a bug in order to boost everyone's interest in fixing it. A problem

that looks minor will more often be deferred.

For example, suppose a bug displays a little garbage text in a corneT of the screen. This

is minor but reportable. It will probably be fixed, but against a deadline, this bug would not

stop shipment of the program. Sometimes, onscreen garbage is merely an isolated problem

(and the decision to leave it alone might be wise, especially just before release). Often

though, it is the first symptom of a more serious underlying problem. If you keep working

with the program, you might discover that it crashes almost immediately after displaying

the garbage. This is the consequence you're looking for; it will get the screen garbage

fixed.

When a program fails, it either:

• falls into a state the programmer didn't expect, or

• falls into error recovery routines.

If the state is unexpected, subsequent code makes incorrect assumptions about what has happened. Further

errors are Likely. As to error recovery routines, these are often the least tested parts of the program. They often

have errors and are poorly designed. Typically, error routines contain more serious bugs than the one that led

there.

When the program logs an error, displays garbage onscreen, or does anything else that the programmer

didn't intend, always look for a follow-up bug.

FINDING THE SIMPLEST AND MOST GENERAL CONDITIONS

Some bugs show up at midnight every leap year, but never appear any other time. Some bugs won't show up

unless you make a complex series of erroneous or unlikely responses. Bug fixing involves tradeoffs:

• If it takes minimal effort to understand and fix a problem, someone will fix it.