Testing Computer Software phần 6 ppt

140

RNG functions on large systems (Knuth, 1981) and microcomputers are often poor. The worst don't even

use the basic algorithm correctly; they use floating point routines to do what should be strictly integer

arithmetic. Others only work with numbers between 0 and 65,535 and repeat their sequence every 65,536th

number. These are unacceptable.

We can't go into the subtleties of creating and testing RNGs in this book. Kaner & Vokey (1984) overview

the problems and testing techniques. Knuth (1981) is the authoritative text. Here are some suggestions:

• Read up on random number generators before you do any testing using random inputs. Don't trust

someone else's generator just because it's there, even if it's provided as part of a respectable

language on a respectable machine.

Keep reading about generators until you understand the suggestions that follow. You don't have to

take the suggestions, but if you don't understand them, you know little enough that you risk wasting

lots of time generating oops-they-weren't-so-random-test cases.

• If you're going to use a generator supplied with your

programming language, sample many (100-tOGQ),

numbers from it and shuffle them. That is, use fur

ther random numbers to reorder them. This is slow,

but it brings many poor RNGs up to a level of

acceptability.

• If you use a language that allows high precision

integer (not floating point) arithmetic, consider

writing your own function to use one of the follow

ing generators. Define the RNG by:

R[N+1] = (A » R[N] + C) modulo M

That is, generate the N+1 st number by multiplying

the Nth by A, adding C and taking the result modulo

M. The larger M is, the better, but slower. Figure

7.3 lists good values for the parameters.

The value of C is not critical (as long as it's odd), but

careful selection can reduce serial correlations. The

values for M = 240 are from Kaner & Vokey (1984).

The rest are from Knuth (1981). To give you an idea

of the care that goes into selecting these parameters,

Kaner and Vokey tested over 30,000 candidate val￾ues for A, and perhaps a hundred values for C.

141

GENERALIZED EQUIVALENCE TESTING

Mathematical functions are no longer the only reference functions available. You can use output from other

products to test quite a few aspects of your product's behavior. For example:

• If your program licenses the same underlying spell-checking code as another program, run the same

word list through both programs.

• To test the adequacy of your hyphenation algorithm (especially a variant of the algorithm that

you're trying to apply to another language), test against a respected program sold in your target

market (e.g., Germany). Create narrow columns of text to force hyphenation, and feed both

programs the same word lists.

• Test your program's inter-character and inter-line spacing by laying out the same text, with the

same fonts, in your desktop publisher and in a competitor's.

• Check the control codes you seiKteo a printer (redirect output to a file) against codes from another

program that is printing an identically formatted document.

If you can capture output from another program, you can test yours against it. It might take more work than

it's worth to set these tests up, and you always run the risk that the other program has bugs, but keep the option

in mind.

Remember to include comparison output from the other program with your bug reports. This is the first

step in reverse engineering the other program, and it might be enough in itself to tell the programmer how

the bug should be fixed. ^ \ .

REGRESSION TESTING: CHECKING WHETHER A BUG FIX WORKED

When you report a problem, you tell the programmer exactly what you did to find it. Some programmers

examine the code thoroughly, find the cause of the problem, fix it, and test the fix. Some address only the

symptoms you reported. They write special-case "fixes" which don't solve the underlying problem but do

keep it from appearing under precisely the circumstances you reported. Some misunderstand your report,

and find, fix, and test the wrong problem. A few change code blindly, don't check their work, and give back

code with the same bug plus whatever they broke making their "fix." There is a continuum of

thoroughness and you have to be ready for it.

It's often claimed that one third of the "fixes" either don't fix the problem or break

something else in the process. Martin & McClure (1983) summarized data showing that

fewer than half of the fixes work the first time they are tested by the programmer (if he tests

them).

When you test a bug fix, you have three objectives:

• Check that the bug was actually addressed. Run exactly the same test that you ran when you found

the problem, the one you described in your report. If the program fails this test, you don't have to

do any further regression testing. On the other hand, if the program passes this test, take a second

to ask whether you're running the right test. Are you sure that you know how to demonstrate the

bug? If you have any doubt, load the old version of the program, the one you know has the bug in