| testing.com > Testing Craft > Techniques (Test Design Tips) > Try the Next Thing |
Summary
A command to a program may have immediate effects but also enable, disable, or modify later commands. It's important to both check the immediate effect and also try later commands.
Question: Which commands? And how?
Answer: The ones that a user would likely perform, in the way a user would perform them.
I was once testing a product with password protection. You could change the password. That dialog looked something like this:
You have to give the new password twice because everything you type is echoed as asterisks. Without the confirmation, you might easily change the password to something you didn't intend.
One of the things I tested was whether the password change was rejected if the first new password you typed was different than the second new password. The result was a popup dialog telling me to try again.
The program seemed to work. However, a tester should always ask, "What's the next thing a user would do?" Well, if a user is trying to change a password, and makes a typo, she's going to try again. Most likely, the next time she won't make a typo.
So I tried that. Oddly, I got the same "try again" popup. I typed the passwords again, this time more carefully. Again the popup. It turned out that my first attempt actually had changed the password, even though the popup said it hadn't. Because the original mistyped new password had now been stored as the old password, the program thought I was mistyping the old password from then on.
A real user probably would have given up in confusion after a few tries. Then, the next time she tried to log on, she wouldn't be able to. Her password had been changed!
This is not an easy bug to work around. What is customer service to do? Say "try all the likely ways you might have mistyped your new password the last time you tried to change it?"
As a related example, here's a bug in prerelease version 0.7.6 of the AbiWord word processor. (Thanks to Robert Sievers for sharing this with me.) As you would expect from a word processor, there's a paragraph dialog:
Notice the two fields in the middle right. I've changed the line spacing in the paragraph from single-spaced to triple-spaced. That seems to work:
But does it really work? Many programs manipulate internal data. AbiWord does two things: it manipulates a big document data structure, and it displays that structure in a pretty form on the screen. By looking at the screen, we're only really checking one of those things. Check all of the effects of an action. You might have no way to examine an internal data structure. If so, you need to check it indirectly, by seeing whether the further actions that first action is supposed to enable actually work. Often, the only way to narrow the possible next actions down to a manageable set is to ask, "What are the next things a user might do?"
Well, a user might tinker further with the line spacing to get it just right. Suppose she decided to widen the spacing a bit by changing it to 3.5 lines. She would discover that the spacing didn't change at all. When she reopened the paragraph dialog, she would see it had reverted to 3.0 lines. It's "stuck".
Testing oblivious of the user's next action might find both of these bugs. The password bug would be found if you happened to test successful attempts to change the password after failing attempts (rather than the reverse order). You might stumble over the spacing bug if you were testing various values in the spacing pulldown menus. But trying the user's next thing increases the chance of finding the bugs that users would otherwise find.
Be the first person to add a comment in the
Wiki Forum at page TryTheNextThing.
(The Forum is explained in its FrontPage.)
In this spot, the author of this page will occasionally summarize the discussion in the Forum.