A Case Study of Failure Improvement (Part 1)

Part 1, Part 2, Part 3

Created and summarized by Brian Marick.

I am testing the part of the wv Word importing code that imports tables. I started with a modestly complex table in a document I'd recently written. This is a form of what Cem Kaner calls "staked application testing".

Since the document contained proprietary information, I can’t show it. But here’s a sanitized version, which can be downloaded:

(For the rest of this document, if you click on the underlined word in the sentence describing a picture, you'll get the downloadable version.)

When I ran wvHtml (which converts Word documents to HTML documents) on the file, the resulting table looked like this:

	Practitioner Track	Research Track	Toolsmith Track	Staff Track
Introductory	Span all columns - all text is greyed
	Span three columns			Tail of this row
	Span three columns - end section
Domain	One column		One column
	One column		One column
	Span three columns
	Span three columns
	Span three columns
	Span all columns

I originally viewed this table only with Internet Explorer 5, not Netscape Navigator. (Note to the Justice Department: this was a random choice. I mainly use Netscape Navigator 4.5.) I used only one browser because I am primarily interested in testing how the wv library imports Word documents, not how it exports HTML. So I thought I could avoid the whole rat's nest of browser compatibility. As you'll see, I could not. It was especially maddening when trying to prepare this page.

The output has the following problems.

1. There is a narrow gray border around the whole table.
2. The row 1 headings have a border, giving a "box within box" effect. They should not have any borders around them. The column 1 headings suffer from the same box within box effect. Really, the whole table does. (The effect is more pronounced on Internet Explorer 5.0 than Netscape 4.5. It is a consequence of the way HTML models tables, but the effect can be eliminated.)
3. The thicker borders at the top of the Introductory and Domain sections of the table are missing.
4. The grey shading for the cells with text is missing.
5. In Internet Explorer 5 on NT, the borders between cells without text are missing. It doesn't matter whether the border is thin or thick. The borders appear with Netscape 4.5 on NT.

wvHtml also printed one instance of:

wvError: (./pcd.c:76) pcd fc is 40000800, complex is 0, index is 0

Where to start?

There is a whole mass of border problems, a problem with the background color, and a funny error message. Since I’m just learning the wvHtml program (and am an HTML table novice), it seems simpler to start with the background color problem, guessing that all the other problems are interrelated.

I could take one of two approaches. (See Tammana and Faught's "Software Defect Isolation".) In a top-down approach, I would try to progressively simplify the example into the simplest case that fails, using a divide-and-conquer strategy. In a bottom-up approach, I would start with simple examples that I suspect would fail.

For the gray shading, I bet that I can easily come up with a simple example. First, I’ll try a gray shading for a whole table (suspecting this might not reveal a bug). My source looks like this:

We go astray...

Surprisingly, the result is not only not shaded but also appears to contain only a single cell. (The single cell only happens with Internet Explorer, not Netscape - but see below).

If I look at the source, it has the right structure for a 2x2 HTML table. Take a look yourself by clicking on the previous link and viewing the source.

Surprisingly, the wvError message occurs as well, even though I thought it must have something to do with borders. It will turn out that this message always appears.

At this point I could have investigated further by varying the table. But I decided to speculate that the HTML output was incorrect. I edited it down by removing the elaborate paragraph tags. (Notice that I'm proceeding top-down now.) That didn't change the behavior. I removed the <div> tags. Same behavior. Then it occurred to me (rather belatedly) that there was no text in the table cells. I put text there and (voila!) a proper 2x2 table appeared. I now know something about HTML tables that I didn’t before.

Internet Explorer 5.0 and Netscape Navigator 4.5 differ in how they display the table you see just above. IE shows a single cell because there's no text in the four cells. Netscape shows four cells because of the <p> and <div> tags, even though there's no text. In the process of editing down the table, I confused myself even more because I looked at the table with Netscape after I'd edited out the tags, at which point it also showed a single cell. I incorrectly jumped to the conclusion that it had always shown a single cell. When I wrote this page and checked it against both IE and Netscape, I became completely bewildered to see the four-cell table in Netscape. It took me a while to discover how IE and Netscape differ.

Note that I've possibly found the reason for the missing cell borders in the original table. I'll work on that later.

All this makes me want to write down a test design idea. If I decide to spend more time looking at HTML output and browser compatibility, I should make sure to check different placements of empty cells to see how wv translates that from Word to HTML. Does it work around incompatibilities?

A proper test case

I next put text in each of the cells:

I made it simple text because I’m trying to isolate a particular bug, not accidentally stumble over new ones. I now have a simpler case that demonstrates the lack of shading (as well as the box-within-a-box effect):

1	2
3	4

Checking that the supposed cause actually makes the difference

Looks like I’m about ready to report this. One thing to do is to check that the same table without shading works.

It does, though with the same error message as before and the same box-within-box effect:

1	2
3	4

Is the cause specific or general?

A final thought occurs to me. A gray shade is an unusual one. Perhaps a simple shade like white would work better. White, however, is not useful. It’s not that it’s not visibly different than no shading at all (given a white page background). It’s that the source for the unshaded HTML document is specific about the background being white:

<td bgcolor="White" width="50.00%" rowspan="1" colspan="1">

(This leads to another note to myself. What happens when the table should be clear, against a background that’s some other color? I should check interactions of backgrounds.)

So I tried it with Black:

(Note that Word thoughtfully prints the black text in reverse video to make it visible. Should the HTML duplicate the appearance of the Word display or the actuality of the document? Probably the latter, especially since Word's HTML export puts black characters against a black background. But that's another design note.)

The result? The same behavior as with the unshaded document:

1	2
3	4

Indeed, there is absolutely no difference between the HTML for the unshaded and black document.

Where do we stand?

Looks like I’ve gone as far as I should here. I sent Caolan McNamara (the developer) a bug report and moved on to isolate other failures.

Related Testing Craft Pages

• There's more about handling bugs in the index of techniques.
• The approach I'm taking in these pages is kind of dumb. For an explanation of why, and for a better approach, see this case study of adding test support code.

Be the first person to add a comment in the Wiki Forum at page FailureImprovementCaseStudy.
(The Forum is explained in its FrontPage.)

Summarized Discussion

In this spot, the author of this page will occasionally summarize the discussion in the Forum.