"A Bug's Life"

July 9, 1998
by Alex Zavatone

The understanding of what constitutes a bug in a piece of software is as complex as the path that it takes towards getting it fixed. A lot of times, what appears to be a bug in the program is "pilot error", or to put it in other terms "the user goofed". As software is written upon libraries of existing code, the bug really may exist in the operating system code or just be a limitation of the system. In any case, in the development of software, bugs are part of the process and the QA team that ferrets them out are the last line of defense between the customers and the nefarious bugs themselves.

At this time, let us have a moment of silence and give thanks to those who toil to make our software bug free. Bless you all.

But before I get carried away, to understand the Zen of a bug, the development processes of software companies should be explained. It is the way that software is built that will determine when and how bugs can be found and addressed.

The Developer's Tale

(commonly omitted from the collections of Chaucer's Canterbury Tales)

There are several ways which software can be developed. At one end of the spectrum is someone who can code sitting down and writing off the top of his or her head (DANGER DANGER). The more involved approach starts with complete product specifications which come from product development meetings based on product requirements, followed by plans of implementation, developmental, alpha and beta phases. It gets complicated to say the least. Now, I've worked at 3 major software companies and several smaller ones and in large part, product development goes like this:

1) A buncha people get together to figure out what the product should do based on a list of desired functionality and requirements. Some of the requirements may even come from surveys of people who will use the product. It is wise to not only have folks from development and marketing in these meetings but also the stars from QA and Tech and Docs for they will have to test, support and document that which comes from these sacred gatherings.

2) From these often endless and incredibly thrilling meetings come the requirements for a "product specification" which the Docs team puts in order. This is the defacto, red-blooded standard for what the product is supposed to do. Often it is considered a "living document" in the hopes that when the desired product functionality changes that the spec will be updated to reflect that. This even happens on occasion.

3) The developer engineers, who often have mystical names such as Yachuk, Calaprice, Balchandani and ever more mundane ones such as Thompson, Edmunds, Jennings and of all things, Smith, form what is called an "engineering coven" where they set about the impossible task of figuring out how long the process will take. They are almost always wrong, probably because eye of newt was in short supply this season, and this means that later on in the product cycle, there will be a pestilence called "Crunch Period". This crunch period has, at times, lasted for up to 1/2 of the life of the project. Locusts often appear at these times.

4) Once the Development Schedule Parchment has been bought off on by project management, the meetings coalesce to determine the Testing or QA Schedule. QA actually does not stand for Questions and Answers but Quality Assurance. This elusive type of engineer actually works their way up from checking underwear on production lines. I was inspector number 12.

5) After a small lamb is sacrificed and the project manager, Dev and QA leads buy off on the unified development and QA schedules, Documentation actually heads off to the rafters to craft their own schedule and then we never see them again till it is time to, as they put it, "proofread the docs." All but the stoutest engineers flee for the hills, disregarding the safety of women and children at these times.

6) Upon the mention of the entity known as the "product manager", thou shalt fear them for they art to be feared. Their job is to make our friendly, cheerful and often cherubic engineers suffer undue pain and emotional distress. (OK, some of the pain is probably due.) The tools of suffering include emergency projects, shortening of schedules, tightening of budgets, cutting of features and the dreaded holding of meetings. But nothing can prepare you for 6 months of "You're coming in this weekend right?"

7) Our beloved developers start to develop. Angels sing.

8) QA, either still on their last project or in hibernation start to thaw for the upcoming task at hand.

9) Bells ring out the "making of the first milestone." Glorious proclamations declare that "Testing May Commence!" Development has sanctified that the embrionic product has reached a point where it is not a waste of effort to start the "Finding of the bugs". At this point, and for the rest of the project cycle, portions of the project are so unstable that they are tagged: "we know it's broken don't even think about looking for bugs here." As the product matures, more areas become open for testing and even areas that merge with other areas become virtual testing fodder. QA becomes fat and happy in these times of plenty.

Time passes.

10) Time passes and the code reaches greater levels of completion and maybe even stability. Bugs are entered into the Database that is "the holiest repository of bugs." Meetings are held. Meetings are held. Code is built and releases are made. Releases not only take on the names of "Alpha candidate" (aka stable enough and sorta complete), stability release and Beta candidate (which means feature complete and possibly stable in the dreams of those who don't know better). Normally Beta releases are those that are thrown from the high towers to the select and sinister few below so that they may take them back and inflict these incomplete creations on their unsuspecting processors. Hence the phrase "Whatever doesn't kill you makes you stronger." It was coined by a poor tired 486 that had been used as a beta test machine once too often. Think of that poor machine the next time you double click.

11) "Yea, though we walk through the shadow of Death, we shall fear no evil" becomes the most heard phrase at the water cooler as Crunch mode starts. It is also spoken before each and every bug review meeting. Why? Because suddenly, it is realized that there is not enough time to finish the project! The schedule can not slip, there are too many bugs, your department solely supports Waiters On Wheels, entire days are set aside for product meetings. Your department has hired its own massage therapist. Hell has frozen over and you are wearing bermuda shorts and flip flops. Here we learn the hideous words known as bug deferrals, feature cuts, and comp time. Dev engineers often leave in the pre-dawn hours as some of the QA enter the building under the cover of darkness. Time has no meaning -- especially in meetings. Days drag on, bugs become "deferred till next release" (forever), product features disappear, "monitor tan" becomes a more than a fashion statement.

12) HALLELULA! Golden Master Candidate! The US version of the software has been deemed Kosher and Olestra free. Software may be sent to that far off place in a far off land called Duplication. Engineers' names are spoken in reverence. Project managers are temporarily forgiven. Your mission is complete Mr. Bond. Now take one short breath and start localizing...

Piece of cake huh? Thought you'd think so. But now that you know how it's done, you may understand the demanding requirements placed on each and every little bitty bug. Which is why it's so important that you play your part in this little life & death struggle to squash them.

Eeek! A Bug!

"Oops, it didn't do what I wanted it to do."

"Well, what did you want it to do?"

These are the two things you need to know when you think you have found a bug. Next question is "Wait a minute, is it really supposed to do that?" And most of the time it is. Ya blew it buster. But just recheck your code, check the state of your sprites, "you puppeted them didn't you?" Check your variables and make sure that what you think you told Director to do is what you really told it to do. And RTFM. Always RTFM. Most of the time, it's pilot error. But when it isn't here's what you gotta do if you wanna play with the big kids.

QA montra: Simple case... Simple case... Simple case. A bug is so much easier to fix if it can be reproduced through a simple case. Would you want to look for the problem in that 46 meg file or in a 200 k one? And you'd be surprised how many times you'll discover your own error when you're isolating the problem into a simple case.

Once you've isolated the bug into a simple case, here's what they need to fix it:

Bug description. Provide a clear, concise description of the bug. Bad example: "It broke," or "Value doesn't do what I want it to". Good example: "The value() function does not return a valid list when parsing multi line strings created from a valid list."
Steps to reproduce. (Get your mind out of the gutter!) Once you have the description, you need very simple steps to reproduce the bug. Here, you know what you did to find the bug but the developer does not. Neatness counts here and so does clarity in your explanation. If you don't do a good job here, the bug will get bounced back to you after the developer spends precious time on it and you'll have to narrow it down yet again, filling your day with even more fun and cutting into your Quake time. Not good. Not good at all. Here I find that if you can't explain it to a 5 year old, your bug is not clear enough. 5 year olds are a tough audience.
Produce a teeny little file that shows the bug. A picture is worth a thousand words and a dir file is like a Tom Clancy novel.

Submit the Bug

So now that you've got your Description, Steps to Repro and maybe even a happy little sample file, you've got to check it on the "other platform", check it in a projector, check if any funkiness like colordepth may cause it to happen, check here, check there, get all your machine vital data, your take on how nasty the bug is, the build or release number of the current version you are testing and then shovel all that info into a new record of the bug database. Can you imagine how difficult this must have been before computers?

Now your happy little bug is entered into the database, but lonely. She is not assigned to a developer to evaluate, she has no "fix priority", she is homeless but not without hope. Daily, meetings commence where managers of all flavors meet to grade, herd, review and dissolve these shall we say "unassigned" bugs. They are checked to see if they are duplicates of previous bugs, they are checked to see if the QA engineer was on controlled substances (DUH!) they are checked for clarity. And then each and every bug is given its fix priority and its own engineer by "the Lord of the Dance", the QA Lead. This is of course done in a bug database.

Every new -- and might I mention proud -- bug sits in waiting for the engineer to click on their database button which lists all the bugs that are now "open in Development" for that engineer. No choice now for the developer to review each bug one by one to see if it's a 'real' bug, if it's a duplicate of something else, if it's fixed already since it was reported, if it's totally useless, if it's so obscure that no one on earth will find it, if the report is even understandable and so on. If the bug was so lucky to pass this gauntlet, the engineer must be able to reproduce the bug. If they can't then the waste of time already put into that bug is big. Think of that the next time you make a report that says "Something's broken". In fact if the bug can't be reproduced because the bug report is unclear or not complete, the waste of time rises because the bug must go back to the QA engineer, get straightened out and then go back to dev engineer for yet another pass. Hence the phrase: "Zav, could you come here a second. Where is he? Buzz, did you forget to fasten the leash?"

When the developer is finally able to reproduce the bug, then the fun begins. God (pick one) willing, he/she/it is able to fix the bug. Sometimes, there are more pressing bugs and your special bug may have to wait till after a "milestone", or if not a major issue, may get deferred, never to be addressed again... but I digress. Once our skillful developer has used his carefully honed skills to implement a fix, the potential fix waits till the next release is built and the code passes basic sanity checks.

Now you know what goes on in each any every little house of horrors that calls itself a software development company. So, a quick review: Narrow it down to a simple case. Explain it clearly. Include coherent steps to reproduce. Sample files? OF COURSE! Smoke'em if ya got em.

Why? For the bugs you find in your daily work will not get fixed till YOU report them in a manner that will get them looked at inside the hallowed halls of 600 Townsend St. SF CA 94103. An unreported bug will come back to haunt you. A reported bug will rest in peace.

Your mission: Find them. Report them. Use the Online Director Bug database.It's the least you can do.

A Director user since 1987, Alex (Zav) Zavatone has worked on the engineering teams for both Director and Shockwave, 4, 5, 6 and 7. Recent investigations find him developing foundation classes for Director with asynchronous process management and other life threatening activities. In its early days, he had something to do with this Internet thing known as "DOUG". A noted ne'erdowell, slacker and laggard, Mr. Zavatone is nonetheless is trusted by children, small animals and the elderly. In his spare time, Mr. Zavatone rehabilitates lame desert trout.