DEV Community

Jesse Phillips
Jesse Phillips

Posted on • Edited on

A Testing System and Why it is Wrong

GIVEN

WHEN

THEN

  • You will know what it means for QA to understand the system.
  • You will be confused why I choose the article title.

Analogy

Jim is a quality professional, he and Sarah Mathews are expected to deliver a small change to the location of some dinosaur bones. Sarah is a Magrathean working diligently on the earth's surface.

Sarah completes their work and references the work item number in the back office cellar, where the lights don't work. Jim reads the work item and asks Carl to send a telegram back to Sarah asking how they are going to test the location of the bones.

Sarah is able to get a response back quickly, even though they had to start work on moving some mountains, "You can't, just make sure the rest of earth is looking good STOP" with this new found knowledge Jim sets off to Earth.

Jim has been doing this for awhile, he has some favorite lakes he enjoys visiting, he is aware the Pacific Ocean keeps running into the Indian Ocean practically every release. There is even a joke around the world all the QA professionals have, "what goes up, must come down." No one really knows why that joke is used, but boy does it resinate with all these mistakes those developers make.

Jim files a new work item. He found an issue, it seems like a big one so it is likely to "come down." He put together some very detailed reproduction steps.

  • Go to earth
  • place goat in grass

Observed:

  • Goat does not eat grass

Expected:

  • Grass is eaten by goat

Jim and Sarah get to talking about the issue, Sarah starts, "the goats aren't supposed to eat the grass, we have a grass eating field to test the goats."

"I know, but I wanted to make sure the goats would eat grass anywhere." thinking he had the upper hand.

"OK, but we need the grass on the planet to preserve the calculations." Sarah was annoyed that this important detail isn't understood during testing.

At this point Jim is getting heated, he has a valid concern and Sarah has not even attempted the representation steps he gave. "I couldn't even find grass! It was like I was on Mars or something."

This is just getting rediculous, where on earth did he go that makes him feel like it was Mars. Sarah and Jim go to earth, they scower the planet to find where Jim was testing and it was nowhere to be found. Sarah walks Jim back through his steps.

"I took a right at Albuquerque and that is when I notice earth and started testing." This is all Sarah needed to hear from Jim and understand completely. Sarah stood up and walked in a circle, "you started testing on Mars."

"No that is always how I get to earth, someone must have moved it and didn't tell me." Albuquerque was a good friend of Jim's, he always takes his lunch by earth and likes to count the sheep being trained for herding.

"We had implemented the solar system, and this has caused earth and Mars to be moved, Jim couldn't find earth last week when you were testing." Sarah had convinced Jim to close out his work item as working as expected.

Jim didn't understand why he needed to know about the entire solar system, the people of earth are going to be so busy watching goats eat and herding sheep, they aren't going to care about all of these moving parts that drive their day night cycles.

Top comments (1)

Collapse
 
alanmbarr profile image
Alan Barr

The round earth concept is a decent mental model for quality. Not sure I can wrap my head around explaining the runtime aspect but still helpful. Thanks for sharing that.