Hello there! 👋
I'm Lucas, a Brazilian QA engineer, and this is Another QA Content nobody asked for (but i made it anyway 😂). My goal with this blog is to talk about things related to the QA world in my own way, giving my opinion and point of view. If you like it, follow me for more posts like this one!
Today i want to talk about a subject that is covered at the first chapter of the CTFL syllabus. The seven commandments principles of software testing. (This was a Moses pun if you didn't get it). No more puns for now, let's open this ocean of knowlegde!
1️⃣ Testing shows the presence of defects, not their absence
Many QA engineers may have been in a situation like the following before: The product owner or even the client says your job is to guarantee the sprint deliverable is bug free.
Testing is a destructive activity, it's goal is to make the software fail. When a tester write a test case, it is meant to find a bug, not to prove there is none.
So, this principle states that it's impossible to guarantee that a testing object is defect free.
Some points to be aware: Maybe a non-QA person will misunderstood this principle as an excuse for a QA that isn't good at his job. And on the other hand, it can be misused by a QA to justify not catching a bug that should have been found easily. I personally think this principle isn't meant to justify a bug that has got into production, i think it's meant to avoid creating false expectation on the product and the QA job.
2️⃣ Exhaustive testing is impossible
As a QA, you will probably never have the time to test every possible scenario of a software.
Imagine a small form with 3 select fields, every field has 10 options. To test every possible scenario, it would take you 10³ = 1000 test cases.
We can use a black box testing technique called equivalent partitioning. Briefly explaining, it consists of dividing test data into partitions in such a way that all the members of a given partition are expected to be processed in the same way.
There can even be a situation in wich time is a concern. In this case it's more recomended to perform a risk analysis, and prioritize tests based on the higher risks for the business.
3️⃣ Early testing saves time and money
This principle is probably the parent of the trending shift-left testing approach. It states that the testing activities should start as soon as possible in a project.
Testing activities are not limited to execute or automate E2E tests when a feature is ready. A tester can also review documentation and requisits, to prevent errors and mistakes from advancing to later stages of the software development life cycle. Remember, the later you find a problem, the more it will cost to fix.
4️⃣ Defects cluster together
This principle is an application of the Pareto Principle. Maybe you don't know it by its name, but you might have heard about it somewhere. Here is a brief about it:
For many outcomes, roughly 80% of consequences come from 20% of causes
In software testing, there's a high probability that 80% of the problems can be found at 20% of the modules/components. So be aware when you find a bug, for there can be more, hiding on the same feature.
5️⃣ Beware of the pesticide paradox
To explain this one i'll borrow a paragraph from a Guru99 article about the 7 principles of testing (you can find the link bellow in the references section).
Repetitive use of the same pesticide mix to eradicate insects during farming will over time lead to the insects developing resistance to the pesticide.
So, if you repeat the same test case again and again, chances are it will become less effective (consider an effective test case one that has a higher probability of finding bugs). Because of that, as a QA you need to review your testing techniques and test cases frequently. In an Agile project where requisits change very often, this is even more crucial.
6️⃣ Testing is context dependent
This principle talks about how different contexts may influence the test in different ways.
Testing in an agile project is done differently than it's done in waterfall, due to the differences in release periodicity.
Testing an e-commerce is done differently than testing an internet banking system, due to the different risks involved.
The correct tool for automating regression tests depends heavily on the context too. As you get more experienced, the more you get to understand the context you're incerted, and plan your QA job to fit it well.
7️⃣ Absence-of-errors is a falacy
Like we have seen in principle number 2, testing shows the presence of defects and not their absence. What this principle states is that no matter how much effort you or your team put on testing activities, the tests alone can never guarantee an error-free software.
A software isn't considered to have an error only when something doesn't work as expected. A system that is overcomplicated for the user (bad usability) or doesn't fullfil the user needs (bad modeling) is also considered to have errors.
💰Bonus
To test your understanding of the seven testing principles presented above, try to solve this question from the CTFL 2018 sample exam:
🤔(Sample Exam C, Question 4): A product owner says that your role as a tester on an Agile team is to catch all the bugs before the end of each iteration. Which of the following is a testing principle that could be used to respond to this (false) statement?
a) Defect clustering
b) Testing shows the presence of defects
c) Absence of error fallacy
d) Root cause analysis
The answer is below.
If you've read this far, i hope the content has added something for you. If it didn't, remember: This is just another ordinary QA content 😬.
🍀 See ya!
❗ Bonus question answer
b is correct. Testing can show the presence of defects but cannot prove their absence, which makes it impossible to know if you have caught all the bugs. Further, the impossibility of exhaustive testing makes it impossible for you to catch all the bugs
🔍 References
CTFL Syllabus 2018
Guru99
Geek for Geeks
Top comments (3)
A few more things I'd add to the pile.
QA testing takes a skillset. A QE (quality engineer) that does QA testing has developed the skillset. As a developer, I do not tell a QE how to do their job... I can't do their job, I don't have the skillset. Could developers learn the skillset? Sure. I can also learn to be a medical doctor. Just takes time and an investment in learning.
Management that thinks it will save money by reducing QE staffing only serves to increase the number of bugs that escape into the wild. Those bugs only to be discovered by end-users.
TDD-style unit tests are primarily for development purposes, not testing purposes. As an artifact, they are useful to run to ensure basic correctness, but that's a tertiary benefit. (The secondary benefit is enabling the ability to fearlessly refactor.) TDD-style unit tests are written by developers, not by QE.
Thanks for posting this. It's often all too easy to forget that QA's job is really never done, and that just because you don't see any errors in your testing, that does not mean there aren't any. I constantly bring up these points myself, yet sometimes people find it hard to understand.
Great article, enlighten me as i just started my role as SDET.