The title "Cargo Cult Quality" is inspired by Professor Richard P. Feynman's Caltech commencement address in 1974 titled "Cargo Cult Science"1.
Even if my younger self didn't know about software testing, I always wanted to be involved with quality and software engineering.
During my university years, I was fortunate to do a thesis on testing an automation testing tool - my answer to the chicken or the egg question for testing tools. That was my first encounter with software testing, and thanks to a background in metrology and instrumentation, I was already familiar with concepts like quality and rigorous processes.
Software testing then seemed like the ideal fit as the solution to quality for software, or so I assumed.
I know now that it may be about software testing, but quality is more than that.
In the past 20 years, I have been lucky enough to work in many countries with great people from every continent and various cultures. I have been and am still learning, sharing, discussing, and experimenting with approaches to quality for software products - or digital user experiences. And, I came to see that many organisations suffer from the same shortcomings or false beliefs about quality.
Here are the most common false beliefs I have encountered and my take on those.
Teabag
It is maybe the most prevalent belief I have faced in my career.
I happened to be some companies' first tester - or whatever job title was popular then. Those companies have been releasing products for years successfully. Yet, they were suddenly in need of a tester. Why so?
Similarly, I have been hired as a consultant to help companies with products on the market for years to improve testing practices. Why so?
The traditional response from those companies was that they needed to improve the quality of their existing or future items. However, while I was still in my early twenties and short of experience, I was just given the simple instruction: test the product to make it better (or, as it's often shortened, find the bugs).
To my amazement, we did not necessarily fix more bugs, and neither did the development team work differently from before. But, the need to demonstrate quality was such that it could impact the timeline. The testing was becoming a bottleneck. Why so?
That is where the teabag belief comes into play. Too many managers believe adding one or more testers to an organisation will enhance quality. Some see a good tester as a Lipton tea bag in a cup of water, infusing quality when dipped into the company - with or without milk and sugar.
Testing or quality specialists are not magicians, but they can help organisations see their products and processes from different angles. As observers of quality culture, their concerns and recommendations are worth listening to.
One size fits all
The second false belief is that there is a single definition of quality, a single approach to quality.
Let me start by sharing two anecdotes that opened my eyes to the various definitions and perceptions of quality.
The first happened in France while working for a major telecommunication company. I was in charge of accepting the software developed by a third party.
After rejecting several deliveries in a row, with a colleague, we requested to have a demonstration by the provider before delivering. After all, we assumed that they were not delivering broken software on purpose. The demonstration included a presentation of the quality process. Their testing strategy had that previously successful tests were still valid and therefore were not rerun but directly reported as successful.
From their point of view, the quality was about fixes and new features; no need to worry about existing features. They were also testing in a different setup from our target. Once this was clarified and they changed their approach, we got better and more regular delivery fitting our quality expectations.
The second anecdote happened during my time in India as an offshore automation testing team lead.
My team was struggling to deliver the level of quality I was expecting. The automated tests ran most of the time successfully, then broke all of a sudden. But still, the team seemed to do its best and did not see any real problem. After all, flaky tests can be blamed on test data and the environment.
It was then when my 1-year-old AC broke in my apartment - New Delhi without AC can be sizzling - and to my surprise, my colleagues praised the AC since it was successfully fixed - by a crafty handyman. I then realised that I'd been out of alignment with my teammates about how each other perceived quality. I was expecting the tests to be robust and maintainable, and they were mainly worried about having tests that work at least once and are fixable.
After aligning our definition of the quality, we improved the robustness of our tests significantly.
It is essential to be aware that the definition and perception, or expectations, of the quality differ from one organisation to another, from one industry to another, and more importantly, from one culture to another. To establish a shared definition, we must first understand what quality means to each stakeholder.
Cargo cult
From my experience, it is the most common software engineering pitfall. It can be considered as an extension of the two prior ones.
Cargo cult2 is from Western anthropology and refers to religious practices in a developing society in which followers attempt magical rituals to bring modern goods from a more technologically advanced culture.
In several companies, I have encountered a similar belief where an organisation decides to go through a transformation, hoping it will lead the organisation to deliver faster and better. They do every ritual they have read about, and yet, in the end, they do not end up with better results. But instead, they have new problems to solve. Why so?
The reason is that most organisations concentrate on the symptoms (reactive) rather than the causes (proactive) of their problems:
- too long release: agile will solve it
- deployment takes too much time: DevOps will solve it
- too many issues raised in production: testing team will solve it
- testing is too slow: automated tests will solve it
Companies then implement these as if they were silver bullets. Those are excellent ideas, and we should all strive for them. However, there are no cookbooks since they deal with quality culture.
An organisation can't improve its culture's quality without undergoing a cultural transformation.
Conclusion
Whatever position you hold in a company, from C-level to junior level, remember that as your organisation grows, its culture and definition of quality will transform. And, you should regularly reflect on:
- Clarify what quality means (definition) for your organisation and the people you work with.
- Identify how you may help to improve quality and how you might assess it (perception).
- Determine how you can be an accelerator of quality. Why not question any process or tool that isn't a lever?
If you are a tester, you have the power to improve quality and make people's lives easier. However, as soon as your job becomes about checking off boxes instead of finding problems or making things work better for users, that starts spinning in circles which doesn't help anyone! You need to see yourself as someone who makes changes happen faster rather than waiting around thinking "We'll get there eventually."
Cover image from the short film Cargo Cult by Bastien Dubois.
Top comments (0)