Better Ideas At Test Design

Even with many years in this industry, I get inspired by courses I take. A course - BBST Test Design - served as inspiration on sharing on this: Having better ideas at Test Design.

And by Test Design, I mean the continuous collection, creation and prioritization of ideas that would help us produce the results from testing that the world around us expects. The ideas that lead us into doing what we do with software; so that we recognize what we recognize; so that we have the conversations around quality that we need to have.

I know from the 25 years I’ve been at this that testing is far from simple. It’s knowledge work, just like application programming, targeting information that helps us address quality concerns.

With a simple model, we could describe testing as a process of doing testing where the input is someone with brains, and the output is learning to do the work better and the information and artifacts we expect in our organizations. We come as we are, and we learn: the software we are testing and its features; the problems and their relevance; each other and communication and collaboration; and the business that pays our salaries.

In the last months, I’ve taken a course on Test Design. On the course, two exercises have thrown me at Open Office Impress, the presentation software, and choosing a single variable to analyze for test ideas. True to my exploratory tester nature, I could not commit to a variable before completing a whole variable tour to find something I would have fun with, finding information.

I chose transparency of elements. I learned quickly to connect it with a default value it could have; with editing, presenting and printing modes and their options; the different element types it could be applied with; and the many places from where you can edit it.

On the first exercise, we listed risks imagining bug reports we might end up writing on it. I generated the list allowing the application to be my external imagination and it increased the creativity I could bring at the task.

On the second exercise, we were asked to apply risk-based domain testing. Equivalence classes, boundary values and the sort, but with the idea that risk - what we expect might fail - will guide us to equivalence classes. Like entering a single digit can (and does) behave quite differently from something with three digits or decimal numbers.

I found a bunch of inconsistencies, and problems, and the application rewarded my tester efforts with a big visible crash dialog that nicely reproduces at will when combining two digit numbers with undo. Yes, single digit is fine but two digit with undo crash the app. And remind me that we don’t have to create bugs intentionally for learning, the software industry has us covered.

It’s not just courses where I find that I already think in quite many dimensions and details allowing me to discover bugs, but that is the experience and reality from the teams, projects and products I work with too.

With the simple process, I am often called to situations where the output isn’t where it should be. We are missing bugs. We are not documenting with test automation. We are thinking simplistically about coverage, and thus missing even the idea that there are bugs to find on other dimensions.

As a tester, I start with adding of results. But as principal, being great at testing isn’t sufficient. I need to make people around me better at testing. I need to fix the practice, while adding some of the results.

To fix the practice, I have a recipe of my own. I don’t do instructions and processes, and I don’t choose tools and enforce guidelines. I start my work from within, joining a team as a tester. As such, I experience what the team misses, and I try to figure out how to learn together ways of not missing that anymore, even when I am gone.

We work towards making testing everyone’s business. Testing is too important to be left for just testers. Developers, product owners, neighbor teams are all welcome to pitch in.

We make improvements continuously, but each individual improvement can be a small adjustment through feedback. We notice the change looking back six months, but day to day it seems we do the same things.

I work to remove myself, so that I can repeat the work with another team needing insightful ways of taking small steps to better.

Fixing the results start from showing what results we have been missing.

I’ve repeated this growth journey across organizations and teams, and the takeaway I still want to leave you with is on where do you learn to have the versatility of the ideas so that you see the results we have been missing. Those ideas stem from your ability to connect information of the past into the product change work ongoing right now.

I recommend you read bug reports. Not just your own, but your colleagues, your organizations, and if possible, whatever the customers directly report in unfiltered form.

I recommend you read lists of generalized bug reports. taxonomies are available in books by Kaner and Beizer with a lot of relevant information

Learn Test Design. BBST course series is brilliant. I grew up to being a tester with Cem Kaner creating the teaching materials and owe a lot of foundational perspectives to his work now packaged as online learning courses.

Finally, work together with others. When you work in a group - an ensemble - you will learn about things you did not know you don’t know, and thus could not ask. It speeds up our learning significantly.

We all need to go and learn to experience what we miss. Better ideas produce better results in testing.

I’m happy to connect on LinkedIn, and write my notes publicly on Twitter. Looking forward to learning to provide better results in testing with you all.