Agile, development, testing: a rant that ends with ChatGPT

Agile, Scrum and the "full-stack developer" - a case study

The team took the standard definition of roles in Scrum (PO, Scrum Master, Developer team) and ran with it - straight off a cliff! They were convinced that only developers should handle automation testing and that manual testers were just unnecessary baggage. The PO even only considered a developer a "full-stack" if they could also do automation testing. Manual testers were only allowed to stay on the team because they were "technically" part of the Definition of Done. But, in true Agile fashion, the team believed that anyone could take on any role at any time. And since testers had no "real" job on the team, the test manager ended up moonlighting as the Scrum Master, while one tester took on the role of writing stories for the dev team. The rest of the manual testers were left to handle "external" responsibilities like customer support.

On a larger scale, the project subscribed to a SAFe methodology, which called for a "division of labour" between multiple Scrum teams. But, as per the prescription¹, there was also a "System Team" in charge of testing and quality-related tasks. This team handled everything from infrastructure to the general framework for testing. They even created shared libraries to make testing efforts uniform across all teams. But, when it came to staffing the System Team, it was no surprise that they were all "full-stack developers". These were "super-full-stack-developers" who were also expected to handle tasks typically assigned to Operations. So, the "full-stack developer" became a god-like all-around professional figure in the team's eyes.

"You're making it sound like a disaster. But what's the big deal? What could be more agile than that?" Or maybe, some people might say it's a huge exaggeration. But trust me, folks, it's not. This is the real deal. The project I worked on last year was a wild ride. I'll admit, it's a bit of an exception, but the project is still going strong and running like a well-oiled machine. My attempts to convince them to embrace the power of (automation) testing were unsuccessful, so I decided to jump ship. But hey, at least I got a good story out of it².

Anyway, let's take baby steps and start by having a little chit-chat with our dear developer about testing. The one with the capital 'D' of course!

The Developer and testing

Listen up folks, I got a bone to pick with y'all. I don't want y'all to model the Developer on yerself or yer colleague, not even on an abstraction of all the colleagues you had a chance to work with. I'll try my best to do it myself, even if I might crack under the pressure in the end. But just a tiny bit, I promise. What I'd rather like you to think about is The Developers Weltanschauung - which is just a fancy word German philosophers use for worldview. And start from this thought: we can safely say that our own (personal) worldview is influenced by the events that occurred in our past and by the impact they had on us. Now let's try to extend this idea to the Developer and try and read his mind.

Now, let me tell you a little story³. Once upon a time, The Developer was brought to the world with one purpose only: to develop software. Without him, software development makes no sense. Yet, for a very long time, in the early days, he wasn't the central figure in his own story. Managers and (to my great surprise!) testers were putting him in a box. Much like a house pet, he received orders and was punished when his behaviour (aka code) was unacceptable.

But things started to change dramatically in the '90s. Software developers took matters into their own hands, they set free from the chains and writing code became an art. Extreme programming, object-oriented programming and a whole lot of other innovations came about⁴. This was the new '60. Hippie developers were going to change the world with start-ups built in basements and college dorms⁵. They were not only challenging the software development process. Notions such as freedom of speech, legal property and such were also being reshaped⁶. All in all, it was just great fun!

In 2001, the Agile Manifesto was printed because software was starting to take over the world, and things were getting a little chaotic. After all, software was not needed just for software's sake, or in, let's call them, software companies. Everything was slowly becoming software - even huge companies needed it. And the creators of the manifesto weren't a bunch of young whippersnappers, they were seasoned pros with an average age of well over 40⁷. Even if the manifesto itself was written in a ski resort (and probably some booze was involved), these were people with a lot of experience. But at its core, the manifesto was all about giving developers the freedom to do their thing. Business types and their complicated plans were allowed in, but only through the back door (they had to pay for everything, after all).

Now, as far as I can see, this suggests that Developers see themselves as code scientists who do things for the sake of science alone. Which is nice, it's noble, and can also make you rich overnight if you're lucky. We also have dedicated methodologies meant to protect and amplify this mentality and creativity now. Even if, in all truthfulness, some developers will still feel like they are put in a box. They will complain that, because of the huge number of meetings, they are not able to do their (real) job.

But what about testing? Well, if I'm right up to this point, the Developer thinks testing is useful if it completes his Weltanschauung. And that means testing is important if (and only if) it can act as a tool to validate his scientific experiments. In short, and somewhat bluntly, if it makes code better. Yet the "makes code better" means that it makes the code a little less likely to crumble under the weight of human error. As my wise friend once said, 'the code needs to answer to itself'. Or was it someone famous? Who cares, the point is the same.

This is, however, nothing less than unit testing (and, ideally, TDD⁸). And if we think about it, this is what most developers will recognize as quality assurance. To give you an example, this is exactly how the "full-stack developers" I mentioned at the beginning understood the matter. They were asked to create automation testing. And you know what they did? They created huge and complex tools to make unit testing easier. Yet, let's be real - this was all about the developer's sense of self-worth. I mean, have you ever heard a developer say 'I just love unit testing'? No, because that's ridiculous. It's like saying 'I love doing taxes' or 'I love going to the dentist'. The "full stack" developer's attitude on the project was rather "Why do things the easy way when you can make it harder and call it 'quality assurance'?"

Enter the Tester

"Who's the tester with the big T? And what does she do?" you might ask. Well, for a while she was like the nagging wife that criticizes her husband for not cleaning the house properly. With the difference that, usually, the wife herself knows how to clean the house (better). Back in the day, testers had their own secret society, hidden away from the developers. They would get the code and go on a bug hunt like it was their own personal witch trial.

But then the Agile revolution happened and suddenly the testers were left out in the cold. Like a friend once said, they were "just testers", people who do what they do until they can land a job as a developer. These were the people who lugged around huge files with test cases and functional requirements like they were the last copies of the Dead Sea Scrolls. They were an odd extension of the businesspeople, that couldn't keep up with the fast pace of the Agile development. And it seemed like everybody would be better off if developers could just automate their jobs.

The Dead-Sea-Scrolls society was in a pickle, but they weren't going to roll over and play dead (sorry, bad pun). So they did what any sane group of testers would do - they created the ISTQB (I Swear This is the Qualification Board). They wrote a manual and created a certification, piling paper upon paper until the problem was buried under a mountain of bureaucracy. The basic main activities needed in each test were identified as: test planning and control, test analysis and test design, test implementation and test execution, evaluating exit criteria and reporting, and test closure activities⁹. In short, they just kept adding more paperwork and more theory to a paper-pervaded role, until it became so confusing that only "highly qualified testers" could get a grip on it.

But then something magical happened. Those same old-fashioned bureaucrats who loved their paperwork suddenly discovered the joys of code and had a blast. It was like the hippie-dippy '60s all over again, but this time for testers. In 2006, testers discovered Selenium and Page Objects¹⁰ and it was like they had found the holy grail of testing. JavaScript-based tools started promising their own revolution in 2017¹¹, and testers were like kids in a candy store. Finally, new design patterns created especially for automation purposes were introduced, and it was like a lightbulb went off above everyone's heads¹². Testers also started to find their way back into the Agile world. They borrowed TDD and extended it to ATDD¹³ and then to BDD¹⁴. It was like they were getting a second chance at life. And, in my opinion, this last step is the biggest "break with tradition". Testing was not about what the management wanted, but about the end user. Testing was not about forcing developers to do what the higher instances wanted them to do. Rather, it became a story about the developer and his audience. It was like a testing fairy-tale come true¹⁵.

So, is the tester with the capital T a specialized developer? I don't want to suggest that the Tester got tired of all the paperwork and became a developer. Not at all. Instead, the Tester brought the paperwork into the digital age. She started talking about "living documentation". Diagrams, flowcharts, user stories, acceptance criteria, test cases, all generated automatically. It's like magic! It all comes from the source code, tests, and other artefacts created during the development process. And, most importantly, it's always accurate and up to date. With all the latest information at their fingertips, it's like they have a crystal ball for the project. No more confusion, no more misunderstandings. It's like the Tester is the all-knowing oracle of the development team. It's like a testing party, man! So who is the Tester? The documentation-loving party animal!

Sex, drugs, and rock'n'roll?

I think it's no secret by now that I'm a total tree-hugging, granola-munching liberal. I firmly believe in the power of "Give peace a chance" and "Live and let live" philosophy. That's why I'm also a firm believer in the division of roles and responsibilities. If we just let people do what they love, everything will be A-OK. So, I invite you to let the testers test away and the developers code to their hearts' content. And to take advantage of each other's strengths. But let's be real, we don't work with unicorns here. Sometimes projects are like a three-legged race with a blindfold on. And, while testers have had a resurgence in recent years, they're often the ones getting the short end of the stick. Can testers do something about it?

Well, let me tell you what I tried to do. There were a lot of other issues on my latest project with the full-stack dev team, apart from what I have already shared with you (or, most probably because of it). It was a hot mess that had been going on for a really long time, let me tell you. First off, there was no documentation to be found. Nada. Zilch. Zip. Zero. Second, no one had a clue what the main features of the project were. And to top it off, the testers were like, "IDE? Never heard of her." But it worked. And everybody was talking about "feature-based development" like they knew what they were talking about. And I, being the fool that I am, fell for it hook, line, and sinker. So I sat down with my books¹⁶, scratching my head thinking to myself, "Are we really gonna do this? Let's reverse engineer this baby and BDD the pants off this project!". Even though you didn't start off on the right foot, we can still catch up. So, I rounded up my trusty team of "Three Amigos¹⁷" and we delved deep into understanding what the project was all about. We wrote test cases in plain English, that even a cucumber¹⁸ could understand, and generate all the missing documentation with the highest possible dose of serenity¹⁹. It was like trying to teach a giraffe how to swim but it seemed that we could make it work.

It was a wild ride while it lasted. But like a house of cards, our team crumbled faster than a cookie in milk. Egos were so big, you could see them from space. People were clawing their way to the top like it was a game of "Survivor: Office Edition." And let's not forget about "The Three Amigos" who were missing one crucial member: the business analyst. The PO was like a beacon of hope in the beginning, guiding us to our destination. But, just like a mirage in the desert, he disappeared into thin air when it came time to actually get to work. The manual testers were so fixated on the UI, you'd think they were trying to solve a Rubik's cube blindfolded. And don't even get me started on the "System Team" who provided infrastructure that was about as useful as a screen door on a submarine. Automation testing was a complete and utter failure.

Was it the Testers' fault or was I just not Tester-y enough? I mean, Martin Fowler was already pointing out in 2005 that there were more people involved in the software development process than just developers, like testers in the "New Methodology.²⁰" But it can't be just a battle between developers and testers, it's more about figuring out what the heck "The Agile" really means. We've got all the tools, and all the knowledge to do things right. But it's like trying to find a needle in a haystack of confusion. So, here's a thought: if you want to go Agile, do your homework first, and don't pretend to switch just because it's trendy. Especially because the whole Scrum, Super-Scrum, Safe and Unsafe Scrum talk is getting as old as the fight between developers and testers. Now, we've got DevOps²¹, GitOps²² and a whole new paradigm knocking at our doors. We have a little chat that promises to make all I've said so far completely and utterly useless.

ChatGDP - the final revenge of the Tester?

What does ChatGPT have to do with it? Is this some sort of sneaky clickbait? Well, I have a confession to make. This text was copy-edited by ChatGPT. I had the general idea, I knew where I wanted to go with it, and even had it all broken down into paragraphs. But it would have taken me weeks to get to where this text is right now. I'm both thoroughly impressed and a little scared. Not just for the writing business - I feel bad for them but I won't lose my job over it…or will I? This thing is seemingly as good at writing code as it is at correcting and rephrasing text! So, what does this mean for the whole Agile/tester/developer debate? Is ChatGPT going to take over the world and leave us all jobless? Only time will tell. But here are my two cents on it. Let's take a giant leap backwards and ponder the elusive concept of "The Agile." Is it about speed? Of course. Being clear? Duh. But when you really break it down, it's all about the code. Like a bad tattoo, it's forever etched in our lives. But what happens when code becomes a thing of the past? Will developers become as relevant as VHS tapes? Will testers have a job security of a mayfly?

One might say, we won't need junior developers anymore, but who will double-check the code and make sure it makes sense? Senior developers, of course! But what happens when we run out of senior developers? Without junior developers, the future looks bleak. But don't worry, ChatGPT.100 will be there to save the day. So, goodbye developers! Time to become a boring manager or find a new job, like a professional couch potato or meme creator. And who knows, maybe carpentry will make a comeback and become the hottest job on the market. But what about testers and their newfound love for coding? Will they have to give it all up and become professional couch potatoes like the rest? Or perhaps they'll have to resort to more drastic measures like becoming professional taste testers for memes. The future is uncertain, but one thing seems sure, they'll have to find a new hobby because coding is so last year.

Or maybe not? Does the Tester finally have a chance to come out on top? It does look like the Tester finally found a silver lining in the cloud of AI takeover. Indeed, I found quite a reassuring paragraph in a recent study about "The Future of Quality Assurance". It goes this way: "However, even with all this computer power and all this data and intelligence, there's still one thing at which every AI sucks: understanding human behaviour.²³" And this is great news for the tester! While machines may rule the world, testing will always be a job for humans. And let's face it, BDD - as in Behaviour-driven - is basically the president of testing anyway. So go forth, testers, and conquer the world (or at least, the world of software). Also 'cause, you know, what could be more Agile than that?

References

• Abu-Faraj, M. (2019). A Survey on Software Quality Assurance. Journal of Engineering and Applied Sciences, 14(15), 5111–5122. https://doi.org/10.36478/jeasci.2019.5111.5122
• Anonymous. (2009). Standards and guidelines for quality assurance in the European higher education area. (5) (PDF) A Survey on Software Quality Assurance. European Association for Quality Assurance in the European Higher Education, Helsinki, Finland.
• Anonymous. (2013). 730/D9, Nov 2013-IEEE approve draft standard for software quality assurance processes. IEEE, New York, USA https://ieeexplore.ieee.org/document/6781526.
• Baumgartner, M., Klonk, M., Mastnak, C., Pichler, H., Seidl, R., & Tanczos, S. (2021). Agile Testing: The Agile Way to Quality. Springer International Publishing. https://doi.org/10.1007/978-3-030-73209-7
• Binamungu, L. P., Embury, S. M., & Konstantinou, N. (2018a). Detecting duplicate examples in behaviour driven development specifications. 2018 IEEE Workshop on Validation, Analysis and Evolution of Software Tests (VST), 6–10. https://doi.org/10.1109/VST.2018.8327149
• Binamungu, L. P., Embury, S. M., & Konstantinou, N. (2018b). Maintaining behaviour driven development specifications: Challenges and opportunities. 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), 175–184. https://doi.org/10.1109/SANER.2018.8330207
• Binamungu, L. P., Embury, S. M., & Konstantinou, N. (2020). Characterising the Quality of Behaviour Driven Development Specifications. In V. Stray, R. Hoda, M. Paasivaara, & P. Kruchten (Eds.), Agile Processes in Software Engineering and Extreme Programming (pp. 87–102). Springer International Publishing. https://doi.org/10.1007/978-3-030-49392-9_6
• Crosby, P. B. (1979). Quality is free: The art of making quality certain. New York: McGraw-Hill. http://archive.org/details/qualityisfree00cros
• Garg, S. (2015). Cucumber Cookbook. Packt Publishing Ltd.
• Goericke, S. (2019). The Future of Software Quality Assurance. Springer International Publishing.
• Humphrey, W. S. (1989). Managing the Software Process. Addison-Wesley.
• Irshad, M., Britto, R., & Petersen, K. (2021). Adapting Behavior Driven Development (BDD) for large-scale software systems. Journal of Systems and Software, 177, 110944. https://doi.org/10.1016/j.jss.2021.110944
• Lopes de Souza, P., Lopes de Souza, W., & Ferreira Pires, L. (2021). ScrumOntoBDD: Agile software development based on scrum, ontologies, and behaviour-driven development. Journal of the Brazilian Computer Society, 27(1), 10. https://doi.org/10.1186/s13173-021-00114-w
• Martin Fowler. (2005). The New Methodology. Martinfowler.Com. https://martinfowler.com/articles/newMethodology.html
• Nicieja, K. (2017). Writing Great Specifications: Using Specification by Example and Gherkin. Manning.
• OIN. (2011). ISO - ISO/IEC 25010:2011 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. https://www.iso.org/standard/35733.html
• Rose, S., Wynne, M., & Hellesøy, A. (2015). The Cucumber for Java Book: Behaviour-driven Development for Testers and Developers. Pragmatic Bookshelf.
• Scaled Agile, Inc. (2017). Scaled agile framework. Retrieved 2017 from http://www. Scaledagileframework.com/.
• Smart, J. F. (2014). BDD in Action: Behavior-driven development for the whole software lifecycle. Manning.
• Solis, C., & Wang, X. (2011). A Study of the Characteristics of Behaviour Driven Development. 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications, 383–387. https://doi.org/10.1109/SEAA.2011.76
• Spillner, A., & Linz, T. (2012). Basiswissen Softwaretest: Aus- und Weiterbildung zum Certified Tester; Foundation Level nach ISTQB-Standard. dpunkt-Verlag.
• Walkinshaw, N. (2017). Software Quality Assurance. Springer International Publishing. https://doi.org/10.1007/978-3-319-64822-4

1. See Scaled Agile, Inc., 2017. ↩

2. Please keep in mind that this text is meant to be a rant, just plain old having fun. Some context and more reliable material will be available in the footnotes in the form of definitions and references. But they are not meant to be, by any means, exhaustive. ↩

3. There are some truly amazing reconstructions of the history of software development and quality assurance available nowadays. Mine is not going to be one of them. What follows is just my caricaturist and biased way of expressing what came to mind while reading, amongst others, Walkinshaw (2017), Goericke (2019) and Baumgartner et al. (2021). ↩

4. To mention just a few, the widespread adoption of the Internet in the 1990s led to the development of new technologies and practices specifically for building web applications. HTML, CSS, and JavaScript were developed to create web pages and dynamic user interfaces, and the HTTP protocol provided a way for these pages to be transmitted over the Internet. Integrated development environments (IDEs) and version control systems (VCS) also became more prevalent in the 1990s. ↩

5. One of the most notable IT startups built this way in the 1990s was Google. In 1996, Larry Page and Sergey Brin, two PhD students at Stanford University, began working on a search engine in their dormitory. They called it "Backrub" at the time, and after months of development, they renamed it Google. The company was officially incorporated in 1998, and the rest is history. Another example of a successful IT startup built in a basement in the 1990s is Amazon. In 1994, Jeff Bezos, an investment banker, started working on an online bookstore in the garage of his home. He launched the company in 1995, and it quickly grew in popularity. ↩

6. See e.g. "The Pirate Bay" and the scandals it provoked. The site was launched a little bit later - in 2003 - but it was the e prototype of a website that provides a directory of torrent files and magnet links for peer-to-peer file sharing using the BitTorrent protocol. ↩

7. On this see e.g. Baumgartner et al., 2021, p. 108. ↩

8. Test-driven development (TDD) is a software development approach in which tests are written for a piece of code before the code itself is written. The tests are then run, and the code is written to pass them. The process is then repeated, with new tests being written as new features are added to the code. ↩

9. For this see Spillner & Linz, 2012. ↩

10. The basic idea behind the Page Objects pattern is to create a separate class for each page or page section of the web application under test. Each class represents the elements and functionality of the corresponding page, and the methods in the class provide an interface for interacting with the page elements. For example, a login page class would have methods for entering a username and password, and clicking the login button. ↩

11. This is when Cypress was first released, but tools such as Puppeteer, Playwright and Test Café followed suit, and are now more popular than ever. ↩

12. Take, for example, the Screenplay pattern, which is based on the idea of modelling the test as a series of actions that a user would perform on the system. And, similarly, App actions in Cypress, which envisages a separate class or module and can be used by the test code to interact with the application. ↩

13. ATDD (Acceptance Test-Driven Development) is a software development methodology that emphasizes the use of acceptance tests to drive the development process. The ATDD process starts with the stakeholders defining the acceptance criteria for a software feature. These acceptance criteria are then used to create acceptance tests that will be used to verify that the feature meets the requirements. The development team then uses these acceptance tests to guide the development process, ensuring that the feature is developed to meet the acceptance criteria. ↩

14. BDD (Behavior-Driven Development) is a software development methodology that focuses on the behaviour of the software system, rather than its implementation. ↩

15. On a more serious note, we can also notice how the definition of "software quality assurance" changed over time in the academic world. Abu-Faraj (2019) sums it up nicely (up to a certain point) this way: "Humphrey (1989) defined it as achieving levels of fitness for use. Crosby (1979) defines it as conformance to requirements. In recent studies, new definitions of software quality appeared. OIN (2011) defined it as the capability of the software product to satisfy stated and implied needs under specified conditions. Anonymous (2013, 2009) defined it as the degree to which a software product depends upon those requirements that accurately represent stakeholder needs, wants and expectations". But one of the latest trends on the matter is called "Total Quality Management" (TQM) (…) [and] it broadly represents an approach to management that aims for long-term success by linking quality with customer satisfaction", with one of its main characteristics being customer focus (on this see Walkinshaw, 2017, p. 30–31). ↩

16. I highly recommend Solis & Wang (2011), Smart (2014), Garg (2015), Rose et al. (2015), Nicieja (2017), Binamungu et al (2020), Binamungu et al. (2018, 2018b), Lopes de Souza et al. (2021) and Irshad et al. (2021), amongst others. ↩

17. The "Three Amigos" refers to a team of three individuals: a business analyst, a developer, and a tester. The team works together to ensure that the requirements for a software project are clearly understood, and that the solution being developed meets those requirements. The "Three Amigos" work collaboratively to ensure that the solution is tested and validated before it is released to the customer. This approach is often used in the context of Behavior Driven Development (BDD) to ensure that the software meets the needs of the end user, and that the development process is as efficient as possible. ↩

18. Cucumber is a tool for behaviour-driven development (BDD), which allows anyone to write tests in a natural language format that is easily understood by non-technical stakeholders, and implement the code accordingly. ↩

19. Serenity is another open-source BDD automation framework, and one of its main features is that it creates reports that provide detailed, meaningful information about the acceptance criteria and the test results. ↩

20. Martin Fowler, 2005. ↩

21. DevOps is a set of practices and tools that aims to bridge the gap between development and operations teams by automating the process of software delivery and infrastructure changes. It relies extensively on continuous integration and continuous delivery (CI/CD), infrastructure as code, containerization, and monitoring and logging. ↩

22. GitOps is a methodology that uses Git as a single source of truth for both application and infrastructure definition, promoting continuous delivery and deployment practices through code versioning and pull request workflows. The idea is to manage the entire lifecycle of an application using Git as the source of truth, making it easier to track changes, revert mistakes and automate the deployment pipeline. ↩

23. Goericke, 2019, p. 203. ↩