I love software process. My passion is finding ways to build software faster, but better technology is only half the battle. The other half is a people and organization problem. Process can do wonders for that half.
Among some of the benefits process can bring to a project are:
- Ensuring everyone is working on the most important thing
- Coordinating efforts to distribute work effectively
- Adding little nudges to improve code quality such as automated testing and code reviews
That last bit is where process can become a disaster though. Process is most effective as a nudge. Building software is a complicated process and it is easy for developers to forget to do a handful out of the hundreds of things they are constantly thinking about. Process can provide safeguards for developers to remember to do those things.
Process can not be used as enforcement.
That is contrary to how many view process. They treat it the way we treat our legal system. Every time something bad happens, the solution simply must be a new process implemented in order to prevent the situation from happening again.
Sometimes that works, but it requires all the developers to believe in the new process. Let’s use automated testing as an example.
Your product has a bug. It was pretty bad. The bug would have been easily caught by automated testing. After fixing the bug, a new process is implemented where all new code requires automated testing. Great! Now all our future code is bug free right?
Not quite.
If the developers on the team don’t believe in automated testing, then they may not write it. A code review can be enforced to check for them, but that won’t work if the reviewer doesn’t believe in automated testing.
The solution to that may be engineering management checks on it. What’s the reporting ratio though? 4 developers to 1 manager? 8 to 1? Not all tests are created equal. A manager with a team of developers who don’t believe in automated testing will only be able to check that they have built some tests. That manager will not have time to check that they have built good tests that would actually prevent bugs from being introduced.
Process can be used to help remind a developer who believes in automated testing that they should write automated tests. Process can not force a developer who doesn’t believe in automated testing to write automated tests that are actually useful.
Another thing to consider is that all process has costs. The only process that should be used is one where the benefits outweigh those costs. For example: companies with large scale software systems will have systems that are complex enough to require a senior developer or software architect review designs for new systems. Performing a review has the cost of time for at least two people: the reviewer and the developer whose work is being reviewed.
Not having this review could result in rewriting large swathes of poorly designed code though. The cost of a few hours for a review process outweighs the cost of the days, weeks, or months needed to fix that code.
Needing to get approval from one or two people makes sense. But what if some reviewers missed things? Do we start adding more reviewers? Does every project need approval from 6-8 people now? How do we make sure you can actually schedule a meeting with all those people? How long does the start of the project get delayed in order to make sure everyone can make the meeting?
Now we’re entering in the territory where the cost of the review is going to exceed the benefits. Most of the problems would have been caught by only one or two people. To get the remaining number of problems by requiring more people to review the software design, we delay the project by a few days (or even weeks). We’re also using up the time of a lot more people. Instead of trusting our reviewers to learn and get better, we have burdened the entire organization with an expensive process.
Many people view process as a way to remove risk from software projects. But software development is an inherently risky activity. Attempting to remove all risk with process will end up introducing massive costs in time. Eventually this cost becomes untenable.
I once worked with a manager who believed it was possible to remove all risk from software. He wanted the 5 lead developers to review the code written by all two dozen developers. So every lead performed code reviews for over two dozen people and every developer had their code reviewed at least 5 times. Doing that properly would have taken a single lead more than 24 hours in a day. Even if there was enough time, how much more benefit would the extra reviews have gained us?
Time is not infinite. Spending it doing one thing means not spending it doing something else. When time is unwisely spent on low value processes, it prevents the development team from working on something that could produce a high amount of value for the business. Competitors who make better decisions around processes would end up bringing more value to their customers in less time. That helps them steal customers.
The quest to lower software risk at all costs has inadvertently created a huge amount of business risk.
I like to think of software process as seasoning for food. If you have a great cut of steak that’s cooked well, then a little seasoning is only going to enhance it. Too much will ruin the steak since all you’ll taste is seasoning. On the other hand: if you have a pile of poo, no amount of seasoning will make it not taste like a pile of poo. The seasoning can only enhance the strong foundation of great food.
Process can only enhance a team that already works well. If a team communicates well and each member trusts each other, then there is a strong foundation to work with. Adding process tweaks the team’s workflows and makes them more productive.
No amount of process can help a dysfunctional team. The problem here isn’t there isn’t enough process or the wrong processes are in place. The problem here is with the team’s culture. Process is an easy thing to fix, which is probably why many people think of process solutions first. But that’s just avoiding the underlying issues that really need to be tackled.
This post was originally published on blog.professorbeekums.com
Top comments (4)
I'll also add that process is pretty bad at dealing with new circumstances. If your team is going to stay the same size and the challenges are going to stay pretty consistent, you might be able to lean on process, but when new challenges pop up out of nowhere, culture needs to take over for process while a new process is created.
We have a small, but growing team and technical challenge, so process is really important, but in constant flux, and it's the culture that needs to guide us in the unknown. Culture also needs to change over time to respond to new circumstances, but it's much less brittle than process.
Great article overall. Definitely jive with the lessons. What do you think @jess and @maestromac ?
Like Ben mentioned, our processes are in constant flux which is great because we're always trying to get better and stay communicative and productive, but it also means we need people to re-buy-in every time.
@pbeekums you expanded a lot on #3: testing and code reviews, any thoughts on the processes surrounding:
1) Ensuring everyone is working on the most important thing
2) Coordinating efforts to distribute work effectively?
I focused a lot on #3 because that's where I've seen people take things too far the most.
1 & 2 are usually tied together pretty tightly. This tends to be very specific to various situations.
For example, when projects are small there is less data. A lot of decisions have to be made intuitively. Whose intuition is trusted the most? I don't think there is a right or wrong answer here. Plenty of companies have succeeded with the single product visionary.
My personal preference though is that things are discussed as a team and that teams make decisions together. I believe it gives everyone a greater sense of ownership and that will help motivate them to do better work in whatever their role is. This does require an entire team to be product motivated though. A great developer who is only interested in fun technical problems will contribute a lot to the implementation, but very little in the discussion on what to build.
Projects with lots of users have lots of data. This makes some things easier because you can just look at the data on what are important things to focus on. If there's a conversion funnel and one step has a significantly larger drop than every other step, it makes sense to work on things to optimize that step. But this is also a reactive way to build things. The product is going to get optimized, but very little actual innovation will happen. Some intuition is going to be required to balance the need to optimize existing functionality versus building new functionality. This goes back to the previous point on whose intuition to trust.
All of this is further complicated by the business. It is really easy to involve an entire team in a discussion around what to build in a consumer application. Ideally every member of the team is also a user of the product. That gives them the necessary perspective in thinking about how to make things better for all users.
That gets much harder in a B2B product since most of the team members won't be consumers. Even if they were, the person buying the software and the person using the software are often not the same people. I think product managers are relied on more here to make decisions because it'll be there job to spend endless hours talking to users, buyers, sales teams, customer support teams, dev teams, investors, etc. and find some way to balance the needs of everyone. A developer is less capable of making product decisions here because they don't have time to get that context and build software. Not unless they like working 20 hour days.
As I've learned the hard way, this is similar for education products. You can never be your own user when you're building content for others.
Once priorities are decided, regardless of how they're decided, we then have the problem of distributing work. There are more factors involved than I can list, but in general I really like standups. I think it's important to discuss where every team member is at and whether progress is going faster or slower than expected. It helps redistribution of work.
This is a hard area to say generic things about so I'll use a specific example. Let's say someone is working on a tough algorithm. They think they had it handled, but hit a bunch of unexpected use cases. Pair programming didn't make sense originally because it seemed easier at first, but now it might. So someone gets pulled off their current task and paired up because the algorithm is the most important piece.
Or maybe a feature was given a high priority, but it has taken 5x more time than expected with no end in sight. At that point the value of the feature versus the time needed to finish it makes it less worthwhile. That's time that can be spent on other things that bring value.
The hardest bit is when there is a big feature that needs to be done quickly. I think most managers these days understand the mythical man month, but there are ways to break things up to manage adding more developers to a project. At that point it's a matter of breaking up various systems, figuring out where potential merge conflicts are going to be, trying to assign as many de-coupled pieces as possible, and determining the absolute maximum number of people that can be involved before the diminishing returns turn negative. That's all very vague, but this really depends on the specific system involved.
Also, I said "standups" and not "daily standups". Sometimes once a day is not enough. I've switched from daily to three times a day to back to daily, all with the same team. It just happened to be necessary depending on the situation we were in.
I just made a very long spiel that can be summed up by: "It depends."
Would be happy to talk about specific situations if you'd like.
This is something I recently started thinking about. As someone who's tasked with handling processes, I can see it time-consuming and disruptive to other's workflow. Reading this gave me a new perspective. I really like that food-seasoning metaphor. Great stuff!