The way adults deliver software on time

Let's quickly introduce this post with a little bit of context. First of all, I'm the CTO of a web agency which deals with many short projects with usually one or two developers working on it. I've also worked on multi-year projects involving dozens of developers but that's a smaller part of my experience.

Here we'll refer to the client as the person whose money you are spending and whose hopes you are entrusted with. They could be the founder of your startup but for me it's literal clients.

And finally, I hate all the technical project management and methodology gibberish so don't expect me to use complicated terms nor to deploy corporate methodology diagrams that make rocket engines look simple.

The goal here is to give you my take on software planning, from the time where you have to give an estimate to the on-time delivery of the product.

Estimating

It would be fair to assume that the immense majority of developers do not like to produce estimates. And while — as the #NoEstimates movements states — it's definitely impossible to estimate everything within a 10-minutes precision before doing the project, the client has a limited amount of money and an event to attend in 3 months where they'd like to present their product. Maybe you can't commit on everything but you definitely have to commit on something. The "when it's ready" philosophy is a sweet delight best consumed by hobbyists and GRRM while us, responsible adults, have to provide estimates.

Note — This requires a lot of experience both from the person(s) doing the estimate and the company projects. Personal experience will be a way to give good estimates and spot all the pitfalls while the Git history or previous time sheet will support the numbers with data. If you are not experienced enough, you need to recognize so and get help from someone more senior than you are, otherwise you'll be putting yourself in a complicated situation. If you don't know, you don't know, it's fine!

Get rough

Usually the first time I have to estimate anything it's when my commercial team comes up to me with a new project. The specifications are more or less detailed, but you'll have to deal with them.

The main idea is to create a first version of your work-breakdown structure. You can use a mind-mapping tool for that, like Coggle or simply a sheet of paper. Lay out all the different blocks of your program (like "front-end", "back-office", "API", "mobile app", etc). Then lay out all the features of each component. If a feature crosses several blocks, put the feature in each block.

Then look at your tree and wonder what you forgot. There is a lot of usual suspects, like the login/logout page, password recovery, email change process, various emails, GDPR cookie consent, analytics tracking, etc.

Then look at your tree and ask yourself what you forgot. Maybe it's an additional invisible block that you'll need, maybe some regulations will mess with your plans, maybe it's unexpressed implicit client needs, etc.

Then look at your tree and ask yourself what you forgot. Until you run dry of ideas.

Then look at your tree and ask yourself what you forgot.

However rest assured, you can get a rough estimate that is going to produce a realistic deadline. In that regard I always love to refer to the Apollo program — which included inventing software engineering on top of doing the software itself. The deadline was very publicly announced by JFK in his "We choose to go to the moon" speech given after his advisors studied different options of space prowesses that could be done within the deadline. Did it cost an insane amount of money? Yes. Was it successful? Also yes. That's something we'll cover later on.

Assess your risks

As explained in "Risk Management Is Project Management for Adults" by Tim Lister, risk management is the very foundation of project management and especially software project management.

At this state of your project, you must identify two things:

• The core feature(s) which are the raison d'être of this project
• The most risky features whose timing is difficult to assert, probably because you lack the experience in those specific tasks

Interestingly enough it turns out that the most risky features are usually the core features. It makes sense because that's what makes this project different from all the other projects.

This risky part merits your utter attention. Try to talk to experts to help you estimate these tasks. Have a look at different libraries or frameworks that could help you doing this. Read blog posts and look for desperate Stack Overflow comments.

Equipped with this knowledge you can probably estimate the magnitude of the task to come. Then prepare some buffer to make it twice and some more buffer for some polishing time.

Please note that if you encounter some risky parts that also happen to be far from the core of the value proposition then you really should negotiate to move those parts out of the scope or at least to limit the commitment on them.

Note — If your project includes research and development then the problem is completely different. Research can't provide warranties on the result nor the timing. Your project can implement features that emanate from academic research but don't ever bet on something that hasn't been discovered yet.

Estimate regular tasks

Now is time to estimate the non-risky part of the business. For each small item in your tree, think how many days it'll take. Don't be shy, everything takes a long time. I've made this mistake a lot in my first years, estimating that some tasks would take small hours or even minutes because I felt that a longer time would feel ridiculous. It's not ridiculous, you need time to work correctly. That's why I estimate everything in days and very rarely do I need to go beyond half a day.

Then you get up your tree and for each node you can sum the work planned on each leave. Don't hesitate to add a small percentage of the sum to the leave. Suppose that you imagined something like this (which is a bit exaggerated for demonstration purposes):

• My Project (8)
  • Login (4)
    • Form (1)
    • Password recovery (2.5)
      • Form (0.5)
      • Email (1)
      • Password change (0.5)
  • Signup (3)
    • Form (1)
    • Email confirmation (1)
    • Email confirmation page (0.5)

As you can see, the project takes 8 days but if you look at just the leaves the sum is 5.5 days. There's 2.5 days added as you move up the hierarchy which account for the growing complexity of the project and thus the difficulty to deal with other parts. That is inspired from the COCOMO model's exponential factor.

Negotiate the deadline

Quite often, the client — or your commercial team before them — will not be satisfied by the presented cost and/or deadline. In that case you must remember that you made those estimates for a reason in a first place and that there is no way to bring them down.

One thing you can do to advance the deadline is work more. That is definitely not something that you can sustain but if the project is a very few weeks long then going home later and working on the week-ends might just save the situation. The downside is that you'll burn out and you'll need a recovery period. I've done this before and I know that after this kind of sprints I'm useless for at least a whole week.

You can also throw in more people, however this needs to be smartly done so that everyone can work on their own tasks without being impeded by the advance of other team members.

Yet the main tool you have is scope reduction. If you want the project earlier then the only real leverage that you have is to reduce the scope. So you can take that tree and start cutting branches. It can be a few leaves here and there. It can also be big branches. You can usually skim the project quite a lot from a lot of fancy but not-so-useful requirements.

Don't forget extremities

Depending on the project size this is more or less noticeable but you'll of course need to worry about the project setup and deployment. Most of the time I'll advise to be feature-oriented but for those two it's incompressible technical tasks.

Setup will be the creation of code repository(es), create the project scaffolding and get something basic running. Basically it's configuring the framework(s) that you'll use for the project.

Deployment is the configuration of the development and production platforms, which is sometimes easy within your usual infrastructure but sometimes will be done according to your client's requirements and should not be overlooked. If the client has complicated IT processes and requires you to do deployment over a special VPN that stays up only 10 minutes at a time when the moon enters specific phases then you'd better worry about this and put it in your estimate.

Planning

Next big stop on the hate track is planning. However, if you followed the methodology thus far you'll find that it's quite simple. This section is more for project managers than developers in theory but in practice everyone should know what is going on and why it's going on.

Basic structure

While on big projects you can start to parallelize things, it is far better to make your tasks as sequential as possible. When you start working on something, make sure that the previous task is done. My suggested order of big tasks is the following:

1. Functional specifications. Using a prototyping tool like Axure. This then gets translated into feature tickets. In my company, project managers do this.
2. Graphic design. Please don't use Photoshop for that, as a developer I prefer to see Sketch designs. In the team we use Avocode to communicate designs which allows for a very high level of precision when integrating the designs. That's also not a developer's job since it's graphical designers doing this.
3. Technical specifications. It doesn't have to be a long formal document but at least sit around with the tech lead to decide the main technical choices (front-end and back-end frameworks usually) as well as the strategy to tackle the risky parts of the project.
4. Development. That's where you do your favorite Agile things and pop up your favorite tools and write beautiful code ✨.
5. Data filling. If the project needs data, then once the features exist the filling of that data can begin but not before.
6. Testing. While testing happens every day on a per-feature basis, before the release a fair amount of testing will be done on the whole platform, which will lead to a round of bug fixing.
7. Release. This means pushing your code onto the production platform. This needs to happen as early as possible. If you can, invert testing and release so you release onto production a first time before the testing process starts.

Reviewing the specs

Once a project manager hands you the specifications, you need to review them. The early estimate that you made earlier is your budget and now you need to make sure that the finished specifications match that budget.

So take your WBS from the estimate and update it for the finished specs. This time you should have a pretty accurate idea of what you're going to produce.

The trick is when the specs are not done well and that edge cases are forgotten. It's important to play with the specs and consider them under all aspects to make sure that they make sense and that all cases are accounted for. Otherwise you'll discover it during the development and that will be unfortunate.

Another important part of reviewing the specs is pointing out what would take an inconsiderate amount of time to be done for little-to-none added value. UX designers and project managers don't always know how programs work under the hood and might choose very inefficient solutions for no reason at all. That's a developer's job to stay clear from these traps and push easy solutions instead.

— XKCD, Tasks illustrating how counter-intuitive the abilities of computer science are

Note — It's very easy to miss things. You don't know what you don't know. Missing things won't appear in your head magically. You need to setup a thought process that will help you highlight those missing things. This is the topic for another article, but some ideas include testing the wireframe extensively with different persona, starting to lay the data model or create sequence diagrams to make sure everything fits.

Data filling

One important aspect is to make sure that all data has a source. It's easy to get crazy in the design process and include lots of texts and images everywhere but every single bit of information has to be accounted for. The source of that data must be crystal clear at this point and the way that data is going to end up in your database must be a part of the specs.

Might be that you need a back-office to fill all the data. In that case you need also to think about how you are going to do the back office and include that in the time you'll have to spend. Usually I use the Django admin website or Wagtail to generate those interfaces because they are so damn efficient and I love them very much 💕.

It might also be that you're going to inject data from CSV file or from an external API. In that case you need to make sure that your data source provides the data you need in a format you can exploit.

In any case you cannot start filling the data before the data model of your app is done. If you start filling spreadsheet files "to win time" before the data model is ready then let me tell you that you'll be damn sure to do something wrong. Don't start filling your data until you have a nominal way to fill it.

Ordering development

Let's say your Spotify-like app has 3 main branches with the two core features in bold.

• User management
  • Login
  • Logout
  • Lost password
  • Profile management
• Music
  • Explore catalog
  • Search
  • Play music
• Billing
  • Credit card management
  • Receipts download

In which order do you make those?

You could split the thing technically and say that you create a music engine, a music catalog, a billing engine and then slap a GUI on top of that all.

You could also do things by order of absolute importance of the feature, so start with the billing then the music and the the user management.

But what happens in either of those cases if you're late and you have to release the project right now? Most importantly, what happens if you want to test what you did? In neither those orders features are self-standing. Your client can't test music playback if they can't find music first.

In your planning, put first things first, take the big blocks and put them out there. In that case, playing music and billing users. Then you need to make sure that you can reach those blocks. So before you start working on the music playback, you need users to be able to navigate to the music. And before that you need them to be able to login. However you don't really need them to be able to change password nor to fill their profile.

Supposing that, while billing is the reason why this app exists, the reason why people will use it is because it plays music. Additionally, you can release it without billing functional, as your commercial policy is probably going to include having a free trial period.

Considering all of this, I'd plan features in this order:

1. Login
2. Explore catalog
3. Play music
4. Search
5. Logout
6. Lost password
7. Profile management
8. Credit card management
9. Receipts download

This way, starting from step 3, you always work on a usable product. Certainly incomplete but definitely usable. And yes one of the core features is pushed to the end but that's a conscious decision taken with business.

Developing

Now we've reached the final part of this article. How to actually develop your software in a way that hits the deadline.

Stick to the specifications

Let's take a paragraph to emphasize on the importance of specifications. If specifications are not precise enough or if they move during the project, the following advices become very complicated to follow. The main if not only reason to change specifications midway is to reduce scope, as explained below. Bad specifications will double or triple the time of a project even if the most senior and battle-hardened developer is on that job.

One thing at a time

Because I'm a big John Wick project management fan, let's review what happens in this video from the beginning of John Wick 3:

As you can see, John Wick is being chased by mean guys that want to kill him. He gets inside a room where he finds old guns. He could do a ton of things but instead of panicking he just spends some time to create his perfect gun for this exact shot. And kills the other guy. He had little time, made a MVP and stayed alive.

The single most useful advice from this article is finish 100% of your task before moving to the next one. You cannot leave anything pending. You cannot have TODO comments in your code. You cannot have something bleeding waiting to be repaired. Once your mind goes elsewhere you forget things and they keep on bleeding forever.

Of course the feasibility of this lies greatly in the choice of what a task is. You need to make sure that your tasks are small and attainable. That they are sealed airtight and don't hang together halfway with another task. Sometimes it's difficult, sometimes interdependencies make the size if the task pretty big but I've never met a case where the task I have to do doesn't fit in my head.

Eventually, when the deadline becomes tight or simply if the company process are approximate, someone will lose their shit and ask you to go in two different directions at a time. You must ignore it and keep on doing one task at a time. In doubt think about this: if someone is running in all directions at once then they will forget whatever they ask of you because they cannot keep track of their nonesense. This way you can do what you decide and they won't be able to reproach it to you.

Another side-effect of this is that you should not write code that you don't need now. All the code that you write is focused to implementing the current feature and future features don't matter at all. Maybe you can keep them in mind to avoid shooting yourself in the foot and orient some decisions but certainly don't write any code for future features. You will loose your focus doing so and get confused. Those future features might even change before you get there.

The technique I recommend using is somehow inspired by the Pomodoro technique, however it is not time based. Rather, you should set a goal for the current step you are taking and do not deviate from it until it is done. When done then take a small break and move onto the next thing.

Testing testing testing

I won't go into any kind of automated/manual testing recommendations but one thing is for sure: you need to test the fuck out of your software. Test it in all the ways it's not intended to work.

Any glitch you see should alarm you. Try to reproduce said glitch and look at the code to see what happens. From personal experience, 90% of "weird glitches" turn out to be damn annoying bugs in production.

Also, you need to get the client or at least some project manager to test your code on a daily basis.

• By finding bugs early you make sure that you have a healthy foundation to work on. The more bugs you have the harder it is to see a single one!
• By having your project manager test your work you make sure that you stick to the spec and thus don't drive into a weeks-long tunnel of misdirection.

Real-life data

We've seen in the previous section that data must only be filled when there is a nominal way to fill the data. This also applies the other way around: don't ever test with mock data. You must test with data that is as real as possible in quantities that are as big as possible.

If possible, even test with larger data sets than what you expect in production, just to make sure that it fits.

The use of real-life data relates to the previous point about bugs. It's very easy to let bugs appears if you only ever work with the same set of data all the time.

Don't trip

Something that is also incredibly easy to do when doing software development is to trip on some technical difficulties and loose days for no apparent reason. This will happen quietly without you noticing, eating up hours of your time.

The best way to avoid this is to regularly review your work and take a step back. Maybe you do it with a coworker, your project manager, your boss but several times a week just look at what you are doing and try to see if you are stuck.

Another technique is that when you know you might get stuck because you are working on something new, it's always good to time-box your work. This, once again, gives you the opportunity to rectify trajectory before it is too late.

If you are stuck then maybe you can change approach, call for help, clear your head with a day off or anything that feels right to you but you definitely have to do something.

Dealing with a late project

The project management triangle is a well-known concept in project management. It'll make sense with what I just wrote above.

As we've seen with the Apollo program earlier, they had a scope, a deadline and a minimal quality (aka not killing astronauts). The reason why it ended up costing a lot of money is because it's the only adjustment factor they had to fit the box. That's also the reason why recent NASA efforts to go back to the moon will certainly not meet the deadline, as American taxpayers are probably not ready to support a money-eating monster.

Increasing cost

Adding human resources to a late software project makes it later

In spite of Brooke's law quoted right above, you can add resources to a late project but you have to be very careful. There is two considerations:

• It will increase the cost of the project, so someone is going to have to pay for this. Maybe your client, maybe your company, maybe a shared responsibility, that's not really the concern for developers. What matters is that the cost increases.
• It will slow down the current team because more people to talk to equals to more time spent not coding. This is why if you do that you must have a project that is already properly loosely coupled and have the new developers to work on those remote parts that don't really interact a lot with existing parts. You want the extra workforce to be able to compensate the productivity loss.

Increasing cost is an option but it has to be carefully used. In a lot of cases and especially if the project is small it's not possible to do so.

Increasing time

Pushing the deadline may seem like an intuitive option but there is actually a lot of problems to be considered before doing so.

• Is the client ready to accept a late project?
• Does your company have the bandwidth for this extra time spent on this project?
• Considering that you are going to do free work rather than paid work, it costs your company money.
• Dragging a project is very bad for the team morale, especially if it's more than 50% above the initial estimate.

Like the increase of cost, the increase of time has some drawbacks, including actually the increase of cost. That's really not a scenario you want to see happening.

Decreasing scope

The good news is that as advised above you've done things one by one and in a meaningful order. Which means that every single commit is a finished product. Maybe not all commits are equally tested but at least the intent is that every commit is a finished product.

Which means that you can, right now, push your product into production and not worry that some parts are bleeding. Everything is all right.

Now that's a big clean scope reduction however that is not necessarily the only leverage you have. Indeed, during the specification process some things were specified in a certain way but maybe if you think about it with the current experience of the project and a developer eye you can find shortcuts which would impede a little bit on the UX but would allow you to finish on time.

In short, when you are late, two options offer to you:

• Cut the scope at the current advancement status at the release date. Maybe you will finish those features later, maybe not. It's down to commercial negotiations.
• Simplify the remaining features so they are potentially less user-friendly but faster to complete.

Let's put this a different way. What is on the paper you signed with your client is the deadline, the price and the broad scope of the project. Meaning that as long as your project does what it has to do, for the price that was agreed and before the deadline that was fixed then the fine details of the specifications are by far the most negotiable part of the project.

Considering this, when your project is late you should probably find a way to cut down the scope because it is by far the less painful way of crossing the finish line.

Conclusion

While estimating and planning are complicated tasks, it is part of our job to accomplish them. In fact, if we call ourselves software engineers we should be able to hold the same level of accountability than other kind of engineers. If a bridge can get built on time then software should be deliverable within the deadline as well.

This starts with making rough estimates at the beginning of your project and then narrowing down the plan as the specifications and code get produced while making sure that the budget and deadline are still consistent with what is being actually done.

A pattern we've seen emerging at different stages of the project is that if you want to deliver your project sooner you have different adjustment variables you can play on but the safest and most efficient is definitely to play on the scope.

The most important lesson is to carefully arrange your tasks and to accomplish them one by one so you keep the complexity of your work manageable and your project in a healthy shape.

In the end our developer work is often more landing page than robots and lasers. I see many juniors that are disappointed by this outlook and many seniors that just hope to move into management. However our industry is plagued with projects blowing up their estimates. Then if our job is so easy, let's try at least to deliver it on time.

---

Acknowledgements

I used the following articles as inspiration and proofing for this article. There is a lot of content that I don't endorse 100% but they make other interesting reads.

• Tradeoffs Of Time Estimates — Professor Beekums
• On dealing with deadlines and estimates — Eddy Ernesto del Valle Pino
• Project Management for Developers 101 — Yaakov Ellis
• How to move to #noestimates — defmyfunc
• How long will it take? Git knows. — Dimitrios Melidonis
• Ways to #NoEstimates — Jay Bazuzi
• How we used the improvement kata to gain 25% of productivity — Philippe Bourgau
• On Estimating Things — Terry Brown
• Software Estimation, Enterprise-Wide part 1 and part 2 — Vitalie Temnenco
• Risk Management Is Project Management for Adults — Tim Lister
• Why software projects take longer than you think – a statistical model — Erik Bernhardsson
• Extreme Programming - A gentle introduction — Don Wells