DEV Community

Cover image for Why developers are SO sh*t at estimating!

Why developers are SO sh*t at estimating!

Luke Garrigan on August 10, 2019

The 10x striver: A short story, about you. Chapter 1 - Planning You're a programmer, you're in the middle of sprint planning ...
Collapse
 
bootcode profile image
Robin Palotai

We also estimate the happy-path, solving if everything goes smooth. When was the last time you said "this should take 5 minutes" and there you are 2 hours later, installing drivers and reading arcane config syntax..

Collapse
 
lukegarrigan profile image
Luke Garrigan

Yeah that’s so true, I’ll be honest, it happens far too often in my case 😂

Collapse
 
bootcode profile image
Robin Palotai

Let's break it down a bit. The process looks like this:

  1. Oh, this looks exactly what I want, and sounds easy.
  2. Start reading docs or installing package. At this point we are hooked, and likely to go on.
  3. First hurdle comes, but now we have invested. Why leave now, success is so near. Like when waiting for bus - I won't leave now, I have waited for long, it surely will arrive soon.

The first sign of barrier should make us stop and think. When we branch out to 3 Chrome tabs, following how to configure various deps. Or get a result from a call that is not expected according to the docs.

I'm not saying we should throw up the white flag, but we have to ask ourself: This seems rather 4 hours instead the 10 minutes I hoped. Is it worth the investment, or should I stop now?

Collapse
 
jaloplo profile image
Jaime López

You are right, totally right with your article. I'm currently in a position where I have to estimate a lot of projects/solutions and estimating the time is something that makes me crazy. Overall because I could be the person in charge of that project and it's completely possible that I forgot some tasks or constraints.

In my company, we only estimate the effort for coding/configuring phase and then complete the rest of the phases (analysis, design, testing, ...) as a percentage of it. For example, I estimate implementing an API could be 5 days of coding, the testing phase will be 80% of it so we will spend 4 days. The whole task will be the sum of all these phases getting a final time of 9 days for this case.

In addition to this, we set a lot of assumptions to avoid forgetting anything. Think of a web application you estimated creating a simple user interface but that doesn't mean anything to your customer. You need to add some assumptions for limiting the scope like: no CSS rules will be developed. This is something the customer will understand or, at the end of the day, you will have as a shield to defend your position.

Collapse
 
lukegarrigan profile image
Luke Garrigan

Thank you for this, I love hearing how people take different approaches to estimate tasks. Your approach is very interesting, we do a relatively similar thing at work although we have Quality Assurance engineers who estimate the effort for testing and that gets bundled on top of the overall estimate.

Collapse
 
jaloplo profile image
Jaime López

Thanks for this. QA team is a good thing to test the whole product but there are a lot of tests in a project lifecycle. You have to include some effort for unit testing, integration tests, smoke test, user acceptance tests, etc. So, there should be something in the estimation to cover testing saying what kind of test are included in the estimation so you can adjust the percentage of the effort of the team.

Do your estimations include the time QA team will spend in the project? This is something you should consider to include inside or outside your estimation and you should co-work with your QA mate.

Collapse
 
dillieo profile image
Sean Patterson • Edited

We take this approach too, but also add another batch of time for iterations. How many times does the mock-up not do justice to the task or a fringe case wasn't caught in the original API design? Those things we take into account for as well when we estimate.

Collapse
 
jaloplo profile image
Jaime López

I learned that estimations are a value we said based on experience, calculus or any other technique we used. So, if a customer requires an API implementation for their business logic, we will estimate the implementation of a number of services and their complexity, let's say 5 services with medium complexity, and not the final solution - a description of what each of these 5 services will do like a black and white box. In the assumptions section (or any other place), we will define what means "medium complexity" so fringe cases will be considered out of the scope of the current project and we should include a note telling need an assessment to give a proper estimation for that issue, problem, requirement or system.

We, as estimators (that exists?), need to limit what we are estimating and considering. We cannot give the final solution in the estimation so we are not able to include fringe cases if we don't study them.

Collapse
 
bootcode profile image
Robin Palotai

I liked Making Things Happen (aka The Art of Project Management), which despite the title, is a totally BS-free book about overseeing a project. Has great advice about estimation. You can pop it open anywhere your current interests are.

Collapse
 
vimmer9 profile image
Damir Franusic

In my case, it's because I'm a perfectionist most of the time, and refuse to do ad-hoc solutions. That being said, I tend to always under-estimate the time needed to finish the project.

On the other hand, when I'm laser focused and free of distractions, I seem to be able to summon a huge amount of coding mantra, and produce more quality code that I ever expected to be possible. To paraphrase the last sentence; I sometimes still manage to surprise myself, even after 20 years of coding.

Collapse
 
lukegarrigan profile image
Luke Garrigan

I love that feeling, somedays you feel like you could just take any ticket and code it up in a few minutes, embracing that inner 10x.

Collapse
 
vimmer9 profile image
Damir Franusic
Thread Thread
 
jaloplo profile image
Jaime López

Me too!!!

Thread Thread
 
zkriszti profile image
Krisztina Závecz

Same here! Trance, deep house, lowkey tech... 😎🎵

Collapse
 
elasticrash profile image
Stefanos Kouroupis

When it comes to frontend nowadays I feel like I am really good at estimate tasks. We estimate a task and it's risk separately. For example this might be a half a day task but it has a big risk (two days). So it's fine if that task spans for 2.5 days. Also modern frameworks like angular made predictions even easier due to their structured nature

Collapse
 
lukegarrigan profile image
Luke Garrigan

Yeah, I agree with this, to be honest, a lot of my dodgy estimations are because of the unknown of the backend, but sometimes the frontend does catch me out!

Collapse
 
firozansari profile image
Firoz Ansari • Edited

Alternative:

...
The project is actually delivered in 4 months.

  • Customer: But it's a sh*ty deliverable.
  • Seller: It's a sh*ty deliverable.
  • IT: But you asked to deliver 8 months work in 4 months.
  • Seller: Can you fix it?
  • IT: Sure, we should fix it in 4 months.
  • Seller: Nah, we could do it in 1 month
  • Customer: Ok, let's start.
Collapse
 
lukegarrigan profile image
Luke Garrigan

Spot on!

Collapse
 
firdiansyah profile image
Akbar Firdiansyah • Edited

Moral of the story.

Great SELLER!!!

Collapse
 
jeikabu profile image
jeikabu

Back when I was a wee CS student a professor I respect a great deal told us to estimate the time and then just double it. At the time I think a lot of us only half-listened to all these "old guys". Now that I'm approaching old guy status I accept most of what they said. 🤣

That said, there's a reason why many companies/projects don't have precise, external roadmaps as to what's coming when. It's easier for things to slip than predict a hard date. Better to delay than "ship" something not ready.

I'm convinced it's more witchcraft than science, really.

Collapse
 
lukegarrigan profile image
Luke Garrigan

Haha the older you get the more you realise that these old buggers actually made some sense 😄

Collapse
 
phizzard profile image
Phil Tietjen

At a previous job, we used to estimate time and log time on our sprint tasks. I am very guilty of underestimating tasks from optimism, pressure, and trying to impress. Eventually I got better by just adding a padding of time from my actual estimates, but if I went over that then I would feel super poopy.

These days where I am uses points and so far it's been far less stressful being constraint to an approximate amount of work than being constraint to a time to be done.

Not that points is the answer but it's been better for me!

Collapse
 
lukegarrigan profile image
Luke Garrigan

I've used a point system also but we end up converting the points into days anyway. Like 1 point would be half a day, 2 points is a day, 3 points is 2 days, etc. I don't know whether that's just company-specific, but I like the idea of using points without the conversion to time!

Collapse
 
phizzard profile image
Phil Tietjen

A lot of it I'd imagine is company-specific. We don't do straight time conversions because points have to account for initial work, work post PR to address comments, and QA.

It's been an adjustment but it's been pretty good so far.

Collapse
 
lankydandev profile image
Dan Newton

Are you trying to describe me??

I need to improve on this as well. I once estimated a 5 month project could be completed in 1, maybe 2 months tops. Yeah, that didn't work out very well.

Since then I have tried to be more pessimistic when estimating.

I have not got much better at it, but I haven't got worse as least 😊😊

Collapse
 
lukegarrigan profile image
Luke Garrigan

😅 that's brilliant, I've never had to estimate an entire project but I can only imagine that's 10x worse, so, so many unknowns!

Collapse
 
leoat12 profile image
Leonardo Teteo

This week I was given the task of solving a bug on the application front-end. It looked simple enough so I said when I asked that you should be solving by tomorrow (Monday). Well, I still have no idea how I'm going to solve it. It is basically some bug around Samsung smartphones' keyboard predictive text that makes the user input act like crazy. I tried a lot of things already, but it seems Samsung was very crafty as to how to turn developer's life like hell. u.u

Collapse
 
codemouse92 profile image
Jason C. McDonald

The planning fallacy only occurs when planning for our own tasks, interestingly, us humans take a much more pessimistic approach when estimating for other people.

Not always. I've known a number of cases where bosses, clients, or even co-workers set unrealistic deadlines for others. (You even mentioned a few such cases.)

By the way, I've written about this phenomenon as well, and broken it down into several root causes and how to fix them.

Collapse
 
lukegarrigan profile image
Luke Garrigan • Edited

Yeah of course, as I mentioned with Elon Musk setting unrealistic deadlines!

What I meant is that if someone from a 3rd party standpoint were to make an honest estimation of time for a particular person to do a task it would likely be more pessimistic.

Collapse
 
stealthmusic profile image
Jan Wedel

It’s important, that when you need to fix estimates in a team, most probably the root cause lies in management. Estimates are estimates, but this is hard to understand for managers. So, the most important part is to acknowledge that even if estimates get better, the work won’t be done faster.

If that step has reached, based on my experience it works pretty good to use points and fixed length iterations. Once team (and it should alway estimate the whole team) has a good common sense on how many points for a comparable task, you just change the meaning of what a point is, based on how many points have been successfully delivered in the iteration. You dont‘t try to improve the estimations itself.
There will still be times where you will still need a week longer than planned. But that’s the way it is.

Ah, and breaking something up in subtasks can help, but it’s not worth to do that upfront if it’s not clear that the task will be done at all. In that case, simple team complexity estimation is sufficient.

Collapse
 
lukegarrigan profile image
Luke Garrigan

I particularly like the part about even if estimates get better, the work won't be done faster, I'll have to use that next time when I'm trying to blame the bad code for me taking so long😄

Collapse
 
stealthmusic profile image
Jan Wedel

Maybe you have to work on your communication, too. If it’s late, than only because we need to meet our company’s quality goals. 😉

Collapse
 
christiangroeber profile image
Christian Gröber

I honestly don't feel too much like what you described - I know that I have the tendency to underestimate how long something might take me, so I just always go wa, way higher than that, so that I can be sure that I'll be done in the time I promised. Now, that might sound great, but I honestly believe that that's what cost me my last web development job. They went with someone else, and even though it took them just as long as I estimated it would take me, me saying it would actually take me that long tipped them off.

Collapse
 
kayis profile image
K

Biggest problem is usually changing goal posts.

I think I estimate pretty well, but the stuff we agree on at the start is never 100% of what I'm actually end up doing.

But I could imagine, if you're a dev for a corp das does everything waterfall style, and not for a startup, you could get the impression that estimation isn't that of a problem.

Collapse
 
s_aitchison profile image
Suzanne Aitchison

I think we also often fail to take into account "coordination overhead" - e.g. when my task depends partially on understanding or agreeing an approach taken by someone else working on another task.. or liaison back and forth with a client or designer. Sometimes these chats and slack convos end up taking a decent chunk of time and I think we're pretty bad at factoring it in!

Collapse
 
lukegarrigan profile image
Luke Garrigan

Completely, that's something that didn't even cross my mind! Thank you Suzanne.

Collapse
 
william1530 profile image
william1530

I rather overestimate than underestimate. We use points to estimate in the team. If the estimates are far apart then the lowest and the highest explain their position, after which we estimate again. Of course some stories can take longer based on uncertainty but I don't feel I have to estimate low in order to impress my team.

Collapse
 
lukegarrigan profile image
Luke Garrigan

You're an exception William, maybe you're one of those 10x's 😄

Collapse
 
alexparra profile image
Alex Parra

Hey Luke!
Nice write up. I’ve faced this so many times and it’s nice to know I’m not the only one.

Also thought you might Michael Wolfe’s answer on this:
quora.com/Why-are-software-develop...

Collapse
 
lukegarrigan profile image
Luke Garrigan

Thank you, yeah you're not alone! 😅

haha, I loved that write up, thank you for sharing!

Collapse
 
eekayonline profile image
Edwin Klesman

This is a solid topic. Thanks for writing it! 🙌🏻

I've created a 3 point estimate Excel.
When I need to estimate effort for a development project I use it to get a more realistic estimation.
It even has a risk indicator that adds an extra percentage depending on the risk level, etc.

It basically allows me to write down my best case, realistic, and worst case estimation for a (sub)task.
This is less overdone than a X times multiplier but it doesn't overcome the fact that experience/historical metrics, and the need for objective estimations still remain very important.

Collapse
 
theodesp profile image
Theofanis Despoudis

One rule I use is, whenever I give a hardcoded estimate, for example half-day work, I double that to be sure, so it becomes a days work.

Collapse
 
_craigmurray profile image
Craig murray • Edited

So true, and not just for developers, this is the case for many many day to day tasks!!

Collapse
 
lukegarrigan profile image
Luke Garrigan

Thank you, yeah it's definitely a universal thing! 😃

Collapse
 
lukegarrigan profile image
Luke Garrigan

Yeah, I agree!

I do wonder though if, for instance, the person in charge of the project used to be a programmer and they think to themselves: I could do it in half a day.

Collapse
 
remast profile image
Jan Stamer

I am all for not estimating at all. See Beyond Estimates by Woody Zhill.

Collapse
 
lukegarrigan profile image
Luke Garrigan • Edited

Oh me too, many companies enforce it though.

But yeah, If I had it my way I'd never estimate again.

Collapse
 
adam_cyclones profile image
Adam Crockett 🌀

3pts ... Oh

Collapse
 
jjtriff profile image
jjtriff

Uf, I'm so glad to have read this. It happens to me all the time.

The tips are great, btw.

Thank you so much ;)

Collapse
 
lukegarrigan profile image
Luke Garrigan

Haha that's good to hear, we're all in the same boat!

You're welcome 😃

Collapse
 
asifnaeem1 profile image
Asif Naeem

Absolutely right, Im still struggling with that problem. If I plan to complete a task in one day, it takes 3 days. Even I try my best to follow the plan

Collapse
 
lukegarrigan profile image
Luke Garrigan

Haha love that

Collapse
 
darkes profile image
Victor Darkes

That's why I'm glad at my work we have the full team saying how long a story should take and agree on the work required.