4 reasons why the “10x Developer” is so controversial

The notion of the “10x Developer” stems from an infamous study that measured and compared the productivity of programmers. The study found that although the programmers had roughly the same amount of experience, (7 years) some programmers were far more productive than others.

In fact, the best programmers averaged more than 10 times the productivity of the worst. Hence, 10x. (note: I said average, debugging time was a ratio of 20:1!)

These radical results have been a source of debate ever since.

I’m very interested in assessing and improving my own skills so I took a deeper look to see what exactly was so hot about this topic.

It boils down to these 4 things:

4 Reasons

Note: links below!

#0 Productivity is hard to measure

The original study tracked several measurable abilities, things like program size and time-to-debug. These things are easy to quantify, but what about the things that are harder to graph?

Here are a couple questions that the study didn’t address:

How maintainable was the code written?
How elegant were the bug fixes?
How well did the programmers deal with murky requirements or requirements gathering?
Were the solutions user friendly and accessible?

Do you consider these abilities to be part of what it means to be a programmer?

#1 Tiny sample size

The original study only looked at twelve programmers, over the course of a handful of hours. That is not nearly enough people to look at, and it also tells you a lot about the types of problems they were working on. How might the results of the study have changed if the test looked at larger tasks or tougher bugs? Would the top performers have burnt out at their accelerated pace?

Despite the small sample size similar studies have corroborated the findings. Moreover, there have been similar studies in other fields that have reported similar gaps in abilities. Lots of studies. (scroll for links!)

So sure, the sample size is terrible…but does that mean it’s incorrect?

#2 Well, common sense….

Are there really programmers who can out produce 10 other developers? Granted, a single person may be able to out-maneuver a team due to communication overhead – but this study measured at the individual level. Can one professional programmer, with the seven years experience, really out perform the independently accumulated efforts of TEN?

Of course not….right? Well..The study didn’t report that the 10x developers did 10 times better than the average programmer. They measured 10 times better than the worst. It doesn’t take much to imagine a scenario where this could be the case, and besides…wouldn’t one of the further studies have debunked this by now?

#3 The study was done over 50 years ago

A lot has changed since 1968, particularly in the realm of programming. Programming techniques, tooling, and languages have matured a lot since then. C hadn’t even been invented yet, would the results have been different in the programmers had been using a more modern toolkit?

Well, like above studies have been done since then and this study was intended to study the programmer – not the program’s productivity. The question isn’t about how productive the technology stack is, it’s about how productive the programmer.