Software 2.0?

Recently an article by Tesla’s Andrej Karpathy titled Software 2.0 circled the techsphere. The author was of the opinion that neural networks are a fundamental shift in how we build programs and that this is an interesting development which warrants a web 2.0 style name.

Now I agree that neural networks are super and the whole ML approach to a lot of “human hard” task such as object recognition, speech synthesis, translation etc is the right one. And this is both from a theoretical point of view as well as a practical “this is what gets results” point of view. The fact that they’re relatively easy to pick up, as opposed to some other well known tools in the ML arsenal (ahem, looking at you kernel methods, Bayesian methods, Gaussian processes etc[1]) is just the icing on the cake.

And I also agree with the title itself. I do think that we are heading towards a new type of product, with vastly increased capabilities and a different sort of human interaction for it. Some of them have almost magical powers. I’m thinking of self-driving cars and conversational interfaces. The former will radically alter transportation in the world, while the latter seem poised to be the third wave of UIs for programs, devices, and things, the same way command lines and GUIs were in their time.[2]

But I don’t think neural networks are radically changing the way we built these things. Sure, they fill in some pieces of the puzzle we didn’t quite have yet. But we’re a long way away from building something like Excel via a neural network.

And until we actually have software building large pieces of software, with loads of state, not just the pure functions we’re approximating right now, I don’t think we can speak of software 2.0. But then again, when we do have that sort of capability, perhaps the world as a whole won’t care about mundane thins such as spreadsheets, image sharing or digital payments.

[1] as an aside, isn’t it interesting that the more CS/AI focused methods have a low barrier to entry. Not just neural networks, but also nearest neighbour, tree models, and to a certain extent graphical models. Whereas statistics inspired methods are heavy on the math and relatively hard to pick up. Not that I think having a theoretical understanding of a tool is not a good thing, but it does limit the people that will adopt the tool to those with (1) the background and (2) willing to put in the work to understand it.

[2] even for regular line-of-business applications this seems to be the case. I was recently surprised to find out that in Google Analytics you can do various queries and reports by asking for them. So you can say “What’s the most visited page this week” and sort of get an answer.