It's been more than 70 years since we, the community of software engineers/developers, have been programming computers by writing code as text. Many interesting programming languages have been invented, and many methodologies around those programming languages have been introduced. Still, after all these decades of focused and concentrated programming by writing code the way we write any other text, typical results are disappointing (to say the least).

Flimsy software

I don't want to sound gloomy, but ever since I started software development (back in 1990), I've been involved in, or seen too many software development projects that were seriously substandard. And I'm choosing my words carefully here, because I am tempted to use much stronger, more colourful language when assessing the state of typical software projects.

Of course, I hasten to add that I myself am not immune to this shameful score. When I go back and look into some code I've written 30, or 20, or even 10 - 5 years ago, I cringe.

I also cringe when I peruse countless other repos (some publicly available on GitHub, some of proprietary nature, reserved for privileged eyes only).

Overall, not a pretty picture. Poorly written code, buggy and super difficult to scan, difficult to read, to reason about, to refactor and to fix, seems to be plaguing our industry. This flimsy software that is so prevalent in many places is causing serious reasons for concern. Yes, some of those apps are frivolous, they may help people upload an image of their cat or create a meme or some other inane social media gimmick, but many other products deal with more serious matters.

I spent some time working on projects for health care institutions. I was appalled at the shockingly bad state of the software code one sees in those half-baked repositories. Same thing after spending a decade in the fintech sector. We are talking about software that automates processes that seriously impact patients, their health and well being, as well as citizens and their financial stability.

What is causing such poor state of affairs?

There is no lack of expertise

I cannot with a straight face sit here and claim that our profession is a young one, a baby, an infant that is yet to learn its ways. We've been at it for several generations already. Plus, the high quality of many easily accessible resources, where top experts offer amazing guidance on how to do software properly, has been available for decades.

Experts of the likes of Kent Beck, Ron Jeffries, Uncle Bob and many many others, have generously offered copious resources, teaching us how to do software the right way.

Seemingly to no avail.

So we cannot blame the lack of readily available expert guidance for the appalling state of software development profession. After 70 years at it, and after seeing so many products and repos ending up a cesspool of terrible code, it's time to take stock of the situation: what's going on here?

Coding is an exercise of frustrating indirection

The reason we write software code is simple: we want to automate data processing. We write computer programs and in the process we are hoping to instruct computers on how to process/transform data.

But we don't know how to talk to computers. Those machines are opaque to us. In the early days, computers were a bit more approachable, as early engineers were manipulating them in a physical fashion -- rewiring their circuitry.

That was quickly proven unwieldy, so scientists invented symbols to stand in for those physical rewiring. Those symbols (i.e. computer code) were entered by programmers in the form of text.

Entering some text and hoping that computers will understand it was a fool's errand. Nothing like that was possible back then (nor does it seem possible even today), so it was necessary to introduce a level of indirection. That's when compilers were born (and later on interpreters). The job of a compiler/interpreter is to process the text entered by programmers and try to translate it into a set of machine instructions.

That approach created a lot of frustrations, to the point where the community came up with a slogan (obviously written in utter exasperation): "Computer, do what I meant, not what I told you!"

Of course, computers are dumb lifeless mechanical/electronic contraptions, so they remain impervious to our pleas.

But the fact remains that indirections are always frustrating. If I know what I want, and instead of doing it myself I have to try to explain it to some dumb machinery, in the best of cases that would end up being quite awkward. In the not so optimistic cases, such indirection becomes exasperating, leading some to the verge of desperation.

For example, if I want to position my subtitle on the web page to a certain area on the rendered screen, I cannot directly manipulate its position in the browser. Instead, I need to find and open the appropriate cascading stylesheet file (CSS), and try to understand where in the almost always large text file is the text that talks about position the subtitle (or any other visible element).

Once I locate that snippet of text, I need to modify it in such a way that it can then instruct some mechanical machinery (in this case, a web browser), where on the screen to position that visible element.

I can't begin to tell you how many times I went through the frustrating exercise of making a change in the CSS file, reloading the page, and seeing either a completely wrong position of that element, or, worse yet, seeing NO CHANGE on the page!

Extremely frustrating. Why? One word -- indirection.

Instead of manipulating code, manipulate the data

To avoid this problem with indirection (skip the middle man), it would make much more sense to bypass all that charade and directly manipulate the data.

For example, instead of second-guessing what some modified text in the CSS file may change on the rendered page, it would make much more sense to me to simply go and directly position some visible element on the page, and then let that direct manipulation of data reverse engineer the CSS file and persist the change.

Similar to that, any other data processing (usually governed by the business policy rules) tends to get very convoluted very quickly if we are trying to implement it by following the indirect model (i.e. write some text and then hoping that everything gets translated properly so that machines could execute the policy rule the way we intended it). It would be much better, less frustrating, less risky, more productive, to simply directly manipulate the data and let the machinery observe what's going on and then reverse engineer our actions and produce the text, or machine code, that will correctly implement desired rule.

Need for innovation

As I've mentioned, after 70+ years spent in the land of code-as-text, and after not being able to make any progress (still suffering from incredibly unreliable, buggy and crash prone apps), it is high time to slow down, assess the situation, and start cutting our losses.

Yes, countless engineers have been rigorously trained to only think in terms of code-as-text, but look where it has gotten us. Isn't it time we start entertaining other possibilities? Obviously, current strategy is not helpful. We are stuck in some dogmatic dream revolving around text that is somehow supposed to become clearly understandable to machines. The innovators among us must now wake up from that dogmatic slumber and invent new, more reliable, less frustrating ways of building software apps.