The Art of Defensive Programming

As a programmer, there are times where you have to dig deep to find a simple solution for a complex problem.

For instance, my boss had come to me, pointing to a problem on a customer-facing web portal. I did deep into the code for it, pulling out error messages logged in various systems... and I see something odd in one table. So I ask if something was added... and there was. But it wasn't fully added... and expectation on if some of the data in the table had to be there or not was faulty. In this case, it was a description of a new location the company had developed. Add a description in... and the portal problem resolved itself.

In times like these, I am reminded back to an article written by David Small in STart Magazine back in the summer of 1986. The magazine was for Atari 16-bit computer enthusiasts, starting with the Atari 520ST. The article was titled "Voodoo Computing: The Pragmatic Art of Defensive Programming".

Nearly 40 years later, a programmer myself, I feel it's time to give that old article a refresh with my own perspective. I borrow a bit heavily from that article, because many of the points made still hold true today.

But like Mr. Small has done before, we must for the moment examine a typical modern computer... and how truly we're lucky it works at all.

Refresh my memory on this

The first really big thing to realize is that memory (unlike storage) is as Mr. Small said, "a frighteningly transient thing." Most every computer, from your PC and the servers you're talking to down to your phone and tablets, use what is called "dynamic RAM". DRAM allows us to pack a lot of memory into a small package, but at the trade-off of having to refresh it with additional circuitry -- at millions, if not billions, of times a second.

It used to be that the CPU inside the computer had to do this, but now, it has dedicated circuitry called a memory controller that does it for the "core" of the CPU that does the actual work. The CPU also has it's own internal RAM stores, usually smaller "static RAM" that doesn't need refreshing and are used for caches -- namely because reaching out to the memory controller and out to that fancy DDR5 uber-clocked DRAM is still damn slow!

It's also still susceptible to power failures, lightning strikes frying the circuitry, cosmic rays, and even a rouge magnet coming around and flipping a few bits of data. It's just that... well, actually, most of the time the RAM is being used. If not for actual programs and their data, but as disk caching.

Intel back in the day thought that error-checking RAM was too expensive, and made it optional... and when it was revealed that it happened, there was a bit of an uproar because such RAM would stop crashes.

Think of it this way: It's over 16 billion dominoes stacked on top of each other. One push, one external influence, and down it goes.

And then there's storage.

Do you have a backup?

You got two types of storage: hard disk drives and solid state drives. Hard disk drives store data by employing tiny tiny magnetic fields on spinning disks, contained in a case in an attempt to stabilize them as much as possible, even though they're spinning tens of thousands of revolutions a minute. Any vibration, even from another drive or you just walking in, can send a shock wave that knocks the head off kilter and the drive has to slow down, recover, and reread (or re-write) a good chunk of data, slowing it's response down. Even then, the drive's discs can lose those magnetic fields, lose the capability to host those magnetic fields, or even have the read-write head used take a nose-dive right into the disc.

Solid state drives fare differently. They remember the data for a long time but every write damages the flash chips used. Too many writes, and the chip is dead. This is why SSDs use techniques such as "write leveling" and "sector remapping" to try to extend the life of the drive as much as possible. Sure, reading is fine... but do you want to trust your only copy of a project to something that can turn from read-write to write-once-read-never?

There are other things that Mr. Small does talk about that are still relevant, but we tackle by adding more computers into the mix. Keyboards, for instance, have a tiny CPU in it, a microprocessor, to handle each key press "bouncing" and sending signals erroneously as the key settles back into it's upright (and hopefully not locked) position. CPUs themselves run "microcode" and some even have tiny CPUs that can monitor and correct errors. Monitors have CPUs now. Some mice even have them.

It's turtles all the way down. And occasionally, something goes wrong, through no fault of your own. Take Andy Ihnatko's common comment that humans "are but imperfect vessels for the perfection of the universe", add that computer demand perfection, and you end up what Mr. Small calls the "ultimate question of the Frustrated Programmer": Is it my code or my PC?

Mr. Small laments that "something wonderful, childlike, and native dies in a programmer the first time a computer genuinely screws up. Up to that point, you are sure the problem is yours, somehow, somewhere. Afterwards, you are never the same. It's like learning the truth about Santa."

So what can we do? What can we follow as sane people?

Mr. Small says to treat working with computers as like performing sorcery. Programming is like casting spells. Take defensive measures to not let the demon out as you bind it to do your bidding, and you'll go far. To that, he describes ten maxims to which I'll expound upon:

Maxim #1: When you're having a bad day, stop working.

I've had many of a time when working on a project involving various Microsoft projects where I know I've coded a routine right. I could find no bugs, even from the examples provided... but somehow, it wasn't working.

But if I put it aside, get up, get a snack or something to drink, and come back to it... it's working. I didn't change a thing. Some cache must of gotten flushed or the system decided to actually run the code I had written instead of a different version five changes back.

The thing here is to take a break. Stop hammering the system. Let it do some idle maintenance. Save your work, save a backup, and go do something else or something useless. I hear social media is a useless thing...

Maxim #2: Comment your code to death

I've had too many times where I was hyper-focused on one thing... and then get interrupted and lose the train of thought. How can you recover from an interruption quickly?

Comment your code. Even a high-level set of steps help. It takes a minute more but saves tons of time, not only for you but the next person who takes over your project after you've left it.

Maxim #3: Steal from the best

Corollary: Googling a problem to find some code that works is good.

Lets face it. A lot of times, you're writing code but using a library that handles the heavy lifting of interfacing with some site. Other times, it's a problem that you're getting frustrated over... but someone's solved it already.

Good programmers know this. They're happy to share their code, and look for others code to pick up their tricks. Maybe there's a better way that's faster.

So let other people look at your code, and take in their input. You may find other bugs that way.

Maxim #4: Use the best tools, and be prepared to pay for them

Seriously. Your time is valuable, and good tools are worth it.

Back in 1986, Mr. Small was making an Atari ST think it was a Macintosh. It was 7000 lines of Motorola 68000 assembly language, or about 120 letter sized pages. It's not a job for a slow printer... nor a job for a slow floppy-disk based compiler. Hard drives and RAM drives (virtual drives in RAM) were a must back then.

Now, we have Visual Studio with how it type-checks on the fly and catches common mistakes. Rarely have I had to go back and correct a mistake where I capitalized a variable in one place when I didn't some place up. We have plugins for Visual Studio that even do deeper analysis. That is well worth it.

Maxim #5: Backup your work.

I've had too many times where I had to save off what I was doing, switch to a different version of some code that had to go in the exact same place in order for it to compile, then switch back. Some of this code can't be put into a repository for some reason or another (maybe even too soon), so... I ZIP up a copy beforehand.

And then you have the time that your PC dies. I had that at a previous job where the hard drives were "just a bit" flaky. Ether I rebuild my work on a new hard drive or I try a recovery program called SpinRite. The latter worked, but afterwards, I backed up everything to a network drive.

Back it up often. You'll going to need it later.

Maxim #6: Keep three backups.

In this hyper-connected day and age, losing your work is essentially doing it twice. You don't have time to spend manually recovering your work, especially to folk who encrypt it for ransom and happen to be on your network because Bob in Accounting thought that cutie in his email was "safe".

No. Save a copy off on your network file share and make sure there's a copy of that off-site. Working on it off of a Git repo? Commit a copy on a remote server. It'll be needed.

Maxim #7: Don't be clever.

Seriously. You don't have time to be clever. The compiler takes care of that for you in most cases, but only if you let it.

For instance, C# has the "var" declaration, shoving any type-checking to run-time. Sure, you can "var" here and "var" there to have it assume the type of a returning function, but that returning type can change and if it's not what you were expecting (because you hid that from yourself by using "var")... well, there's your problem.

Besides, half the time you won't remember that one clever trick anyway... and that trick may not work in the future when someone has to debug your code because something changed. You did comment it, right?

Maxim #8: If it works, don't fix it.

The compiler doesn't care how it's formatted. If it's a kludge but it works, leave it alone and do something else.

Do you need to sweat the indenting, the pretty printing? A modern IDE like Visual Studio will do that for you for most languages, and some like Python they'll assist you on getting it right. Let the computer be the neat freak. If you insist on being the neat freak, stick to Python, which mandates it.

Maxim #9: Write defensive code.

Mr. Small says "Always give your code the maximum chance to work. Or: It'll always thing of something you don't."

Remember my example at the top of this article. Other programmers looked at the table in the past and saw the description always had data in it. But the program that manages the table allows it to be flagged as void of data. That flag is called "NULL", and no matter what programming language you're in that has such a concept, it's always something you have to pay special attention to.

Those other programmers assume it would always have a value. When it didn't it threw an error that resulted in a bad user experience.

Using variable? Initialize it when you declare it. Watch if you reuse the variable for something other than what you intend for it. And watch for special values (like NULL).

Maxim #10: Keep it simple, stupid.

Mr. Small said "Structured programming is useless in the real world. You don't need to program in a structured way."

To be honest, I think that falls a bit more under an old Navy saying: Keep it simple, stupid. UNIX philosophy follows this with "Do one thing, but do it well."

I see a violation of this numerous ways. For instance, a simple "pull data from a vendor server and store it in a database" program. Do you need to wrap the core of the program in one object library while calling it from a runner program? Or how about doing the pulls and pushes as calls through a set of web services when you have direct access to both ends? How about calling an endless set of routines down to only make a web service call to a second back-end server that pulls the data itself, but is only behind a firewall?

Do you really need to spend all that time with all that "plumbing?"

The more complicated the program is inside, the more problems you'll encounter, and the more time you (or whomever is debugging it in the future) waste when something does go wrong. Keep it simple and straight-forward.

In conclusion

Thus ends my update. If you've learned something from this, you'll already be on your way to a better programmer... or at least make sure the next one won't be trying to track you down and strangle you for how bad your code is.