This article was originally posted on November 26th 2018 at: https://nickjanetakis.com/blog/optimize-your-programming-decisions-for-the-95-percent-...
For further actions, you may consider blocking this person and/or reporting abuse
The other day, while working on someone else's machine, I found myself not being able to remember a basic shell command that I had been aliasing. I Googled it and found the answer. I felt a bit silly that I couldn't remember something basic I used to know, but it was easy to remedy. Aliasing hadn't decayed my ability to think through the problem itself.
I think there are some who over-optimize, and that is to be avoided. There's a sensible stopping point, but if you over-anything you're going to have problems.
Great post.
Thanks.
Yeah and thinking through is the most important part. Now you get the best of both worlds. For the 95% you can use your aliases and be productive, and in the 5%, you knew what went wrong and it was a quick Google search away.
I think it all boils down to just doing things and letting the "doing" guide your actions. Instead of thinking about "I need 100 aliases set by 10am SHARP!", just pay attention to your own patterns. If you find yourself Googling the same command a few times, make an alias for it then.
Couldn't agree more. When you notice a pattern of searching for the same things over and over, then you should do something about it and create aliases (if that's a simple command), shell aliases (if it's a series of commands) or even boilerplate code or modules, if you reuse the same components/functionality over and over.
Over the years I've realized that keeping 'work notes' has increased my productivity. Having a bunch of text files in a folder that I can
grephas saved me countless hours googling for solutions that I've already googled before.I'm not saying googling is bad. But sometimes it can be a time sink. You know "replace all instances of word x with y" --> "grep cookbook" --> "my cool vim macros for finding/replacing text" --> "my better emacs macros for finding/replacing text" --> "how I manage my vim plugins" --> "look at my cool .vimrc" --> "Why I moved from vim to VSCode" --> .....................................
:)
This is a very insightful article. I, too, have been a victim of premature optimization. Programmers have been "raised" to think that everything must be generalized; everything must be abstracted. By following that way of thinking, we have doomed ourselves with the points you raised in the article.
It drives me insane sometimes when I overthink about a certain feature I want to add to my code. I often get lost in the forest by thinking of the "best way to implement this feature for reusability, performance, and maintainability" rather than making actual progress.
Yeah, it's very non-intuitive too, because reusability, performance and maintainability are fantastic things to have.
The non-intuitive part is you only get there by making progress and uncovering the problems that lead to non-generic, slow and unmaintainable code but at the same time, some of those problems aren't even problems that need to be solved.
For example, I'm totally cool investing my time and energy into making a unique user system for a long running project I'm working on. It's not like I'm sitting there creating 15 new applications a day that each require a generic user registration system based off that single unique project.
Exactly. It's just so strange how the supposed "good things" to strive for are the exact same things that may inhibit progress.
This was a fantastic article, and I'm going to share it with my team to read. I recently was doing a code review with a newer developer and I noticed that they had made some design choices in how they wrote that code that optimized one particular section to the detriment of a couple other, more frequently used sections. My advice to them was to write code for the rule, not the exceptions. I hadn't considered how that applies more broadly to optimizing workflow, but that's definitely rings true in my experience.
Personally, I often try and over optimize before I've even begun. Every time I think of a new side project, I do what you described and try and make it generic and reusable so others can use it too. Consequently I get overwhelmed and rarely build any of the ideas I was thinking about. I really appreciate the reminder to not over-optimize in areas like that.
Thanks for reading.
Yeah, this sometimes happens to me too. I think it partially stems from there being this trend where you need open source everything you work on from the beginning.
Instead of just getting your hands dirty and solving the problem you have first, you start thinking about trying to carefully code the perfect solution from the beginning because "but people are going to read my code!" or you get side tracked for 5 hours trying to decide on which license to pick.
By the time all of that happens, you throw your hands up in the air and call it quits before you even had a chance to write 1 line of code.
This is the web developer freelance business equivalent of spending a month creating business cards, an LLC and a fancy portfolio site when all you really need to do is talk to a potential client (which you could do in literally 10 minutes without any of that stuff).
I do this every time in my side projects! I get overwhelmed in architecting some ultra reusable thing up front to solve the problem I'm interested in. Instead what happens is I get lost in that forest of decisions and make 10% of the generic solution. I know I should just prototype the actual problem first and then worry about whether it's reusable later but I keep getting stuck in made up architectural issues. It's a difficult habit to break.
Great post!
Thank you for making a case for using a beefy single server instead of overengineering everything for the serverless cloud from the start. I agree 100% that it is better to thoroughly understand the challenges of scalability, how it applies to your own situation, or if it applies at all, instead of betting everything on the latest trending technology du jour that might lock you in and make you regret it down the road.
A single strong database and web server can go a long way.
I've been also making all the mistakes you mention in terms of software development. I'm 100% OK with customizing the heck out of my dev environment though :-D
My personal obsession with overthinking problems and trying to find the best approach was part unhealthy perfectionism, but also part the way I grew up with PCs and how I learned writing programs for them. I know what it means to write code for an under-powered platform where optimization was a necessity in order to get usable performance. There was little RAM, little (or no) storage. Updating the screen could be very slow unless you knew how to write directly to video RAM. Preferably in machine language. That became pretty much a non-issue over the years, but when smartphones, iOS and Android became a thing I felt thrown back to that time. Once again we were dealing with relatively weak hardware and its shortcomings that can be worked around with some trickery. I wrote a couple of graphics processing apps and hit the limits of Dalvik and the ARM CPU pretty quickly. Fixed point integer computation instead of floating point was necessary. Native code close to the metal instead of Java interpreted in Dalvik was the solution.
These are not examples for premature optimization though. It was necessary.
These examples are more of something that might have shaped my mindset over the years to look for ways to optimize things before I even run into issues just yet.
This, and also what Some Dood mentioned in his comment: our abstract methodical thinking were we are conditioned into generalizing things and finding ways to create reusable code. I also know that it can be incredibly appealing for developers to create reusable code for the sake of being reusable, as if that library was the product and not the application using the library. This is a trap I ran into time and time again. That's basically when I was caught up in programming land, and didn't care much about end user land.
In itself aiming for generalization and reusability is a good and useful practice. But like anything in life, too much of a good thing can turn into a bad thing.
I tend to follow a modified form of YAGNI: use the simplest approach that would not block you in the future, i.e. you don't code for the myriad of future hypotheticals but you do keep paths open towards potential solutions for those hypotheticals.
As to the editor discussion, I generally use a (good) IDE, which has lots of default yet optional features.
As developers, we’re constantly bombarded with information about how companies most of us admire (in terms of technical achievements at least) like Google or Uber did this amazing thing that solves all their problems. Since it’s in our nature to follow others by example, it is especially enticing to follow their lead, regardless of the domain we’re working on.
This post really opens your eyes on this matter. Keep your feet grounded, focus on your own domain, and solve your own problems in a way that also leaves the door open for the near and possible future.
Thank you for writing this.
That is what my current (soon to be previous) work environment was like. Everything had to be 100% uniform. If it made sense to it to one app, we should do it to another app by default. It drove me nuts.
The part of researching the better architecture or dev stack to handle 1 billion requests has happened to me 3 or 4 times in weekend draft apps i developed.
Guess what, most of the time the apps were never ready to be used because of the what if's 😁
Advice: Code on whatever you know, get your software running and then if you need handle the optimizations!
Your app doesn't need to be fast from ground up because at the end of the first deploy it has only one user -> you
Donald Knuth never said “(or at least most of it) in programming.” These are your words or somebody else's. It's very unethical to quote wrong things and not even mention the source. Here's the original paper, please read it first, page 268: web.archive.org/web/20130731202547...
Having a generic understanding git/bash/ cli tools of the work environment is useful for a team like mine where we practice Mob Programming everyday to achieve 10x performance. We practice Pair Programming sometimes and solo for obvious tasks only.
All that doesn't lead us to over engineer or over design the code base on requirements that don't exist. You're more likely to do that if your work by yourself alone, and that's the main problem. We can be pretty lean and write only the simplest code in the direction of scale once that's necessary.
Only skills with practice and mentoring will give you that, it's not to optimise for the 95% because that's your knowledge of what 95% is. You don't know what you don't know.
Thanks for the heads up on the quote. I guess someone should remove it from his official "quotes" page on Wikiquote. It's listed at en.wikiquote.org/wiki/Donald_Knuth.
I changed it to what it is in the paper you linked.
Oh wait! That's a different paper. It looks like he said it somewhere else using different words. TIL.
The reason I raised the concern is that I've been caught on the trap. The internet can be misleading sometimes, see hackernoon.com/the-danger-of-relyi...
I didn't read that paper and I don't know where to find it. It would be good to check the original to see if that's exactly what's written there or link the source (even if it's wikiquote). I've seen multiple publications slightly changing the quote and then what we know today is a complete different one. Think an effect like the wireless telephone you played at school. Even website like Wikipedia get this wrong cause nobody bothers to check the source or the source is not available anymore!
One of the few things I remember from my college days was a lecturer who said 'optimise for the common case'. It's always stayed with me over the years and I try and keep it in mind when I find myself over-thinking or dwelling on 'what if...' :-)
Thank you for very nice post especially it is comming from an experience from yourself.
I really like Kent Beck qoute “Make ir work, make it right, make it faster” wiki.c2.com/?MakeItWorkMakeItRight...
I believe the execution is really matter and if you overthink/over-engineer your solution, you won’t release something that work. You will have a great plan WOW congratulations but it is no relased yet. I’m always trying to find the right solution for the existing problem instead of find a solution for waht if.
Last thing I want to share a nice joke
A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd.
First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.
twitter.com/brenankeller/status/10...
One exception to that is when dealing with critical or embedded code. You usually have to optimize for the edge cases, otherwise your end product may fail at unexpected times, even putting lives at risk.
Other than that, I totally agree, try to ship as fast as you can and continuously improve on following feedbacks instead of bashing your head against stuff that may not even matter at the end.
many Experienced developers tend to see things in more abstract way than those new comer so that they can sense and smell things that could messed-up in the future...and then the "What If" questions come up.
great post btw...
I find myself in a infinite loop when I start customizing.
For example, This is what happened today:
I decided to write my posts in markdown instead of using a word processor or note taking app, actually vs code has enough features that you can write markdown pretty easily. But then I find out it doesn't support checkbox syntax, now the loop started and after a few wasted hours I found myself installed a bunch of plugins in VS code then sublime text then plugins for it and so on. Later I frustrated and get back to vanilla vs code.
through thanks for this great article
Well, this post really hit me through the skull. I'm just like this, trying to build something out of the thought that "how can i pull it off and make sure in time it would be scalable in the near future" and guess what, I haven't had something written and still stuck reading articles.
Great article! thanks!
I really needed to read that... shut up and take my unicorns!
Great post! It feels do good to know that I am not alone.
Excellent article.
You might like (or maybe know) this other excellent one:
programmingisterrible.com/post/139...
I just started to learn DevOps and my projects are small, and i was about to learn kubernetes 🤣, and when i read this article i realised that i really don't need to do this right now.
Having gone through all of those things, I thank you for your post Nick.
What you wrote about is so important.
Really great post Nick, this topic should be made into a sticker for constant reminding 😂