Going The Other Way

Have you ever "gone the other way", by which I mean where most people use product A then usually "grow up" or "evolve" and use product B, and you went the other way! B to A. A made up example of B to A would be ridesharing apps -> calling a taxi service on the phone.

In the software world, when you go the other way you notice how all the blog posts, online help is not really helpful at all. They assume you'd only ever go A -> B.

Examples of this I have found are: moving from AWS to Heroku for hosting or from Ledger-CLI to YNAB4 for budgeting.

I'll cover the first of those examples in this article. I will talk about going from AWS to Heroku for the tryjobalerts.com site hosting and data processing.

William Shock. Yes... Bill Shock.

I'm guessing many of you have tried the generous free year of AWS services, and then got your first bill on the 13th month and said "errr. OK". That happened to me and prompted me to cull or move out a load of stuff running in AWS. On the flip side, I was amazed at how cheap SES is - I think my bill was $0.10, and how expensive Elastic Search is, where my bill was $50 for a hardly used cluster. I would have not been suprised to see those numbers reversed.

However before the bill, I had started off moving EC2 servers to Heroku. The reason wasn't cost - Heroku is typically a little more expensive as it provides additional wrapping and services over cloud infrastructre. I moved for the obvious reason that I'm lazy. Which is a codeword for good time prioritisation. I don't want to manage a operating system, server or CI pipeline. Instead I want an expert to take care of that for me for an extra $5 a month.

The point where I switched from EC2 to Heroku wasn't a well thought out logical decision, based on sitting on a leather couch, musing about software architecture. It was an emotional decision based on my node app crashing, and all my services for tryjobalerts.com being out of action for 5 minutes and counting. What if someone tries to edit their settings and they get an error page? how embarassing. Although not that unusual these days, I didn't want to be the one.

What to do next? Well, in the time honoured solution of "turning it off and on again", I restarted the EC2 instance and BaaM! the node app crashed again. Whatever it did, it took out the machine in such a way I couldn't connect to it via SSH to fix the problem. By this point I rashly decided to move to Heroku. I just didn't have the heart to fix this shit when I wanted to be coding. And I knew this would likely happen again and again, and I'd probably end up on the slow march towards Kubernetes.

Was that a hasty decision? Well maybe it was, but it had been a subconcious thought for quite a while. Setting up nginx, let's encyrpt, PM2 etc. etc. was mildly intersting but feels like a distraction from what I am really trying to do. I am happy for someone else to deal with that, and if everything dies, build them up again. I'll concentrated on the code, thank you very much.

I gave Heroku a try. I thought, let's just see how easy it is. If it "just works" I'll switch over. Otherwise I'm downloading k8s (joking!). I was working on this while the site was out like a light, no one could access their settings, sign up or receive an email - so no pressure.

Moving to Heroku was redicuously easy. I just had to create a file called Procfile with this in it:

web: node distjs/node/index.js
worker: node distjs/bot/index.js

And set up a few things in the Heroku UI - secrets and such like. I also had to change the port express run on from a fixed port 3001 to the number Heroku provides as an environment variable. Within 15 minutes I say I had it working, sans DNS entries.

I kept SES and PSQL on AWS, and everything worked lovely.

I kept things this way for a few weeks. I then got that bigger than expected bill from AWS, including a fair amount for the PSQL. Nothing too catastophic but it didn't seem like good value for money.

Heroku has a weird limit for PSQL Hobby Basic

I noticed Heroku offers a "Hobby Basic" $9/month PSQL with up to 10M rows. It turns out I use a decent amount of space in TEXT fields, but not too many rows. So I migrated to that. Yes unbelievably I am migrating something from AWS to Heroku to save money.

The limit is weird, being based on number of rows rather than storage size. I wonder how much excellent data you can store as blobs in 10M rows - maybe I should keep quiet as the Heroku Product Owner could be reading.

I searched online for the best way to move the data over. I like Stack Overflow because if someone has had a war story you hear about it instead of experiencing it first hand. However most of the advice online is how to get your data out of Heroku into AWS. Going the other way.

Therefore I tried a couple of ways. It turned out it was easy enough to use psql command line tools pg_dump and pg_restore. It worked first time and took about 30 minutes to do - most of the time was me doing housework while waiting for pg_restore to do it's work.

(Incase it's helpful to anyone, the tip here is don't use the Heroku CLI, get the connection details from the Heroku portal, and use the psql standard tools.)

It is quite amazing to me that Heroku turns out to be cheaper on the PSQL database. It might be because they don't offer as much power as RDS, although they are using AWS in some capacity - perhaps they manage the VM themselves.

It could be that Heroku is happy to lose a bit of money on that Hobby Basic tier, or they more aggressively share resources than AWS does in RDS, so under load it would crack sooner.

Heroku's Hobby Basic PSQL doesn't offer rollbacks, and that could be another reason it's cheap. It means I need to take care of backups myself, however this can be automated quite easily with a periodic dump of the DB.

Don't be too cheap though

The big mistake I made when moving to Heroku was to try and use the free dynos. Only do this for non-production MVPs. By using the free ones, I had to do a lot of work setting up SSL with Cloudflare, and I never quite got it to work.

I then realised I needed the paid dynos anyway because the free ones cannot be restarted (!) without a deployment. So I upgraded, and then the SSL was sorted. No Cloudflare needed. Had I done that in the first place I'd have saved a couple of hours of frustrating DNS hacking.

It's always a fine line. I mean you try to save money where you can, but sometimes you don't know in advance what the free-tier traps are. Maybe Heroku could make this more obvious, but they did warn that free dynos are mainly for hobby use, so fair enough.

Herokus! Herokus! £2 a lb!

I sound like a Heroku salesman at this point. There are other "Platform as a Service" offerings out there. I had just used Heroku before in 2014, and liked that it still existed, so I used that. However there could be better options for you, check them out. Or maybe a regular old cloud like AWS is what you needed all along.

Regardless, it's OK to go the other way, choose whatever makes sense for the project you are doing.

Join JobAlerts to get free daily emails with the latest tech jobs from a curated list of excellent sources. Click here to learn more.