A couple of weeks ago I had a conversation with a coworker about whether it would be possible to make a read-only API completely static, i.e. pre-generate all the resources instead of generating them on demand.
A static API could be hosted on a service like github pages, so you wouldn’t need much maintenance to keep the thing running, even if the underlying data needed to change.
Over christmas I had a go at making an API like this. This API lets you lookup jargon used by the civil service (where I work) and in this blog post I’ll share what worked well and what didn’t. You can find all the code on my github.
Storing the data as version controlled CSVs
The source data for the API is stored in CSV files on github. This works great, because the data is easy to manage (you can even load it into a spreadsheet), and you can view it in a tabular format on github, because github renders CSV as HTML.
For an open data API, this makes it very easy for anyone to find out when data has changed and to submit changes to it.
Creating RESTful routes
This blog post explains a bit more about how github pages routes URLs to files.
Basically you can create any URL structure, but a big downside to the API being static is it can’t respond to query string parameters. This means you can’t use query parameters to paginate through large datasets.
An alternative is to make each page a separate resource, for example:
In the end I decided to avoid pages altogether because my dataset was not that big, and it would just be annoying to have to reconstruct it on the client side. So the routes for my API are just:
- https://matmoore.github.io/jargon/ (everything)
- https://matmoore.github.io/jargon/{ACRONYM}/ (a single lookup)
These resources are generated by running a ruby script that transforms the CSV into JSON.
Deployment
It’s very easy to deploy to github pages. I decided to use a separate gh-pages branch because I wanted to keep a clear separation between the CSV source and the generated JSON.
I repurposed some code from middleman-gh-pages in order to easily regenerate the API and publish to this branch. This is something you could automate using CI. In theory you could build the whole thing into a marketplace app, instead of bundling the code with the data like this, but this will do for a proof of concept.
Vendor lock-in
The solution is a bit too dependant on github. In particular it relies on github pages routing URLs in a particular way, and github controls all the headers returned by the API.
By using github pages you are accepting their usage limits which means that your API could still go down if it gets too much traffic, and they don’t recommend using it for commercial purposes:
GitHub Pages is not intended for or allowed to be used as a free web hosting service to run your online business, e-commerce site, or any other website that is primarily directed at either facilitating commercial transactions or providing commercial software as a service (SaaS).
However, I think the script that generates the API could be made to support other platforms like Gitlab Pages.
Other limitations
The static nature of the API meant I could only offer exact lookups. This means you have to enter an acronym as upper case, with punctuation matching the dataset. It would have been nice if the API allowed fuzzy matching and could return a useful 200 response instead of a 404 if there is no match.
A static API like this is also a bad fit for anything that requires authentication/access control or if you want to apply your own rate limits.
Conclusion
Ultimately I wouldn’t recommend this solution for anything important, because it’s inherently inflexible. There are certain things you just can’t do, and a free service like github pages is never going to be as reliable as a paid platform like Heroku, since it can be closed down at any time without affecting paying customers.
However, for a personal project with zero budget this is a fairly hassle free way of providing a simple API. I think github pages could also work well if you want to quickly prototype an API, or provide mock responses to test against.
Top comments (0)