90% of software engineering is integrating janky APIs, and I love it

(Title photo: Ophelia photos - Creative Commons)

I stumbled upon this twitter thread and corresponding Hacker News comments today: 90% of software engineering done today is integrating poorly documented APIs | Hacker News. This take doesn’t resonate with me at all. I actually think this attitude is toxic.

Nobody says: "90% of building a bridge is making shapes out of cement of varying quality". It is well understood that civil engineering is about which cement to use, designing a sound architecture, computing structural stresses, organizing a large workforce, establishing a project plan. It is hard, challenging and rewarding. Cement will always be of poor quality, because it is not unobtainium that can magically be molded into the specified bridge.

The same goes for software engineering and our own digital cement: APIs. In fact, combining cruddy APIs to solve problems is so challenging and fulfilling that I can’t imagine doing anything else.

Two types of software engineering

These days, I am in the business of writing a CRUD e-commerce application, using PHP, wordpress and mysql of all things. In the past 20 years I've worked on anything from bare-metal embedded systems to quantum cryptography ethernet bridges to search engines and "not google-scale" data engineering. This has taught me that there are two very separate kinds of software engineering.

Software engineering as copying bytes

One is what I call "copying bytes". You get input bytes, relevant to the business in some way. As a software engineer, your job is copy these input bytes to their specified destinations.

For example, you receive an order from the customer (line items + shipping address), and you have to forward these line items to the fulfillment department, recording a few log entries along the way. You might calculate some tax or apply a coupon along the way, slightly modifying those bytes, but roughly, you are just sending them along the way god designed them.

Software engineering as “inventing” bytes

The other kind of software engineering is what I call "inventing bytes", or "transforming bytes in really complicated ways". This is the realm of funky algorithms, data science, mathematics, complicated stuff. For example: taking an input 3D model and outputting some CAM instructions or searching for a string in complicated index structures. Often, the hard thinking is done by people at universities or research labs.

Writing code is usually the tiniest part of the undertaking. One could wonder if these “byte inventors” are software engineers at all. Maybe we should call them "3D model transformation engineers", or "search index data structure engineers". Often, their code needs to be rewritten by someone who knows how to indent files (or say, port it from hacky matlab to java).

The APIs for inventing bytes are weird. Frankly, I don't really dabble in invented bytes, so I can't say too much as to the documentation of these APIs.

You might actually copy bytes your entire career

If you are not in the business of thinking clever ideas or making pretty things like video games, but in the business of making money, you probably fall into the "copying bytes" category of software engineering.

Which bytes to copy, when, where, what log entries to add along the way, how to scale the copying, how to ensure the copied bytes are transactionally aligned with other copied bytes, that stuff is complicated. But it's turtles all the way down, one API to copy bytes building upon another. Ultimately you hit the hardware, and boy are the hardware APIs to copy bytes utterly nonsense.

You don’t need to write interesting code to be an engineer

You might think it's inane: you learned how to balance trees, how to schedule processes, and how to do some differential curve geometry; how to write compilers, how design database engines and how to implement distributed consensus; why don't you get to write all those beautiful algorithms as part of your day job? Thing is, clever people have already done so, and often they made a business out of it. Now you can pay them for their clever thinking and their diligent work, hoping that they were indeed more clever than you, and go back to calling APIs to copy whatever bytes have just been computed.

People want to transfer data for all kind of different reasons, in different ways, with different constraints. That means that even the best written API documentation will have to be generic enough to cover the use case it was built for, and with some chance, what people are using it for. It will rarely exactly fit your use case. The documentation will thus not be perfect, and might not cover what you are looking for at all.

I actually think API documentation at large is miles better than it ever was. All the APIs I had to deal with lately come with copypasta for 15 languages, a snappy searchable index, a plethora of tutorials and articles and videos, an active github. This is miles better than having to guess how much whitespace padding a file placed on a PC-NFSD spool needs for the COBOL parser to be happy.

Solving problems by combining crappy APIs is the most fulfilling thing I do

But here's why I find my job copying bytes with subpar APIs singularly fulfilling:

Copying bytes is not easy, especially in today's distributed world. Figuring out how to do it reliably, securely and at scale requires a lot of thought. Figuring out which data to transfer to whom needs business understanding, knowledge of people and systems thinking. Building and maintaining a solution requires product and project planning, managing teams, dealing with changing requirements. It requires keeping a codebase healthy, mentoring other engineers, aligning with stakeholders, dealing with legacy code.

The APIs I ultimately use are the least of my problems. Some are nice, some are terrible, some are really problematic, but they don't have much to do with the "engineering" part of my job. The engineering part is all about drawing pictures with boxes and arrows. The boxes are other people's systems, the arrows are other people's APIs. What arrows I draw and where they point to and what the little bubbles along them contain, and which boxes I chose to use, that's what I do as an engineer.

A concrete example: checking out an order

As a concrete example of a hard problem, fully implemented by glueing together shoddy APIs: fulfilling an order on an e-commerce store.

• seamlessly show payment process and checkout status to the customer using a slick UX in their browser (APIs: some shoddy REST, some janky web socket or whatever, some hacked react and some dubious CSS. Ultimately just shoveling the status code from the payment processor to the user's browser, potentially adding some curly braces and escaping a string or two.)
• log the transaction and provide metrics and analytics to business and ops (some inane pixel data, some putrid tracking JS, some shoddy SQL, maybe some janky JSON going to a SQS queue)
• process the payment information, order information and display it in the customers account section / backend CRM / fedex (I'm not even going to list how subpar the APIs are here)
• transform the shipping address into a proper shipping label with barcode (have fun with zebra and ZPL)
• dispatches the line items to the required fulfillment centers, provides picking information to the warehouse workers, with tracking, metrics, inventory handling, ...

Designing something like this so that it works correctly, efficiently, with visibility, reliability, scalability, and fulfills business needs, and sparks joy for every human involved – now that's actual engineering.

Do you think glueing together APIs is engineering?

Do you think software engineering today is boring? Do you hate glueing APIs together? What do you think constitutes "real" software engineering? What is the ideal day-to-day of a software engineer? Please be sure to comment below!