Parsing AWS AppSync Responses, Elm GraphQL Libraries, and Only Doing Front-End

Some context on my 1st week of the new job, dealing GraphQL null-able types, describing some of the challenges of being a front-end dev when you are not full stack, and the harder than immediately apparent choice of choosing an Elm GraphQL library.

GraphQL Null-able Types

Working with AWS AppSync at work (AWS’ awesome GraphQL API hosting service), and I forgot that it can return wildly different responses when you involve null-able types, specifically SomeResponse! vs SomeResponse. This results in JSON that can have both your data possibly and errors which is an impossible situation.

I haven’t touched GraphQL in over a year, but finding Mutation examples and docs on how the request syntax differs from the schema, is just as hard to find now as it was back then. Disappointing; I’m experienced with it, but I can’t imagine n00bs trying to figure it out. It’s days like this where I’d rather use REST and JSON.

On this particular project, I’m “just” a front-end dev, so I am not worried handling all of the back-end infrastructure, nor writing any of the API services, nor modelling the database. Experienced front-end developers know this comes with a set of trade offs. Specifically, if you write the API, you can change it, when you need to change it, and make it work however you need for your UI. If working with the API is challenging, you can prioritize working on it, then return to the front-end.

If, however, you’re a front-end dev only, you just have to deal with what you’re given, and work around the schedule of those on the back-end, and negotiate changes that may or may not be possible. This is a dependency; NOT BAD, just a trade off. It adds delays and latency, but can sometimes add increased speed of delivery in other ways. Lots of pro’s/con’s here.

Types that can be null in GraphQL query and mutation’ function responses are particularly challenging because you have 4 possible scenarios to handle:

• the call worked, but you received null
• the call worked, and you received a response type
• the call worked, but you received a list of errors
• the call worked, but you received data and a list of errors

Assuming you’re not using libraries, returning back data as a normal HTTP request in Lambda works as you’d expect, and AppSync handles the heavy lifting of converting that an a GraphQL response, which is great. AppSync also handles the various error scenarios.

However, anyone who has used a nulls in languages like JavaScript, Java, and/or Python, or even Maybe’s/Option’s in an ML based languages like Elm and ReScript knows, they can make things rough compared to a Result/Either type because Maybe’s don’t tell you why the data was found. Amusing in a macabre way because that was my first interview question.

This is a lot of the reason developers like Go coming from Exception based languages because it fixes 2 problems those languages have:

1. Why did this function fail?
2. Who caused the program to fail?

In Go, you immediately know; assuming the error returned has reasonable context in the error message, your function call returns data or an error message. Secondly, because you immediately know if the function failed or not, can abort your current function call in a procedural way, and return that error upwards, with lots more confidence who caused it, and where. While similiar to exceptions in that some deeply nested code can halt the entire program, this provides better control, simpler types, and less hunting for where error occurred after the fact with less ceremony needed in handling those errors.

The Result/Either type in ML based languages, specifically ones that have types, even TypeScript, gives you this same style of error handling.

Can this function fail: yes or no? If no, it’ll return a Result type rather than a String or a Number.

// this won't fail
getFullName = (first, last) =>
  `${first} ${last}`

// but this could, so return a result
getPerson = id => {
  fetch(…)
  .then(…)
  .then( data => ({ ok: true, data }) )
  .catch( error => ({ ok: false, error }) )

Does this function work: yes, or no? If no, why? The why will be contained in the Result’s Error sub-type, usually a string, explaining either why the function failed, or something it depends on that it called that failed.

GraphQL null-able types, or Maybes, act the same, the but don’t tell you why they’re null/Nothing.

"""
GraphQL query
"""
someFunction(id: ID): Response!
"""
… why could a response be null?
"""

When you add that into a GraphQL response that also has a possible null return value, instead of handling the 2 scenarios that Go or us Functional Programmers like: “The return value tells you yes or no”, now you have 4 scenarios like I mentioned above.

The solution, if I were in a full-stack position, would be:

1. Go to the GraphQL schema
2. Change the query/mutation function’s return type from Response! to Response
3. Deploy
4. Change my UI code to be much simpler and just handle either I got a response with errors, or I got a response with my Response in it.

Choosing an Elm GraphQL Library

Since that isn’t changing, that brings me to my next point: which GraphQL library to choose. When I first started learning GraphQL, I used JavaScript first because the documentation was much more clear for transitioning from simple curl calls to JavaScript fetch calls to make basic GraphQL query and mutations. The libraries, though still heavily based on React, did provide either escape hatches to just import Node.js style code in the case of Apollo, or other simple wrappers on top of Node.js fetch.

Once I could quickly make calls against a server, and see the raw responses were matching up to my expectations, I moved to finding an Elm equivalent. Two years ago or so, Dillon had his GraphQL library and marketed it well.

What made it stand out to me was it generated all the Elm types based on your GraphQL schema. You take your GraphQL schema, run a Node.js cli command against it, and voila, you have all of the types, queries, and/or mutations ready to be used in your Elm code. Anyone coming from Swagger in any other language generating REST API code, this should be familiar, and desirable as the GraphQL Schema/Swagger.json is the source of truth for API calls.

The only challenge I had, and I recently had again, was the selection types. The great feature of dillonkearns/elm-graphql library is the ability to get a response from your GraphQL call, and it’s already parsed to the response type. If you call someGraphQLQuery and it returns a ResponseThing, then your Elm code already has both the someGraphQLQuery function and ResponseThing type ready to rock. However, the feature is the expectation you’re going to convert it your own type.

Challenges of Being a Front-End Dev vs. Full-Stack

From those of you familiar with Domain Driven Design (DDD – learn more here ), this would be the API is one Bounded Context, and the UI is your Bounded Context. In front-end development, we often need types in a certain shape to build a UI; lists need items in an Array with id’s and names, inputs need items with user input + original valid values, etc.

API developers do not have those concerns, even if the UI developer is the only consumer. They are not the ones building a UI around the JSON/GraphQL responses they create. They are not eating their own dog food.

What often happens is UI developers will do 1 of 2 things:

A: They’ll create completely different types on the front-end, even in non-typed languages like JavaScript, and convert the back-end ones into it. This could be for something as simple as converting a Python’s API snake case of first_name to be the camel case norm in JavaScript to firstName, or it could be both renaming, adding different properties, and even ignoring some from the back-end.

B: Or they’re orchestrate many calls because they are missing data, and coalesce those into a single type. Need a user, and their list of permissions, and that requires 2 AJAX calls? Create a function to abstract Promise.all, then mash ’em together in the response.

Dillon’s GraphQL library supports both A and B styles out of the box; you’re expected to take what the GraphQL schema sends back, and convert them to front-end types. I remember when I first used Dillon’s library, for the 1st 3 months I was like “Why in the world am I having to define my types twice, this is so dumb…” If you’re a full-stack dev; meaning you’re building your back-end for front-end API’s, then you don’t have to convert because those back-end types were specifically tailored for the front-end. I then realized it wasn’t dumb, but a great feature, and I was dumb.

However, the syntax for doing that always makes my brain melt… because I’m dumb. Getting what Dillon’s library calls a SelectionSet to compile and make sense was always an hours to many days long affair. To be clear, this is my inability to be smart and instead rely on willpower; Dillon’s library is great, I’m just not smart enough to understand the SelectionType types. I’ve used it twice in production over a year, and always eventually figured it out, so it’s not a deal killer, it’s just one of those things in Elm that I struggle to wrap my head around.

This is also partly GraphQL’s problem. If you look at 90% of the GraphQL documentation and marketing, it’s for UI and API developers who are dealing with way too many API’s, and need just specific pieces of data from many places. The GraphQL queries and mutations allow you to select exactly what you want, and JUST getting that data back, solve my UI developer B problem mentioned above with just a single API call.

"""
Note I'm getting both a Person and Address, 
and only getting 2 pieces of specific data from both. 
I don't care about the rest of the data, 
nor do I care if getting both person and 
address in the same call is hard; in GraphQL, 
I make a single request and let the back-end 
handle the logistics of getting all the data, 
not my problem.
"""
query(someThing: ID) {
  person {
    id
    name
  }
  address {
    street
    zip
  }
}

That’s not what you’re doing as a UI developer often, however, if you’re building BFF’s; in that scenario, an API was specifically created ONLY for the UI to consume, typically BY the UI developer creating the UI. There is no need to utilize a query and than tailor the data you want in the response. Instead, you call a function and get a response type getting exactly what you need.

If you’re doing that, Dillon’s library is still a good choice in that you just create a similiar (sometimes exact, but not always) type nearby and just map (e.g. convert) to that using his SelectionSets. As you spend more time in the UI, those types in the UI do sometimes drift away from the GraphQL ones as you learn more. You can either re-sync the API by going back to the API, and updating it, or just leave it since the performance impact may be negligible.

If you do update it, the workflow is:

1. go update your schema
2. then go fix your API to match it
3. then re-run Dillon’s code gen
4. use Elm’s compiler to tell you what to update

It’s a great workflow, fast, and allows you to confidently respond to change, often that change initiated by you as you learn more.

Enter Vendr’s elm-gql . This one is specifically built for the full-stack workflow mentioned above. I’m building a UI with types, and I expect the API to deliver those to me. There’s no need to convert anything because the UI is driving the development here, and the API is there to serve that.

That’s not what I’m dealing with, though. I’m only doing the front-end. I do have some influence on the back-end. While I’ve only know him for a week, my back-end developer is super nice, takes initiative with implementation questions often, and handles all that while also handling all the Ops and Database development too. If you’ve ever built a SaaS or worked at a startup, you know developers in those situations wear many hats, regardless of what their title is. It’s a lot of work that never stops in volume and intensity, with lots of context switching.

Still, he’s not at my beck-and-call, and there are reasons the schema is the way it is. So I need to be flexible. I can’t use Vendr’s because I don’t control the schema, and my influence isn’t measured in minutes to change something. I tried Dillons’, but the SelectionSet yet again made my head explode in trying to grok the types, exacerbated by the large amount of null-able types in my schema which then results in a lot of Maybe values. One or 2 Maybes is fine, but I try to avoid them as much as possible because it makes the code not fun to deal with, and you end up in a lot of weird conversations with the UI/UX Designer about “Well, this edge case could happen according to the compiler, but I’m too tired right now to give you a valid user flow scenario on how it could happen, or how the user could recover from it…”

So I ended up using Guillaume Hivert’s library.

It just just enough abstraction over the basics of converting your HTTP calls to GraphQL queries and mutations, but ALL of the parsing of responses is on you. I’m well versed in parsing JSON in Elm. I’m also familiar with the compiler errors as well as runtime errors you get with JSON that doesn’t match up to what you designed. At some point I’ll probably have to move beyond the unit tests and add contact tests, maybe via Pact.js.

This brings is back around to my original problem: handling a multitude of possible success and error responses from GraphQL “because of null-able types” that make my UI extremely hard to build. This allows me to narrow the types to “It either worked or not; only 2 possible scenarios”, and I can get the flexibility in parsing the JSON to enforce those types, and be extremely clear why it failed.

You’ll get JSON back from AppSync with your schema looking like:

“It worked”

{ data: yourReponse:{ …an object… } }

“It failed”

{ errors: […reason(s)…] }

“It failed with data”

{ data: yourReponse: null, errors: […] }

“It failed with data”

{ data: yourReponse: {…} , errors: […] }

Because I have control in the JSON parsing, I can convert those 4 to 2:

type Response
  = ItWorked (Maybe Data)
  | ItFailed (List String)

Note to self, if I ever get an back-end job again, and we’re using GraphQL over REST, build a GraphQL linter that prevents you from putting ! in your schema, lelz.

Thanks to the Elm Slack for helping answer my GraphQL questions!