tapir: Yet another way to REST

Here at ClearScore, the backend developers utilize a number of tools and libraries to create the software that runs behind the scenes. In more cases than not, the software consists of services that communicate "RESTfully" via HTTP(S)—both with other backend services and with the frontend. Of the libraries in use at ClearScore, http4s was one I was excited and happy to learn I'd be using when I joined the company.

In the past, I've used various libraries to accomplish the development of HTTP services, including Spray, Akka HTTP, and Play in Scala; Spring Boot and Java Servlets in Java; Rocket and Actix-web in Rust; as well as simply using PHP or Ruby on Rails. But focusing on Scala—which is what the majority of ClearScore's backend is written in—http4s was attractive to me because of two main things: its simplicity and its adherence to functional programming.

For example, here's an HTTP service that returns the current time:

import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

import cats.effect.{ExitCode, IO, IOApp}
import org.http4s.HttpRoutes
import org.http4s.dsl.io._
import org.http4s.implicits._
import org.http4s.server.blaze.BlazeServerBuilder

object Time extends IOApp {

  val service = HttpRoutes.of[IO] {
    case GET -> Root / "time" =>
      Ok(theTime())
  }

  override def run(args: List[String]): IO[ExitCode] = {
    BlazeServerBuilder[IO]
      .bindHttp(port = 8080, host = "0.0.0.0")
      .withHttpApp(service.orNotFound)
      .serve
      .compile
      .drain
      .map(_ => ExitCode.Success)
  }

  def theTime(): String = {
    LocalDateTime.now().format(DateTimeFormatter.ISO_TIME)
  }
}

Yeah, not very exciting, but simple is what we want (trust me 🙂). Take note, this particular implementation is using the Cats Effect library: put simply, this provides us with an effect type, IO, to run side-effecting input and output asynchronously (e.g. like interactions with our HTTP server or with external services, like the computers hardware clock). http4s is compatible with other effect types, but that's not important right now. Ignoring most of the code, the cool thing, in my opinion, is just four lines in this example:

val service = HttpRoutes.of[IO] {
    case GET -> Root / "time" =>
        Ok(theTime())
}

service is not a complex thing at all. At its core, it's a partial function that accepts as input a thing of type Request[F] and outputs as a result a thing of type Response[F], where F is IO in my example (but again, that's just a minor detail for now). The beauty, again, lies in the simplicity of the detail: service is easy to reason upon what it can and will do. This implies testing it is straightforward.

def test(): Unit = {
  val routes = service.orNotFound // i.e. anything outside our partial function will '404'

  val response: Response[IO] = routes(Request(GET, uri"/time")).unsafeRunSync()

  assert(response.status == Status.Ok)
  assert(response.bodyAsText.compile.string.unsafeRunSync().nonEmpty)

  val notfound: Response[IO] = routes(Request(GET, uri"/nonexistent")).unsafeRunSync()

  assert(notfound.status == Status.NotFound)
}

So why is this blog post about something else if http4s is so great?? Well, let's consider the following points.

First, any web service can grow in size, where by size I'm referring to the possible API endpoints a service can present to the outside world. Partial functions are good, but they can get unwieldy due to the size of your service. Fortunately, in http4s you can break your routing functions into smaller logical units and merge them together just before serving them up. But what does a function offer if the types are too vague? A Response[IO] doesn't tell you too much about what kind of response it will provide, unless you look directly at each and every individual implementation.

Another point that is particularly relevant to ClearScore—and most likely many other software companies today—is that we practice proper documentation of our REST API's, especially for internal use. This is because we have a lot of services. Because we are only human, we can't possibly know how every service works, all the time. Fortunately, we utilize quite a common practice in web-based software development: Swagger/OpenAPI-based documentation. In fact, every service in ClearScore has to have an HTML accessible Swagger UI clearly showing you what a REST API can do. As a new starter at ClearScore, this has been extremely useful for learning about the different services in our ecosystem.

Unfortunately, being human also means we still have other flaws: maintaining the Swagger documentation (JSON or yaml) can be painful or simply forgotten.

Enter tapir.

tapir

tapir (Typed API descRiptions) is a library from the awesome people at SoftwareMill. At ClearScore, we already depend on one of their other popular libraries, sttp. Recently though, through the introduction via one of my work colleagues ^1 , I also learned of tapir

tapir presents another level of abstraction to HTTP services, specifically those written in Scala. With http4s in mind ^2 , tapir provides a typed DSL that sits not entirely on top of http4s, but more so, nicely just over the routing logic it's designed around. By doing so, the interoperability between tapir and http4s is trivial.

So what does my service instead look like in tapir?

import sttp.tapir._
import sttp.tapir.server.http4s._

// assume we have the following implicity in scope
// serverOptions: Http4sServerOptions[IO], fs: Sync[IO], fcs: ContextShift[IO]

val timeEndpoint = endpoint.in("time").get.out(stringBody)
val service = timeEndpoint.toRoutes(_ => IO((theTime().asRight[Unit])))

Hmm, does that look better? Let's break it down and see. So the first thing to understand is tapir uses a completely different model for building your API. In my own words, I'd say it bears resemblance to the builder pattern that some of us are used to in Java and maybe in some Scala libraries. endpoint acts as an empty starting point to build our service routes. In my example, timeEndpoint is:

• an endpoint with an input on the path "time",
• that will be a GET method, and
• that will output a String HTTP body response

The last line calls toRoutes to "provide the logic" for our essentially templated function. In this case, our GET path templates a function of Unit => IO[Either[Unit, String]]. tapir uses Either to represent the possibility of an error occurring with your logic. In our simple example, our side-effecting method theTime is forced into a right and delayed in an IO. Obviously, your real logic would do a better job at handling effects and error handling. Alternatively, tapir also provides toRouteRecoverErrors which will handle any unexpected errors from your effect and hide the use of Either from you.

Just a quick aside: I realize having a function of Unit => IO[Either[Unit, String]] for our GET isn't really helpful if we actually want to report your error to the API caller. Don't worry, tapir can map the left side of an Either to an HTTP error status code and a response using errorOut. For now, to keep things simple, I will ignore proper error handling in my examples.

In the end, service has the same type as before (HttpRoutes[IO]) and can be used without difference in our Time app. So at this point, maybe you're not entirely convinced about using a different method to create effectively the same thing yet? Let's look at a slightly bigger example. Imagine our service did this instead:

val service = HttpRoutes.of[IO] {
  case GET -> Root / "time" =>
    Ok(theTime())
  case request @ PUT -> Root / "time" =>
    ???     
  case GET -> Root / "time" / "zone" =>
    ???
  case request @ PUT -> Root / "time" / "zone" =>
    ???
}

In tapir, we could do the following:

val timeEndpoint = endpoint.in("time")
val timeZoneEndpoint = timeEndpoint.in("zone")

val a = timeEndpoint.get.out(stringBody).toRoutes(_ => IO((theTime().asRight[Unit])))
val b = timeEndpoint.put.in(stringBody).toRoutes(time => ???)
val c = timeZoneEndpoint.get.out(stringBody).toRoutes(_ => ???)
val d = timeZoneEndpoint.put.in(stringBody).toRoutes(zone => ???)

val service = a <+> b <+> c <+> d

Let's highlight some benefits:

• Unlike the repeated paths in our http4s route, in the tapir example we define just once a path for "time". Furthermore, only once do we define a sub-path to "zone" (i.e. "/time/zone"). In terms of refactor, this is a major plus, especially when we decide to add or change logical paths.
• In the http4s route, I've skipped the entity body deserialization steps that would have to occur in each PUT route. It's also not clear what kind of body content-type we are expecting. In tapir, I've clearly stated the body content we expect is a String ^3 .
• Finally, as highlighted in the previous example, now that I've defined my routes once, the logic just needs to be plugged in (i.e. where my ??? placeholders are). tapir has done its type safe magic to state what kind of logic it expects based on the inputs and outputs I've defined. I think this is a major win because it means our logic can be referentially transparent. It's just a function that happens to match the types defined. With a better set up (i.e. using dependency injection of some sort), we could easily swap and/or test our HTTP routes and logic.

Take these benefits and multiple them a few more times with respect to the size of your REST API. I'm quite confident using the builder pattern might save you some time reasoning about what it is your REST API should do and how it should do it. Furthermore, the type safety enforced by the plug and play logic will make developing your API feel more like a block building experience.

One glaring downside to the tapir approach is that we need to combine the individual routes to create our service. In comparison to working with an increasingly complex partial functions, I've come to prefer tapir when working with larger REST APIs. In fact, that is usually my deciding factor: if the REST API will be considerably large, I'll defer to tapir to wrangle together all the varying server logic into orderly units. If the REST API will only serve a handful of endpoints, most of which probably belong to the same or related logic implementations, then I stick with writing smaller routes using the http4s DSL.

Documenting your API

The second thing that attracted me to tapir was an extension to the core library that allowed all those type safe routes I handcrafted to automagically get documentation. Let's rewrite the last code example to quickly demonstrate.

import sttp.tapir.docs.openapi._
import sttp.tapir.openapi.circe.yaml._

val timeEndpoint = endpoint.in("time")
val timeZoneEndpoint = timeEndpoint.in("zone")

object Endpoints {
  val timeBody = stringBody.description("ISO time")
  val zoneBody = stringBody.description("Zone ID")

  val getTime = timeEndpoint.get.out(timeBody).description("Retrieves the current time")
  val setTime = timeEndpoint.put.in(timeBody).description("Changes the current time")
  val getZone = timeZoneEndpoint.get.out(zoneBody).description("Retrieves the current time zone")
  val setZone = timeZoneEndpoint.put.in(zoneBody).description("Changes the current time zone")
}

import Endpoints._

val service = getTime.toRoutes(_ => IO((theTime().asRight[Unit]))) <+> 
              setTime.toRoutes(???) <+> 
              getZone.toRoutes(???) <+> 
              setZone.toRoutes(???)

val openAPI = Seq(getTime, setTime, getZone, setZone).toOpenAPI(title = "Time API", version = "V1")

// println(openAPI.toYaml) would print a completely auto-generated OpenAPI specification in yaml

I've slightly organized things a bit differently, mainly for readability. However, there are only really two things I've made in addition: I added human-readable description's to my endpoints and response/request bodies; and I merged them together into tapir's OpenAPI type. As commented in the code, if I wanted to view or write the OpenAPI specification, I could println it to get the following:

openapi: 3.0.1
info:
  title: Time API
  version: V1
paths:
  /time:
    get:
      description: Retrieves the current time
      operationId: getTime
      responses:
        '200':
          description: ISO time
          content:
            text/plain:
              schema:
                type: string
    put:
      description: Changes the current time
      operationId: putTime
      requestBody:
        description: ISO time
        content:
          text/plain:
            schema:
              type: string
        required: true
      responses:
        '200':
          description: ''
  /time/zone:
    get:
      description: Retrieves the current time zone
      operationId: getTimeZone
      responses:
        '200':
          description: Zone ID
          content:
            text/plain:
              schema:
                type: string
    put:
      description: Changes the current time zone
      operationId: putTimeZone
      requestBody:
        description: Zone ID
        content:
          text/plain:
            schema:
              type: string
        required: true
      responses:
        '200':
          description: ''

Imagine if I had many more API endpoints than this to maintain and/or refactor 😳. To top it all off, tapir also provides a library to route a Swagger UI instance from within your http4s application to serve your API spec.

import sttp.tapir.swagger.http4s.SwaggerHttp4s
val swaggerRoute = new SwaggerHttp4s(openAPI.toYaml).routes // and then just add this to your http4s instance

Auto generated documentation and effectively plug and play web application coding based on Scala types, hopefully that's enough reason for you to consider trying out tapir on your next http4s application.

^1 Thanks to Andrew Davidson, who introduced me to tapir.

^2 tapir supports http4s, Akka HTTP, and Finatra.

^3 tapir supports other robust content types, for example, it has libraries for Circe JSON integrations.

"Tapir" by brainstorm1984 is licensed under CC BY-ND 2.0