Proper RPC with Prim+RPC

I did not think that RPC would become a popular topic in 2023 but I'm happy to find that I was very wrong. It seems that RPC has become the topic to tackle with the introduction of many new frameworks that blur the line between the client and server. It's overwhelming even, especially for me, as someone who has been working on my own RPC framework that offers another take on how RPC should be handled in JavaScript and TypeScript.

My framework however does things a little differently: it aims to use the JavaScript language itself. That is to say, this framework allows you to write regular JavaScript functions on the server and call those functions on the client without additional wrappers, without a compiler step, with full type definitions, support for handling files, callbacks, errors, and advanced types, and the ability to support your favorite existing server and client, over the transport of your choice.

I'd like to introduce Prim+RPC: a JavaScript library that bridges otherwise incompatible JavaScript environments. The goal of this RPC library is to let developers focus less on message transport and more on the message being sent, using the frameworks and tools you already love using.

I've worked across many frameworks over the past few years on many projects and those tools have not stood still. In other words, the JavaScript ecosystem is changing every day. New tools are coming out consistently and it's incredibly exciting but staying up to date and using the latest tools can lead to framework fatigue. More specifically, changes to foundational frameworks like those responsible for server and client communication have been a pain point for me, for some time. They can sometimes be difficult to learn and set up, or they require a separate schema to be defined, or come with their own validation logic that needs to be followed, or require client regeneration on API changes, or they enforce a paradigm shift in how we think about defining APIs.

Each new tool has its unique set of features designed to make API development easier. And it truly can be easier once that framework has been learned and is embraced by a community. It's an exciting time to be developing with JavaScript because these tools are introducing many new features that are intended to make development easier. However, these server frameworks also sometimes enforce new ideas that go beyond transport such as validations, schemas, generated clients, and wrappers around the code that I actually want to call on the server from the client.

With all of the innovations in these frameworks, one thing has stayed consistent in my search for new server/client frameworks: my need to call a function on the server and get the result of that function on the client.

So that's the library I set out to create. A framework that, once set up, allows me to write a regular JavaScript function and call that function directly or, at least, appear to the client as if it's calling the function directly. The result is fairly close to that intention. I can write plain JavaScript, using my favorite libraries, and allow Prim+RPC to send function calls and receive results.

Let's get straight to an example. Let's say that I just have a function on a server that I want to make available to the client.

// this is the function that we want to call on the server
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}

// this is how we want to call the function on the client
const greeting = sayHello("Backend", "Frontend")

Prim+RPC makes this possible without introducing a lot of additional wrappers. Let's see how with a real demo. In this example, we set up the Prim+RPC server and client in a single file for testing purposes: this way we can compare the result on the client with the result of the actual function call. Our example function will just return a simple greeting but we could define whatever function we'd like here. Let's see how we can call this function in Prim+RPC:

import { createPrimClient, createPrimServer, testing } from "@doseofted/prim-rpc"

// write a regular JavaScript function
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true // expose function to Prim+RPC

// set up plugins for your transport of choice (this is a demo)
const plugins = testing.createPrimTestingPlugins()

// pass your functions and transport plugins to the server
const module = { sayHello }
const { methodHandler, callbackHandler } = plugins
const server = createPrimServer({ module, methodHandler, callbackHandler })

// pass your transport plugins to the client
const { methodPlugin, callbackPlugin } = plugins
const client = createPrimClient<typeof module>({ methodPlugin, callbackPlugin })

// call your function: it's automatically translated into RPC
const greeting = await client.sayHello("Backend", "Frontend")
const expected = sayHello("Backend", "Frontend")
console.log(greeting === expected) // true

In this example, a function sayHello() is given to the Prim+RPC server. The client then calls this function and receives a result. It's important to note that the client was never given this function but it can still read the result of the function through the handler plugins that we configured (method and callback handlers as they are known in Prim+RPC). The function call was automatically translated into an RPC using a JavaScript Proxy, processed by the server, and the result was parsed back into the expected format.

You can try this example yourself by running this command:

npx giget@latest gh:doseofted/prim-rpc-examples/simple-test prim-rpc-examples/simple-test

Of course, this code snippet is just a demo in a single file. We could've just called the function directly. Where Prim+RPC is useful is crossing the boundary between client and server. That's why Prim+RPC includes plugins for communication across HTTP, WebSocket, Web Workers, and can support additional transports by writing custom plugins. We could set up an HTTP server with Prim+RPC like so:

import Fastify from "fastify"
import { createPrimServer } from "@doseofted/prim-rpc"
import { createMethodHandler } from "@doseofted/prim-rpc-plugins/fastify"

// here's our regular JavaScript function again
function sayHello(x: string, y: string) {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true
const module = { sayHello }

// in this example, we'll use Fastify as an HTTP server
const fastify = Fastify()
const server = createPrimServer({
    module,
    methodHandler: createMethodHandler({ fastify })
})
await fastify.listen({ port: 1234 })

// expose type definitions for use on the clinet
export type { module }

The Prim+RPC server has now been configured at http://localhost:1234/prim. Our function is now available to be called remotely. We just need to set up a client to communicate with it. Let's set up a Fetch plugin to connect the client and server.

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser"
import type { module } from "./my-server"

const client = createPrimClient<typeof module>({
    endpoint: "http://localhost:1234/prim",
    methodPlugin: createMethodPlugin()
})

const greeting = await client.sayHello("Backend", "Frontend")
console.log(greeting) // "Backend, meet Frontend."

When we run this code, the client makes a request to our server at the given endpoint and receives the result from Prim+RPC without having access to the function directly. It even has full typed support with TypeScript by passing the module types (not the module itself) to the client. This is powerful because you get full code completion in your editor without generating a client or needing to define a schema specific to a framework. You can even use these types across projects and repositories by uploading these types to a registry (like NPM or Verdaccio). This could even allow you to define a public API with Prim+RPC and for users of the API to have a fully typed client without needing to download an API-specific client for JavaScript.

You can download this full example with the following command:

npx giget@latest gh:doseofted/prim-rpc-examples/client-server prim-rpc-examples/client-server

While the function we're using as an example here just prints out a greeting (sayHello()), we can do much more than that. We can pass callbacks to our function, throw errors, upload files, and more. This example only demonstrates the basics of Prim+RPC. I've written a full guide on how to get started with Prim+RPC and more examples are available on the website.

There are a lot of exciting RPC-related tools being released right now and I'm excited to add another to that list. Prim+RPC is now available as a prerelease and I aim to work toward a full release. I'm personally very excited to use this tool in my projects and I hope that you are as well. If you are excited about the project, consider giving it a star on Github and trying it out today!

I'll be writing more about Prim+RPC occasionally among other design/development topics. Follow my blog for more!