The New Transparent RPC for JavaScript

Getting data from a JavaScript server is too complicated. I'm not saying that it's difficult but complicated. We have many options to do so but they all require us to rethink how we acquire that data. Let's say that I have some function defined on the server:

function createProfile({ name, email, password, photo }) {
    // ... (implementation irrelevant)
    return newProfile.id
}

This function does exactly what I want. I haven't yet thought about the server, I just know that I want to create a profile and get the ID back. Let's consider how this function gets to the client.

We could set up an HTTP server but we'll need to wrap our function in a route, mapping our function to HTTP-specific verbs and errors. We could try GraphQL but we'll need to define a schema for our function in its IDL. We could try gRPC but we'll need to create protocol buffers, implement them, and generate a client. We could try tRPC but we'll need to wrap our function in its procedures and router. But even then, most of these options require special processing for files or additional setup for a type-safe client or a lot more work to support real-time events from the server.

What's the Alternative?

None of these options are necessarily difficult and there are additional frameworks to make them easier but they all add a lot of wrappers to the function that I want to call when I just want to call the function itself. And this is just one function, what if I have hundreds?

If I have a server written in JavaScript and the client is JavaScript, shouldn’t I be able to call the function, like it’s JavaScript?

createProfile({
    name: "Ted Test",
    email: "ted@example.com",
    password: "redacted :)",
    photo: new File([...], "headshot.jpg")
})

That's what we're going to do today. We'll use an open-source library called Prim+RPC: a thin layer between the server and client of your choice that translates function calls and results so that you can simply call your server functions on the client as intended.

💡 This article is a much shorter adaptation of the setup guide on the Prim+RPC website. If you'd like a more in-depth look at this project, check out the full guide.

Star on GitHub

Let's Get Started!

I've set up a starter project for us, so we don't have to start from scratch. It is a monorepo composed of two modules: a Node server where our function will be defined and a client that will call the function, both serialized using Prim+RPC's plugins for the Fetch API.

You can download the project with the command below or download it directly:

npx giget@latest "gh:doseofted/prim-rpc-starter#follow-along"

It's also available and easy to get started on StackBlitz:

You can start the project by running npm install to install dependencies and then npm run dev to start both the server and client parts of the project. The client will be available at http://localhost:3000.

We'll be greeted with a message "Not implemented". Let's fix that!

Server Setup

Let's set up the server first. First things first: let's define a function. We'll work with something simple to get started. Add this function to server/module.ts:

export function sayHello(x = "Backend", y = "Frontend") {
    return `${x}, meet ${y}.`
}
sayHello.rpc = true

This is just a regular JavaScript function. The only thing that we added is a property .rpc that signals to Prim+RPC that we want to expose this function as RPC. Since Functions in JavaScript are also Objects, this is valid syntax (we could also omit this property and add our function to an allow-list).

Moving on to the server framework itself, replace the contents of server/index.ts with the following. I'll explain what this does:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import * as module from "./module"

const prim = createPrimServer({ module })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

export type Module = typeof module

First, we pass the module containing our function to createPrimServer(): a framework-agnostic utility to receive RPC and respond with RPC results. We then pass the Prim+RPC server to a method handler, primFetch(), which wraps our RPC into Fetch API Request and Response objects. We also add two necessary CORS headers on every Request so that our client in the browser is allowed to request resources from the server.

💡 If you are running this example from Stackblitz, you may need to change the access-control-allow-origin header value to https://localhost:3000 if you experience any issues running the project (using secure HTTP).

The returned fetch utility is a function that takes a Request and returns a Response, as expected by server frameworks like Deno and Bun. Since Node doesn't yet support this natively, we use an adapter, createServerAdapter(), for Node's createServer utility. Now we can expose our function over HTTP, using port 3000. As a last step, we export the type of our module which will later become a type import on the client (exposing types only, not the actual code).

We did it. We're ready to move on to the client. As a sanity check, we can test this out now by sending a simple request. With the server running, try out this command :

curl "http://localhost:3001/prim/sayHello?0=Backend&1=Terminal"

Our Prim+RPC server is now available at http://localhost:3000/prim (you can also change the path prefix if you'd like).

Client Setup

The client is very easy to set up. Replace the contents of client/prim.ts with the following. I'll explain what this does:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import type { Module } from "../server"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
})
export default client

We have set up the Prim+RPC client, a framework-agnostic tool for sending RPC and receiving RPC results. We pass a type parameter to our client with the type of our module: the Prim+RPC client will apply RPC-specific transformations to our types. Note that we don't pass the module code itself, only the TypeScript types. We then pass the server address to the client as well as a method plugin that will turn our function calls into Requests for the Fetch API. We export this client to be used throughout our app.

That's all there is to it. We can now use this client to call our function. Replace the contents of client/index.ts with the following:

import { client } from "./prim"

const greeting = await client.sayHello()
console.log(greeting) // "Frontend, meet Backend."

const app = document.getElementById("app")
if (app) app.innerText = greeting

We didn't need to generate code for a client or wrap our function call in some request specific to the client, we just called the function and received a result.

Since our server and client are in two separate environments and it may take a (very short) amount of time to resolve, our result is wrapped in a Promise so as not to block execution on the main thread (which means we await the result). But the Prim+RPC client automatically transforms our Module types for us, so we know.

That Was Easy

Now, we can add more functions to the server and we don't need any more setup to call those functions on the client. It's just ready to be called.

We could stop here. But we can do much more with Prim+RPC: we can support file uploads and downloads, additional types, add validation, and even pass server context to our functions when needed.

Read on to learn how we can use these features.

⭐ Prim+RPC is an open-source project, currently in prerelease. If you find the library useful, consider giving it a star on GitHub and sharing it with others.

Star on GitHub

Adding Validation

When calling a function over the Internet, we don't know what could be passed to the server. Prim+RPC doesn't validate arguments by default but we can add this easily. Let's validate using Zod as an example. First, we'll install it on the server:

cd server
npm add zod

Let's modify our server/module.ts so we validate and overwrite the given arguments:

import { z } from "zod"

export function sayHello(x = "Backend", y = "Frontend") {
    ;[x, y] = z.tuple([z.string(), z.string()]).parse([x, y])
    return `${x}, meet ${y}.`
}
sayHello.rpc = true

And now we have validation. If you'd like to ensure functions are never defined without validation, check out Prim+RPC's security guide for more details.

File Support

Prim+RPC can support files as arguments and return them to the caller. With the Fetch plugin, we don't even have to do any extra work. It works out of the box! Let's demonstrate this with an example that converts a Markdown file to HTML, using micromark. First, we'll install it on the server:

cd server
npm add micromark

And replace server/module.ts with the following:

import { micromark } from "micromark"
import { File } from "node:buffer"

export async function markdownToHtml(markdownFile: File | string) {
    const markdown = typeof markdownFile === "string" ? markdownFile : await markdownFile.text()
    const html = micromark(markdown)
    return new File([html], "snippet.html", { type: "text/html" })
}
markdownToHtml.rpc = true

On the client, we just call it. Update client/index.ts:

import { client } from "./prim"

const markdown = "[**Backend**, meet **Frontend**.](https://prim.doseofted.me/)"
const htmlFile = await client.markdownToHtml(markdown)
console.log(htmlFile.name, htmlFile instanceof File)

const app = document.getElementById("app")
if (app) app.innerHTML = await htmlFile.text()

We could also access returned files from a URL, for easy file downloads. Try the following command or visit the given URL in your web browser:

curl "http://localhost:3001/prim/markdownToHtml?0=_Hello%20there_"

Extended Types

Prim+RPC, by default, can support arguments and return values serializable by JSON (with files processed separately). But what if we want more? Libraries like superjson can add support for additional types like Map, Set, Date, BigInt, and more. And Prim+RPC supports custom serialization like superjson! Let's install it in our project:

cd server
npm add superjson
cd ../client
npm add superjson

Now we can change the .jsonHandler option in server/index.ts. Replace the contents of this file with the following:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import jsonHandler from "superjson"

const prim = createPrimServer({ module, jsonHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

export type Module = typeof module

Serialization and deserialization go hand-in-hand so let's add this to the client too. Replace the contents of client/prim.ts with the following:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
    jsonHandler,
})
export default client

Now can create functions that use these new supported types. Let's try it out. Replace server/module.ts with:

export function whatIsDayAfter(day: Date) {
    return new Date(day.valueOf() + 1000 * 60 * 60 * 24)
}
whatIsDayAfter.rpc = true

And call it in client/index.ts:

import { client } from "./prim"

const tomorrow = await client.whatIsDayAfter(new Date())
console.log(tomorrow, tomorrow instanceof Date)

const app = document.getElementById("app")
if (app) app.innerText = tomorrow.toDateString()

See the JSON Handler documentation to learn how to set up other popular JSON alternatives.

Support Callbacks

Prim+RPC can support callbacks on your function as well. We just need to set up a callback handler. While functions return once, callbacks on a function can fire multiple times so we need to maintain a connection to the server. For this, we'll use a callback handler backed by a WebSocket connection.

First, let's install ws which will act as our WebSocket server:

cd server
npm add ws
npm add -D @types/ws

Replace server/index.ts with the following. And stick with me: this is one-time setup and we don't have to touch WebSockets again after this:

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createCallbackHandler } from "@doseofted/prim-rpc-plugins/ws"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import { WebSocketServer } from "ws"
import jsonHandler from "superjson"

const wss = new WebSocketServer({ noServer: true })
const callbackHandler = createCallbackHandler({ wss })
const prim = createPrimServer({ module, jsonHandler, callbackHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
}
const fetch = primFetch({ prim, postprocess })

const fetchAdapter = createServerAdapter(fetch)
const server = createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

server.on("upgrade", (request, socket, head) => {
    wss.handleUpgrade(request, socket, head, ws => {
        wss.emit("connection", ws, request)
    })
})

export type Module = typeof module

Are you still with me? We have just set up a WebSocket connection and passed that WebSocket server to Prim+RPC. Now the server can handle callbacks on your functions!

However, the client doesn't yet know how to tell the server that it has callbacks. Let's do that now using the WebSocket callback plugin:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import { createCallbackPlugin } from "@doseofted/prim-rpc-plugins/browser-websocket"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin(),
    callbackPlugin: createCallbackPlugin(),
    jsonHandler,
})
export default client

And now we can use callbacks on our functions. Let's create a function that uses them in server/module.ts:

export function typeMessage(message: string, typed: (letter: string) => void) {
    let timeout = 0
    const letters = message.split("")
    for (const letter of letters) {
        setTimeout(() => typed(letter), ++timeout * 300)
    }
}
typeMessage.rpc = true

This will type the message that we give it, one-by-one. Not a great use of server resources but a good demo. Let's try this on the client (client/index.ts):

import { client } from "./prim"

const app = document.getElementById("app")
if (app) {
    client.typeMessage("Hello!", letter => {
        app.innerText += letter
    })
}

And now we see the message typed out on the page!

Pass Server Context

Up to this point, we have not really touched the server inside of our functions. Instead, we've passed everything that our function needs as arguments. However, we can't always just ignore the server: it contains information that we may need that doesn't have direct comparisons to calling a function in JavaScript. But we can still access the server context from our functions. And that's what we're going to do next.

In this example, we’ll set a secret cookie from the client that is required to access our secret function. Without the cookie: no function access. However, our function won’t have to touch the cookie at all.

First, let's install a tiny helper package:

cd server
npm add cookie
npm add -D @types/cookie

And now we can use this package on our server. We'll define a contextTransform option that will be given the Request object (from the Fetch API) for every function call that we make to the server. The returned value from this function will become the context (this value) of our function.

Replace server/index.ts with the following:

💡 If you are running this example from Stackblitz, you will need to change the access-control-allow-origin header value to https://localhost:3000 for this example to work in the browser (using secure HTTP).

import { createPrimServer } from "@doseofted/prim-rpc"
import { primFetch } from "@doseofted/prim-rpc-plugins/server-fetch"
import { createCallbackHandler } from "@doseofted/prim-rpc-plugins/ws"
import * as module from "./module"
import { createServer } from "node:http"
import { createServerAdapter } from "@whatwg-node/server"
import { WebSocketServer } from "ws"
import jsonHandler from "superjson"
import { parse, serialize } from "cookie"

const cookieOpts = { httpOnly: true, sameSite: "none", secure: true } as const
function contextTransform(req: Request, res?: { headers: Headers }) {
    const secret = "can't-touch-this"
    return {
        setSecret(given: string) {
            res?.headers.set("set-cookie", serialize("secret", given, cookieOpts))
        },
        get allowed() {
            return secret === parse(req.headers.get("cookie") ?? "").secret
        },
    }
}
export type ServerContext = ReturnType<typeof contextTransform>

const wss = new WebSocketServer({ noServer: true })
const callbackHandler = createCallbackHandler({ wss })
const prim = createPrimServer({ module, jsonHandler, callbackHandler })
function postprocess(res: Response) {
    res.headers.set("access-control-allow-origin", "http://localhost:3000")
    res.headers.set("access-control-allow-headers", "content-type")
    res.headers.set("access-control-allow-credentials", "true")
}
const fetch = primFetch({ prim, postprocess, contextTransform })

const fetchAdapter = createServerAdapter(fetch)
const server = createServer(fetchAdapter).listen(3001)
console.log("Prim+RPC is available at http://localhost:3001/prim")

server.on("upgrade", (request, socket, head) => {
    wss.handleUpgrade(request, socket, head, ws => {
        wss.emit("connection", ws, request)
    })
})

export type Module = typeof module

Inside of the contextTransform function, we return setSecret() and allowed which will be used from our function. We have also updated the CORS headers to allow credentials so that the browser can utilize the cookies we set. Using TypeScript, we get that return type and assign it a type ServerContext. We'll pass this as a type hint to our function.

❗ This is of course a demo. In a real application, you will want to use some form of cryptography.

Replace server/module.ts with the following:

import type { ServerContext } from "./index"

export function secretMessage(this: ServerContext, secret?: string) {
    if (secret) {
        this.setSecret(secret)
        return ""
    }
    if (!this.allowed) throw new Error("No secret, no entry.")
    return "Access granted! The answer is 42."
}
secretMessage.rpc = true

Notice that we only accessed the functions that we defined in the server's context so we didn't even have to touch the cookie at all. The this parameter is only a type hint and is not an argument that we have to pass: Prim+RPC will handle that for us.

Now we can make one small change to the client. Since we're passing cookies down to the client, we must set the credentials option of the fetch function to “include” so that cookies can be set properly.

Let's do that real quick in client/prim.ts:

import { createPrimClient } from "@doseofted/prim-rpc"
import { createMethodPlugin } from "@doseofted/prim-rpc-plugins/browser-fetch"
import { createCallbackPlugin } from "@doseofted/prim-rpc-plugins/browser-websocket"
import type { Module } from "../server"
import jsonHandler from "superjson"

export const client = createPrimClient<Module>({
    endpoint: "http://localhost:3001/prim",
    methodPlugin: createMethodPlugin({ credentials: "include" }),
    callbackPlugin: createCallbackPlugin(),
    jsonHandler,
})
export default client

And now we can call our function in client/index.ts:

import { client } from "./prim"

await client.secretMessage("can't-touch-this")

const app = document.getElementById("app")
if (app) app.innerText = await client.secretMessage()

Since we have set the secret in a cookie, we don't technically need to set the cookie anymore. If we just comment out that line:

import { client } from "./prim"

// await client.secretMessage("can't-touch-this")

const app = document.getElementById("app")
if (app) app.innerText = await client.secretMessage()

You'll find that we can still see the secret because we had already set the secret before and no longer need to pass any arguments to our function. It's handled in the server context!

This can be a powerful tool for setting up authentication, adding redirects, or otherwise integrating with the server of your choice.

Summary

All of the frameworks that I mentioned in the beginning have their place. But when I'm getting started on a new project, how messages get passed from server to client is far from the first thing on my mind. It's important but it shouldn't dictate the logic happening on my server or restrict what I want to do.

Prim+RPC provides a powerful library for calling functions remotely and remains invisible once it is set up. I can define a function on the server of my choice and call that function on the client, all using JavaScript. Yet requests are simple enough to make outside of JavaScript as well.

⭐ Thank you for following along with me. If you like this method of retrieving data from the server, consider giving Prim+RPC a star on GitHub.

Star on GitHub

Try adding your own functions to this project, maybe build out a simple API. If you get stuck, there are several examples on the Prim+RPC website as well as this post.