Synchronized immutable state with time travel

I am working on @webstudiois - an open-source visual development platform.
As part of the UI for the Webstudio Designer, a very interactive interface, I needed an undo/redo functionality and created a library for synchronizing state across the client, server, and multiple clients in future releases.

Challenge 1 - immutability

Is immutability a solved problem in EcmaScript? Not by a long shot.

We have got far, starting with initial attempts using abstractions like immutable which has a severe downside with interoperability. Then we went over to manually cloning objects and arrays using built-in language features like the spread operator, which makes it super unreadable to change complex structures. For example, nested objects or arrays with objects because you have to go inside whatever you want to change and then clone every object on your path to keep the object immutable.

Structured cloning is an API I look very much forward to, though it's not solving the same problem Immer does.

Immer is the first popular library to solve immutability in an interoperable way. It lets you work with regular objects, so you have no lock-in on Immer and can use vanilla functions that mutate anything you give them without causing any trouble. The most crucial feature of Immer for me, though, is patches generation.

import produce from "immer"
const nextState = produce(draft => {
  // Mutable behavior is allowed on the draft.
  draft[1].done = true
  draft.push({title: "Tweet about it"})
})(previousState)

Challenge 2 - syncing UI state to the server

When using immutable data structures, you mainly have only two options on how to save the changes to the server:

1. Send custom updates to the server using any APIs.

   This is probably the most popular approach, which may work fine in simple cases, but won't scale well as your application grows.

   You have to rely on each update implemented correctly because it usually ends up with custom logic for each update that prepares the data to be sent to the server. You also need to handle errors to let you reflect the server state on the client, making it hard to do, especially with an optimistic UI. Over time, this approach provides a large surface for potential errors.

   It can't be undone easily.

   When data you sent to the server was manually prepared for this update, undoing that update will also require you to write custom code that would undo the same update. In some cases, it becomes very complex, especially when you have to talk to multiple endpoints, and every endpoint also has to have a way to undo the change.

2. Sending entire snapshots to the server

Challenge 3 - sending the minimal updates to the server

It is super easy to update the server using snapshots, reflecting everything you want to be permanently stored. This makes sure the data on the server and the client is consistent, and it's super easy to keep it this way over time.

Downsides:

1. You have to potentially send massive payloads to the server, making it impossible to sync frequently and quickly
2. You can't easily build collaborative features because if two clients can update the same state, you have to figure out how to make a conflict resolution work with a snapshot. We already lost the information about who changed what and when.

Doing it this way, you can quickly run into conflicts even by opening two separate tabs. Also, you are limited in the ability to use iframes or web workers because if any of them require access to the state, they can't get easily synchronized without sending the entire state to them. This is a real deal-breaker since it will negate any potential benefit of using a web worker.

Challenge 4 - undo/redo the UI state

If some centralized store-like abstraction does not manage your UI state, undoing it becomes problematic because you have to make sure that every representation of a certain state is being updated.

In addition to that, you need to be able to distinguish multiple types of states:

• Data state - essentially, what the server knows about your app and should be undoable
• UI state - current state of the UI that is not reflected on the server and should not be undoable, e.g., a derrived state
• Ephemeral state - a state that is temporarily reflected on the UI, and should not be undoable, e.g., a selected element

Having store-like abstraction to manage states requires you to manually define which state and action is undoable, which state is being reflected on the server, and which one is ephemeral if it's not a component's local state.

There are many edge cases, and managing that in a large app is not trivial.

Solution with autogenerated patches

All of the above challenges were a problem for me. I need a minimal amount of changes to be sent over the network. I need a way to easily undo/redo the changes. I need the ability to reflect changes inside an iframe. I need to reflect changes across tabs and windows for a collaborative editing experience.

Let me introduce you Immerhin.

Immerhin is a very thin abstraction on top of Immer that lets you use basically any state management (Redux-like stores etc.). I am personally using this tiny library react nano state with it.

Immerhin lets you declare a transaction to identify a single user action, which may update multiple states. Thanks to Immer, it enables you to mutate all those states without losing immutability. It uses Immer's patches and revise patches to send the minimal amount of changes to any consumers and enables undo/redo out of the box.

Immerhin's current development state is not battle-tested and still lacks many features, so consider this as a project to follow for future releases or use for non-production projects. You can follow the repo or Webstudio's Twitter account for the updates on this.

Here is a small Codesandbox demo that shows how it can be used to have a shared state between 2 separate lists and have an undo/redo functionality with no extra logic to manage for undo/redo.

Here is how to use Immerhin:

Create container

Container is an object that has .value reflecting the current value and a .dispatch(nextValue) which will update all container subscribers. Immerhin only cares about those two properties. How you do the rest is up to you.

Example using react-nano-state:

import {createContainer} from 'react-nano-state';
const container1 = createContainer(initialValue);
const container2 = createContainer(initialValue);

Create a transaction

Transactions let Immerhin know which containers to update and which patches to apply. They are used to implement automatic undo and sync.

import store from 'immerhin';

// - generate patches
// - update states
// - inform all subscribers
// - register a transaction for potential undo/redo and sync calls
store.createTransaction(
  [container1, container2, ...rest],
  (value1, value2, ...rest) => {
    mutateValue(value1);
    mutateValue(value2);
    // ...
  }
);

Sending the changes

It's up to you how you send the changes. Immerhin provides a sync() function that gives you all accumulated changes so far and clears the stack.

// Setup periodic sync with a fetch, or do this with Websocket
setInterval(async () => {
  const entries = sync();
  await fetch("/patch", { method: "POST", payload: JSON.stringify(entries) });
}, 1000);

Undo/redo the changes

Since Immerhin knows everything from transactions, you just need to call store.undo() or store.redo() to update those containers with revised patches from Immer.

Downsides with Immerhin

Like with any solution, there are tradeoffs. Here are a few I could come up with so far:

1. Potentially, if one of the consumers did not receive the patches, their state is getting out of sync. Currently, Immerhin doesn't provide a way to work around this. If you have great ideas, please share them on Twitter or over the issues. An ordered id on each transaction could potentially be used to throw an error if something is missing and the client has to refetch.

2. Immer patches contain a path, but a path is essentially relative to the original object. Should there be inconsistency, the path is going to be wrong. Potentially we could use IDs instead of paths. Also if the previous problem is solved, this is not a huge deal because as soon as the update is out of sync, it can be rejected.

3. Immer uses patches spec identical to JSON patch with a difference on the path being an array, not a string.

Future of Immerhin

There is a bunch of missing features I really want to implement in the future, and Webstudio will need:

1. Conflict resolution
2. Sync mechanism between iframes and tabs

If you click on the "Try Webstudio" button on webstudio.is, you will see the designer interface that uses Immerhin. While the UI is very alpha, I am curious if you will already find obvious bugs with undo/redo functionality.