E2E Reactivity using Svelte with Phoenix LiveView

This post is based on a talk given at ElixirConf 2022

Svelte is a reactive JS framework that is delightful to use - it finds a perfect balance between simplicity and power. But since it's running in a browser, there's still the issue of fetching data from the server, and keeping that data state consistent between clients and backend servers.

In contrast, LiveView (part of Elixir's Phoenix framework) takes a contrarian view and insists that state should be managed only on the server. This makes sense since data is stored in a database close to the server, and not accessible by the browser! However there are drawbacks to the user experience, as events need to make a round trip from the browser to the server and back whenever a user changes state.

If only there is a way to combine the snappy UX of client side rendering in Svelte, with the strong-consistency state management of LiveView. Well dear reader, what if I told you...

But first, why Svelte?

Few JavaScript frameworks I've used have captured "simplicity" in the way that Svelte is designed. In a gross generalization, code runs in the browser primarily to store state, capture events and manipulate DOM. Sometimes this is executing simple logic based on user interaction with the page, such as form entry validation. In other cases, there is complex interplay between elements displayed on the webpage. Often some part of the markup needs to be re-generated and rendered, as a result of changes.

A gross generalization: code runs in the browser primarily to store state, capture events and manipulate DOM

A lot of modern frameworks create significant abstractions on top of this basic utility, such as "lifecycles" for rendering, or finite state machines to manage data. They also like to reinvent markup, as though HTML/CSS/DOM are not sufficient for modern browsers on their own (cough JSX cough).

Svelte uses vanilla HTML/CSS/JS and adds just enough extra syntax to render JS variables in HTML, conditionally render blocks based on variables' state, loop and iterate over data, etc. Components logically organize and scope HTML/CSS/JS into a single file. This means you can grab some CSS from a css-tricks post and use it, without concerns about conflicting with styles outside of the component.

What is slick about Svelte is that it doesn't need a "virtual DOM" to determine which markup to re-render, but instead precompiles the page into JavaScript functions that may be called by the Svelte engine to generate updated markup, which is injected to existing DOM. This approach is not all that different than jQuery code you've probably written which manipulates DOM directly through selectors, and keeps rendering very snappy and efficient for the browser.

Svelte gives developers precise control over which JS variables should be treated as reactive, and this can include logical blocks in addition to plain variables, which means components can react to derived state.

On top of the reactive features, Svelte adds a lot of gravy - context/stores for state management across components, tweening and transitions for buttery smooth UX animations to give that native-app feeling, and actions for interfacing third-party libraries into the reactive system.

Everything everywhere all the time

All of this reactivity to local state changes is great, except for a glaring omission - a lot of state changes happen on the server, not just in the browser! Keeping state on the client up-to-date with a server creates a lot of challenges around consistency, especially if the client can update state as well. Heck, entire areas of academia are devoted to algorithms that try to keep state consistent across distributed nodes. And since the protocol of HTTP requires separate requests/responses for exchanging information, most solutions for keeping the browser up-to-date involve polling the server to refetch state.

This feels pretty clumsy compared to the smart reactivity of our client-side framework, which just figures out that state has changed and updates itself.

Enter Phoenix LiveView - a framework for server-rendered HTML that takes a similar reactive approach to re-rendering where LiveView does the hard work of tracking changes and sending the relevant diffs to the browser. This makes the server the single source of truth, regardless if state changes on the client or the server.

At the end of the day, a LiveView is nothing more than a process that receives events as messages and updates its state. The state itself is nothing more than functional and immutable Elixir data structures. The events are either internal application messages or sent by the client/browser.

While this solves hairy problems around data consistency, rendering everything on the server has a detrimental impact on UX, as browser events must be handled by the server as well. Here's an example of how to show modal with animation in LiveView, which IMO is a ridiculous amount of code. While JavaScript code can still execute in the browser, they are triggered through "hooks" supplied by LiveVew. In practice this makes it hard to design UX elements that leverage browser capabilities heavily, and using third-party libraries requires retrofitting into a LiveView hook.

But underneath the hood, something magical is happening - LiveView is using authenticated WebSockets to send diffs of HTML from the server to the browser for rendering.

When powers combine

What if we could take the best of both of these frameworks - using Svelte to drive a snappy browser UX while leveraging LiveView to keep Svelte property state up-to-date with changes from the server?

In this post, I describe exactly how to do that! Not only will we wire LiveView's lifecycle interface into Svelte's component lifecycle to update state, but we will also inject LiveView's WebSocket into Svelte components to make it easy to send updates from the client back to the server without needing an XMLHttpRequest.

This will be fun, let's get started.

NOTE: the full code for this post can be found here

I bootstrapped a new Phoenix app named "swiphly" using mix (note that LiveView is enabled by default in newer versions of Phoenix):

mix phx.new . --app swiphly --database sqlite3

Because Svelte is precompiled, we need a way to include it into the javascript package build pipeline of Phoenix. The latest version of Phoenix leverages esbuild, but a few modifications are necessary to use esbuild plugins. You can find the full instructions in the Phoenix documentation, but the commands I used are:

npm install esbuild svelte esbuild-svelte --save-dev
npm install ../deps/phoenix ../deps/phoenix_html ../deps/phoenix_live_view --save

I also need to add a custom ./assets/build.js file since we are overriding the built-in config. This will include an esbuild plugin to compile Svelte components into our package:

const esbuild = require('esbuild')
const sveltePlugin = require('esbuild-svelte')

const args = process.argv.slice(2)
const watch = args.includes('--watch')
const deploy = args.includes('--deploy')

const loader = {
  // Add loaders for images/fonts/etc, e.g. { '.svg': 'file' }
}

const plugins = [
  // Add and configure plugins here
  sveltePlugin(),
  // ... other plugins such as postCss, etc.
]

let opts = {
  entryPoints: ['js/app.js'],
  mainFields: ["svelte", "browser", "module", "main"],
  bundle: true,
  minify: false,
  target: 'es2017',
  outdir: '../priv/static/assets',
  logLevel: 'info',
  loader,
  plugins
}

if (watch) {
  opts = {
    ...opts,
    watch,
    sourcemap: 'inline'
  }
}

if (deploy) {
  opts = {
    ...opts,
    minify: true
  }
}

const promise = esbuild.build(opts)

if (watch) {
  promise.then(_result => {
    process.stdin.on('close', () => {
      process.exit(0)
    })

    process.stdin.resume()
  })
}

And that's about it, now Phoenix will precompile all ./assets/**/*.svelte files, and include them in the static JavaScript bundles under ./priv/static/assets. So far so easy!

Two lifecycles become one

Now for the fun part. We need a way to instantiate Svelte components as part of LiveView's rendering engine, and we'll do this with a Svelte-specific LiveView component with a custom hook for creating and updating a Svelte JavaScript object.

First, the LiveView component

We define a LiveView component in Phoenix, which creates a div with two data attributes:

• data-name which will contain the name of the Svelte component to use for rendering, and
• data-props to hold variable state that should be passed as properties to the Svelte component.

It also wires a phx-hook (Phoenix hook) to the element, which will map LiveView lifecycle methods into Svelte component lifecycle methods. We'll look at that shortly.

Data properties will be encoded as JSON when rendered by Phoenix, and parsed as JSON in the browser.

defmodule SwiphlyWeb.SvelteComponent do
  use SwiphlyWeb, :live_component

  def render(assigns) do
    ~H"""
    <div id={@id || @name} data-name={@name} data-props={json(@props)} phx-update="ignore" phx-hook="SvelteComponent"></div>
    """
  end

  defp json(nil), do: ""
  defp json(props) do
    # encode props as JSON, using Jason or other
  end
end

We can use this LiveView component when rendering from a controller action:

defmodule SwiphlyWeb.EndToEnd do
  use SwiphlyWeb, :live_view

  @impl true
  def mount(params, session, socket) do
    # do some server side logic
    # add state to the socket assignments
    socket.assign(:foo, "bar")
    {:ok, socket}
  end

  @impl true
  def render(assigns) do
    ~H"""
      <.live_component module={SwiphlyWeb.SvelteComponent} id="foo" name="MySvelteComponent" props={%{foo: @foo}} />
    """
  end
end

This renders the component defined above by referencing the module SwiphlyWeb.SvelteComponent, and passes server state from the socket assignments into a props variable of the module. There is a bit of indirection happening between the controller and the component, but it scopes only the state to the component that it needs to render.

So far, we have an HTML div that gets rendered by Phoenix as a LiveView component, with state encoded as a data attribute. The magic of the LiveView component is that it is aware of server-side state changes - whenever the props variable changes, LiveView detects this and will call a method in our custom hook in the browser.

Joining two frameworks

Let's take a look at the custom hook now, we define this in ./assets/js/hooks.js. Recall that we specified this hook in the LiveView component div with a phx-hook attribute above.

There's a lot of code here, so we'll start at the top. Svelte components compile into JavaScript objects, we can import them directly.

We specify which Svelte component to render from the LiveView component above by setting data-name to the variable name of the import. Note that the variable name of the import must match the data-name attribute of the LiveView component div. Yes this is souper hacky, but due to limitations in esbuild, a dynamic require isn't possible isn't possible 🤦.

The interface for a custom LiveView hook is the link between LiveView and Svelte lifecycles - we are instantiating the Svelte component during mounted(), updating its props during updated(), and cleaning up during destroyed().

Svelte objects are pretty simple as well, providing a constructor that takes a target DOM element and a list of properties. The DOM element I am using is simply the div being rendered by the LiveView component. I can reference an instance of this object through this._instance when updating properties or removing the component. And recall that LiveView will call these hook methods whenever state changes on the server socket assignments!

import MySvelteComponent from "./components/MySvelteComponent.svelte"

const components = {
  MySvelteComponent,
}

function parsedProps(el) {
    const props = el.getAttribute('data-props')
    return props ? JSON.parse(props) : {}
}

const SvelteComponent = {
    mounted() {
        // 
        const componentName = this.el.getAttribute('data-name')
        if (!componentName) {
            throw new Error('Component name must be provided')
        }

        const requiredApp = components[componentName]
        if (!requiredApp) {
            throw new Error(`Unable to find ${componentName} component. Did you forget to import it into hooks.js?`)
        }

        const request = (event, data, callback) => {
            this.pushEvent(event, data, callback)
        }

        const goto = (href) => {
            liveSocket.pushHistoryPatch(href, "push", this.el)
        }

        this._instance = new requiredApp({
            target: this.el,
            props: {...parsedProps(this.el), request, goto },
        })
    },

    updated() {
        const request = (event, data, callback) => {
            this.pushEvent(event, data, callback)
        }

        const goto = (href) => {
            liveSocket.pushHistoryPatch(href, "push", this.el)
        }

        this._instance.$$set({...parsedProps(this.el), request, goto })
    },

    destroyed() {
        this._instance?.$destroy()
    }
}

export default {
    SvelteComponent,
}

In updated() we are calling into Svelte internals, using $$set to update properties without re-instantiating the entire object. Similarly with $destroy, we call this internal Svelte function to release component resources and remove from DOM.

Bonus round

Now that we have end-to-end reactivity from the backend to the client, this is like a dream come true! But wait, there's more...

I'm also wrapping two methods from LiveView and adding them to the Svelte component props - pushEvent and pushHistoryPatch. pushEvent lets us send data to the server on the WebSocket, without needing XMLHttpRequest/fetch/axios, and is authenticated and encrypted. pushHistoryPatch helps with navigating to other pages, and skips full page reloads if not necessary.

Let's see how this looks in MySvelteComponent

Here's a simple component containing a form and button, with requests to the server passing a name, data and callback() parameter. That's all you need, no paths or parameters, methods or headers.

<script>
    export let foo, request
    let form
    let formResult = ''

    function createFoo() {
        const data = new FormData(form)
        request('create', Object.fromEntries(data.entries()), (result) => {
            if (result?.success) {
                setTimeout(() => { form.reset() }, 555)
                formResult = 'Foo created'
            } else if (result?.reason) {
                formResult = `Error creating foo: ${result.reason}`
            } else {
                formResult = 'An unknown error has occurred'
            }
        })
    }

    function deleteFoo() {
        request('delete', { foo_id: 123 }, () => {})
    }
</script>

<form
    class="grid grid-cols-2 gap-4 form-control"
    bind:this={form}
    on:submit|preventDefault={createFoo}
>
    <!-- some form fields -->
    <div class="font-semibold text-accent">{formResult}</div>
</form>
<div class="button" on:click={deleteFoo}>

Pattern matching FTW

And on the server, Elixir can pattern match on the name and properties to decompose logic:

defmodule SwiphlyWeb.PushEvent do
  # same code as above...

  @impl true
  def handle_event("create", %{"foo" => "bar"}, %{assigns: %{}} = socket) do
    with {_, _} <- Foo.create_bar(params) do
      # stuff...
    end
  end

  @impl true
  def handle_event("create", %{"foo" => "!bar"}, %{assigns: %{}} = socket) do
    with {_, _} <- Foo.create_notbar(params) do
      # stuff...
    end
  end
end

What's amazing about this setup is that I don't need to optimistically update local state, since the server will push the updated property state back down the hook! In practice, this cuts out half of the JS code I would normally have in my Svelte component.

One more thing

Let's review what we have so far:

• Created a "cybernetic webapp" (Svelte's own words) with reactive components for a snappy browser experience
• Hooked Svelte component properties to server-side state managed by Phoenix LiveView components for live re-rendering based on changes from the server
• Leveraged the LiveView WebSocket to easily send changes from the client to the server as events, without messing with AJAX/REST

Pretty nice, huh!

But what if state is changing from another browser client, and that update hits a different server than I'm connected to? This is a pretty common scenario at production scale, as I'll have many servers handling browser requests. Think of any collaboration app with many users interacting with the same content.

Phoenix has another trick up its sleeve - a flexible pubsub system that is easily clustered across servers. By adding a few extra lines of code to publish an event when state is updated, all servers are notified and can update the state they are holding for their browser clients.

End-to-end reactivity can include n:n browsers and servers, not just 1:1

This is truly incredible! And because Phoenix uses a separate process for each connected browser client, this approach scales beautifully as well.

Happy coding!

- JumpWire team