Electron Adventures: Episode 66: Dynamic Rendering

In the previous episode we managed to significantly improve performance of creating rows, but it's not good enough. For a 16MB file, we still need to create 1M rows with 20M elements, each with some characters of formatted text.

Considering that we'd only ever display a few kB on screen at once, this is a huge waste.

Dynamic Rendering

The idea is to calculate which rows are visible and which are not, and only display the visible ones. For everything else, just render a placeholder of the same size.

This is far from the most performant way, as huge number of placeholders still take a while to generate and update, but it's already surprisingly effective.

For this we'll do all the calculations ourselves, assuming every row has the same height and placeholder rows have identical height to fully displayed rows. There are many ways to handle more general case, using Intersection Observer API, but they'd be a lot more complex and potentially also slower.

src/AsciiView.svelte

But first, something I forgot to do in the previous episode, Ascii View needs to be

<script>
  export let data

  let ascii = ""
  for (let d of data) {
    if (d >= 32 && d <= 126) {
      ascii += String.fromCharCode(d)
    } else {
      ascii += "\xB7"
    }
  }
</script>

<span class="ascii">{ascii}</span>

<style>
  .ascii {
    white-space: pre;
  }
</style>

src/Slice.svelte

The Slice component can render either the real thing or a placeholder. It's controlled by visible prop.

<script>
  import { printf } from "fast-printf"
  import AsciiSlice from "./AsciiSlice.svelte"

  export let offset
  export let data
  export let visible
</script>

<div class="row">
  {#if visible}
    <span class="offset">{printf("%06d", offset)}</span>
    <span class="hex">
      {#each {length: 16} as _, i}
        <span data-offset={offset + i}>
          {data[i] !== undefined ? printf("%02x", data[i]) : "  "}
        </span>
      {/each}
    </span>
    <AsciiSlice {data} />
  {:else}
    &nbsp;
  {/if}
</div>

<style>
  .row:nth-child(even) {
    background-color: #555;
  }
  .offset {
    margin-right: 0.75em;
  }
  .hex span:nth-child(4n) {
    margin-right: 0.75em;
  }
</style>

src/MainView.svelte

There's a few things we need to do.

First, let's save the main node, and some properties with range of visible components:

  let main
  let firstVisible = 0
  let lastVisible = 200

Second, we need to pass the correct visible flag to the slices. We also need use: callback to initialize main variable, and some callbacks to update firstVisible and lastVisible variables on scroll and resize events:

<div
  class="main"
  on:mouseover={onmouseover}
  on:scroll={setVisible}
  use:init
>
  {#each slices as slice, i}
    <Slice {...slice} visible={i >= firstVisible && i <= lastVisible} />
  {/each}
</div>

And finally a simple calculation which rows are visible.

  function setVisible() {
    let rowHeight = main.scrollHeight / slices.length
    firstVisible = Math.floor(main.scrollTop / rowHeight)
    lastVisible = Math.ceil((main.scrollTop + main.clientHeight) / rowHeight)
  }

  function init(node) {
    main = node
    setVisible()
  }

How well it works?

It correctly handles scrolling, and resizing window. Somehow it even handles Cmd+Plus and Cmd+Minus shortcuts for changing font size as they issue scroll event.

As scrolling event is heavily throttled, it actually takes a while during scrolling to render rows. This isn't great, and browser doesn't have any kind of scrollstart event. We could emulate it with creative use of requestAnimationFrame.

Or we could just display 100 rows on each side of the visible part to

However even this absolutely simplest approach works quite well already!

And of course, the performance! 1MB file loads in ~2s, down from 42s we originally had.

This isn't amazing, as we'd like to be able to comfortably deal with 100MB+ files, but we have easy way ahead - just group rows into 100-row chunks and conditionally display or not display those.

We could also have no placeholders of any kind, and put big height on it, and just position: each displayed row absolutely.

Results

Here's the results:

Now that we fixed performance we can do the long promised file loading, but first I want to do a detour and try another framework you've probably never heard of.

As usual, all the code for the episode is here.