Samuel Earl

Posted on Jul 13, 2022

Scatterplot Overlay - Building a Geospacial App with SvelteKit, Deck.gl, and Mapbox - Part 2

Add a Scatterplot Overlay with Deck.gl

As cool as having a map is, an empty map is not that useful. Let's see if we can add a Scatterplot overlay with the Taxi data set to the map we created in the previous post.

Deck.gl is a WebGL overlay suite for JavaScript, providing a set of highly performant data visualization overlays.

Deck.gl comes with several prepackaged layers that we can use, in conjunction with our map, to display geospatial data. The simplest one is the ScatterplotLayer, which we will use.

An introduction to Deck.gl

Deck.gl allows you to visualize stacked layers of data. Each layer can be of a different type (we'll see two: scatterplots and hexagons, but there are many more) and can have millions of data points. In the finished visualization, the layers appear on top of each other, and they support interactions like pan, zoom, rotate, as well as mouse hover, clicks, etc.

Deck.gl doesn't have to work with a map but it was designed with geospatial applications in mind. Like the map we designed in the previous post, Deck.gl layers have the same concept of view state and user interactions.

There are a couple of ways to make Deck.gl work with mapbox-gl, but this is how we are going to work with them:

We will have a deck.gl <canvas> element and a mapbox-gl <div> element in the HTML. Then we will make sure that the deck.gl canvas stays on top of the mapbox element. This will allow us to use a static map (i.e. one where the interactive option is set to false), which will get all the view state information from Deck.gl.

Here is the updated code, which adds Deck.gl to our code and causes the map and Deck.gl to work together:

<MapStylePicker currentStyle={mapStyle} on:change={handleStyleChange}/>
<div class="deck-container">
  <div id="map" bind:this={mapElement}></div>
  <canvas id="deck-canvas" bind:this={canvasElement}></canvas>
</div>

<script>
  import { onMount } from "svelte";
  import mapboxgl from "mapbox-gl";
  import { Deck } from "@deck.gl/core";
  import MapStylePicker from "$/components/MapStylePicker.svelte";

  let mapElement;
  let canvasElement;
  let map = null;
  let deck = null;
  let accessToken = import.meta.env.VITE_MAPBOX_API_ACCESS_TOKEN;
  let mapStyle = "mapbox://styles/mapbox/light-v9";
  let viewState = {
    longitude: -118.2443409,
    latitude: 34.0544779,
    zoom: 2,
    pitch: 0,
    bearing: 0,
  };

  onMount(() => {
    createMap();
    createDeck();
  });

  // Create the map.
  function createMap() {
    map = new mapboxgl.Map({
      accessToken: accessToken,
      container: mapElement,
      interactive: false,
      style: mapStyle,
      center: [viewState.longitude, viewState.latitude],
      zoom: viewState.zoom,
      pitch: viewState.pitch,
      bearing: viewState.bearing,
    });
  }

  /**
   * Use the `setStyle` function from `mapbox-gl` to update the map style:
   * https://docs.mapbox.com/mapbox-gl-js/example/setstyle/
   */
  function handleStyleChange(event) {
    map.setStyle(event.target.value);
  }

  // Create the deck.gl instance.
  function createDeck() {
    deck = new Deck({
      canvas: canvasElement,
      width: "100%",
      height: "100%",
      initialViewState: viewState,
      controller: true,
      // Change the map's viewState whenever the view state of deck.gl changes.
      onViewStateChange: ({ viewState }) => {
        map.jumpTo({
          center: [viewState.longitude, viewState.latitude],
          zoom: viewState.zoom,
          bearing: viewState.bearing,
          pitch: viewState.pitch,
        });
      },
    });
  }
</script>

<style>
  .deck-container {
    width: 100%;
    height: 100%;
    position: relative;
  }

  #map {
    position: absolute;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
    background: #e5e9ec;
    overflow: hidden;
  }

  #deck-canvas {
    position: absolute;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
  }
</style>

This has all the same functionality as our previous version, but Deck.gl has been incorporated. We added the following things:

A couple more HTML elements along with CSS styles for those elements.
The @deck.gl/core library.
A couple of variables for the canvasElement and the deck instance.
A createDeck() function that will create the Deck.gl instance, which gets called during the onMount hook.

The finished code

I am going to show you the finished code for this post and then explain some things about the finished code below.

Add Data

Create a new folder called data in your src directory and create a new file inside your data folder called taxi.js. The taxi.js file will contain location data for each time a taxi either picked up or dropped off passengers on Manhattan Island.

Then go to https://github.com/uber-common/vis-academy/blob/master/src/demos/data/taxi.js, click the Download button, and copy/paste the raw data into your taxi.js file. At the bottom of the taxi.js file you will need to change the module.exports = dataRows.map... statement to export default dataRows.map....

Once that is done, import the taxi data into your index.svelte component:

import taxiData from "$/data/taxi.js";

Now we need to process this data into a usable format. Since we are only going to be working with a ScatterplotLayer for now, we only care about the latitude, longitude, and another bit of data called pickup, which is used for coloring the dots.

We add a let points = []; variable and a processData() function and call it when the component mounts to process the data.

Add Deck.gl Layers

Import the ScatterplotLayer:

import { ScatterplotLayer } from "@deck.gl/layers";

Then add the following code:

// Whenever the layer data is changed and new layers are created, then rerender the layers.
$: renderLayers({ data: pointsArray });

// Use the `deck.setProps()` method to set the layers in deck.gl.
// See https://deck.gl/docs/api-reference/core/deck#layers.
function renderLayers(props) {
  // If `deck` is null then return early to prevent errors.
  if (!deck) return;

  deck.setProps({
    layers: createDataLayers(props)
  });
}

function createDataLayers(props) {
  const {data} = props;
  return new ScatterplotLayer({
    id: "scatterplot",
    getPosition: d => d.position,
    getFillColor: d => [0, 128, 255],
    getRadius: d => 5,
    opacity: 0.5,
    pickable: true,
    radiusMinPixels: 0.25,
    radiusMaxPixels: 30,
    data
  });
}

The $: renderLayers({ data: pointsArray }); reactive property is responsible for watching the pointsArray variable and calling the renderLayers() function whenever the data in the pointsArray variable changes.

The renderLayers() function is responsible for taking the pointsArray data (which is passed as the props parameter) and getting the updated scatterplot layer from createDataLayers(). That updated scatterplot layer is then updated in the browser by calling deck.setProps().

The createDataLayers() function is responsible for creating the updated scatterplot layer and returning it.

Now you should see a working Deck.gl overlay that displays the taxi data as a scatterplot overlay. You should see blue dots laid out across the top of Manhattan Island. The data points are a little small, so you might need to zoom in to see them.

Let's go over just a few properties in the new ScatterplotLayer() code above:

data {Array}
The data for the layer. In this case, it's our Taxi data set.

getPosition {Function}
Function that gets called for each data point. This should return an array of [longitude, latitude].

getFillColor, getRadius {Function}
These also get called for each data point and they return the color and radius, respectively, for each point.

pickable {Bool}
This indicates whether this layer will be interactive.

Polish

Dynamic Color

Right now, our getFillColor accessor returns a constant, but we can have color change depending on the data we pass. Let's display the magenta data points where a taxi picked up a passenger and orange data points where a taxi dropped off a passenger:

// Variable declarations:
const PICKUP_COLOR = [114, 19, 108];
const DROPOFF_COLOR = [243, 185, 72];

...

// Inside the createDataLayers() function:
getFillColor: d => d.pickup ? PICKUP_COLOR : DROPOFF_COLOR,

Control Panel

We are going to create two control elements that will allow us to (1) change the size of the dots in our scatterplot and (2) show or hide the scatterplot.

First, create a LayerControls.svelte component inside the src/components directory and paste this code inside:

<div class="layer-controls">
  <h4>Layer Controls</h4>
  <hr>
  <div class="controls-container">
    {#each Object.keys(settings) as key (key)}
      {#if controls[key] && controls[key].type}
        {#if controls[key].type === "range"}
          <div class="range-input-group">
            <label>{controls[key].displayName}</label>
            <div style="display:inline-block; float:right;">
              {settings[key]}
            </div>
            <div>
              <input
                type="range"
                id={key}
                min={controls[key].min}
                max={controls[key].max}
                step={controls[key].step}
                value={settings[key]}
                on:change={(event) => handleValueChange(key, Number(event.target.value))}
              />            
            </div>
          </div>
        {:else if controls[key].type === "boolean"}
          <div class="checkbox-input-group">
            <label>{controls[key].displayName}</label>
            <input
              type="checkbox"
              id={key}
              checked={settings[key]}
              on:change={(event) => handleValueChange(key, event.target.checked)}
            />
          </div>
        {/if}
      {/if}
      {#if controls[key].displayName === "Hexagon Upper Percentile"}
        <hr>
      {/if}
    {/each}
  </div>
</div>

<script>
  import { createEventDispatcher } from "svelte";

  export let settings;
  export let controls;

  const dispatch = createEventDispatcher();
  let max = 100;

  function handleValueChange(settingName, newValue) {
    // Only update if we have a confirmed change.
    if (settings[settingName] !== newValue) {
      // Create a new object so that shallow-equal detects a change.
      const newSettings = {
        ...settings,
        [settingName]: newValue
      };

      dispatch("layerSettingsChange", newSettings);
    }
  }
</script>

<style>
  .layer-controls {
    border: 1px solid lightgray;
    box-shadow: 2px 2px 2px gray;
    font-family: sans-serif;
    font-size: 12px;
    line-height: 1.833;
    width: 220px;
    position: absolute;
    top: 10px;
    right: 10px;
    padding: 20px;
    z-index: 100;
    background: white;
  }

  h4 {
    margin: 0;
    line-height: 1rem;
  }

  hr {
    border: none;
    border-top: 1px solid gray;
  }

  .checkbox-input-group {
    display: flex;
    justify-content: space-between;
    align-items: center;
    margin-bottom: 5px;
  }
</style>

In the index.svelte file, import the LayerControls component:

import LayerControls from "$/components/LayerControls.svelte";

...and add the LayerControls component to your HTML:

<div class="deck-container">
  <MapStylePicker currentStyle={mapStyle} on:change={handleStyleChange}/>
  <LayerControls {settings} controls={CONTROLS} on:layerSettingsChange={updateLayerSettings} />
  <div id="map" bind:this={mapElement}></div>
  <canvas id="deck-canvas" bind:this={canvasElement}></canvas>
</div>

Now add the following two variables below your other variable declarations and the updateLayerSettings() function below your other functions inside your <script> tag:

const CONTROLS = {
  radiusScale: {
    displayName: "Scatterplot Radius",
    type: "range",
    value: 10,
    step: 1,
    min: 1,
    max: 100
  },
  showScatterplot: {
    displayName: "Show Scatterplot",
    type: "boolean",
    value: true
  },
};
// The reduce function will return an initial value of `{radiusScale: 30, showScatterplot: true}`.
// Each time a user changes a layer control, the properties in this `settings` object will be updated and the layer data's appearance will be updated.
let settings = Object.keys(CONTROLS).reduce(
  (accu, key) => ({
    ...accu,
    [key]: CONTROLS[key].value
  }),
  {}
);

function updateLayerSettings(event) {
  settings = event.detail;
}

Now update the renderLayers() reactive property and the createDataLayers() function so the layer data will be updated when a user changes any of the control settings:

$: renderLayers({ data: pointsArray, settings: settings });

...

function createDataLayers(props) {
  const { data, settings } = props;
  return [
    settings.showScatterplot && new ScatterplotLayer({
      id: "scatterplot",
      getPosition: d => d.position,
      getFillColor: d => d.pickup ? PICKUP_COLOR : DROPOFF_COLOR,
      getRadius: d => 5,
      opacity: 0.5,
      pickable: true,
      radiusMinPixels: 0.25,
      radiusMaxPixels: 30,
      data,
      ...settings
    })
  ];
}

Now, you should have a control panel that will allow you to control display aspects of your scatterplot layer.

Mouseover Interaction

We are going to add a mouseover event to our scatterplot that will display a small tooltip with either the term "Pickup" or "Dropoff" for each data point.

These steps are similar to the control panel steps above, but the user events flow in the opposite direction. Instead of capturing a user event in HTML components and passing it to the WebGL layer, we capture a user event in the WebGL layer and pass it to HTML elements.

All of this code will be in the index.svelte file. First, add the tooltip code to the HTML:

<div class="deck-container">
  {#if hover.hoveredObject}
    <div class="tooltip" style:transform={`translate(${hover.x}px, ${hover.y}px)`}>
      <div>{hover.label}</div>
    </div>
  {/if}
  <MapStylePicker currentStyle={mapStyle} on:change={handleStyleChange}/>
  <LayerControls {settings} controls={CONTROLS} on:layerSettingsChange={updateLayerSettings} />
  <div id="map" bind:this={mapElement}></div>
  <canvas id="deck-canvas" bind:this={canvasElement}></canvas>
</div>

Then add the CSS for the tooltip:

.tooltip {
  position: absolute;
  padding: 4px;
  background: rgba(0, 0, 0, 0.8);
  color: #fff;
  max-width: 300px;
  font-size: 10px;
  z-index: 9;
  pointer-events: none;
}

Add the following variable below your other variable declarations:

let hover = {
  x: 0,
  y: 0,
  hoveredObject: null
};

Now we need a way to capture an onHover event when a user hovers over a data point in our scatterplot layer. In the createDataLayers() function, add the onHover listener like I have done in the code below:

function createDataLayers(props) {
  const { data, settings } = props;
  return [
    settings.showScatterplot && new ScatterplotLayer({
      id: "scatterplot",
      getPosition: d => d.position,
      getFillColor: d => d.pickup ? PICKUP_COLOR : DROPOFF_COLOR,
      getRadius: d => 5,
      opacity: 0.5,
      pickable: true,
      radiusMinPixels: 0.25,
      radiusMaxPixels: 30,
      data,
      onHover: (hoverProps) => handleHover("scatterplotLayer", hoverProps),
      ...settings
    })
  ];
}

...and add a handleHover() function at the bottom of the <script> tag:

function handleHover(layerType, hoverProps) {
  let label;
  if (layerType === "scatterplotLayer") {
    label = hoverProps.object ? (hoverProps.object.pickup ? "Pickup" : "Dropoff") : null;
  }
  // Set the coordinates for the tooltip.
  hover.x = hoverProps.x;
  hover.y = hoverProps.y - 20;
  hover.hoveredObject = hoverProps.object;
  hover.label = label;
}