Container Queries: Adaptive Images

Originally published on crunchingnumbers.live

A powerful application of container queries is adaptive images. It's about striking the balance between displaying an image that looks good (the larger the better) and one that loads fast (the smaller the better).

Currently, we are limited to srcset, which selects the optimal image based on the global screen size. This may work for splash images that cover the whole width, but what about images that we show in a partial area?

Enter container query. We can compare every candidate image's width, height, and aspect ratio (I assume that we have these information via metadata) to those of the container, then use the best fitting image's URL.

In practice, we create a component and pass an array of images. (Here, by an image, I mean a POJO with just URL and metadata, not the actual image.)

<ContainerQuery as |CQ|>
  <div
    local-class="image-container"
    {{did-update
      (fn this.setImageSource CQ.dimensions)
      CQ.dimensions
    }}
  >
    <img local-class="image" src={{this.src}} />
  </div>
</ContainerQuery>

In the backing class, setImageSource calls findBestFittingImage to set this.src. The latter function exists in a utility so that we can write fast unit tests.

export function findBestFittingImage(images, containerDimensions) {
  const { aspectRatio, height, width } = containerDimensions;

  const imagesRanked = images.map(image => {
    const { url, metadata } = image;

    const imageHeight = metadata.height;
    const imageWidth = metadata.width;
    const imageAspectRatio = imageWidth / imageHeight;

    const arMetric = Math.abs(imageAspectRatio - aspectRatio);
    const hwMetric = ((imageHeight - height) ** 3 + (imageWidth - width) ** 3) ** (1/3);
    const hwTiebreaker = ((imageHeight - height) ** 2 + (imageWidth - width) ** 2) ** (1/2);

    return {
      url,
      arMetric,
      hwMetric: Number.isNaN(hwMetric) ? Infinity : hwMetric,
      hwTiebreaker
    };
  })
  .sort((a, b) => {
    if (a.arMetric > b.arMetric) return 1;
    if (a.arMetric < b.arMetric) return -1;

    if (a.hwMetric > b.hwMetric) return 1;
    if (a.hwMetric < b.hwMetric) return -1;

    return a.hwTiebreaker - b.hwTiebreaker;
  });

  return imagesRanked[0].url;
}

The formulas for arMetric, hwMetric, and hwTiebreaker aren't anything special. I'm using l^p norms to quantify the difference between an image and the container. I can put them into words by saying I'm making 3 assumptions:

1. Users prefer images whose aspect ratio is close to the container's.
2. Users prefer images whose height and width are larger than the container's.
3. If all images are smaller than the container, users want the image that comes closest to the container.

That's it! With a bit of JavaScript and math, we solved a problem that MDN says is not possible (I paraphrased):

[H]ence the need to implement solutions like srcset. For example, you couldn't load the <img> element, then detect the viewport width with JavaScript, and then dynamically change the source image to a smaller one if desired. By then, the original image would already have been loaded, and you would load the small image as well, which is even worse in responsive image terms.

Why can't we just do this using CSS or JavaScript? - MDN

Here are my code for adaptive images and tests. There's more to do so I encourage you to extend my work. I made an ideal assumption about where I'm getting image URLs and metadata. Do things change when they come from a network request? Are there better ranking algorithms? I look forward to see what you can do with container queries.