Better web video with AV1 codec

Learn how to instantly improve online viewing experience for your users by embracing the new AV1 video format that is already supported by Chrome and Firefox. This short guide will also show how to replace your GIF's with videos, using AV1 or H.264, to make your files twenty to forty times smaller.

The bets are placed. Both YouTube and Netflix have named AV1 a video codec for the future: Google's video service is already using it on TestTube (new, experimental features for YouTube). Netflix has been calling AV1 "our primary next-gen codec" for a while now. At Evil Martians, we have already tried AV1 at our landing page and at the landing page of Amplifr. In this article, we will share our experience with a new video format and give step-by-step instructions for optimal encoding strategies.

Codecs and Containers

With static images, you don't have to think twice: opt for JPEG or PNG supported by all browsers, or experiment with a more compact Google-developed WebP for newer browsers. You can almost always (barring some dirty hacking tricks that tools like imgproxy can protect you against) be sure that an image file with .png extension is, indeed, a PNG.

With video files, it is a bit more tricky than that. File extensions (.mp4, .wmv, .webm or .mov) barely represent containers, up to three different formats are used to make a video file happen:

1. Video codec: determines the compression strategy for your video, this is where the trade-offs are made between quality and quantity. On the web, some popular video codecs are H.264, HEVC, VP9, and now AV1.
2. Audio codec: does the same for audio. If your video does not have sound, you can do without it. Otherwise, the popular choices are MP3, Opus, and AAC.
3. Containers store both video (compressed by some video codec) and audio streams (compressed by some audio codec), and can also add extra details like subtitles and meta information. Popular containers are MP4, MOV, WebM.

So, when you see .mp4 extension, the only thing you can be sure about is that the MP4 container had been used to package a file. The choice of codecs depends entirely on a creator: it can be H.264/AAC, or AV1/Opus, or something else.

Meet AV1

AV1 is a video codec that was first released almost a year ago: in March 2018. It is designed to compete with previous codec generations such as HEVC/VP9 and H.264/VP8.

To get familiar with technologies used in a new generation video code, feel free to read "Introducing AV1" and "AV1: next generation video".

With all the low-level trickery involved, AV1 is capable of generating files that are up to 30-50% smaller than H.264/VP8 and up to 30% smaller than HEVC, even though, due to being still mostly experimental, it has some problems (at the time of this writing):

• The encoder is not optimized yet. As a result, encoding is extremely slow (an upcoming encoder written in Rust attempts to solve this issue). The format is not yet ready for live streaming. However, it is perfectly suitable for web, as your average landing page will usually have a short embedded video that rarely changes.
• While supported by Chrome and Firefox, AV1 lacks implementations for Safari and Edge (although Microsoft already has AV1 support in early beta). So you need to have at least two versions of all your videos: AV1 for Chrome/Firefox and H.264 for everything else. Ideally, you should have a third, HEVC version for your Safari users on desktop and mobile and we will show how to prepare all three of those files.

The central promise for of AV1 is maintaining high image quality even at low bitrates, thus allowing for smaller files without apparent compression artifacts.

How to use AV1 right now

Now we are going to show a sequence of steps required to produce the quality video content for the web with AV1. First, you need to choose a container: in theory, it does not matter, but MP4 is recommended and seems to be the most popular at the moment. For the audio codec, we will use Opus with AV1 as an efficient and free alternative.

To ensure the best cross-browser compatibility, we will produce not one, but three files:

1. For desktop Chrome and Firefox (31% of users as of August 2019): MP4 container with AV1 video codec and Opus audio codec.
2. For Safari and Edge (16% of users): MP4 with HEVC and AAC.
3. For other browsers: a larger file in MP4 container with H.264 for video and AAC for audio.

You can also go with just options 1 and 3, you will still ensure that all your users can see the video.

For conversion, I recommend using FFmpeg in a terminal. There are plenty of GUI tools for video compression, but CLI allows for steps that are easily reproducible and can be automated with a script. Make sure that you are using the most recent version of FFmpeg, as versions below 4.1 do not support AV1 in an MP4 container. Here are the steps to install it.

For Mac:

1. Make sure you have Homebrew.
2. brew install ffmpeg

For Linux, we recommend using the latest build of FFmpeg from the official website, as at the time of this writing not all package managers contain most recent, AV1-enabled versions:

1. wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz
2. tar -xf ffmpeg-release-amd64-static.tar.xz
3. sudo cp ffmpeg-4.1-64bit-static/ff* /usr/local/bin/

For Windows, you can use this guide.

Once ffmpeg executable is available in your command line, let's generate the H.264 file (to ensure compatibility with older browsers). Since all our files will use MP4 as a container, I will use .av1.mp4, .hevc.mp4, and .h264.mp4 file extensions. Here is the command you will need to use (don't worry, we will walk you through all the options in a moment).

# Replace SOURCE.mov with a path to your source video file

ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libfdk_aac -c:v libx264 -crf 24 -preset veryslow -profile:v main -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.h264.mp4

Now open the resulting video.h264.mp4 file and check your quality. If you are satisfied with the result, but the size still seems too big, try adjusting the -crf option (try -crf 26 or -crf 28): it will reduce the file size, but also the quality, so try to find an acceptable trade-off. That process, frankly, is more art than science.

You can convert H.264 to AV1 if you don't have a single uncompressed source.

Now it is time to generate the AV1 file. The command below will take longer than the one for H.264, but that is to be expected: at the moment, AV1 codec does not use the full power of CPU. That is a curse, but also a blessing: if you are about to encode several files at the same time, it is safe to do so, as multi-threading is currently not supported by default.

ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libopus -c:v libaom-av1 -crf 34 -b:v 0 -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -strict experimental video.av1.mp4

Play around with the -crf setting for an optimal size/quality balance.

Now the same for HEVC:

ffmpeg -i SOURCE.mov -map_metadata -1 -c:a libfdk_aac -c:v libx265 -crf 24 -preset veryslow -pix_fmt yuv420p -movflags +faststart -tag:v hvc1 -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.hevc.mp4

Then copy all three resulting files (video.h264.mp4, video.hevc.mp4, and video.av1.mp4) to the root of your web project.

Understand compression options

For now, the commands above look like black magic spells, but all those keys are used for a purpose. Here is what they do:

• -i SOURCE.mov sets the source video file for input. FFmpeg will take video and audio streams from this file, convert them, and pack into a new container.

• -map_metadata -1 will remove video metadata (like the name of a tool that was used initially to create a video). Sometimes metadata is useful, but for web development it is rarely the case.

• -c:a libopus or -c:a libfdk_aac selects an audio codec.

• -c:v libaom-av1 selects a video codec, a library to compress images into a video stream.

• -crf 34 stands for Constant Rate Factor and sets your size/quality balance. Think of it as the quality slider for JPEG, but it goes in the opposite direction (0 stands for best quality and bigger size). CRF scale is different for H.264 and AV1: H.264 goes from 0 to 51, AV1 from 0 to 61. As a result, you will have different CRF ratios for AV1 and H.264. According to this guide by Facebook, here are the optimal mappings between H.264 and AV1 CRF values: 19 → 27, 23 → 33, 27 → 39, 31 → 45, 35 → 51, 39 → 57.

• -preset veryslow forces H.264 and HEVC codecs to generate smaller video file even if it will be much longer.

• -profile:v main that we use in our H.264 command selects the video codec profile. We can only use "Main", as our video will not be played in Safari otherwise.

• -b:v 0 a sets minimum bitrate to force the constant quality mode in AV1.

• -pix_fmt yuv420p (pixel format) is a trick to reduce the size of a video. Basically, it uses full resolution for brightness and a smaller resolution for color. It is a way to fool a human eye, and you can safely remove this argument if it does not work in your case.

• -movflags +faststart moves the important information to the beginning of the file. It allows browser to start playing video during downloading.

• -tag:v hvc1 enables native HEVC video support on Apple operating systems.

• -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" is a way to ensure the produced video will always have an even size (some codecs will only work with sizes like 300x200 and 302x200, but not with 301x200). This option tells FFmpeg to scale the source for the closes even resolution. If your video dimensions were even in the first place, it would not do anything.

• -strict experimental option needs to be used for AV1 as AV1 encoder is still experimental.

• video.av1.mp4 sets the name for the output file.

Play nice with browsers

Now you need to make sure that browsers will display the right file depending on whether it is supported or not. Luckily, we can set a type attribute on a source element, and only the supported file will be played. For more <video> tag options, look here.

<video controls width="600" height="400">
  <source
    src="video.hevc.mp4"
    type="video/mp4; codecs=hevc,mp4a.40.2"
  />
  <source
    src="video.av1.mp4"
    type="video/mp4; codecs=av01.0.05M.08,opus"
  />
  <source
    src="video.h264.mp4"
    type="video/mp4; codecs=avc1.4D401E,mp4a.40.2"
  />
</video>

Source tags work similarly to if...else statements: the browser will read the list of <source> tags top to bottom and play the first one with supported video type.

Type attribute describes a file format: which container (video/mp4 for MP4), video codec (av01.0.05M.08 for AV1, hevc for HEVC and avc1.4D401E for H.264), and audio codec (opus for Opus and mp4a.40.2 for AAC) should be used.

Bonus: how to convert GIFs to AV1 and H.264

In modern times, using GIF for video fragments is a poor practice. GIFs take 20 to 40 times more space than H.264 or AV1. They also require more CPU and power and drain more battery than modern video formats. If you need short animation sequences on your website in 2019, always opt for video codecs. Luckily, FFmpeg supports GIF files as an input source.

Here's how to convert your GIF to H.264:

ffmpeg -i IMAGE.gif -map_metadata -1 -an -c:v libx264 -crf 24 -preset veryslow -profile:v main -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" animation.h264.mp4

And here's how to go even further and convert it to AV1:

ffmpeg -i IMAGE.gif -map_metadata -1 -an opus -c:v libaom-av1 -crf 50 -b:v 0 -pix_fmt yuv420p -movflags +faststart -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -strict experimental animation.av1.mp4

Now we can use animation.h264.mp4 and animation.av1.mp4 in our HTML. Just replace VIDEO_WIDTH, VIDEO_HEIGHT, and PATH_TO_VIDEO:

<video
  autoplay
  loop
  muted
  playsinline
  width="VIDEO_WIDTH"
  height="VIDEO_HEIGHT"
>
  <source
    src="PATH_TO_VIDEO/animation.av1.mp4"
    type="video/mp4; codecs=av01.0.05M.08"
  />
  <source
    src="PATH_TO_VIDEO/animation.h264.mp4"
    type="video/mp4"
  />
</video>

autoplay and loop attributes will emulate the expected behavior of a GIF: looping an animation after the website was loaded. playsinline will forbid Safari from opening the video in full-screen mode.

And that concludes our practical guide!

Even though AV1 codec is still considered experimental, you can already leverage its high-quality, low-bitrate features for a sizable chunk for your web audience (users with current versions of Chrome and Firefox). Of course, you would not want to leave users for other browsers hanging, but the attributes for <video> and <source> tags make implementing this logic easy, and in pure HTML, you don't need to go at length to detect user agents with JavaScript. Mastering a few FFmpeg commands also seems like an easy way to improve the video viewing experience for your visitors. We already use AV1 in production on a couple of our projects and have not encountered any significant problems (except for video compression times, but, again, we are dealing mostly with short static sequences).