Introduction
As part of our development workflow, I wanted to run Playwright tests in headless mode inside a Linux environment. I also needed to run it inside a Docker container. Another thing I wanted was for the EC2 instance to have GPU support so the tests can run with hardware acceleration.
As far as I can see, the way Google Chrome developers chose to support hardware acceleration under Linux is through Vulkan (here and here)
According to Nvidia, there's no official support for Vulkan inside Docker. Although it seems that FAQ hasn't been updated because I was able to find a Docker container with Vulkan support here.
The ingredients of our "recipe" are the following:
- GPU enabled EC2 instance
- Nvidia drivers
- Docker Engine
- Nvidia Container Toolkit
- Vulkan enabled Docker container
Steps
Start by launching an EC2 instance with the following features:
- Instance type: g4dn.xlarge
- AMI: ami-058165de3b7202099 (Canonical, Ubuntu, 22.04 LTS, amd64 jammy image build on 2022-06-09)1
- Storage: At least 20 GiB
SSH to the instance and refresh the system:
sudo apt-get update -y && sudo apt upgrade -y
Don't forget to reboot after the upgrade finishes
The installation of the Nvidia drivers is documented here. I went with installation option 2 (Public NVIDIA drivers). As I'm using a G4dn instance, I did a search using the following parameters:
As you can see in the image, the latest version at the time of this writing is 515.65.01. Now we can download and install this version:
-
wget https://us.download.nvidia.com/tesla/515.65.01/nvidia-driver-local-repo-ubuntu2204-515.65.01_1.0-1_amd64.deb
(To get the link, select the driver version in the search page. Click on the Download button. A new page should show up with a button labelled "Agree & Download". Copy the link from that button) sudo dpkg -i nvidia-driver-local-repo-ubuntu2204-515.65.01_1.0-1_amd64.deb
-
sudo cp /var/nvidia-driver-local-repo-ubuntu2204-515.65.01/nvidia-driver-local-22D4AC2B-keyring.gpg /usr/share/keyrings/
(the previous dpkg step suggested to copy the keyring) sudo apt-get update && sudo apt-get -y install cuda-drivers
Reboot the machine after the driver installation. To make sure the drivers are working as expected, run nvidia-smi
. You should get an output similar to this:
To verify that the Vulkan drivers are available, we need to install vulkan-tools: sudo apt-get -y install vulkan-tools
. After running vulkaninfo
you should get something like this:
Next step is to install the latest Docker Engine, following these steps
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt update && sudo apt-get install ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update && sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
- Check docker is installed successfully:
sudo docker run hello-world
- Add the user to the docker group so we don't use sudo to execute Docker commands:
sudo usermod -aG docker $USER
. Log out and log back in so the changes take effect - Make sure Docker is enabled after a reboot:
sudo systemctl enable docker.service && sudo systemctl enable containerd.service
Next, we need to install the Nvidia Container Toolkit (instructions here)
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker
- Verify that Docker has access to the GPU (notice the
--gpus all
flag):docker run -it --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
. There should be some text similar to thenvidia-smi
command executed previously - Verify that Docker has access to the Vulkan drivers:
docker run --rm -it --gpus all nvidia/vulkan:1.2.133-450 vulkaninfo
. There should be some text similar to thevulkaninfo
command executed previously
Now to run Playwright inside the container, run the following steps:
- Start the container with a mounted volume to the current folder, so files are saved:
docker run -v $PWD:/tests -w /tests --rm -it --gpus all nvidia/vulkan:1.2.133-450 bash
Inside the container run the following commands to install NodeJS:
-
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
(it seems they need to update the container with new keys. If we don't do this, the next commands will fail) wget https://deb.nodesource.com/setup_16.x -O nodesource_setup.sh
sh nodesource_setup.sh
apt install -y nodejs
To verify that Playwright is working correctly I use the following files:
package.json:
{
"devDependencies": {
"@playwright/test": "^1.24.2"
}
}
playwright.config.js:
const config = {
use: {
channel: "chromium",
launchOptions: {
args: [
'--enable-features=Vulkan,UseSkiaRenderer',
'--use-vulkan=swiftshader',
'--enable-unsafe-webgpu',
'--disable-vulkan-fallback-to-gl-for-testing',
'--dignore-gpu-blocklist',
'--use-angle=vulkan'
]
}
}
}
module.exports = config
gpu.spec.js:
import { test, expect } from '@playwright/test';
test('It should take a snapshot of the GPU Chrome page', async ({ page }) => {
await page.goto('chrome://gpu', { waitUntil: 'domcontentloaded' });
await page.screenshot({ path: 'gpu.png' });
await expect(page.locator('text=Graphics Feature Status').first()).toBeVisible();
});
Place those 3 files in the /tests folder
To install Playwright run npm install
Now we need to install the Chromium browser which we'll use for our tests. I'm adding to the command a custom path to install the browsers:
PLAYWRIGHT_BROWSERS_PATH=/tests/.cache/ms-playwright npx playwright install chromium
- Install needed dependencies:
apt install -y libglib2.0-0 libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdbus-1-3 libxcomposite1 libxdamage1 libpango-1.0-0 libcairo2 libasound2 libatspi2.0-0
- Run the Playwright test:
PLAYWRIGHT_BROWSERS_PATH=/tests/.cache/ms-playwright npx playwright test gpu.spec.js
As a result the test will save a file called gpu.png. Opening that file should show something like this
If instead you see "Software only, hardware acceleration unavailable" then something went wrong and Playwright is not using hardware acceleration. Go through the steps again, making sure nothing is missing. Good luck!
Top comments (0)