🖥️🎥 Automated screen recording with JavaScript

When I built the macOS defaults recording feature, I wanted to automate the maintainer work as much as possible, meaning that I want to script the screenshots and recordings, and rerun them when a new macOS version comes out. I found two packages that helped me a lot:

• Record the entire Mac screen using aperture-node
• robot.js to move the mouse and use the keyboard programmatically (should work on any OS)

Here is how I used them:

const aperture = require('aperture')()
const robot = require('robotjs')
const delay = require('delay')
const { compressVideo } = require('../../utils')

async function record() {
  // ...

  robot.moveMouse(pos1.x, pos1.y)

  // Action!
  await aperture.startRecording({ highlightClicks: true, cropArea })

  robot.moveMouseSmooth(pos2.x, pos2.y, 2)
  await delay(1000)
  robot.moveMouseSmooth(pos3.x, pos3.y, 5)
  await delay(100)
  robot.moveMouseSmooth(pos1.x, pos1.y, 10)
  await delay(500)

  const tmpRecordingPath = await aperture.stopRecording()
  // End recording

  try {
    await compressVideo(tmpRecordingPath, outputPath)
  } catch (compressVideoError) {
    throw new Error(compressVideoError)
  }
}

Let's explain what happens here.

robot.moveMouse(pos1.x, pos1.y)

The robot.js moveMouse method... move the mouse. It does it directly with no delay.

The x value is set from the left border of the screen. The y value is from the top border.

robot.moveMouseSmooth(pos2.x, pos2.y, 2)

The moveMouseSmooth do it "human-like". It's not perfect but it's good enough. The 3rd parameter adjusts the speed of the mouse movement.

To make sure last action is ended before doing another one, I add some delay between actions using delay.

Other robot.js methods I've been using:

const { width, height } = robot.getScreenSize()

robot.keyTap('g', ['command', 'shift'])

const pic = robot.screen.capture(x, y, width, height)

Simple as that!

Let's move forward to Aperture.

Aperture is a low level Swift script that uses the AVFoundation framework with great performances. It was built to fulfil an Open Source screen recorder needs called Kap.

The Node API is pretty straightforward:

const options = {
  cropArea: {
    x: pos2.x - recordWidth / 2, y: 0,
    width: recordWidth, height: recordHeight
  },
  highlightClicks: true
}

await aperture.startRecording(options)

The cropArea x value is set from the left border of the screen. The y value from the bottom border. I had to be careful about that, as it's not the same referential as robot.js!

const tmpRecordingPath = await aperture.stopRecording()
//=> '/private/var/folders/3x/jf5977fn79jbglr7rk0tq4d00000gn/T/cdf4f7df426c97880f8c10a1600879f7.mp4'

The stopRecording method gives us a path where the video is saved.

Then we can post process our screen recording. In my case, I built a method to resize, compress and move it to another folder.

Unfortunately, I didn't find a robust solution for resolution enforcement. So I can't guarantee the results are 100% the same on different setups.

That's it! Isn't it simple? Let me know what you think in the comment section 🙂

If you wanna go deeper in automated screen recording, take a look at macOS defaults recorder!