Learning to Read Music with the Web MIDI API

Learning to Read Music with the Web MIDI API

My two daughters both take piano lessons. Their music school has some nice online learning web apps and tools that help them with music theory and sight reading. There's one web app that they use (pictured below) that displays a note on the staff, and below the staff there is a picture of a keyboard. They have to click the key on the keyboard that corresponds with the note on the staff. It's been a good tool for them to practice recognizing notes quickly.

I've noticed that my youngest daughter will sometimes get up from the computer, walk over to our digital piano, play the note there, and then come back and click the key on the screen. That got me thinking - maybe these skills would transfer better if she could play the note directly on our digital piano to match what she sees on the screen. So I set out to build a simple web app that would do just that - display a note on the staff, and while connected to a digital piano or keyboard, use the Web MIDI API to let her know if she played the right note that corresponds with what's on the staff. To build this web app, I had to learn about MIDI and the Web MIDI API. In this article, I'll share what I learned and how the web app was created.

What is MIDI?

Before I can talk about the Web MIDI API, it's important to have a basic understanding of what MIDI is. If you were on the internet in the mid 1990s, then the word MIDI may be closely associated in your mind with the sounds emanating on auto-play from Geocities websites. Something like this perhaps. MIDI isn't sound or audio though.

MIDI stands for Musical Instrument Digital Interface, and it's a technical standard that has been around since the 1980s. It's used for communication between digital musical instruments, computers, audio devices, etc. For the purposes of this article and the demo app, the most important aspects of MIDI you need to understand are MIDI messages.

MIDI Messages

There are a few types of MIDI messages, but I only ended up using Channel Voice Messages. There are different events -- or pieces of information -- that can be represented by channel voice messages, such as Note On, Note Off, and Polyphonic Key Pressure. When MIDI messages are sent, they're transmitted on MIDI channels. Up to 16 channels are supported. For the purposes of this demo app, we are only using channel 1. A Note On message gets sent when a key is pressed on the digital piano. This Note On message consists of three pieces of numeric information: the type of event (144 represents a Note On event on channel 1) the note number (from 0 - 127, where middle C is 60), and the velocity (how hard the key was pressed).

Note On Message Example

With that understanding of what information is sent when a key is pressed and what it means, the next step is to learn about the Web MIDI API, so that we can make sense of that information in the browser.

What's the Web MIDI API?

The Web MIDI API allows us to interact with MIDI-enabled devices via the browser. This could mean using a MIDI-enabled device as an input to a web application, or it could mean sending MIDI messages from a web application to a MIDI-enabled device. The API is pretty straightforward and easy to get started with.

Compatibility

Before moving on, it's important to note that the Web MIDI API currently doesn't have wide browser support. It's currently only supported in Chrome, Opera, and the Android Browser.

If you want to use the features of the Web MIDI API for more than just experimentation, and you need broad browser support, check out JZZ, a MIDI library for Node.js and web browsers. I haven't used it myself, but it was often referenced as a good alternative when I was reading about the Web MIDI API.

Building the Demo App

I'll walk you through some of the key parts of the code, but you can find the full source code for the demo app here: https://github.com/jankleinert/get-your-notes-on. A live version of the app is hosted here - if you've got a MIDI device, hook it up to your computer and try it out! Here's a screen capture of the web app in action:

Checking for Browser Support of the Web MIDI API

The first step is to check if the browser supports the Web MIDI API. As shown in the code snippet below, we'll check navigator.requestMIDIAccess, and if that is true, then we'll call navigator.requestMIDIAccess().

if (navigator.requestMIDIAccess) {
  console.log('WebMIDI is supported in this browser.');
  navigator.requestMIDIAccess().then(onMIDISuccess, onMIDIFailure);
} else {
  console.log('WebMIDI is not supported in this browser.');
}

onMIDISuccess Callback

onMIDISuccess() is the function that is called if we successfully are able to get MIDI access. In that function, we do a few things:

• Shuffle the array of notes (level1Notes), so that the user doesn't see the notes in the same order each time they use the app.
• Draw the first note in the array on the staff.
• Get any available MIDI inputs (there will typically only be one). When an onmidimessage event fires, we'll call getMIDIMessage.

function onMIDISuccess(midiAccess) {
  shuffleArray(level1Notes);
  drawNote(level1Notes[noteIndex]);

  var inputs = midiAccess.inputs;
  var outputs = midiAccess.outputs;

  for (var input of midiAccess.inputs.values()) {
    input.onmidimessage = getMIDIMessage;
  }
}

// 60 represents middle C                           
var level1Notes = [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72];

Listen for noteOn

In the getMIDIMessage() function, we look at the three pieces of information that are passed as part of the channel voice message. As we learned earlier, a noteOn message on MIDI channel 1 has a value of 144. So that is what we're looking for in the switch statement. The other two pieces of information represent the note number and velocity. We update some text on the web app to show these three pieces of information, and then we call noteOnListener and pass in the note number.

function getMIDIMessage(message) {
  var command = message.data[0];
  var note = message.data[1];
  var velocity = message.data[2];

  switch (command) {
    case 144: // noteOn
      document.querySelector('.note-info').textContent = 'Command: ' + command +
        ' , Note: ' + note + ' , Velocity: ' + velocity;
      noteOnListener(note);
      break;
    }
}

In a more complex application, you could do a lot more in this function. You could listen for noteOff messages as well, for example.

noteOnListener

In the noteOnListener() function, we do the following:

• Check if the note that was played was the correct note. Set the color of the note to green or red depending on the outcome.
• After 1.5 seconds, reset the staff and display the next note in the array.
• Display the score once all the notes have been played.

That's pretty much all there is to it! If you want to look at the code, you'll find it here:

PRs welcome, especially if you'd like to add support for bass clef notes! :)

Want to Learn More?

Here's a list of resources and links where you can learn more about the Web MIDI API.

• Slides from my talk on this topic at Node+JS Interactive 2019
• Web MIDI API info from midi.org
• An excellent article on Getting Started with the Web MIDI API. I learned a lot from this one!