How LyricsX keeps track of progress of media players

LyricsX is an open source software for macOS to download and display lyrics of current playing track on Music (previously iTunes), Spotify, Audirvana, Vox, Swinsian, or the Now Playing indicator in the OS. It gets time-tagged lyrics files from local storage or internet, and then display the lyrics in sync with the player.

As a crucial component of the development of Lyricova Jukebox, I have researched multiple implementations of real time lyrics display programs, and I found the mechanism behind LyricsX particularly interesting. Here in this article, I’d share with you how LyricsX tracking the player progress in a unique and resource-saving way.

Common practices

Before we go to LyricsX, let’s first take a look at how the majority of other open source software work on this. In short, there are 2 major ways of doing it: rely on the player to send out update events, or query the player for progress at a regular time interval. Both ways are not ideal when it comes to the extreme cases where the user adjusts the player progress a lot, and the lyrics having a lot of lines in a short period of time.

The first way, relying on the player to send out progress update events, is not reliable because the update interval totally depends on the platform. Some platforms may issue updates several times a second, but some may just send once a couple of seconds. In the worst case, some lines might just be skipped over due to the long update interval. Even worse some might just lack of such feature exposed to the public, rendering this way unusable in those cases.

The second way, query the player for progress on a regular time interval, could indeed solve the problems mentioned above, but just in an ideal system where the query itself is as simple as reading a variable. In contrary, it is almost never the case in real life. Usually to query the progress, you had to go through some kind of RPC, or running a tiny script. Either way it is, it already sounds a lot complicated and resource-consuming, in a worse case, some may even need to send a request through the internet for the playing progress. Depends on the precision, you might want to run the query multiple times a second. But if a single query can already take half a second, it is almost impossible to achieve a better precision this way.

How did LyricsX do it?

The way LyricsX solved all the problems above is really cleaver. Instead of keep asking for the player for the current progress, it maintains a timer inside itself, and try to sync it with the player by using only a few basic events.

State

The program maintains a global state of the player, with the following properties:

• Player status : Obviously, this can be either playing or paused. For the Stop state of some players, there is no track standby for playing, hence no timeline for lyrics, and thus out of our consideration.
• When the state is playing:
  • Internal t_0 time : This is calculated by current time in internal clock - player progress, serving as a reference point of the current play session. The internal clock can be anything that is fast enough to retrieve, like the system clock of the real world time, or a clock to measure code performance. We only care about the relative difference here.
  • Playback speed : 1.0 for normal speed, 0.5 for half speed, 2.0 for twice the speed, so on and so forth.
• When the state is paused:
  • Playback progress : the progress of the player when it is being paused.

The global state is updated in the following player events:

• Status change (on play, on pause, on stop)
• Manual time change (on seek)
• Rate change (on rate change)

You can see the event set we have is pretty basic, and mostly essential for such a software to run.

Sync the UI

With the player state in hand, it is much simpler to get the current player progress through just a simple system call and a calculation.

progress = ( t() - t_0 ) / rate

where t_0 is a function to get the current time in the internal clock.

Apart from that, since we now maintain a copy of the player state, we can simply assume that the player will continue the playback in the indicated speed as long as the state doesn’t change. With that, we can put more optimizations in place, such as generating animation sequence timelines ahead. All those preparation work only need to be done when the player state changes, which is much less frequent the update of playback progress.

In Lyricova Jukebox

In my attempt to port this logic in Lyricova Jukebox, I registered the play, pause, seeked and ratechange events, and for the internal clock, we used performance.now() for its high precision and the convenience to use in requestAnimationFrame().

For the timeline, LyricsX used a seemingly reliable internal method DispatcherQueue.schedule() provided by Apple. However, the scheduling function in web – window.setTimeout() – does not have the honor to have a high enough precision for us. We then resorted to have a requestAnimationFrame loop running when the media is playing. Thus, to start a loop when the player state turns to Start, and stop the loop when it turns to Stop.

In the requestAnimatioFrame loop, we check for the current time against the end time of the current keyframe. Since the loop is run every single frame in the browser, we keep the logic in the loop simple, and only advance a keyframe when the current frame ends. The frame pointer will only be updated in the hooks mentioned above to make sure the pointer is always pointing to the right frame.

If you are interested in the implementation details, you can take a look at the source code of LyricsX and its submodule MusicPlayer which is responsible for translating events from different players.

Alternatively, if you are more comfortable with React and TypeScript, I have ported this logic to my Lyricova Jukebox, which operates on the <audio /> tag of the browser.

• ddddxxx/LyricsX
• ddddxxx/MusicPlayer
• blueset/project-lyricova: usePlayerState()

Acknowledgement

Thanks a lot to ddddxxx, the main maintainer of LyricsX, for the brilliant idea and implementation of the software.

The post How LyricsX keeps track of progress of media players appeared first on 1A23 Blog.