Using Machine Learning to Catalog Modular Synthesizers - Idea

TL;DR: Join me tomorrow at 1pm UK time on Twitch to start building a cool app to recognise modular synthesizer components from an image.

A colleague of mine, Simon Metson is a big modular synthesizer geek. He solders up and builds modules for modular sythesizers as well as plays in a "improvised synthesizer band".

He came to me with an idea: Could we create an app that could use machine learning to analyse a photo of a modular synth rack and catalog all the modules in it?

There is a website called Modulargrid that allows people to drag and drop from a library of modules onto a rack to document / illustrate their creations.

Here is Simon's rack on there:

Wouldn't it be great if you could just take a photo of your synth creation and have an AI analyse the image and identify all the modules in the rack and create an entry on Modulargrid for you?

So that is what we are going to try and build! We will be using the Tensorflow Object Detection API to see if we can train a neural network to be able to detect modules in images.

Come and join myself and Si live on the IBM Developer Twitch channel tomorrow at 1pm UK time for the first part in the series of us building it.