DEV Community

Cover image for Acoustic activity recognition in JavaScript

Acoustic activity recognition in JavaScript

Charlie Gerard on December 22, 2019

For my personal projects, I often get inspired by research papers I read about human-computer interaction. Lately, the one I've been looking into ...
Collapse
 
supunkavinda profile image
Supun Kavinda

Okay, one question.

low-frequency whistling = brushing teeth.

How does this happen? Does the software recognize frequency, or it also recognizes the tone?

Collapse
 
devdevcharlie profile image
Charlie Gerard

I didn't train it with whistling so it's predicting whatever is closest to what it was trained with, which must be brushing teeth in this case.
It's looking at all the data coming from the WebAudio API

Collapse
 
supunkavinda profile image
Supun Kavinda

Yep. I asked if it is trained for frequency or the tone? or both?

Thread Thread
 
devdevcharlie profile image
Charlie Gerard

If by tone you mean notes, notes are named frequencies.

The data you get back from the WebAudio API is the frequencies picked up by the microphone copied into a Uint8Array using the getFrequencyByteData method on the AnalyserNode.

Thread Thread
 
supunkavinda profile image
Supun Kavinda

By tone, I meant tone: which makes a C note from violin and piano distinct.

However, I got what you meant. Nice and creative work!

Thread Thread
 
josepjaume profile image
Josep Jaume Rey

Notes are just particular frequencies that are given a name. In any case, the tone you're using is a bit off.

Thread Thread
 
techpeace profile image
Matt Buck

Traditionally, that's referred to as timbre. It's the volume of specific frequencies in a note's harmonic series that account for differences in timbre in pitched instruments. Here's a great video explaining how that works.

Collapse
 
veselinastaneva profile image
Vesi Staneva

Awesome post! Kudos for making such an interesting experiment with JavaScript 👏 👏 👏

Actually, my team just completed an open-sourced Content Moderation Service built with Node.js, TensorFlowJS, and ReactJS that we have been working over the past weeks. We have now released the first part of a series of three tutorials - How to create an NSFW Image Classification REST API and we would love to hear your feedback(no ML experience needed to get it working). Any comments & suggestions are more than welcome. Thanks in advance!
(Fork it on GitHub or click🌟star to support us and stay connected🙌)

Collapse
 
brunooliveira profile image
Bruno Oliveira

Amazing!! Your side projects are always awesome

Collapse
 
mvasigh profile image
Mehdi Vasigh

This is super cool! An awesome intersection of a bunch of different things I've been wanting to play with. Thanks for putting this together.

Collapse
 
codermikky profile image
Sai Deep Konduri

Impressive

Collapse
 
user1m profile image
Claudius Mbemba

Really cool @devdevcharlie !

Collapse
 
husslerrr profile image
Brown-West

Nice work, trying to remake this for practice purposes, how do I go about it ?

Collapse
 
anthonyofboston profile image
Anthony

Thank you. I used this tutorial and was able to create my own acoustics activity recognition app for detecting drones anthonyofboston.github.io/