In the current age of digital music streaming, keeping track of one's preferences can become a daunting task. With the multitude of artists and genres available on platforms like Spotify, understanding your musical tastes can feel like traversing an ever-expanding universe.
This was the problem I wanted to solve: I needed a way to analyze the predominant genres in my Spotify playlists.
The idea was simple: create an application that can pull artists from a Spotify playlist, then use EveryNoise.com to fetch their respective genres, and finally provide a sorted list of genres that appear the most in the playlist.
The real challenge was using the latest AI tools to build this application. I wanted to leverage the power of OpenAI's GPT-4 API to streamline the development process.
This is the end result if you want to skip the reading and check the code directly:
Markkop
/
discover-your-noise
Scrap EveryNoise.com to discover the most dominant genres from a Spotify playlist
🎧 Discover Your Noise
This is a simple Node.js script that counts the genres of songs in a Spotify playlist. The script uses the Spotify Web API to get the playlist's artists, and then it uses Puppeteer to scrape the genres of each artist from everynoise.com.
Prerequisites
Before you begin, ensure you have met the following requirements:
- You have installed Node.js and npm.
- You have a Spotify account.
Installation
To install Genre-Scrapper, follow these steps:
git clone git@github.com:Markkop/discover-your-noise.git
cd discover-your-noise
npm install # or yarn install
Obtaining Spotify API Keys
In order to use this script, you will need a Spotify Client ID, Client Secret, and an access token. Here's how you get them:
- Go to the Spotify Developer Dashboard.
- Log in to your Spotify account.
- Click on 'Create an App', fill out the form, and click 'Create'.
- Now you are in your application's dashboard, you can see your…
ChatGPT to the Rescue
My journey in solving this problem started with building a Node.js package using Puppeteer.
My aim was to access EveryNoise.com, search for an artist, and fetch their related genres.
With the HTML content from EveryNoise.com as a reference, I asked ChatGPT to provide the necessary code. This prompt allowed me to focus on the exact requirement: building a scraper to pull genre data for a specific artist.
This is the conversation I had with the AI:
chatgpt.com
If you don't want to open the link, these are the prompts and responses I mainly used:
🕸️ EveryNoise Scrapper
Prompt
Make a nodejs package with puppeteer to access this page, search for an artist, like "beyhude" and get its related genre results. Don't explain anything, just give me the code.
Website: https://everynoise.com/lookup.cgi
// HTML Content of https://everynoise.com/lookup.cgi
<html><head>
<-- ! and so on -->
Result
const puppeteer = require("puppeteer");
const getGenres = async (artist) => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(
"https://everynoise.com/lookup.cgi?who=" + encodeURIComponent(artist)
);
const genres = await page.evaluate(() => {
const anchorTags = document.querySelectorAll("div>a:not([title])");
return Array.from(anchorTags).map((a) => a.textContent);
});
await browser.close();
return genres;
};
getGenres("beyhude").then(console.log).catch(console.error);
📻 Spotify Integration
After creating the scraper, the next step was integrating it with Spotify. My new task for ChatGPT was to modify the code to access a Spotify playlist, get the genres for all artists on that playlist from EveryNoise.com, and sort the genres based on their frequency of occurrence.
Prompt
Now, change so it accesses a spotify playlist and get the genres for all artists on that playlist from Every Noise and then sort the genres from the most found one to the less found.
Result
const puppeteer = require("puppeteer");
const SpotifyWebApi = require("spotify-web-api-node");
// Configure the Spotify API client
const spotifyApi = new SpotifyWebApi({
clientId: "<CLIENT_ID>",
clientSecret: "<CLIENT_SECRET>",
redirectUri: "<REDIRECT_URI>",
accessToken: "<ACCESS_TOKEN>",
});
const getGenres = async (artist) => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(
"https://everynoise.com/lookup.cgi?who=" + encodeURIComponent(artist)
);
const genres = await page.evaluate(() => {
const anchorTags = document.querySelectorAll("div>a:not([title])");
return Array.from(anchorTags).map((a) => a.textContent);
});
await browser.close();
return genres;
};
const getPlaylistArtists = async (playlistId) => {
const data = await spotifyApi.getPlaylist(playlistId);
const tracks = data.body.tracks.items;
const artists = tracks.map((track) => track.track.artists[0].name);
return [...new Set(artists)];
};
const countGenres = async (playlistId) => {
const artists = await getPlaylistArtists(playlistId);
const genres = await Promise.all(artists.map((artist) => getGenres(artist)));
const genresFlat = genres.flat();
const count = genresFlat.reduce((acc, genre) => {
acc[genre] = (acc[genre] || 0) + 1;
return acc;
}, {});
const sortedGenres = Object.entries(count)
.sort((a, b) => b[1] - a[1])
.map(([genre]) => genre);
return sortedGenres;
};
countGenres("<PLAYLIST_ID>").then(console.log).catch(console.error);
🌀 Too many loops
However, during the execution, I encountered a memory error - an EventEmitter memory leak. I knew the issue was the parallel calls made by the Promise.all, but wanted to solution from ChatGPT:
Prompt
I got the error
(node:91817) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added to [process]. Use emitter.setMaxListeners() to increase limitShow me the code I have to change to fix it
Response
The warning MaxListenersExceededWarning: Possible EventEmitter memory leak detected. is usually a sign that you're creating a lot of instances of something (in this case, puppeteer browsers), and each of them is registering an event listener.
The issue lies in the fact that a new browser instance is created for each artist with puppeteer.launch(), and these are not being properly managed.
A good practice would be to use a single browser instance and create a new page for each artist. This way, you can avoid the warning and optimize the script. Here's how you can modify your code:
🤝 Helping the AI
ChatGPT blamed puppeteer.launch(). Indeed, the app doesn't need to open a new browser for each artist, but the problem was opening all of them at the same time.
So I gave a hint to ChatGPT and applied its solution.
Prompt
(I didn't bore to format this one)
I think the issue is on const genres = await Promise.all(
artists.map((artist) => getGenres(browser, artist))
); since we're using the same browser for all artists. Make it so we get one each time instead of all the same time
Response
const countGenres = async (playlistId) => {
const artists = await getArtistsFromPlaylist(playlistId);
let genresCount = {};
for (const artist of artists) {
const browser = await puppeteer.launch();
const genres = await getGenres(browser, artist);
await browser.close();
for (const genre of genres) {
if (genre in genresCount) {
genresCount[genre]++;
} else {
genresCount[genre] = 1;
}
}
}
const sortedGenresCount = Object.entries(genresCount).sort(
(a, b) => b[1] - a[1]
);
return sortedGenresCount;
};
This iterative process of problem-solving and code refinement, guided by prompts to ChatGPT, helped me successfully build the core functionality of the application.
This is the v1 commit with the whole code.
But I would like to change a few lines of code and apply good practices, so the next prompt was:
Refactor this code to apply clean code patterns and early returns
And the result was this refac: apply code good practices commit.
Bonus: generating a README
Of course I needed a README for the project, so I asked ChatGPT to generate one for me.
Prompts:
Make a github readme on how to obtain these env. variables in details, focus on client id, secret and access token and how to run this script
// index.js
{I pasted the code here}add a section on how to obtain the playlist id
Give me 5 emojis that could represent this project
I made some minor changes to the generated README, but it was a great starting point.
Leveraging Genie for Code Refactoring
After setting up the basic functionality, it was time for some enhancements. Initially, the playlist ID was set as an environment variable, which wasn't exactly user-friendly. I decided to refactor the code to allow users to input a playlist ID or URL as an argument. This is where Genie, a Visual Studio Code extension powered by OpenAI's GPT-4 API, came into play.
ai-genie
/
chatgpt-vscode
Your best AI pair programmer in VS Code
A Visual Studio Code - ChatGPT Integration
Prompt OpenAI's GPT-4, GPT-3.5, GPT-3 and Codex models within Visual Studio Code
This repository is meant for documentation, bug reports and feature requests
Testimonials
❄️ Featured by Snowflake on Medium blogpost
🎌 Blogpost VSCode に ChatGPT の拡張機能を入れてコードレビューやバグを発見してもらう
💙 Reviews on Twitter
❤️ ChatGPT the pair programmer - VS Code on Youtube
💙 Generative AI on LinkedIn
Level up your developer experience with Genie
- ✨ Use your own Azure OpenAI Service deployments
- ⚡ Generate commit messages from your git changes
- 💬 Store your conversation history on your disk and continue at any time.
- 💡 Use Genie in Problems window to explain and suggest fix for compile-time errors.
- 🔁 See diff between your code and Genie's suggestion right within editor with one click.
- 👤 Rename and personalize your assistant.
- 📃 Get streaming answers to your prompt in editor or sidebar conversation.
- 🔥 Streaming conversation support…
Genie is a comprehensive tool that assists developers within the Visual Studio Code environment. It provides numerous features that aid in code fixing, drafting, and refactoring, all backed by OpenAI's potent language models like GPT-4.
Genie's suggestions and the real-time diff feature were helpful during the refactoring process.
These were the prompts I used to refactor the code:
Change this So I can pass the playlist ID directly in the command line
Make it optional so it fallbacks to the env option
Change it so it also accept a spotify URL like this one: https://open.spotify.com/playlist/76yxaCsawHQZuuVQgSGlvG?si=a0d56a455c2f429f
Refactor to it to use functions and return early patterns
And the end result you can see in commits
feat: accept playlist id as command line argument
feat: get ID from playlist URL as command line arg
Creating a UI with RefactorGPT and AutoPR
The next step was to add a user interface to the application. I decided to experiment with RefactorGPT, a tool that merges files and sends them with a prompt to the GPT API.
However, the result is just a chatbot response.
While that works to convert the script to a UI, I wanted an automated one.
I wanted to have the IA to do the hard work for me.
So I searched on Twitter for gpt 4 pull request and found the gpt-assistant project which is basically an autonomous GPT agent, like AutoGPT, that knows how to use a browser.
I tried to use it, but not only AutoGPTs aren't that useful yet, but also I didn't want to make it follow the whole manual flow to create a Pull Request like a human would do.
Then I found this tweet talking about AutoPR.
AutoPR is designed to autonomously write pull requests in response to issues and seems to be a promising tool. Unfortunately, being in its early stages, it was difficult to get it to work as expected.
I setup its Github Action, but a single refactor took too long and costed me $2 from GPT-4 API calls. I stopped the action before it finished.
Refactor countGenres
#1
Refactor the countGenres function on index.js, splitting its responsibilities to other functions and renaming the function name if needed.
Concluding Thoughts
At this juncture, I've decided to pause the development of the application. It has achieved what I initially set out to do - analyze the predominant genres in my Spotify playlists. The application is functioning as expected, and the user interface, while a nice addition, would only serve as a bonus.
The ecosystem is rapidly evolving and it's only a matter of time before tools like AutoPR become more efficient and affordable. I'm eagerly awaiting that day, when creating applications could simply mean creating issues on Github. I estimate this reality is only a few months away.
Until then, I'll be monitoring the advancements in AI-assisted development tools and refining my skills. When the time is right, I look forward to revisiting this application, enhancing its features, and sharing my journey with you. In the world of tech, the only constant is change, and I can't wait to see where it takes us next.
Disclosure: This blog post was partially restructured with the assistance of AI.
Top comments (0)