Ever scrolled through Netflix, Disney+ or Hulu and wondered what to watch next?
You can:
- Curate your own watchlist
- Delve into recommendations from streaming services
- Use one of countless websites and apps
Like many, I tried all of the above. They worked, to some extend. But I wanted more.
The Idea
What if you could find all scores and streaming information for movies and TV shows on one single page? This was the vision for GoodWatch:
Looks nice, I thought. So I built it.
The Prototype
I quickly built a Remix web app that fetched movie and TV data from TMDB and grabbed scores from different sites in real-time. After only three weeks, I was proud to deliver a working prototype that brought immediate value.
I used it almost daily. Loading a movie page took a few seconds though, so I needed a better solution.
The Need for Data
I compared many workflow engines and ended up choosing the excellent Windmill.
It allows me to define data pipelines like this:
Baseline data is fetched from the wonderful TMDB API. Then it gets enriched with ratings from IMDb, Metacritic, and Rotten Tomatoes, plus metadata from TV Tropes.
The Challenges
One of the biggest challenges is to map the correct URL's for each data source. Some (not so) clever techniques are needed to find the correct matches.
Another trade-off is how to deal with update frequencies. The database is around 15 GB big and data upgrades need to be carefully orchestrated.
It's self-explanatory that each service's rate limiting policies are respected.
The Stack
- Frontend: Developed with Remix, hosted on Vercel
- Data Handling: Utilizes Windmill for data pipelines, with a primary database powered by PostgreSQL. Auxiliary data storage is handled by MongoDB, with Redis for caching to optimize performance
- Hosting and Deployment: All services are deployed on Hetzner Cloud using Docker Compose
The Obvious
Visit GoodWatch and see how it makes movies and TV shows much easier to discover.
The project is open-source and available on GitHub: goodwatch-monorepo
Join the community on Discord: GoodWatch on Discord
The Future
The ultimate goal is to create the best recommendation engine since Jinni shut down in 2015.
I'm aiming to build a tool that understands your unique taste to help you discover hidden gems.
Feel free to ask any questions in the comments or suggest features you’d love to see!
This is a series of blog posts:
- What do you want to watch next? This is why I built GoodWatch. (you are here)
- A Data Pipeline for 1 million movies and 10 million streaming links
Top comments (4)
I am aiming to build a movie recommender when there are multiple people in a room. Is there an open API I can use so that instead of just presenting them with IMDb score, I present them with your score.
That's a great app idea. I don't have an API yet, but I'll add that to my bucket list.
I could imagine it would work like this:
Request:
Response:
Would that match with your expecations?
So cool! Glad Windmill helped you on it
Such a fantastic idea and interesting to know more about the infra behind the application. Showing the services and countries available to stream a touch of genius. Nice work!