This is a submission for the Cloudflare AI Challenge.
What we built
Douglas Adams came up with the idea of the Babel fish in Hitchhiker's Guide to the Galaxy. It's a fish that, once you've stuck it in your ear, allows any language to be translated to you in real time.
babel_fish is a real time language translation app built with Cloudflare, Svelte, and multiple language models. The app allows you to share a room with your friends in different languages. All messages get translated to your chosen language in real time!
@alicelovescake and I thought this would be a perfect app for trying out the fancy new Cloudflare Workers AI!
Demo
Le code
Frontend
The Svelte application contains the frontend code as well as all of our API endpoints interacting with the language models.
amorriscode
/
babel-fish
A real time translation app
babel_fish <><
This is a submission for the Cloudflare AI Challenge.
babel_fish is a real time language translation app built with Cloudflare, Svelte, and multiple language models. The app allows you to share a room with your friends in different languages. All messages get translated to your chosen language in real time!
This repo contains the Svelte frontend which includes API routes for interacting with the AI. The chat engine is handled by the backend.
Built with:
- 💅 Svelte
- ⚙️ Cloudflare Workers
- 🤖 Cloudflare Workers AI
- 📣 ElevenLabs
🚧 Requirements
- An ElevenLabs API key
- A running version of the backend
🛠️ Development
Note:
wrangleris used to run the dev server so we have access to Cloudflare Workers AI
-
Create an
.envfile and fill in the variablescp .env.example .env -
Clone the frontend
git clone https://github.com/amorriscode/babel-fish -
Install dependencies
npm install -
Run the wrangler dev server
npm run…
Backend
Our web socket backend uses Cloudflare Workers and Durable Storage. It's a simple logic engine that powers the chat rooms.
amorriscode
/
babel-fish-ws
Chat engine for the real time translation app babel_fish
babel_fish <>< ws
This is a submission for the Cloudflare AI Challenge.
babel_fish is a real time language translation app built with Cloudflare, Svelte, and multiple language models. The app allows you to share a room with your friends in different languages. All messages get translated to your chosen language in real time!
This repo contains the websocket backend that powers the babel_fish frontend;
Built with:
- ⚙️ Cloudflare Workers
- 📦 Cloudflare Durable Objects
🚧 Requirements
- A paid Cloudflare account to use Durable Objects
🛠️ Development
-
Clone the repo
git clone https://github.com/amorriscode/babel-fish-ws -
Install dependencies
npm install -
Run the dev server
npm run wev
Journey
Tired: trying out one new technology during a hackathon
Wired: trying out tons of new technology during a hackathon
I've been admiring Cloudflare from afar for a while now. I love what they've been doing with Pages, Workers, and now AI Workers. This hackathon finally gave me a good excuse to play around with the platform!
Aside from all the fancy tooling Cloudflare provide, this was also my first time using Svelte (sorry Rich!). I actually really enjoyed the experience. I'd love to spend more time writing Svelte; I have a lot left to learn.
So... AI?
We used a ton of Cloudflare's available AI models to build babel_fish.
-
@cf/openai/whisperfor voice-to-text -
@hf/thebloke/mistral-7b-instruct-v0.1-awqfor language detection -
@cf/meta/m2m100-1.2bfor language translation
Since a true Babel fish doesn't produce text we also used ElevenLabs for text-to-speech.
Challenges
We only had a single day to work on our project. The tight timeline and all new tech made it an exciting 24 hours.
The biggest challenge of this project actually ended up being the backend audio processing. With the initial POC, user messages weren't transcribed and sent over the wire until they were completely finished speaking. We were able to improve this by using AudioWorklets. Now we regularly check for gaps in a user's message and periodically process and broadcast messages.
The rest was easy because Cloudflare and Svelte made it so. 😎
Thanks to @cloudflaredev and @thepracticaldev for putting on this hackathon. If you're interested in this kind of thing, come work on experimental AI projects with me at Stripe!
<><
Top comments (0)