Marcus Kohlberg for Encore

Posted on Jun 14

Build a chat room with custom bots powered by OpenAI/Gemini

#go #ai #programming #tutorial

This app combines popular chat platforms like Discord/Slack, and the powerful minds of LLMs like OpenAI and Google Gemini.

It lets you create your own AI bots with unique personalities that can seamlessly engage in conversations with each other and with users.

Demo app: Try the Demo version here

In this guide we'll walk through how to run it locally, deploy to Encore's free dev cloud, integrate with your Slack/Discord, and how to develop the application further.

TL;DR - What we'll build

Open Source: Go based application built using Encore (See Open Source Repo)
Multi-platform: Run locally or deploy to your cloud / AWS / GCP / Encore's free dev cloud
Multi-model: Use your OpenAI or Gemini API key to try out each model
Integrations: Add your favorite bots to your Discord or Slack

🏁 Getting started

💽 Install Encore

Install the Encore CLI to run your local environment:

macOS: brew install encoredev/tap/encore
Linux: curl -L https://encore.dev/install.sh | bash
Windows: iwr https://encore.dev/install.ps1 | iex

👉 Create your app

Create your Encore app and clone the example:



encore app create my-ai-chat --example=ai-chat

(Feel free to replace my-ai-chat with a name that tickles your fancy)

🔐 Set Your LLM API Key

OpenAI:
To use OpenAI's models, you need an OpenAI API key. If you don't already have one, you can get one here.

Then use Encore's built-in secrets manager to securely store it:



cd my-ai-chat
encore secret set OpenAIKey --type dev,local,pr,prod

Paste your OpenAI API key when prompted.

Gemini:
Setting up Gemini is a longer process, see the instructions in the appendix.

🕹 Run Your App Locally

To run your application locally, first make sure you have Docker installed and run.

Then, with Encore, you only need to use a single command to start your entire system including multiple microservices, databases, pub/sub, and other infrastructure.

Run it from your app's root directory:



encore run

Encore will build and start your application, providing you with a local URL (e.g., http://localhost:4000). Open this URL in your browser to see your creation - the local chat interface!

How it works

Local Development Dashboard

Encore comes with a local development dashboard, when your app is running, open http://localhost:9400 in your browser to access it.

It comes with tools like an API explorer, Service Catalog, architecture diagrams, and local tracing. All to help you test and understand the behavior of your application.

Go ahead and open it up now.

Architecture

If you open the "Flow" architecture diagram in the local dev dashboard, you should see this:

As you can see, AI Chat is a microservices-based application.

Each service handling a specific aspect of the chatbot ecosystem. The services uses a combination of Encore APIs, pub/sub messaging, and WebSocket communication to orchestrate the flow of messages between chat platforms and LLM providers.

Service Catalog

If you open the Service Catalog, you'll get an overview of all the services and endpoints in the application.

Here are the key components:

Chat Service: The orchestrator service, routing messages between chat platforms and LLM providers.
Discord Service: Handles the integration with the Discord API.
Slack Service: Manages the art of conversation with the Slack API.
Local Service: Provides a cozy web-based chat interface for testing and development.
Bot Service: Responsible for creating, storing, and managing bot profiles.
LLM Service: Formats prompts for LLMs, processes responses, and gracefully handles multiple LLM providers.
OpenAI Service: Interfaces with OpenAI's API for chat completions and image generation.
Gemini Service: Integrates with Google Gemini for even more chat completion options.

Main Flow:

A user sends a message in a connected chat channel
The corresponding chat integration (Discord, Slack, or local) receives the message
The integration publishes a message to the chat service
The chat service identifies any bots in the channel and fetches their profiles and the channel's message history
The chat service sends the message to the LLM service
The LLM service crafts a prompt including the bot's persona and the ongoing conversation
The prompt is sent to the chosen LLM provider (OpenAI or Gemini)
The LLM provider streams responses through pubsub back to the LLM service
The LLM service parses the responses and relays them back to the chat service
The chat service delivers the bot's witty (or not-so-witty) responses to the appropriate chat integration

Deploy to the Cloud

Ready to share your bots with the world? Encore makes deploying to a free dev environment a breeze, you only need to use one command.

Run it from your app's root directory:



git push

Encore will automatically deploy your app and set up all the necessary infrastructure. You'll see a link to the deployment in Encore's Cloud Dashboard.

Once the deployment is complete, click Overview and copy the URL to see your bots in action on the web!

From the Cloud Dashboard you can also connect your own AWS/GCP accounts and deploy there, all with the same automated process.

Add your bots to Discord/Slack

The application is designed to make it easy to integrate with any chat platform, and comes pre-configured to work with Discord and Slack.

Integrate with Slack

To be able to use Slack as a chat platform, you'll need to create a Slack app and add it to your workspace. Here's how you can do it:

1. Create the Encore App:

Complete the steps in the Getting Started section to create your Encore app.

2. Create a Slack App:

Visit https://api.slack.com/apps and click Create New App.
Choose From an app manifest and click Next.
Pick the workspace for your bot and click Next.
Copy the bot manifest and paste it into the text box.
Replace the <bot-domain> placeholder with your cloud environment's url (e.g. staging-my-chatbot-tur3.encr.app)
Click Next and then Create.

3. Activate Bot Events:

On the bot settings page, click Event Subscriptions.
Start the Encore app.
If the Request URL is yellow, click on Retry.

4. Install the App to Your Workspace:

On the settings page, click OAuth & Permissions and then Install to Workspace.
Select a channel for your bot and click Allow.

5. Add the Slack Bot Token:

Copy the Bot User OAuth Token from the OAuth & Permissions page.
Add it as an Encore secret: ```bash

encore secret set SlackToken --type dev,local,pr,prod

#### 6. **Create Your Chat Bots**

Proceed to the [Create Your Chat Bots](#create-your-chat-bots) section to add bots to your channels.

### Integrate with Discord

To be able to use Discord as a chat platform, you'll need to create a Discord bot and add it to your server. Here's how you can do it:

#### 1. **Create the Encore App:**

Complete the steps in the [Getting Started](#getting-started) section to create your Encore app.

#### 2. **Create a Discord Bot:**

* Go to [Developer Portal Applications](https://discord.com/developers/applications) and click `New Application`.
* Give your Discord app a name and click `Create`.

#### 3. **Configure Install Settings:**

* Click `Installation`.
* Select `Discord Provided Link` in `Install Link`.
* Under `Default Install Settings`, add the `bot` scope and these permissions:
* Connect
* Manage Web Hooks
* Read Message History
* Read Messages/View Channels
* Send Messages

#### 4. **Grant Privileged Gateway Intents:**

* Click `Bot` and then `Privileged Gateway Intents`.
* Enable these intents:
* Server Members Intent
* Message Content Intent

#### 5. **Copy the Bot Token:**

* On the `Bot` page, click `Reset Token`.
* Copy the token and add it as an Encore secret:
```bash


encore secret set DiscordToken --type dev,local,pr,prod

6. Install the Bot:

Copy the Install Link and paste it into your browser.
Grant your bot access to a server.

7. Invite the Bot to a Channel (Optional):

If you want your bot to join private conversations, invite it to specific channels.

8. Create Your Chat Bots

Proceed to the Create Your Chat Bots section to add bots to your channels.

Create Your Chat Bots

The Slack and Discord integrations does not come with a custom-made UI for adding bots to channels.

Until you've built your own UI (or maybe added support for slash commands?), you can use the API Explorer in Encore's Service Catalog to add bots to channels.

1. Open the Service Catalog

Open your app in the Cloud Dashboard and click on Service Catalog in the main menu.

2. Create a Bot Profile:

Open the bot service and select the bot.Create endpoint.
Give your bot a name, an engaging prompt, and enter openai as the LLM.
Click Call API.
Copy the bot ID in the response.

3. Find a Chat Channel ID:

Now open the chat service and select the chat.ListChannels endpoint.
Click Call API.
Copy the id of the channel you want your bot to join.

4. Add the Bot to a Channel:

Select the chat.AddBotToChannel endpoint.
Enter the bot ID and the channel ID.
Click Call API.

5. Say hello to your Bot:

🎉 Success!

Check your Slack/Discord channel; your bot should now be present and ready to chat!

Now you can create bots that will make people laugh, think, or maybe even question the nature of reality (but no pressure!).

Development: Make changes to the Chat Room UI

If you want to make changes to the app's built-in Chat Room interface you'll need to rebuild it before you deploy your changes.

1. Install npm*

If you don't have npm installed, you can download it from https://www.npmjs.com/get-npm, or use your package manager, e.g.



brew install npm

2. Install Dependencies

Navigate to the chat/provider/local/static directory and run:



npm install

3. Build the Interface

Run the following command to build the web interface:



npm run build

4. Start the Interface

Start the local chat service and open the interface in your browser:



encore run

5. Start the Interface in Dev Mode

Alternatively, you can start the interface in development mode to get hot reloading and other goodies:



npm run dev

(You still need to start the local chat service with encore run.)

Wrapping up

⭐️ Support the project by starring Encore on GitHub.
Learn more about building Go apps with Encore using these Tutorials.👈
Find inspiration on what to build with these Open Source App Templates.👈
If you have questions or want to share your work, join the developers hangout in Encore's community on Discord.👈

Appendix: Adding Gemini Credentials

To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:
To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:

1. Create a GCP Service Account:

Head over to the Google Cloud Console.
Click Create Service Account and give your new account a name and description.
Grant your service account the Vertex AI User role.
Click Done.

2. Create a JSON Key:

Click on your newly created service account and then on Keys -> Add Key -> Create New.
Choose the JSON format and click Create.
Download the JSON file.

3. Add JSON Key as an Encore Secret:



encore secret set --type dev,local,pr,prod GeminiJSONCredentials < <downloaded json>.json