Building a Terminal-based Chatbot in Rust from Scratch

Introduction

This is going to be a learning experience for me, and I hope it will be for you too. It's easy to get stuck in your comfort zone, and I find it helpful (and refreshing!) to break out occasionally.

I'm a JavaScript/TypeScript developer mostly. It's my day job, and it's what I find fastest and easiest for side projects. But I'm getting that itch to learn something new, so this project is designed as an intro to Rust.

Rust is a quite low-level programming language with some really interesting concepts surrounding memory management. This post will serve as a light introduction to the language in a practical way. We're going to build a functional chatbot together, that you can use in your terminal.

You can find the full code here, so just in case you miss something during the post, you will always have a full working reference.

Right, let's dive in!

Setting up the Rust toolchain

NB: If you already have Rust setup, you can skip this section.

The Rust toolchain is managed by rustup. Let's get that installed:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

The toolchain includes these three binaries:

• rustup - The Rust toolchain installer and version management tool.
• rustc - The Rust compiler, used to compile Rust code into executable binaries.
• cargo - The Rust package manager and build system, used for managing Rust projects and dependencies.

They should be installed now at ~/.cargo/bin, and we'll add this to our PATH as a convenience when running them. In a bash environment, you can append the following line to your .bashrc or .bash_profile file:

export PATH="$HOME/.cargo/bin:$PATH"

To get that applied, either restart your terminal client or run:

source ~/.bashrc

Now, confirm that you have everything installed correctly:

rustup -V
# rustup 1.26.0 (5af9b9484 2023-04-05)

rustc -V
# rustc 1.76.0 (07dca489a 2024-02-04)

cargo -V
#cargo 1.76.0 (c84b36747 2024-01-18)

If you don't see something like the above, please check the official docs to get your toolchain setup correctly.

If you do, congratulations, you're now a Rustacean (well...nearly).

Creating the Project

In Rust, cargo is used to manage projects their builds. We're going to use it to create the project, which we'll call ask, and then cd into the project folder.

cargo new ask --bin
cd ask

Notice we used the --bin flag, this is because we want to setup the project to build a binary file instead of a library.

If you inspect your project now, you will see that cargo has kindly created the following files for us.

cargo.toml
.gitignore
- src
  main.rs

The cargo.toml is our project's definition. Inside here, we can define what dependencies it uses and some other package information. At the moment, it should look like this:

[package]
name = "ask"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

This is all quite obvious, so I won't go further into it. The main.rs file is similarly uninteresting at the moment.

fn main() {
    println!("Hello, world!");
}

Let's just check everything's working properly before we move onto the meat of this post - the chatbot itself. In order to build the project and run it, we use cargo once again. The command cargo run will build and run the main.rs file for us. I wonder if you can guess what will be output to the terminal...I know, it's a tough one...

cargo run
   Compiling ask v0.1.0 (/Users/andy/dev/ask)
    Finished dev [unoptimized + debuginfo] target(s) in 0.82s
     Running `target/debug/ask`
Hello, world!

This is a momentous occasion. We've setup Rust correctly, and built our first Rust binary.

We could also have just run cargo build, but that wouldn't have run it too, and we wanted to see the output in this case. You can therefore think of cargo run as cargo build plus executing the binary that is produced. In this post, we'll mostly be using cargo run.

Scaffolding Out the Chatbot: Dependencies

For the initial scaffolding, we're going to bring in a few packages, or "crates". I'll explain why we need each of them, but first, let's add them to the dependencies section in our cargo.toml:

reqwest = { version = "0.11", features = ["json"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
dotenv = "0.15.0"

• reqwest - This is an asynchronous HTTP client used for making HTTP requests. The "json" feature is enabled to facilitate the handling of JSON data, which we'll need for the OpenAI API.

• tokio - An asynchronous runtime for Rust, providing features for writing asynchronous applications. The Rust standard library provides async/await, but it doesn't provide the runtime itself. I suppose this will come in a later version, but for now we need to use tokio. Don't worry, it's just one easy declaration.

• serde - A serialisation and deserialisation framework. The "derive" feature allows us to apply serialisation and deserialisation to our rust structs, so that they can be converted - for example - to and from JSON without us having to manually implement this formatting.

• serde_json - An extension of serde for JSON data handling, providing utilities to convert between JSON and Rust data structures. Often used in conjunction with reqwest and serde for web API interactions.

• dotenv - A library for loading environment variables from a .env file, where we'll be storing the OpenAI API key.

Ok, we've added our dependencies, now let's start building out the main.rs file.

Scaffolding Out the Chatbot: Boilerplate

We'll add some boilerplate first to handle things like:

• loading the API key from .env
• setting up the reqwest client
• declaring the use of the tokio async runtime

Replace the existing contents of your main.rs file with this:

use reqwest::Client;
use dotenv::dotenv;

#[tokio::main]
async fn main() -> Result<(), String> {
    dotenv().ok();

    let client = Client::new();

    Ok(())
}

I love the way Rust has borrowed some of the best features from other programming languages. Here you'll see parts that remind you of PHP, JS, C, and others.

But what is #[tokio::main]? This is called an attribute, and is used to apply metadata to functions, structs or other items. Here it's telling the Rust compiler to enable asynchronous functionality in the function immediately following - in this case it's the main function.

dotenv().ok(); is fairly self-explanatory, we're pulling in the environment variables from .env (which currently doesn't exist, but we'll change that in a second).

Then we're instantiating an HTTP client for use later.

Finally, we'll return Ok(()). Note that in Rust you can do this: return Ok(());, but you can also return just by removing the semicolon and the return keyword, which is what we're doing here.

Ok, moving on! We're finally going to start doing some real programming.

Implementing the Chat Interface

Adding Structs for OpenAI Communication

Firstly, we need to define some structures (structs) that will help us manage the data we send to and receive from the OpenAI API. These structs will represent the request and response formats expected by the API.

Insert the following code below your imports and above your main function:

#[derive(Serialize)]
struct OpenAIChatRequest {
    model: String,
    messages: Vec<Message>,
}

#[derive(Serialize, Deserialize, Clone)]
struct Message {
    role: String,
    content: String,
}

#[derive(Deserialize)]
struct OpenAIChatResponse {
    choices: Vec<ChatChoice>,
}

#[derive(Serialize, Deserialize)]
struct Conversation {
    messages: Vec<Message>,
}

#[derive(Deserialize)]
struct ChatChoice {
    message: ChatMessage,
}

#[derive(Deserialize)]
struct ChatMessage {
    content: String,
}

Here’s what each struct represents:

• OpenAIChatRequest - The data structure for sending a request to OpenAI, containing the model you want to use and the messages of the conversation.
• Message - Represents a single message in the conversation, with a role (user or assistant) and the message content.
• OpenAIChatResponse - The structure expected in the response from OpenAI, primarily containing a list of choices. Conversation - Used to keep track of the ongoing conversation in memory.
• ChatChoice and ChatMessage - Nested within OpenAIChatResponse, representing individual responses from the AI.

Implementing the Conversation Logic

Let’s build the core interaction loop where the user can input questions, and the program adds them to the conversation and requests responses from OpenAI.

Replace the main function with the following:

#[tokio::main]
async fn main() -> Result<(), String> {
    dotenv().ok();
    let client = Client::new();
    let mut conversation = Conversation {
        messages: Vec::new(),
    };

    println!("\nWhat do you want to talk about today?\n");

    loop {
        println!("You:");
        let mut question = String::new();
        stdin().read_line(&mut question).map_err(|e| e.to_string())?;
        let question = question.trim();

        if question.eq_ignore_ascii_case("exit") {
            break;
        }

        conversation.messages.push(Message {
            role: "user".to_string(),
            content: question.to_string(),
        });

        let response = ask_openai(&client, &mut conversation).await?;
        println!("\nAI: {}", response);
    }

    Ok(())
}

In this updated main function, we've created a loop that prompts the user to input their question. If the user types "exit", the loop breaks, and the program ends. Otherwise, the question is added to the conversation and sent to the OpenAI API for a response.

I won't go into all the details here, because there are already a lot of things going on here, and The Rust Programming Language book is far better than me at explaining. I highly recommend giving it a read - it's really a work of art. And frankly, it needs to be because some of these memory management concepts take a fair amount of brain power initially. However, I suspect that once you "get it", Rust will become very natural.

async fn main() -> Result<(), String>: {

The async keyword makes main an asynchronous function. This basically allows you to use .await on "futures", which are like Promises from JS.

Result<(), String> is the return type of the function, and is a really common Rust pattern for error handling. In this case, main either returns Ok(()) indicating success, or an Err(String) containing an error message. This is a simplification for (my) educational purposes; in larger applications, you might use more specific error types.

let mut question = String::new();

Here, Rust's ownership rules ensure that question is properly allocated and deallocated, preventing memory leaks. The question instance is owned by the loop block and is dropped (freed) when it goes out of scope. This is a core feature of Rust's memory safety and frees us from having to handle memory release explicitly.

Using .map_err(|e| e.to_string())? converts errors from one type to another (reqwest::Error to String). The ? operator is syntactic sugar for early returns in case of an Err.

Crafting the ask_openai Function

Now, we need to define the ask_openai function, which sends the current state of the conversation to OpenAI and retrieves the AI's response.

Insert this function below your main function:

async fn ask_openai(client: &Client, conversation: &mut Conversation) -> Result<String, String> {
    let request_body = OpenAIChatRequest {
        model: "gpt-3.5-turbo".to_string(), // Specify the OpenAI model you want here
        messages: conversation.messages.clone(),
    };

    let response = client
        .post("https://api.openai.com/v1/chat/completions")
        .header("Content-Type", "application/json")
        .header(
            "Authorization",
            format!("Bearer {}", env::var("OPENAI_API_KEY").expect("OPENAI_API_KEY not set")),
        )
        .json(&request_body)
        .send()
        .await
        .map_err(|e| e.to_string())?;

    let response_body = response
        .json::<OpenAIChatResponse>()
        .await
        .map_err(|e| e.to_string())?;

    if let Some(choice) = response_body.choices.last() {
        conversation.messages.push(Message {
            role: "assistant".to_string(),
            content: choice.message.content.clone(),
        });
        Ok(choice.message.content)
    } else {
        Err("No response from AI".to_string())
    }
}

In ask_openai, we construct the request with the current conversation and send it to OpenAI using the reqwest client. We then parse the response and add the AI's reply back into the conversation. If the API call is successful, the function returns the AI's response; otherwise, it returns an error.

The function takes &Client (a reference to Client) and &mut Conversation (a mutable reference to Conversation). Rust's borrow checker ensures that while client is immutably borrowed (you can't modify it within ask_openai), conversation can be modified.

Rust's clever borrow checker automatically infers the lifetimes here, ensuring that the references to client and conversation do not outlive the data they refer to. This prevents dangling references, a common source of bugs in low-level languages like C/C++.

let response_body = response
    .json::<OpenAIChatResponse>()
    .await
    .map_err(|e| e.to_string())?

The .await pauses the function's execution until the network request completes, without blocking the entire thread. Notice that we have to map the error to a String at the end. This is because we're only returning simple strings as errors. For more complex error handling, I've heard good things about the anyhow crate, which as I understand it will simplify handling more complex error structures and will avoid the need to cast the response to a string as I'm doing here.

Oh, I nearly forgot, you need to get an OpenAI API key if you haven't already, and store it in a .env file at the root of the project. Head over to OpenAI and login there to get hold of your key.

OPENAI_API_KEY=your_open_ai_key

Remember to add .env to the .gitignore because you don't want to commit your API key otherwise you could wake up to a nasty financial surprise!

Right, I think we're ready to build and run this thing!

Building and Running the Project

Let's test it out:

cargo run
   Compiling ask v0.1.0 (/Users/andy/dev/ask)
    Finished dev [unoptimized + debuginfo] target(s) in 2.49s
     Running `target/debug/ask`

What do you want to talk about today?

You:
Let's talk about you!

AI:
Sure, what would you like to know about me?

You:
What's your name?

AI:
I am a language model AI created by a team of developers. You can call me Assistanto.

You:
Haha, you're so funny Assistanto.

AI:
Thank you! I'm here to help and hopefully bring a smile to your face as well.

You:
Bye!

AI:
Goodbye! Feel free to come back anytime if you have more questions or just want to chat. Have a great day!

Well, it looks like Assistanto is working. Congratulations if you've made it this far!

There's so much we haven't touched on here. Testing, publishing, etc. And we haven't had to use lifetimes at all either. Rust is a rich and expressive language that is also extremely powerful and actually quite fun too.

I hope you enjoyed this little tutorial, I certainly enjoyed learning some Rust in the process. Next up, I'm not sure whether to dive deeper into Rust or perhaps write a chatbot in Go by way of comparison. Maybe you can suggest something in the comments?