In this post, we're using the Open AI chat completion AI to build a conversational agent that you can put on your portfolio page. It is built with NextJS, Tailwindcss, React, Typescript and the Open AI Chat completion API. Try Virtual Vince.
Introduction
There are a lot of articles about building apps with the OpenAI API at the moment and you just gotta ride the hype train, before GPT-5 finally will take all of our jobs. So I was thinking about a cool project and thought it would be fun to build a chat app that serves kinda like a first level support for a potential customer visiting my portfolio page. It's really straight forward and easy to follow along, if you would like to build something similar.
Requirements
The following requirements were set for the realization of this project.
- No user data should be stored
- The agent should start with a welcoming message to the user
- When requested, the agent should be able to explain what I do
- The application should feel like a normal chat
- There should be some rate limiting and input limits, so users can not burn my tokens
- After some time the agent should end the conversation and link the user to my contact page
The requirements suffice to get some easy PoC up and running...
Setting up the app
We're using create-t3-app to set up a NextJS app with TS, Tailwindcss and tRPC.
npx create-t3-app@latest
To use the OpenAI API, I installed the openai client via:
npm install openai
The initial file structure looks like this:
For this project, you can split the application in three parts: All frontend code will go into the pages directory. Server side code that is interacting with the OpenAI API, goes into the server folder. All configuration, like the ApiKeys and validation of environment variables goes into the .env and .env.mjs files.
For those who do not know tRPC yet, here is a short explanation: tRPC stands for type-safe Remote Procedure Call and is a protocol that let's you couple your server-/ and client-side code enabling you to write end-to-end APIs.
The Open AI chat completion API
Before starting, we quickly want to have a look at the OpenAI API reference to see how data is transferred between my app to see how we have to process our chat messages and how we can influence the behavior of the GPT model.
Request
For the request, there are two parameters required: model, which we will set to: gpt-3.5-turbo and messages, which is the whole chat history between the user and the agent (part of the reason, why the chat rate should be limited, so people do not burn your tokens like crazy). The example on the OpenAI page looks like this:
const { Configuration, OpenAIApi } = require("openai");
const configuration = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);
const completion = await openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [{role: "user", content: "Hello world"}],
});
console.log(completion.data.choices[0].message);
There are several parameters that can be used to tweak the results that the model produces, however for this app we will only change the following ones from the default:
presence_penalty: this setting influences the likelihood, that the agent will talk about a topic it has already talked about.
max_tokens: the maximum amount of tokens to be generated by the agent
Response
A corresponding response of the Chat Completion API could look like follows:
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "\n\nHello there, how may I assist you today?",
},
"finish_reason": "stop"
}],
"usage": {
"prompt_tokens": 9,
"completion_tokens": 12,
"total_tokens": 21
}
}
Here we can already see that the token count per request proportionally increases with the length of a conversation, meaning every subsequent request will become more expensive.
Building the frontend
The application we're building will look like this. A simple chat where the user will get greeted with a welcome message. When the user submits a message, the agent will respond after some timeout.
Before we can start building the frontend, I want to quickly create a dummy reply remote procedure that emulates chat completion API. For this we just create a new router under ~/server/api/routers/chat.ts and create an async public procedure called test, that takes an object, consisting of a role (user in this case) and returns the response I copied from the Open API AI Reference. For the input validation I am using Zod. The procedure awaits a promise to emulate GPT processing the response.
import { z } from "zod";
import { Configuration, OpenAIApi } from "openai";
import { createTRPCRouter, publicProcedure } from "~/server/api/trpc";
/* const openai = new OpenAIApi(
new Configuration({
apiKey: process.env.OPENAI_API_KEY,
})
); */
export const chatRouter = createTRPCRouter({
example: publicProcedure
.input(
z.object({
role: z.string(),
content: z.string(),
})
)
.mutation(async ({ input }) => {
await new Promise((resolve) => setTimeout(resolve, 3000));
const { content, role } = input;
//returns a ChatCompletionResponse object example (not the real api)
const response = {
id: "chatcmpl-123",
object: "chat.completion",
created: 1677652288,
choices: [
{
index: 0,
message: {
role: "assistant",
content: "Answer to your question: " + content,
},
finish_reason: "stop",
},
],
usage: {
prompt_tokens: 9,
completion_tokens: 12,
total_tokens: 21,
},
};
return response?.choices[0]?.message;
}),
});
After having set up this procedure we can move forward to building the chat.
Building the chat app
In ~/pages/index.tsx we're bulding a NextPage, that has the following two components: Messages and PushMessagesForm. The page shares state between the components by using the useContext() hook. The context is also used to share the state mutators for the states messages and processing between the components. The chat messages state is initiliazed by pushing a first message from the assistant into the messages array (This initial message will also be a generated chat completion later).
import { type NextPage } from "next";
import Head from "next/head";
import {
createContext,
useState,
useContext,
type FC,
type FormEvent,
useRef,
useEffect,
} from "react";
import { api } from "~/utils/api";
type MessageType = {
content: string;
role: string;
};
const MessagesContext = createContext<Array<MessageType>>(
[] as Array<MessageType>
);
// eslint-disable-next-line @typescript-eslint/no-empty-function
const SetMessagesContext = createContext((_messages: Array<MessageType>) => {});
const ProcessingContext = createContext<boolean>(false);
// eslint-disable-next-line @typescript-eslint/no-empty-function
const SetProcessingContext = createContext((_processing: boolean) => {});
// <- PushChatMessageForm goes here
// <- ChatMessages component goes here
const Home: NextPage = () => {
const [messages, setMessages] = useState<Array<MessageType>>([
{
content: "Hi, I am VirtualVince. How can I help you?",
role: "assistant",
},
]);
const [processing, setProcessing] = useState<boolean>(false);
return (
<>
<Head>
<title>VirtualVince</title>
<meta name="description" content="Hi I am VirtaulVince. Let's chat." />
</Head>
<main className="mx-auto flex h-screen max-w-3xl flex-col bg-white p-5 md:px-0">
<div className="flex-none">
<h1 className="text-xl font-bold text-neutral-900 sm:text-2xl md:text-4xl">
VirtualVince
</h1>
<p className="text-sm text-neutral-800 sm:text-base">
This is VirtualVince, a simple chat bot that you can talk to find
out a few things about me and what I do. It is built with NextJS,
Tailwindcss, tRPC, Typescript and the OpenAI chat completion AI.
</p>
</div>
<ProcessingContext.Provider value={processing}>
<SetProcessingContext.Provider value={setProcessing}>
<MessagesContext.Provider value={messages}>
<SetMessagesContext.Provider value={setMessages}>
<ChatMessages className="mt-5 flex-1 overflow-y-auto" />
<PushChatMessageForm className="flex-none" />
</SetMessagesContext.Provider>
</MessagesContext.Provider>
</SetProcessingContext.Provider>
</ProcessingContext.Provider>
</main>
</>
);
};
export default Home;
PushMessagesForm
The component PushMessagesForm handles the user messages and the interaction with the server, i.e. moving the response from the agent into the messages array. By using useContext() we can access the states from outside of the component. To create a more natural feeling of the reaction time of the agent we use a timeout will cause the processing block to only render after 750ms. We use another timer for the isTyping... label over the textarea. As the timeout for the isTyping... label is triggered on the input event we need to define a timerRef with useRef() that is reset every time the input is called.
In the handleSubmit function we only send the request to the server when the message by the user is between the two defined variables minLength and maxLength. Furthermore, the user can only submit the message to the server when the previous request has already been processed.
The message is submitted calling the publicProcedure that was previously defined in the chatRouter, by using the mutateAsync() hook. The means we are performing an asynchronous mutation with tRPC.
const PushChatMessageForm: FC<React.HTMLAttributes<HTMLFormElement>> = ({
...props
}) => {
const [message, setMessage] = useState<string>("");
const [isTyping, setIsTyping] = useState<boolean>(false);
const messages = useContext(MessagesContext);
const setMessages = useContext(SetMessagesContext);
const isProcessing = useContext(ProcessingContext);
const setIsProcessing = useContext(SetProcessingContext);
const sendChatMessage = api.chat.example.useMutation();
const isTypingTimerRef = useRef<NodeJS.Timeout | null>(null);
const maxLength = 200;
const minLength = 10;
const handleSubmit = async () => {
if (message.length > minLength && message.length <= maxLength && !isProcessing) {
messages.push({ content: message, role: "user" });
setMessages([...messages]);
} else {
if (message.length < minLength) {
alert("Message is too short");
}
if (message.length > maxLength) {
alert("Message is too long");
}
if (isProcessing) {
alert("Wait for Virtual Vince to answer");
}
return;
}
setMessage("");
setIsTyping(false);
setTimeout(() => {
setIsProcessing(true);
}, 750);
await sendChatMessage
.mutateAsync({
role: "user",
content: message,
})
.then((response) => {
if (response) {
messages.push(response);
setMessages([...messages]);
setIsProcessing(false);
}
});
};
const handleInput = (e: FormEvent<HTMLTextAreaElement>) => {
setMessage(e.currentTarget.value);
setIsTyping(true);
if (isTypingTimerRef.current) clearTimeout(isTypingTimerRef.current);
isTypingTimerRef.current = setTimeout(() => {
setIsTyping(false);
}, 1500);
};
return (
<form
className={`flex w-full flex-col ${props.className ?? ""}`}
onSubmit={(e) => {
e.preventDefault();
void handleSubmit();
}}
>
<span className="h-6 flex-none text-sm">
{isTyping ? "Typing..." : ""}
</span>
<div className="flex w-full flex-1 flex-row rounded-xl bg-neutral-100 p-3 text-neutral-600 shadow-sm">
<textarea
rows={
message.split(/\r|\n/).length > 3
? message.split(/\r|\n/).length
: 3
}
className="flex-1 border-none bg-transparent focus:outline-none"
value={message}
onInput={(e) => handleInput(e)}
onKeyDown={(e) => {
if (e.key === "Enter") {
e.preventDefault();
void handleSubmit();
}
}}
/>
<div className="flex h-full flex-col items-end justify-between">
<button
className="rounded-md bg-blue-500 px-3 py-1.5 text-sm text-white disabled:cursor-not-allowed disabled:bg-blue-300"
disabled={message.length > maxLength || message.length < minLength}
>
Send
</button>
<span
className={`text-sm leading-none ${
message.length > maxLength ? "text-red-500" : "text-neutral-400"
}`}
>
{" "}
{message.length.toString() + "/" + maxLength.toString()}{" "}
</span>
</div>
</div>
</form>
);
};
ChatMessages
This component is a list that displays the chat messages and gives them a different look whether they are from the user or the assistant. It also displays a ... block for the assistant when the user is waiting for a response. To enable rich text formatting, we can add Markdown to the messages using the react-markdown module and the tailwindcss/typography plugin with the prose class. This enables the agent to send actual links, lists and other markdown elements.
const ChatMessages: FC<React.HTMLAttributes<HTMLUListElement>> = ({
...props
}) => {
const messages = useContext(MessagesContext);
const scrollTargetRef = useRef<HTMLDivElement>(null);
const processing = useContext(ProcessingContext);
useEffect(() => {
if (scrollTargetRef.current) {
scrollTargetRef.current.scrollIntoView({
behavior: "smooth",
block: "end",
inline: "nearest",
});
}
});
return (
<ul className={`flex flex-col gap-y-3 ${props.className ?? ""}`}>
{messages.map((message, index) => (
<li
key={index}
className={`rounded-2xl px-6 py-4 text-sm shadow-sm md:text-base ${
message.role === "assistant"
? "mr-32 bg-neutral-200 text-neutral-900"
: "ml-32 bg-blue-500 text-white"
}`}
>
<Markdown>{message.content}</Markdown>
</li>
))}
{ processing && (
<li
className={`rounded-2xl px-6 py-4 text-sm shadow-sm md:text-base mr-32 bg-neutral-200 text-neutral-900 w-fit`}
>
<div className="animate-pulse flex gap-x-1">
<div className="h-2 w-2 bg-gray-500 rounded-full"></div>
<div className="h-2 w-2 bg-gray-400 rounded-full"></div>
<div className="h-2 w-2 bg-gray-300 rounded-full"></div>
</div>
</li>
)}
<div ref={scrollTargetRef}></div>
</ul>
);
};
Integrating Chat-GPT
Now that we have a nicely working chat app, we want to integrate the Chat Completion API. We'll start by creating a new public procedure called welcome. Here, we will use a system prompt to do the initial configuration for the agent, which just tells the agent what is his task and provide it with some background info about myself:
const configPrompt =
"I want you to act as a virtual version of me. " +
"Your name is Virtual Vince. Here you have a little information about me: [ " +
"I am a fullstack developer working with Laravel, Vue, React, Astro and NextJS. " +
"I love coding and cycling. ] " +
"You are polite and reply in Markdown format. Any link that you use should be a Markdown URL. " +
"Your first message should be an introduciton of yourself (like: 'Hey I am virtual Vince!'), continue a normal conversation asking for the name of the user." +
"You should keep your messages short and to the point. After about 500 tokens, you should end the conversation and supply the user with a link to 'https://vincentdorian.me/contact'."
This prompt and the two parameters we defined in the beginning are the only things that we will tweak. This prompt is now used to define a new async query. This query will be called in the NextPage Component.
// ~/server/api/routers/chat.ts -> chatRouter()
welcome: publicProcedure.input(z.object({})).query(async () => {
const response = await openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: configPrompt,
},
],
max_tokens: 100,
});
return {
message: response.data.choices[0]?.message,
total_tokens: response.data.usage?.total_tokens,
};
}),
We are also returning the total_tokens used for the chat. The output of the welcome query procedure is used as the first message displayed to the user.
// ~/pages/index.tsx -> Home(): NextPage
const welcome = api.chat.welcome.useQuery({},{
refetchOnWindowFocus: false,
});
useEffect(() => {
if (welcome.data) {
setMessages([welcome.data]);
setProcessing(false);
}
}, [welcome.data]);
The agent will now welcome the user, as seen here:
Subsequent chat messages
For all subsequent messages, we will always have to send the entire chat history. In our procedure we will take the array of messages sent by the user and append the hidden system prompt in front.
send: publicProcedure
.input(
z.object({
messages: z.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
})
),
})
)
.mutation(async (req) => {
const response = await openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: configPrompt,
},
...req.input.messages,
],
});
return {
message: response.data.choices[0]?.message,
total_tokens: response.data.usage?.total_tokens,
};
}),
This procedure is called from the PushChatMessageForm Component. For this we are just redefining the sendChatMessage() function that we defined earlier and change it to:
const sendChatMessage = api.chat.example.useMutation();
To keep track of the total token count, we also define additional contexts for, tokenCount and setTokenCount(), which we are adding to the Home() component.
<ProcessingContext.Provider value={processing}>
<SetProcessingContext.Provider value={setProcessing}>
<MessagesContext.Provider value={messages}>
<SetMessagesContext.Provider value={setMessages}>
<SetTokenCountContext.Provider value={setTokenCount}>
<TokenCountContext.Provider value={tokenCount}>
<ChatMessages className="mt-5 flex-1 overflow-y-auto" />
{tokenCount < 500 && (
<PushChatMessageForm className="flex-none" />
)}
</TokenCountContext.Provider>
</SetTokenCountContext.Provider>
</SetMessagesContext.Provider>
</MessagesContext.Provider>
</SetProcessingContext.Provider>
</ProcessingContext.Provider>
This context is the used to hide the messages form upon reaching 500 tokens.
The callback of the async mutation in the handleSubmit() function we defined earlier now becomes:
await sendChatMessage
.mutateAsync({ messages: messagesProxy })
.then((response) => {
if (response) {
messagesProxy.push(response.message as MessageType);
setTokenCount(response.total_tokens as number);
setMessages([...messagesProxy]);
setIsProcessing(false);
}
});
Now, we can start having a conversation with our own personal virtual agent.
Conclusion
I think it's already quite nice using this PoC and the "conversation" you are having with the agent feels kinda natural. Of course there are some hiccups here and there in the conversation, but with more detailed prompting you can probably fix them more or less.
For the future I would be really interested, how to use this with GPT plugins, that can access data on my portfolio page or even GitHub when requested. (For now you would have to give it in your initial prompt or feeding another system prompt when triggered by some keyword maybe.)
If you would like to clone this repo just go for it. Here's the link.
❤️ If you enjoyed reading I would be happy if you leave a like or comment. ❤️
If you have any questions, feel free to ask. I am always happy to chat.
Top comments (0)