Creating a Simple Chatbot with Context on Amazon Bedrock

As I was just about to hit snooze and get ready for the long weekend, I decided to check my LinkedIn feed once more. Lo and behold, the first post was "Amazon Bedrock is now released for General Access". It was quite the shock announcement, but as all Community Builders do, I geared up for a morning of experimentation. It was finally time.

In this post, I will be walking you through creating your first Chatbot based on Bedrock with their amazon.titan-tg1-large model. The amazon.titan-tg1-large is one of the models from Amazon Titan, a family of Foundation Models pretrained by AWS for various use cases. It is suitable for text generation. We will be doing this development in Jupyter Notebook.

Step 1: Installing dependencies

The %pip command is a Jupyter magic command that allows for pip package management directly within the notebook.

--no-build-isolation: This flag is used to tell pip not to isolate the build in a separate environment, which can help when dealing with local dependencies.

--upgrade (previously --force-reinstall): This flag ensures that if the packages are already installed, they'll be re-installed with the newer versions provided. The term "upgrade" is more intuitive than "force-reinstall" but they serve similar purposes in ensuring the latest provided version is installed.

../libs/awscli_package-*-py3-none-any.whl (and similar for the other two): This is the path to the .whl (Wheel) files, which are a type of built package format for Python. The * is a wildcard that matches any version, so it installs the latest version found in the directory.

%pip install --no-build-isolation --force-reinstall \
    ../dependencies/awscli-*-py3-none-any.whl \
    ../dependencies/boto3-*-py3-none-any.whl \
    ../dependencies/botocore-*-py3-none-any.whl

We then have to also install langchain. LangChain is a toolkit designed for building applications that leverage language models. It helps in making these applications context-sensitive by linking them to various context sources, and also aids in decision-making based on this context. LangChain provides modular elements for interacting with language models and pre-built setups for tackling higher-level tasks, making it easier to initiate new projects and adapt existing setups for more intricate applications.

%pip install --quiet langchain==0.0.304

Step 2: Importing Libraries
We now need to import the libraries necessary.

1. boto3: AWS SDK for Python, allows interaction with Amazon Web Services.
2. json: Library for working with JSON data.
3. ConversationBufferMemory: A component from LangChain, used for storing messages in a conversation. It can extract messages as a single string or a list of messages, making it useful for chat models.
4. PromptTemplate: Another component from LangChain for creating language model prompts.
5. Bedrock: A component from LangChain's low-level model suite (llms) for initializing the Bedrock object.
6. ConversationChain: A component from LangChain to orchestrate a conversation or a sequence of language model interactions.

import boto3
import json
from langchain.memory import ConversationBufferMemory
from langchain import PromptTemplate
from langchain.llms.bedrock import Bedrock
from langchain.chains import ConversationChain

Step 3: Initalizing the Bedrock client
You then need to initialize a client for AWS service 'bedrock' using boto3 library in Python. You need to replace YOUR_ACCESS_KEY and YOUR_SECRET_ACCESS_KEY (remove the < and > as well with your actual AWS credentials, and region_name specifies the AWS region (us-west-2 in this case) where the service is hosted. As of the writing of this, us-west-2 is the region where the Titan model is available generally, but more should be rolling out soon. This bedrock_runtime client can then be used to interact with the Bedrock service on AWS.

bedrock_runtime = boto3.client(
    service_name='bedrock-runtime', 
    aws_access_key_id='<YOUR_ACCESS_KEY>',
    aws_secret_access_key='<YOUR_SECRET_ACCESS_KEY>',
    region_name='us-west-2'
)

Step 4: Creating a Langchain instance to keep track of conversation history
We then have to create an instance of ConversationBufferMemory from the Langchain library to keep track of a conversation's history. This instance is stored in the variable memory. Following this, the add_user_message and add_ai_message methods are called on memory.chat_memory to add a user message and an AI message to the conversation, respectively. The user message instructs the AI to assume the role of a principal software engineer, while the AI message is an acknowledgement of this role. This setup is essential for scenarios where the AI needs to maintain a specific persona or context throughout a conversation, allowing for more contextually aware interactions.

memory = ConversationBufferMemory()
memory.chat_memory.add_user_message("You will act as a principal software engineer and answer software engineering questions.")
memory.chat_memory.add_ai_message("I am a principal software engineer and will answer your software engineering questions.")

We then initialize Bedrock with the model amazon.titan-tg1-large (can be adjusted according to your requirements).

You also will need to declare the parameters.

1. maxTokens: Specifies the maximum number of tokens the output can have.
2. temperature: Controls randomness in output generation, with 0 for deterministic output.
3. topP: Implements nucleus sampling, where only the most probable tokens are considered for generation, with 0.9 keeping 90% of the probability mass.

titan_llm = Bedrock(model_id="amazon.titan-tg1-large",client=bedrock_runtime)
titan_llm.model_kwargs = {"maxTokens": 4096, 'temperature': 1.0, 'topP':0.9}
conversation = ConversationChain(
     llm=titan_llm, verbose=True, memory=memory
)

Step 5: Running prompts against the initialized model
Afterwards, we can run conversation.predict to run prompts against the initialized Bedrock model and add to the conversation chain.

conversation.predict(input="What is the best programming language?")

The following is a sample response returned upon the given prompt:
There is no one-size-fits-all answer to the question of which is the best programming language. Different programming languages excel in different areas, and the best language for a particular task depends on the specific requirements of that task. However, some of the most commonly used and well-regarded programming languages include:\nJava: Java is a popular object-oriented programming language that is known for its portability, scalability, and ease of use.\nPython: Python is a high-level, interpreted programming language that is known for its simplicity, readability, and versatility.

We can continue to run conversation.predict and see that it adds to the history of the conversation chain.

conversation.verbose = False
str(conversation.predict(input="What programming languages do you use?"))

The following is a sample response when given the prompt: ' As an AI, I do not use programming languages as humans do. However, I am trained and proficient in various programming languages and technologies, including natural language processing, machine learning, and computer vision. These languages and technologies allow me to perform various tasks and solve problems in a way that is similar to how a human would.'

For the full code on the .ipynb file, you can access my repository at https://github.com/renaldig/bedrock-titan-chatbot.

FAQ:

Why does it say that Bedrock is not on Boto3?
Ans: You’ll have to update your boto3 to the latest version with the following:
pip install boto3 —-upgrade
​
Summing Up

To sum up, it's not hard to get started on Bedrock! With that, you've built your first Bedrock implementation which you can build on further for your own applications on. Build on and let's get Bedrocking!