For those about to Bedrock…

Gen AI and LLM security is a big topic, and I have just started experimenting with AWS Bedrock and building simple projects. As I learn, I’ve been trying to think about security principles at each step. OWASP, as usual, has great material on the security challenges facing companies who implement Generative AI. A recently released document breaks down the issues in a way that helps things make sense to a new AI practitioner — Solutions Landscape — OWASP Top 10 for LLM & Generative AI Security.

This resource separates LLM Applications into four different categories:

-Static Prompt Augmentation Applications

-Agentic Applications

-LLM Plug-ins and Extensions

-Complex Applications

Some of the security challenges and remediations for each application type are explained. I absolutely am not an expert on much of anything, and most certainly not AI and LLM’s. That being said, I wanted to understand and organize my thoughts as I build, so I am sharing my experiences building a Bedrock project, some of the security considerations, and some of the potential mitigations.

The project I am going to document in this article falls into the Static Prompt Augmentation category. This is the natural language prompt with generated output that most of us are very familiar with. Some security considerations for this type of build are prompt injection, data leakage, and misuse. I did several versions of this project both in the console and with CDK.

The project description: this should be very similar to the “Explain like I’m five” scenario. The user provides text and the age they would want a summary of that text geared toward.

Basic steps-using the console:

Step 1. Go to the Bedrock page, and take a look at the models available. Some models are available in certain regions only and you will need to request them before you use them:

Security notes here:

 — The models are run in your own environment, so there should not be any data leakage outside of your environment. Your data also is not used for training models. This gives us the opportunity to use models without worrying about our valuable data being sent to our competitors or countries we are not allowed to interact with.

 — Models have distinctly different characteristics, for example, one model will return something like “hacking advice”, but another will tell you that it cannot answer that.

 — Guardrails, in the same vein as the last point, let you customize some of the characteristics you want to control. For example, we can have denied topics, content filters, sensitive information filters, contextual grounding, and more.

 — Model cards and Risk cards are available for many of the models. The Model card gives us information about the model, including intended uses, limitations, training datasets and more. Risk cards address possible threat vectors and mitigations.

Step 2. Request access

I requested and received access to Titan Text G1 — Express. It is a very fast process and won’t slow your developers down.

Security notes here:

--We need to have IAM permissions to enable foundation models for ourselves and our users. So, no misguided users can enable something super expensive or otherwise not allowed. I would never be a misguided user myself, of course.

Step 3. Build your Lambda- in this case, my Lambda function will call the Bedrock model, give it a prompt, some input text and an age, then should receive an age appropriate summary. I’ll include my Lambda code, but warning- I am not a developer, merely a security person.

import boto3
import json

AWS_REGION_BEDROCK = "us-east-1"

client = boto3.client(service_name="bedrock-runtime", region_name=AWS_REGION_BEDROCK)

def handler(event, context):
    body = json.loads(event["body"])
    text = body.get("text")
    age = body.get("age")
    if text and age:
        titan_config = get_titan_config(text, age)
        response = client.invoke_model(
            body=titan_config, 
            modelId="amazon.titan-text-express-v1", 
            accept="application/json", 
            contentType="application/json"
        )
        response_body = json.loads(response.get("body").read())
        result = response_body.get("results")[0]
        return {
            "statusCode": 200,
            "body": json.dumps({"summary": result.get("outputText")}),        
        }
    return {
        "statusCode": 400,
        "body": json.dumps({"error": "text and age required!"}),  
    }

def get_titan_config(text: str, age: int):

    prompt = f"""Text: {text} \n
        You are a subject matter expert. From the text above, summarize the story for a person of this {age} age.\n    
    """

    return json.dumps(
    {
        "inputText": prompt,
        "textGenerationConfig": {
            "maxTokenCount": 4096,
            "stopSequences": [],
            "temperature": 0,
            "topP": 1,
        },
    }
)

Once your Lambda code is entered, you will want to deploy it and possibly bump up the timeout period to 30 seconds or so.

Finally, you will want to give this function the permissions to invoke Bedrock. Click on the role name shown under Configuration>Execution role. You will end up in IAM>Roles. Click on the Policy name in Permissions policies:

Edit this policy to add the permissions to invoke Bedrock by adding in

{
"Effect": "Allow",
"Action": "bedrock:InvokeModel",
"Resource": "*"
},

Save it and hit Next, then Save Changes. You can go back to your newly created Lambda now.

Security Notes here:

--IAM permissions and roles are needed here for everything, so all good.

--I could use AWS Inspector for Lambda scanning in case I wrote some problematic code or had some insecure dependencies.

Step 4. Now I am going to add in some API Gateway action.

You will need to add an API, resource, method, and configure the Lambda integration.

After you are done setting up, you will get an invoke URL. You can test a POST in the tool of your choice.

Security Notes here:

--API Gateway has so much functionality we can apply to help with security. We would definitely want throttling, authentication, and more.

Step 5. Monitoring

CloudWatch- if you’re like me, and it takes you more than a few tries to get something functional, you’ve already been looking at logs in CloudWatch. There is a preconfigured dashboard for Bedrock with Invocation count, latency, token usage and more.

Inspector — I briefly mentioned this above, but we can use Inspector to scan our Lambda for vulnerabilities in this project. In other projects, we might have resources like containers or EC2 that Inspector can scan.

GuardDuty — GuardDuty can grab events from CloudTrail and see a few different suspicious events both in Bedrock and SageMaker. For Bedrock, in particular, it could detect removal of guardrails or suspicious model invocation. GuardDuty looks like it also can see potentially malicious activity from Bedrock agents. Some mentioned activities are crypto mining and command and control traffic. I don’t have anything related to display in GuardDuty yet, but I may become evil at some point and test out something fun that GuardDuty can alert on.

Step 6. Govern- OWASP’s Security Solutions Reference Guide, mentioned above, has this as a stage in the LLMOps process. Some of the items we would look at here include compliance audit, data security checks, risk assessments, and additional considerations. Using a foundational model would give us more information regarding bias, training data, and other items.

Using CDK to build these projects provides the usual infrastructure as code security benefits, such as versioning, consistency, automated checks, and more. It’s also making it much easier to test other foundation models.

So let’s go back to the security concerns mentioned in the OWASP Security Solutions Reference Guide. For this type of a project, Static Prompt Augmentation, our particular vulnerabilities revolve around prompt injection, data leakage, and misuse. These topics are wide ranging of course, but some basics are:

Prompt injection- some possible mitigations include input validation, secure coding, patch management, use of guardrails.

Data leakage- in this project, I’m using a foundational model without any of my proprietary data. This limits some leakage potential. In the same vein, Bedrock uses encryption at rest and in transit.

Misuse - this is another huge arena. Some topics we would think about are AI focused adversarial testing, bias testing, incident response planning, auditing, and monitoring.

I will be exploring some other types of architecture next, most likely some sort of agentic scenario. It will be interesting to look at the differences in security concerns between all the different permutations of AI we can have. I’ll keep you posted, and thanks for reading!