Uploading Files to S3 Using a Signed URL

Amazon S3 is a powerful storage service provided by AWS. Sometimes, you might want to allow users to upload files directly to an S3 bucket without giving them direct write access. This can be achieved using a pre-signed URL.

A pre-signed URL allows you to grant temporary access to an S3 object, enabling users to upload or download files securely. In this post, I'll guide you through the process of generating a signed URL using an AWS Lambda function and testing it using Postman.

Step 1: Setting Up the Lambda Function to Generate a Signed URL

First, let's create a Lambda function that generates a signed URL for uploading files to S3.

1. Create a Lambda Function:

   • Navigate to the AWS Management Console.
   • Go to the Lambda service and create a new function.
   • Choose the "Author from scratch" option and configure your function details.
   • Set the runtime to Node.js.

2. Add the Necessary IAM Permissions:

   • Your Lambda function will need permissions to interact with S3. Attach the following policy to your Lambda's execution role:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
       }
     ]
   }

1. Write the Lambda Function Code:

Below is the code to generate a pre-signed URL:

   const AWS = require('aws-sdk');
   const s3 = new AWS.S3();

   exports.handler = async (event) => {
       const bucketName = '<YOUR_BUCKET_NAME>';
       const objectKey = event.queryStringParameters.fileName; // File name sent as a query parameter
       const expiresIn = 60 * 5; // URL expiry time in seconds (5 minutes)

       const params = {
           Bucket: bucketName,
           Key: objectKey,
           Expires: expiresIn,
           ContentType: 'application/octet-stream', // Change this if you expect different file types
       };

       const signedUrl = s3.getSignedUrl('putObject', params);

       const response = {
           statusCode: 200,
           body: JSON.stringify({
               uploadUrl: signedUrl,
           }),
       };

       return response;
   };

1. Deploy the Lambda Function:
   • Deploy the Lambda function and note down the API endpoint if you're using an API Gateway trigger.

Step 2: Testing the Signed URL Using Postman

Now that you have the Lambda function ready, you can test the signed URL generation using Postman.

1. Generate the Signed URL:
   • Open Postman and create a new GET request.
   • Use the API endpoint provided by your Lambda function.
   • Add the query parameter fileName with the desired file name.

Example URL: https://<YOUR_API_GATEWAY_URL>?fileName=test-file.txt

• Send the request. You should receive a JSON response containing the signed URL.

   {
"uploadUrl": "https://.s3.amazonaws.com/test-file.txt?AWSAccessKeyId=..."
   }

1. Upload the File Using the Signed URL:
   • Create a new PUT request in Postman.
   • Paste the signed URL you received in the previous step as the request URL.
   • In the "Body" tab, select "binary" and choose the file you want to upload.
   • Send the request.

If everything is set up correctly, the file should be uploaded to your S3 bucket.

1. Verify the Upload:
   • Check your S3 bucket to ensure the file was uploaded successfully.

Conclusion

Using S3 signed URLs is a secure and efficient way to allow users to upload files directly to your S3 bucket without exposing your AWS credentials. This method can be especially useful in web and mobile applications where you need to handle file uploads securely.

With just a few steps, you can set up a Lambda function to generate signed URLs and test them using Postman. This approach can be extended to various use cases, such as file downloads or secure access to S3 objects.