Introduction
Amazon Simple Storage Service (Amazon S3) is a cloud object storage service that offers industry-leading scalability, data availability, security, and performance. You can use Amazon S3 to store and retrieve any amount of data from anywhere on the web.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. You can use API Gateway to create RESTful APIs and WebSocket APIs that enable real-time two-way communication applications.
Of course, there are multiple ways to upload file to AWS S3 bucket, however, in this section, we will discuss on how to create an API Gateway endpoint for uploading files to Amazon S3. We will also go through on how to test the API Gateway endpoint. I assume that you have some basic knowledge of AWS services and tools, such as IAM, S3, and API Gateway.
Tool(s)/Framework(s)
Cloud Provider — Amazon Web Service(AWS)
AWS Services — API Gateway, Simple Storage Service(S3), Identity & Access Management(IAM)
High-Level Design
Let me start with high level design & what we are trying to achieve here. First, we will create s3 bucket & then we will configure AWS API Gateway using which we should be able to upload any types of file to S3 bucket.
S3 File Upload Via API Gateway
Implementation
→ S3 Bucket Creation & Configuration
- Let’s navigate to AWS console & head towards S3.
- Click on “Create bucket” button
- Enter only minimal required details & keep all other attributes as it is without any change. Bucket name — enter valid bucket name, for Ex: “apigateways3bucket” AWS Region — choose appropriate AWS region in which you want to create bucket
- Click on “Create bucket” button which will result in bucket to be created.
- we can verify that bucket has been created from S3 dashboard as shown below.
S3 Bucket — After creation on S3 Dashboard
Now that we have created S3 bucket, Let’s move on to create & configure API Gateway.
→ API Gateway Creation/Configuration
- Navigate to API Gateway from AWS Console
- Click on “Create API” button
- Choose “Rest API” & click on “Build”
- Now, let’s enter the required details & click on “Create API” button.
- “Choose the protocol” — Choose “REST”
- “Create new API” — Choose “New API”
- Enter “API name” as “s3upload” & choose “Endpoint Type” as “Regional” as shown below under “Settings” section.
- We can see that API Gateway resource has been created successfully
- Also, we can navigate back to the API Gateway dashboard & can see that new API Gateway with name “S3 Upload” is created.
- Now, we have to configure resources with request path where
- Under “Resources” panel of API page, select /.
- Choose “Create Resource” from the “Actions” menu.
- Now, we will create two resources where each resources signifies the S3 bucket & the object.
- Create a resource under “/” (root) with “Resource Name” as “folder” and “Resource Path” as “{folder}” & click on “Create Resource”
- Create another resource under “/{folder}” (root) with “Resource Name” as “object” and “Resource Path” as “{object}” & click on “Create Resource”
- Once these resources are created which would look something like this.
- Once resources are created, now let’s create HTTP methods which will help us to upload & download the files to/from S3.
- Select “/{object}” from “Resources” & choose “Create Method”
API Integration — Create Metod
- Now, from Actions dropdown, choose ‘ Put ’ method & provide required configuration as given & shown below.
- Integration Type — AWS Service
- AWS Region — Choose the appropriate region
- Execution Rolle — Choose execution role created earlier
- Under “URL Path Parameters” section, add below parameters.
- Name — bucket, Mapped From — method.request.path.folder
- Name — key, Mapped From — method.request.path.object
- Now, from Actions dropdown, choose ‘ Get ’ method & provide required configuration as given & shown below.
- Integration Type — AWS Service
- AWS Region — Choose the appropriate region
- Execution Rolle — Choose execution role created earlier
- Under “URL Path Parameters” section, add below parameters.
- Name — bucket, Mapped From — method.request.path.folder
- Name — key, Mapped From — method.request.path.object
API Integration — Get Configuration
- Let’s setup binary media type to allow file upload/download via HTTP request. Click on “Add Binary Media Type” & enter “*/*”
API Integration — Binary Media Type
Testing
Now, you might be thinking so much of configuration but believe me, you will really enjoy & may explore other functionalities on the way which may excite you more & more.
Huuhhh, enough of configuration now, it’s time for testing :)
Let’s navigate to the “Put” method which would looks something like shown below. Let’s click on “Test” link.
Enter the required input parameters
{folder} — apigateways3bucket (which we have created ealier)
{object} — testfile (object name by which object would be created under S3 bucket)
Request Body — enter the saple text as file content here
API Integration — Test Inputs Parameters
Once all parameters are set, we are all set to click on “Test” button
Now, navigate to the AWS S3 dashboard & check the bucket. it will have the file uploaded in previous step.
AWS S3 — verification after upload
Hurrrrayyy, finally we were able to upload file to S3 using AWS API Gateway.
Conclusion
In this section, we have learned how to create an API Gateway endpoint for uploading files to Amazon S3. We have also learned how to test the endpoint. By using this approach, we can leverage the benefits of both Amazon S3 and API Gateway, such as scalability, availability, security, performance, and ease of development. We can also avoid the limitations of uploading files directly to API Gateway, such as the payload size and encoding restrictions. This way, we can create a robust and efficient solution for uploading files to the cloud.
I hope you have enjoyed this tutorial and found it useful. If you have any questions or feedback, please feel free to connect.
If this post was helpful, please clap for few times or follow to show your support which keeps me constantly motivated to share my learnings.
Learning, sharing & growing together.
Top comments (0)