Accessing S3 Buckets from CloudShell

After one year since AWS CloudShell was released, it worth to comment about a connection issue between this technology and S3 Buckets.

Issue

We have a S3 bucket (in this case, named mytestbucket0123) that we need to access through AWS CloudShell.
But when trying to list all objects on a bucket from CloudShell, executing aws s3 ls s3://mytestbucket0123 we’re getting the following error

"An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied"

Verification steps

As a starting point of this situation, we’ll need to analyze the scene and do the following verification questions:

1. Has the user got the required permissions to list objects of our S3 Bucket? To answer this we have several ways: first check on IAM that the user has assigned those permissions.
   
   The user has attached the AmazonS3ReadOnlyAccess Policy, so it has ListObjects required permission. So let’s verify that the user can already list the s3 bucket objects (from the AWS console for example). Listing objects on the bucket seems to work fine from the AWS Console with the same user (from server machine) as per screenshot below.
   
   So, it doesn’t seem to be user IAM permissions at least. Which leads us to the following step.

2. Has the S3 Bucket got any Bucket policy enabled? We need to search, on the AWS console, the S3 bucket and look the “Bucket policy” section in the “Permissions” tab.
   In this case, the S3 Bucket has the following Bucket Policy

   
   
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "Statement1",
               "Effect": "Deny",
               "Principal": "*",
               "Action": "s3:*",
               "Resource": "arn:aws:s3:::mytestbucket0123",
               "Condition": {
                   "NotIpAddress": {
                       "aws:SourceIp": "<Public IP Address>"
                   },
                   "StringEqualsIfExists": {
                       "aws:SourceVpc": "<VPC-Id>"
                   }
               }
           }
       ]
   }
   

    This bucket policy denies access to all users (no matter they have the required IAM permissions), except they access from a specific IP Address or connect from our VPC (which, in this case is the AWS Account’s default VPC). That means the CloudShell is not accessing to the S3 Bucket from the VPC… So let’s ask the next question.

3. **Does CloudShell terminal connect to the S3 Bucket through a public IP Address?** To answer this, we need to execute the following command into the CloudShell terminal: `curl ifconfig.me`![The result of executing  curl ifconfig.me is 18.224.171.123, which means Public IP Address](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ycs8gjesegoplzoizvlg.png) It seems that CloudShell is trying to access to the S3 Bucket from a non whitelisted Public Address (which belongs to AWS Reserved IP Addresses).

### Cause explanation
After the verification steps, we can realize this is a CloudShell’s current limitation: it cannot connect to AWS Resources using VPC Endpoints or with VPC restricted access. Besides, the IP Address allocated per each CloudShell terminal may change over the time. 

### Solution 
To solve this issue we need to whitelist the Cloudshell’s user agent. Although this may not be a 100% secure resolution, it can be a temporary fix until AWS CloudShell can improve (or no CloudShell access is required). At the time of this writing, the user agent is "[aws-cli/2.4.5 Python/3.8.8 Linux/4.14.248-189.473.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off command/s3.ls]”. Then edit the S3 Bucket Policy to the following (excluding also the CloudShell user agent). 

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Deny",
"Principal": "",
"Action": "s3:",
"Resource": "arn:aws:s3:::mytestbucket0123",
"Condition": {
"StringEquals": {
"aws:UserAgent": "[aws-cli/2.4.5 Python/3.8.8 Linux/4.14.248-189.473.amzn2.x86_64 exec-env/CloudShell exe/x86_64.amzn.2 prompt/off command/s3.ls]"
},
"NotIpAddress": {
"aws:SourceIp": ""
},
"StringEqualsIfExists": {
"aws:SourceVpc": ""
}
}
}
]
}

Finally, when trying to perform on the terminal the command  `aws s3 ls s3://mytestbucket0123`  finally works!

![aws s3 ls working correctly from CloudShell after S3 Bucket Policy update](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1eylou7y1pq6kr50ckd7.png)