rsync and SSM agent port forwarding for file transfers without private key

THIS IS AN INTERESTING HACKING PROJECT DON'T DO THIS IN A HIGH COMPLIANCE ENVIRONMENT

Out of curiosity I was trying to see about copying files using the SSM agent. What I found unfortunately was that you kind of need an authorized_keys available pem/private key file for the easy SCP route to work. At that point I felt like well that kind of defeats the purpose of the whole don't keep SSH keys laying around on your instance by using SSM. Then I came across an interesting gist talking about how to do file transfer using netcat. Pondering for a moment I realized rsync could possibly work. Note that all commands here assume an Amazon Linux 2 system. Installation of the SSM agent may be required for other systems.

Permissions

For all this magic to work permissions will needed to be added to the source instance transferring the file:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ssm:StartSession",
            "Resource": [
                "arn:aws:ec2:*:[account_number]:instance/*",
                "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSession"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "ssm:TerminateSession",
            "Resource": "arn:aws:ssm:*:[account_number]:session/*"
        }
    ]
}

The destination will also need the arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore managed policy (or similar policy that gives the proper permissions) as part of instance profile (I generally recommend this for easy EC2 instance management).

Port Forwarding

It turns out that SSM agent has a plugin for port forwarding which you can find the code for in around this area. For this to work we first need to install the session manager plugin to start the actual session via the AWS CLI on the source machine:

$ sudo yum install -y https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm

Next on the destination machine is the installation of the rsync daemon. This allows for rsync traffic to work over a TCP port instead of the usual SSH method (which again, would require potential private key access):

$ sudo yum install rsync-daemon

From there I worked together a simple configuration which has an ec2-upload directory that owns files as the ec2-user to an uploads path in the home directory. read only = false is needed to allow the uploads to work.

# /etc/rsyncd: configuration file for rsync daemon mode
[ec2-upload]
  path = /home/ec2-user/uploads
  comment = Instance Upload
  uid = ec2-user
  read only = false

Due to session manger console access being done through ssm-user, a bit of sudo was required to create the uploads folder and own it for the ec2-user:

$ sudo mkdir /home/ec2-user/uploads
$ sudo chown ec2-user /home/ec2-user/uploads

Finally it's time to start the rsyncd service:

$ sudo systemctl start rsyncd

Now on the source machine:

$ sudo aws ssm start-session --target [instance-id-here] --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["873"],"localPortNumber":["873"]}' &

[instance-id-here] should be replaced in entirety with the instance ID of the destination instance. Due to 873 being a restricted port root access is required via sudo for port binding to work. The & at the end sets it as a background process so other commands work (otherwise it blocks terminal interaction). You'll want to know the PID of the resulting process so you can shut it down later via kill -15 or kill -9. So now that it's hopefully setup, simply run rsync against a file with rsync://localhost/[module-here] which is ec2-upload in this case. For an example here we'll go ahead and transfer the Google home page:

$ wget google.com
$ rsync index.html rsync://localhost/ec2-upload/

Now on the destination system:

# ls -lah /home/ec2-user/uploads/
total 20K
drwxr-xr-x. 2 ec2-user root      24 Jun 10 04:02 .
drwx------. 4 ec2-user ec2-user  89 Jun 10 03:31 ..
-rw-r--r--. 1 ec2-user nobody   19K Jun 10 03:46 index.html

Success! No PEM file required!

Conclusion

Note that if you open up 873 on your security group well... anyone with access to the port will now be able to upload files with no remorse. Doing this through the SSM agent does some tunneling through the SSM service to make opening this port on the security group unnecessary (so some slight security). Though this also means you shouldn't expect insane transfer speeds due to increased network hops. IAM also enables restrictions on StartSession to help with too much open access. All and all there are other solutions such as EFS or an S3 proxy that might be a better option, but this could potentially be helpful for more niche cases.