Originally published at https://pcarion.com/article/ssh-tunnel

There are quite a few instances when you want a public URL to hit your development machine:

• you want to expose a webserver running on your local machine to the internet so that a colleague or a customer may have a look at it.
• you are using a service, like twillio, which allows you to setup webhooks URL: the service will call those URLs to notify you when something happens.
• you want to test an OAuth integration, with Facebook or Twitter, and you want to provide public https callback URLs.

An alternative option is to install your code on a public server and have those requests hit this server. During development, it is much more convenient to have those requests hit your local machine so that you can debug and see the logs in real time.

You can also use a service like ngrok: this is a tool, developed by Alan Shreve, very easy to use and perfect for webhooks.

We'll describe here how you can setup your own solution on ec2.

Such a home made solution has several advantages:

• you can have persistent URLs - ngrok gives a different domain each time you use it unless you become a paying customer.
• you can automate the process (see the end of this post) meaning that you can quickly tear down your EC2 instance and restart it quickly when you need the URLs again.

At $0.0069 per hour for a nano instance, the service will cost you $1 per month if you use it for about 144 hours. This is more than enough for a "night and weekend" project.

Description of the solution

The solution looks like this:

A machine (C) on the public internet will be able to access multiple URLs, on the same public URL, to access one or more services on your local machine (A).

For instance:

• the URL https://api.mydomain.com will access http://localhost:3000 on your dev machine
• the URL https://api.mydomain.com will access http://localhost:3000 on your dev machine

Warning: a custom made solution is definitely more complex than using ngrok, so you should think twice before taking that route. You need to be comfortable with bash scripting and DNS setup to implement that solution.

This solution requires:

• a domain name: this domain, and subdomains, will be used to configure public URL(s) to access your local machine
• a public DNS: as you want to have public URL to access your server, a DNS is required.
• a machine on the public internet: this machine will act as a bridge between the public internet and your local machine. You can use Digital ocean or AWS EC2, with full root access
• a sshd daemon running on that server: SSH is the swiss army knife for that kind of setup, and you need to be able to fully configure the SSH server, especially to setup a reverse tunnel
• a SSL certificate. To protect your connection, you need to setup SSL certificates so that the public URL can be available only through https. We will use Let's encrypt for that.

The Gateway server (B) is the machine on the public internet and you need to configure a set of services on that server:

• configure the SSD service to allow TCP port forwarding
• install a reverse proxy - HAProxy - to forward different subdomain to different ssh tunnels
• configure let's encrypt to allow SSL traffic over https

In the rest of this document, we will use EC2 to install this gateway server.

EC2 instance configuration

The setup is not intended for a production service and the traffic on that machine should be very low. For that reason, you can use a very small instance to run that server : I am using a t2.nano instance with an ubuntu OS.

security group

The only setup to pay attention to is the network/security group definition.

By default, the setup would allow only the port 22 for your ssh access:

You need more inbound ports for your server:

• 22 for your SSH access
• 80 for your incoming http access
• 443 for your incoming https access
• different ssh tunnels ports, like 8080, 8085, 8090, etc... The number of ports depend on the number of services you want to expose through the ssh tunnel

You can create a security group with those inbound ports:

private key file

To use the SSH connection, you need to create a key pair . Download the associated .pem file, copy it in a safe place and do a chmod 400 on that file to avoid the error:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'ssh_tunnel.pem' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "ssh_tunnel.pem": bad permissions

server configuration

At the end of this document, we will show how to automate entirely the setup process.

The following description is useful if you want to have a better understanding of what the automated script will be doing.

Using your pem file, you can ssh to the server and proceed with the configuration.

First, make sure that all the packages are up to date

sudo apt-get update

setup the sshd server for the reverse ssh tunnel

You need to make sure that the sshd server is running and has the proper configuration.

This command will check the sshd configuration:

$ sshd -T | grep -E 'gatewayports|allowtcpforwarding'
gatewayports no
allowtcpforwarding yes

For the ssh tunnels to work, you need both parameters to be set to yes.

Open the file /etc/ssh/sshd_config and add or set those 2 lines:

AllowTcpForwarding yes
GatewayPorts yes

Then, you need to restart the ssh daemon for those parameters to be taken into account:

service sshd restart

And then you can check that all is in order:

$ sshd -T | grep -E 'gatewayports|allowtcpforwarding'
gatewayports yes
allowtcpforwarding yes

T*est of the reverse ssh tunnel setup*

Before moving forward with the server configuration, you can already check that the ssh setup is working properly.

In order to test that a local service can be reached from the public internet, you need to start some sort of local server on your development machine.

A simple HTTP server will do the trick: from a directory containing no sensitive data, you can start a simple python server:

python -m SimpleHTTPServer 8000

You can check from your local browser from http:127.0.0.1:8000 that the server is working, and ... not exposing sensitive data.

Now is the time to invoke the SSH port forwarding voodoo incantation, from your local machine:

ssh -i "tunnel.pem" ubuntu@ec2-a-b-c-d.us-west-1.compute.amazonaws.com -N -R 8080:localhost:8000

This SSH command is much simpler than it looks like:

• ubuntu@ec2-a-b-c-d.us-west-1.compute.amazonaws.com this is the public DNS Address of your server and you can find the value in your aws/ec2 console
• -N: by default, ssh will create a shell on the remote machine. We don't need that here
• -R: with this option you are asking ssh to answer on the remote side (your gateway)
• 8080:localhost:8000: any connection on port 8080 on the gateway will be tunneled to the the port 8000 on your local machine (where the webserver we started previously is listening on).

If all work as expected, you can open your browser at: http://ec2-a-b-c-d.us-west-1.compute.amazonaws.com:8080

and you should see the file served by your local server!

Your reverse ssh tunnel is working.

More information about reverse ssh tunneling can be found here.

DNS setup

The next step is that you want a nicer URL to access your service right?

You need to configure the DNS for your domain yourdomain.com and create a A record for api.yourdomain.com , with a TTL of 600, with a value of a.b.c.d which is the IP address of your gateway.

I may take time for the DNS configuration to propagate, but once it is done, you can then access your local web server through the URL: http://api.yourdomain.com:8080.

Better, but can still be improved: you may want to setup multiple subdomains which would allow you to host multiple local services, or have multiple machines using this tunnel (each using a specific subdomain).

To do that, you need a reverse proxy on your gateway.

HAProxy setup

Let's install HAProxy:

$ install haproxy
$ sudo apt-get install -y haproxy
$ haproxy -v
HA-Proxy version 1.8.8-1ubuntu0.9 2019/12/02
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>

You then need to configure its main configuration file: /etc/haproxy/haproxy.cfg

The setup is pretty basic: base on the domain being accessed (like api.yourdomain.com), you serve data from a local server (127.0.0.1:8080), which, through the ssh reverse tunnel will connect back to your local machine

Example of subdomain setting in haproxy.cfg

frontend account
    bind *:80
    mode http
    acl host_api hdr(host) -i api.yourdomain.com
    use_backend account if host_api

backend api
    mode http
    server node1 127.0.0.1:8080

With that configuration, you can then verify that the URL http://api.yourdomain.com serves also your local HTTP server data.

We won't go any further yet as the configuration of HAProxy is very dependent on the next step.

SSL - letsencrypt

Nowadays, https is almost always required to access a server.

You can easily get a free SSL certificate using Let's encrypt.

The installation of the required tool is easy:

sudo add-apt-repository -y ppa:certbot/certbot
sudo apt-get update
sudo apt-get install -y certbot

There is a rate limiting with let's encrypt services so... you are limited in the number of trial and errors to configure your certificates.

There are 2 conditions to ensure before starting the let's encrypt setup:

• you need to make sure that the DNS setup is done and propagated for http://yourdomain.com domain name - the let's encrypt server relies on that to ensure that you are the rightful owner of the domain
• you must stop haproxy or any service using the port 80 as this port will be used by let's encrypt, using their own cerbot server to retrieve the certificate

Once those conditions are met, you can start the certificate retrieval process:

sudo certbot certonly --standalone -d yourdomain.com -d api.yourdomain.com -d www.yourdomain.com --non-interactive --agree-tos --email you@email.com

You need to change the command line above with:

• the list of subdomains you want to have a certificate for in the -d arguments
• your email address as the -email argument

If all goes well, you should see this kind of output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for yourdomain.com
http-01 challenge for api.yourdomain.com
http-01 challenge for www.yourdomain.com
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/yourdomain.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/yourdomain.com/privkey.pem
   Your cert will expire on 2020-04-25. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

import certificates to HAProxy

The certificates generated above will be used by HAProxy and they need some massaging to be usable there:

sudo mkdir /etc/haproxy/certs
DOMAIN='yourdomain.com' sudo -E bash -c 'cat /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/letsencrypt/live/$DOMAIN/privkey.pem > /etc/haproxy/certs/$DOMAIN.pem'

The last step is to configure HAProxy, with all your domain and the https setup.

We will cover that in the next section

Automation

As you can see, there are a lot of sets involved to set up a server but there is a way to automate the entire process.

ec2 template

You can speed up the creation of the ec2 instance by using a template.

You can create a template from the instance you just created and make sure that:

• you are using the right security group with all the inbound ports you want to use
• you use the key-pair previously created

With that template, the creation of the instance becomes very easy with basically one click.

automation steps

• create an instance using the ec2 instance template
• once the instance is booted, connect to it using your pem file to make sure your ssh setup is correct
• retrieve the IP address of this new instance and update your DNS setup. Confirm with a DNS lookup that the IP address has been updated

• you can confirm that the DNS is correct ifyou can connect to your instance with your domain name:

  ssh -i "tunnel.pem" ubuntu@yourdomain.com

• update the following script (see below) and run it with:

  ssh -i "tunnel.pem" ubuntu@yourdomain.com 'bash -s' < setup_tunnel_host.sh

automation script - setup_tunnel_host.sh

Here is the script that you can run to automate all the steps describe above...

You need to set a couple of variables at the top of that script

• this script will setup a SSH tunnel for you domain: BASE_DOMAIN_NAME
• your email address for let's encrypt: LETE_EMAIL
• the script setup 2 subdomains with 2 ports (it's easy to update the script to use more ports/subdomains):
  • SUBDOMAIN1/ REMOTE_REDIRECT_PORT1
  • SUBDOMAIN2/ REMOTE_REDIRECT_PORT2

On your local machine, you can then initiates both tunnels with:

ssh -i "tunnel.pem" [ubuntu@yourdomain.com](mailto:ubuntu@chirloute.com) -N -R 8080:localhost:8000 
ssh -i "tunnel.pem" [ubuntu@yourdomain.com](mailto:ubuntu@chirloute.com) -N -R 8090:localhost:3000 

You must update the local port to match the server running on your local machine.

Enjoy!