Simplifying Remote Docker Container Connections in .NET Aspire with SSH.Net

Previously, I have shared my experience so far in my journey towards .NET Aspire using Photo Search use case by the help of multi-modal models. The key part for me was being able to use remote docker host which has worked well out of the box for most part. My focus so far is on the local development environments and therefore exploring different ways to achieve the original goal of running containers remotely on local network using SSH.

the posts so far exploring local development experience with .NET Aspire:

• Combining .NET Aspire, Python, Docker (Remote), and Machine Learning Models for Summarising Photos
• A Custom Reverse Geocoding Resource & Container Startup Dependencies in .Net Aspire

As a recap, it all started with the question whether or not the scenario illustrated in the following image could be supported out of the box or not:

In this brief post, we will start with some of the challenges of the approach used so far, and then introduce the SSH Port forwarding method and how it improves on the shortcomings of the initial approach.

Previous Approach: Overriding Connection Strings When Using a Remote Host

The approach so far has been as following:

• For the containers:
  • Ensure the container port binding uses 0.0.0.0 to listen on all interfaces so that it can be reached from the local network.
  • container.WithContainerRuntimeArgs("-p", $"0.0.0.0:{publicPort}:{publicPort}");
• For Aspire AppHost
  • Ensure DOCKER_HOST environment on the development machine is set.
    • The remote Docker host allows ssh connections from the development machine (e.g. ssh-copy-id is run to add our public key to the remote server)
  • If creating a custom resource, ensure the connection strings and exposed endpoints use the IP of the Docker Host instead of localhost.
  • For built in resources:
    • Either override the injected connection string environment variables.
    • Or use connection string redirection.

The challenges with this approach are:

• Having to modify connection strings at runtime when using remote docker host can get messy as we add more resources.
• Connection string redirection succeeded for PostgreSQL but not for RabbitMQ so had to fall back to environment variables override.
• When we update the connection strings, the exposed endpoints in the Aspire Dashboard can still point to localhost or removed as they are no longer valid.

For instance, this has worked for PostgreSQL connection string redirection to remote host:

var pgConnectionStringRedirection =
            new CustomPostgresConnectionStringRedirection(dbName, host, publicPort.ToString(), pgUsername, pgPassword);

postgresContainer.WithConnectionStringRedirection(pgConnectionStringRedirection);

SSH.Net to the rescue

As Docker CLI is using SSH to run the containers on the remote Docker Daemon, instead of making containers listen on all interfaces, we can instead do SSH port forwarding from our development machine into Docker Host machine.

Wouldn't it be cool to orchestrate this from Aspire AppHost project so that it happens for us when the development orchestration starts and stopped when we spin down our development environment.

The following diagram is a simplified illustration of this concept. The ports used for containers are forwarded using SSH from development machine to Docker Host.

Implementation

Given we have already had a successful SSH connection which allows managing containers on a remote host, forwarding container ports vis SSH not only made sense but also simplified the setup.

When the AppHost starts we need to forward the ports as illustrated in the code snipped below. Instead of calling WithContainerRuntimeArgs("-p", $"0.0.0.0:{publicPort}:{publicPort}"); on the resources, we need to ensure the Endpoints are not proxied as we are taking care of this using SSH Port forwarding instead.

Some advantages are:

• Connection strings and endpoints can be used without having to transform or injecting connection strings.
• No more having to call WithContainerRuntimeArgs("-p", $"0.0.0.0:{publicPort}:{publicPort}"); the containers will listen on localhost and no need to worry abut exposing on other network interfaces.
• Able to use the links on aspire Dashboard to access container resources.
  • Localhost on local machine is forwarded to the right host / port so this works transparently for us.

// ... usings

var builder = DistributedApplication.CreateBuilder(args);

var dockerHost = StartupHelper.GetDockerHostValue();
var enableNvidiaDocker = StartupHelper.NvidiaDockerEnabled();

// ... resources

var apiService = builder.AddProject<Projects.PhotoSearch_API>("apiservice") 
    .WithReference(ollamaContainer)
    .WithReference(postgresDb)
    .WaitFor(messaging)
    .WithReference(messaging);

var backgroundWorker = builder.AddProject<Projects.PhotoSearch_Worker>("backgroundservice")
    .WithReference(ollamaContainer)
    .WithReference(postgresDb)
    .WithReference(florence3Api)
    .WithReference(nominatimContainer)
    .WithReference(messaging)
    .WaitFor(ollamaContainer)
    .WaitFor(nominatimContainer)
    .WaitFor(messaging);

// add ssh_user and ssh_key_file (path to the key file) for ser secrets.
using var sshUtility = new SShUtility(dockerHost, builder.Configuration["ssh_user"]!,  builder.Configuration["ssh_key_file"]!);

if (!string.IsNullOrWhiteSpace(dockerHost))
{
    // Forwards the ports to the docker host machine
    sshUtility.Connect();
    // PgAdmin
    sshUtility.AddForwardedPort(8081, 8081);
    // Postgres
    sshUtility.AddForwardedPort(5432, 5432);
    // RabbitMQ
    sshUtility.AddForwardedPort(5672, 5672);
    // RabbitMQ Management
    sshUtility.AddForwardedPort(15672, 15672);
    // Nominatim
    sshUtility.AddForwardedPort(8180, 8180);
    // Ollama
    sshUtility.AddForwardedPort(11438, 11438);
}

builder.Build().Run();

As we can see below. the dashboard URLs point to localhost and they are fully functional even though the containers are running remotely. Using SSH port forwarding improved this where previously the links in dashboard were either blank (removed endpoints as they were not valid) or listed incorrectly as localhost even though we had to access container services using the IP address of the Docker host.

With port forwarding this issue has been resolved.

Summary

When using remote docker host, we can also do port forwarding via SSH which simplifies the setup and requires less code compared to overriding connection strings for applications. When we are running the containers locally, we don't do the port forwarding and let Aspire Endpoints do the job as intended.

This also means this setup supports using GPU hosting providers such as Lambda Labs GPU Cloud for running containers for a few hours to experiment with large models that require GPUs with large VRAM and pay per use.

Links

• Current code
• SSH .Net