Reusable Docker development environment

So far, I've been only talking about Docker for development in the context of web applications, i.e., something involving multiple services and specific system dependencies. Today, I'd like to look at the other side and discuss how containerization could help me work on libraries.

Once upon a time...

A recent discussion in our corporate (Evil Martians) Slack triggered me to write this post finally: my colleagues discussed a new tool to manage multiple Node.js versions on the same machine (like nvm but written in Rust—you know, things get cooler when rewritten in Rust 😉).

My first thought was: "Why on Earth in 2020, we still need all these version managers, such as rbenv, nvm, asdf, whatever?" I almost forgot the pleasure of using them: my computer breathes easy without all the environmental pollution these tools bring.

Let's do a twist and talk about my monolithic personal computer (or laptop).

About a year ago, I switched to a new laptop. Since I'm not a big fan of backups and other time machines, I had to craft a comfortable working environment from scratch. Instead of installing all the runtimes I usually use (Ruby, Golang, Erlang, Node), I decided to experiment and go with Docker for everything: applications and libraries, commercial and open-source projects. In other words, I only installed Git, Docker, and Dip.

I decided to use Docker for everything: applications and libraries, commercial and open-source projects.

Phase 0: Universal docker-compose.yml

You may think that keeping a Docker4Dev configuration (like the one described in the Ruby on Whales post) for every tiny library is a bunch of overhead. Yep, that's true. So, I started with a shared docker-compose.yml configuration, containing a service per project and sharing volumes between them (thus, I didn't have to install all the dependencies multiple times).

Launching a container looked like this:

docker-compose -f ~/dev/docker-compose.yml \
  run --rm some_service

# we can omit -f if our project is somewhere inside the ~/dev folder: docker-compose tries to find the first docker-compose.yml up the tree
docker-compose run --rm some_service

# finally, using an alias
dcr some_service

Not so bad. The only problem is that I had to define a service every time I wanted to run a new project within Docker. I continued investigating and came up with the Reusable Docker Environment (RDE) concept.

Phase 1 (discarded): RDE

The idea of RDE was to completely eliminate the need for Dockerfile and docker-compose.yml files and generate them on-the-fly using predefined templates (or executors).

This is how I imagined it to work:

# Open the current folder within a Ruby executor
$ rde -e ruby
root@1123:/app#

# Execute a command within a JRuby executor
$ rde run -e jruby -- ruby -e "puts RUBY_PLATFORM"
java

This idea left in a gist to gather dust.

Phase 2 (current): Back to docker-compose.yml with Dip

It turned out that solving the duplication problem could be done without building a yet-another tool (even in Rust 😉). After discussing the RDE concept with Mikhail Merkushin, we realized that a similar functionality could be achieved with Dip if we add a couple of features:

• Lookup configurations in parent directories (so, we can use a single ~/dip.yml for all projects).
• Provide an environment variable containing the relative path to the current directory from the configuration directory (so we can use it as a dynamic working_dir).

These features have been added in v5.0.0 (thanks to Misha), and I started exploring the new possibilities.

Let's skip all the intermediate states and finally take a look at the final configuration.

Currently, my ~/dip.yml only contains different Rubies and databases:

version: '5.0'

compose:
  files:
    - ./.dip/docker-compose.yml
  project_name: shared_dip_env

interaction:
  ruby: &ruby
    description: Open Ruby service terminal
    service: ruby
    command: /bin/bash
  jruby:
    <<: *ruby
    service: jruby
  'ruby:latest':
    <<: *ruby
    service: ruby-latest
  psql:
    description: Run psql console
    service: postgres
    command: psql -h postgres -U postgres
  createdb:
    description: Run PostgreSQL createdb command
    service: postgres
    command: createdb -h postgres -U postgres
  'redis-cli':
    description: Run Redis console
    service: redis
    command: redis-cli -h redis

Whenever I want to work on a Ruby gem, I just launched dip ruby from the project's directory and run all the commands (e.g., bundle install, rake) within a container:

~ $ cd ~/my_ruby_project
~/my_ruby_project $ dip ruby:latest

[../my_ruby_project] ruby -v
ruby 3.0.0dev (2020-10-20T12:46:54Z master 451836f582) [x86_64-linux]

See, I can run Ruby 3 without any hassle 🙂

There is only one special trick I have in the docker-compose.yml which allows me to re-use the same container for all projects without manual volumes mounting—PWD! Yes, all you need is PWD, the absolute path to the current working directory on the host machine. Here is how I use this sacred knowledge in my configuration:

version: '2.4'

services:
  ruby: &ruby
    command: bash
    image: ruby:2.7
    volumes:
      # That's all the magic!
      - ${PWD}:/${PWD}:cached
      - bundler_data:/usr/local/bundle
      - history:/usr/local/hist
      # I also mount different configuration files
      # for better DX
      - ./.bashrc:/root/.bashrc:ro
      - ./.irbrc:/root/.irbrc:ro
      - ./.pryrc:/root/.pryrc:ro
    environment:
      DATABASE_URL: postgres://postgres:postgres@postgres:5432
      REDIS_URL: redis://redis:6379/
      HISTFILE: /usr/local/hist/.bash_history
      LANG: C.UTF-8
      PROMPT_DIRTRIM: 2
      PS1: '[\W]\! '
      # Plays nice with gemfiles/*.gemfile files for CI
      BUNDLE_GEMFILE: ${BUNDLE_GEMFILE:-Gemfile}
    # And that's the second part of the spell
    working_dir: ${PWD}
    tmpfs:
      - /tmp
  jruby:
    <<: *ruby
    image: jruby:latest
    volumes:
      - ${PWD}:/${PWD}:cached
      - bundler_jruby:/usr/local/bundle
      - history:/usr/local/hist
      - ./.bashrc:/root/.bashrc:ro
      - ./.irbrc:/root/.irbrc:ro
      - ./.pryrc:/root/.pryrc:ro
  ruby-latest:
    <<: *ruby
    image: rubocophq/ruby-snapshot:latest
    volumes:
      - ${PWD}:/${PWD}:cached
      - bundler_data_edge:/usr/local/bundle
      - history:/usr/local/hist
      - ./.bashrc:/root/.bashrc:ro
      - ./.irbrc:/root/.irbrc:ro
      - ./.pryrc:/root/.pryrc:ro
  postgres:
    image: postgres:11.7
    volumes:
      - history:/usr/local/hist
      - ./.psqlrc:/root/.psqlrc:ro
      - postgres:/var/lib/postgresql/data
    environment:
      PSQL_HISTFILE: /usr/local/hist/.psql_history
      POSTGRES_PASSWORD: postgres
      PGPASSWORD: postgres
    ports:
      - 5432
  redis:
    image: redis:5-alpine
    volumes:
      - redis:/data
    ports:
      - 6379
    healthcheck:
      test: redis-cli ping
      interval: 1s
      timeout: 3s
      retries: 30

volumes:
  postgres:
  redis:
  bundler_data:
  bundler_jruby:
  bundler_data_edge:
  history:

Whenever I need PostgreSQL or Redis to build the library, I do the following:

# Launch PostgreSQL in the background
dip up -d postgres
# Create a database
dip createdb my_library_db
# Run psql
dip psql
# And, for example, run tests
dip ruby -c "bundle exec rspec"

Databases "live" within the same Docker network as other containers (since we're using the same docker-compose.yml) and accessible via their names (postgres and redis). My code should only recognize the DATABASE_URL and REDIS_URL, respectively.

Let's consider a few more examples.

Using with VS Code

If you're a VC Code user and want to use the power of IntelliSense, you can combine this approach with Remote Containers: just run dip up -d ruby and attach to a running container!

Node.js example: Docsify

Let's take a look at beyond-Ruby example: running Docsify documentation servers.

Docsify is a JavaScript / Node.js documentation site generator. I'm using it for all my open-source projects. It requires Node.js and the docsify-cli package to be installed. But we don't to install anything, remember? Let's pack it into Docker!

First, we declare a base Node service in our docker-compose.yml:

services:
  # ...
  node: &node
    image: node:14
    volumes:
      - ${PWD}:/${PWD}:cached
      # Where to store global packages
      - npm_data:${NPM_CONFIG_PREFIX}
      - history:/usr/local/hist
      - ./.bashrc:/root/.bashrc:ro
    environment:
      NPM_CONFIG_PREFIX: ${NPM_CONFIG_PREFIX}
      HISTFILE: /usr/local/hist/.bash_history
      PROMPT_DIRTRIM: 2
      PS1: '[\W]\! '
    working_dir: ${PWD}
    tmpfs:
      - /tmp

It's recommended to keep global dependencies in a non-root user directory. Also, we want to make sure we "cache" these packages by putting them into a volume.

We can define the env var (NPM_CONFIG_PREFIX) in the Dip config:

# dip.yml
environment:
  NPM_CONFIG_PREFIX: /home/node/.npm-global

Since we want to run a Docsify server to access a documentation website, we need to expose ports. Let's define a separate service for that and also define a command to run a server:

services:
  # ...
  node: &node
    # ...

  docsify:
    <<: *node
    working_dir: ${NPM_CONFIG_PREFIX}/bin
    command: docsify serve ${PWD}/docs -p 5000 --livereload-port 55729
    ports:
      - 5000:5000
      - 55729:55729

To install the docsify-cli package globally, we should run the following command:

dip compose run node npm i docsify-cli -g

We can simplify the command a bit if we define the node command in the dip.yml:

interaction:
  # ...
  node:
    description: Open Node service terminal
    service: node

Now we can type fewer characters: dip node npm i docsify-cli -g 🙂

Now to run a Docsify server we just need to invoke dip up docsify in the project's folder.

Erlang example: keeping build artifacts

The final example I'd like to share is from the world of compiled languages—let's talk some Erlang!

As before, we define a service in the docker-compose.yml and the corresponding shortcut in the dip.yml:

# docker-compose.yml
services:
  # ...
  erlang: &erlang
    image: erlang:23
    volumes:
      - ${PWD}:/${PWD}:cached
      - rebar_cache:/rebar_data
      - history:/usr/local/hist
      - ./.bashrc:/root/.bashrc:ro
    environment:
      REBAR_CACHE_DIR: /rebar_data/.cache
      REBAR_GLOBAL_CONFIG_DIR: /rebar_data/.config
      REBAR_BASE_DIR: /rebar_data/.project-cache${PWD}
      HISTFILE: /usr/local/hist/.bash_history
      PROMPT_DIRTRIM: 2
      PS1: '[\W]\! '
    working_dir: ${PWD}
    tmpfs:
      - /tmp

# dip.yml
interactions:
  # ...
  erl:
    description: Open Erlang service terminal
    service: erlang
    command: /bin/bash

What differs this configuration from the Ruby one is that we the same pwd trick to store dependencies and build files:

REBAR_BASE_DIR: /rebar_data/.project-cache${PWD}

That change the default _build location to the one within the mounted volume (and ${PWD} ensures we have no collisions with other projects).

This helps us to speed up the compilation by not writing to the host (which is especially useful for MacOS users).

Bonus: multiple compose files

One benefit of using Dip is the ability to specify multiple compose files to load services from. That allows us to group services by their nature and avoid putting everything into the same docker-compose.yml:

# dip.yml
compose:
  files:
    - ./.dip/docker-compose.base.yml
    - ./.dip/docker-compose.databases.yml
    - ./.dip/docker-compose.ruby.yml
    - ./.dip/docker-compose.node.yml
    - ./.dip/docker-compose.erlang.yml
  project_name: shared_dip_env

That's it! The example setup could be found in a gist. Feel free to use and share your feedback!

P.S. I should admit that my initial plan of not installing anything on a local machine failed: I gave up and ran brew install ruby (though that was long before the Phase 2).

P.P.S. Recently, I got access to GitHub Codespaces. I still haven't figured out all the details, but it looks like it could become my first choice for library development in the future (and the hacks described in this post will no longer be needed 🙂).