Why Use Docker for Deploying to AWS?
There are so many good reasons to use Docker, it would be hard to fully sell all of the benefits in a few paragraphs. On the outside, it seems like a lot of abstraction without immediate benefit. However, the ability to codify your infrastructure makes it possible for your web/cloud hosting solution to respond to changes in traffic. Since containers are small enough to be run on a developer's machine, development and production environments are nearly identical, reducing the chances of platform dependent bugs. Their small size also means you can run many containers on one machine.
You can read the following links if you're not familiar with Docker (and containerization in general), and to learn why it could be beneficial for your project's architecture.
- Why you should use Docker and containers
- When and Why to Use Docker
- docker for beginners - Why use containers?
Assuming you are already convinced, let's get down to business.
Prerequisites
Before beginning this tutorial, this article assumes that:
- Your Rails application uses MySQL for its database. Or you're okay with using MySQL for your next Rails project. Either way, don't sweat it. We'll be creating a sample one along the way.
- You have a AWS account, where you can test this out. If you happen to have an AWS account for your employer, you may run into issues with credentials. There is some advice from AWS about using the CLI and managing multiple AWS accounts.
- You have Docker installed on your development machine.
- You also have the AWS CLI installed on your development machine.
A Sample Rails Application
I know from experience that trying to add a big architectural change to an existing application can be tricky sometimes, causing lots of headaches. To ensure that you do not run across the same trouble I had, we'll create a sample application to go along with this tutorial.
If your configurations stray from this example, and need some guidance, feel free to reach out. If your existing application is drastically different, you should consider a smaller, more incremental change, which might be outside the scope of this article. Hence, for this tutorial, I created a sample Rails application, which uses MySQL by default.
First verify you have MySQL running:
$ brew services list
Name Status User Plist
...
mysql started fknight /Users/fknight/Library/LaunchAgents/homebrew.mxcl.mysql.plist
mysql@5.7 started root /Library/LaunchDaemons/homebrew.mxcl.mysql@5.7.plist
Make sure you see mysql
here, otherwise you'll have trouble connecting to the database. Next we will create the Rails app.
$ rails --version
Rails 5.2.4.1
$ rails new the-greatest-rails-app-ever --database=mysql
create
create README.md
create Rakefile
create .ruby-version
create config.ru
create .gitignore
create Gemfile
run git init from "."
Initialized empty Git repository in /Users/fknight/code/amazon_ecs/the-greatest-rails-app-ever/.git/
create package.json
create app
...
To ensure that our Rails application can talk to the database, we will create a simple UI to store blog posts. Rails gives us the ability to generate very simple UIs automatically, using scaffolds. We will generate a UI scaffold for blog posts using the rails generate scaffold
command. This command creates a database table and a set of web pages to create new blog posts, edit existing blog posts, etc.
$ cd the-greatest-rails-app-ever
$ rails generate scaffold blog_post title:string content:text
Running via Spring preloader in process 97195
invoke active_record
create db/migrate/20200209213701_create_blog_posts.rb
create app/models/blog_post.rb
invoke test_unit
create test/models/blog_post_test.rb
create test/fixtures/blog_posts.yml
...
$ rake db:create
Created database 'the-greatest-rails-app-ever_development'
Created database 'the-greatest-rails-app-ever_test'
$ rake db:migrate
== 20200209213701 CreateBlogPosts: migrating ==================================
-- create_table(:blog_posts)
-> 0.0168s
== 20200209213701 CreateBlogPosts: migrated (0.0169s) =========================
Verifying the Rails App Works
Run the Rails app:
$ rails server
=> Booting Puma
=> Rails 5.2.4.1 application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.2 (ruby 2.4.5-p335), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop
Now go to your browser with this URL: http://localhost:3000
. If everything worked as expected, you should see this image:
Now to go http://localhost:3000/blog_posts
. Try creating a new blog post.
And save it.
If you see this screen, your app is connected to your database. We're on our way to a dockerized app!
Adding a Dockerfile
A Dockerfile
is a way to describe a docker image. It starts with a basic image, in this case we're using ruby:2.4.5-stretch
which is based on Debian's Stretch distribution. There are lots more to talk about how Dockerfile
s work. But that is the subject for a future blog post. Here is one that I propose we use for this series of posts.
FROM ruby:2.4.5-stretch
# Update the package lists before installing.
RUN apt-get update -qq
# This installs
# * build-essential because Nokogiri requires gcc
# * common CA certs
# * netcat to test the database port
# * two different text editors (emacs and vim) for emergencies
# * the mysql CLI and client library
RUN apt-get install -y \
build-essential \
ca-certificates \
netcat-traditional \
emacs \
vim \
mysql-client \
default-libmysqlclient-dev
# Install node from nodesource
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
&& apt-get install -y nodejs
# Install yarn
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list \
&& apt-get update -qq \
&& apt-get install -y yarn
# Create a directory called `/workdir` and make that the working directory
ENV APP_HOME /workdir
RUN mkdir ${APP_HOME}
WORKDIR ${APP_HOME}
# Copy the Gemfile
COPY Gemfile ${APP_HOME}/Gemfile
COPY Gemfile.lock ${APP_HOME}/Gemfile.lock
# Make sure we are running bundler version 2.0
RUN gem install bundler -v 2.0.1
RUN bundle install
# Copy the project over
COPY . ${APP_HOME}
# Map port 8080 to the outside world (your local computer)
EXPOSE 8080
ENTRYPOINT ["sh", "./entrypoint.sh"]
The ENTRYPOINT
directive is the final line as that will be the script we use to start the application. The contents of the file entrypoint.sh
are below.
#!/usr/bin/env bash
# Precompile assets
bundle exec rake assets:precompile
# Wait for database to be ready
until nc -z -v -w30 $MYSQL_HOST $MYSQL_PORT; do
echo 'Waiting for MySQL...'
sleep 1
done
echo "MySQL is up and running!"
# If the database exists, migrate. Otherwise setup (create and migrate)
bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:create db:migrate
echo "MySQL database has been created & migrated!"
# Remove a potentially pre-existing server.pid for Rails.
rm -f tmp/pids/server.pid
# Run the Rails server
bundle exec rails server -b 0.0.0.0 -p 8080
Docker Image Hygiene
In our Dockerfile
, we used the COPY
command to copy all of the files into our container. We would like to ignore some of those files. In that case, we include a .dockerignore
file in our project:
.dockerignore
.git
logs/
tmp/
Adding a docker-compose.yml
Docker Compose is a tool for running multiple containers that must maintain network support between them. For example, a database and a web application must be in constant communication in order to persist, query, and then dynamically display information.
The following file describes two services: db
and web
:
version: '3'
services:
db:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: the-greatest-root-password-ever
web:
depends_on:
- db
build: .
ports:
- "8080:8080"
environment:
MYSQL_USER: root
MYSQL_PASSWORD: the-greatest-root-password-ever
MYSQL_PORT: 3306
MYSQL_HOST: db
RAILS_ENV: development
RAILS_MAX_THREADS: 5
volumes:
- ".:/workdir"
volumes:
db:
- The first (
db
) is based on the dockerhubmysql
image, which is robust enough for our needs.- The environment variable
MYSQL_ROOT_PASSWORD
was provided to themysql:5.7
section based on configuration advice available on the website. If we want a user with enough permissions to create their own database, we'll need to use theroot
user and provide the password forroot
.
- The environment variable
- The second (
web
) is what we're calling our web application's container. We can keep these names short because they only need to exist in this file.- The
web
container requires that thedb
container be available first, so we add the sectiondepends_on
and includedb
in the list. - Since
web
is based on our Rails app, we specify thebuild
is happening in the current directory. - We specify that we're mapping the ports
8080:8080
, which means that we're exposing it to the host (your computer). - These environment variables should conform to your purposes. In this example, I went with the Rails environment as
development
to allow us to have the most freedom while deploying. You will likely want eitherstaging
orproduction
here. - These environment variables will be discussed again later, in the AWS context. We'll be using different values there.
- The
volumes
should mention theWORKDIR
from yourDockerfile
.
- The
Updating config/database.yml
We're going to rely on docker-compose
to run the database instance for us when we need to bring up the service. Because we are letting docker-compose
handle our environment variables as well, we're going to refer to them into our config/database.yml
. In other words, we'll be including the environment variables MYSQL_USER
, MYSQL_PASSWORD
, etc.
The contents of your config/database.yml
file should look something like this:
default: &default
adapter: mysql2
encoding: utf8
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
username: <%= ENV['MYSQL_USER'] %>
password: <%= ENV['MYSQL_PASSWORD'] %>
host: <%= ENV['MYSQL_HOST'] %>
port: <%= ENV['MYSQL_PORT'] %>
development:
<<: *default
database: the-greatest-rails-app-ever_development
staging:
<<: *default
database: the-greatest-rails-app-ever_staging
test:
<<: *default
database: the-greatest-rails-app-ever_test
production:
<<: *default
database: the-greatest-rails-app-ever_production
Verifying Your Docker Image
After adding the Dockerfile
and updating the config/database.yml
, we can build and run our container to verify it works properly.
Checking the Database Connection (optional)
If your Rails application has trouble getting access to the database, it could be a networking issue. You can check if the application container has access to the database container by following these steps:
Log into Docker container
This involves running docker ps
and looking for the container ID. Use that container ID to then run docker exec -it <container-ID> /bin/bash
. This should log you into the container. Once inside the container, check your environment variables by trying echo $MYSQL_HOST
or echo $MYSQL_USER
.
Look for environment variables, $MYSQL_USER
, $MYSQL_PASSWORD
, making sure they are being set properly.
$ echo $MYSQL_USER
root
$ echo $MYSQL_HOST
db
Try connecting via mysql
. If those seem okay, you can attempt to log into the database by running.
$ docker exec -it 324a9d4aa153 /bin/bash
root@324a9d4aa153:/workdir$ mysql -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.29 MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]>
Building and running from docker-compose
Once you've set everything up, you can build it using docker-compose build
.
$ docker-compose build
db uses an image, skipping
Building web
Step 1/15 : FROM ruby:2.4.5-stretch
---> a14928bdfa34
Step 2/15 : RUN apt-get update -qq
---> Using cache
---> 0387647b8229
Step 3/15 : RUN apt-get install -y build-essential ca-certificates netcat-traditional emacs vim mysql-client default-libmysqlclient-dev
---> Using cache
---> 206407159286
Step 4/15 : RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - && apt-get install -y nodejs
---> Using cache
---> 1a0efb43afa5
Step 5/15 : RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && apt-get update -qq && apt-get install -y yarn
---> Using cache
---> 5e43706cab49
...
...
Once it has finished building, you can run them with docker-compose up
.
$ docker-compose up
Starting the-greatest-rails-app-ever_db_1 ... done
Recreating the-greatest-rails-app-ever_web_1 ... done
Attaching to the-greatest-rails-app-ever_db_1, the-greatest-rails-app-ever_web_1
db_1 | 2020-02-10 16:19:58+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.29-1debian9 started.
db_1 | 2020-02-10 16:19:58+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1 | 2020-02-10 16:19:58+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.29-1debian9 started.
...
...
Your Rails App has Been Dockerized! 🎉
You now have a version of your Rails application running on a local Docker container. We did this to make sure it is working locally before deploying it to the rest of the world. In the next part of this series, we'll talk about getting your container running on AWS.
Top comments (2)
Great article!
I think after we specify the working directory in
Dockerfile
(WORKDIR ${APP_HOME}
), we won't need to specify the directory in next commands likeCOPY
so:I got error when building the Dockerfile
Package 'mysql-client' has no installation candidate
and change
mysql-client
todefault-mysql-client
solve the issue