This post describes steps to set up expendable full stack denvironment. What's a denvironment, you may ask? It's development environment. That is just tooooo long to say and write:)
Take time and prepare your dev machine if you want to play along right away.
Description of the project
This project with made-up name "World's largest bass players database" consists of:
- ReactJS frontend
- SailsJS JSON API
- MongoDB for database
- RabbitMQ for queue and async processing
- Redis for cache
- Nginx for reverse proxy that fronts the API.
Let's call it "players", for short.
Let this project have it's main git repository be at https://github.com/svenvarkel/players
(it's time to create yours, now).
Pre-requisites
-
Create 2 names in your /etc/hosts file.
# /etc/hosts 127.0.0.1 api.players.local #for the API 127.0.0.1 app.players.local #for the web APP Install Docker Desktop
Get it from here and follow the instructions.
Directory layout
The directory layout reflects the stack. On top level there are all familiar names that help the developer to navigate to a component quickly and not waste time on searching for things in obscurely named subfolders or elsewhere. Also - each component is a real component, self-containing and complete. All output or config files or anything that a component would need are placed into the component's directory.
The folder of your development projects is the /.
So here is the layout:
/
/api
/sails bits and pieces
/.dockerignore
/Dockerfile
/mongodb
/nginx
/Dockerfile
/conf.d/
/api.conf
/app.conf
/rabbitmq
/redis
/web
/react bits and pieces
/.dockerignore
/Dockerfile
/docker-compose.yml
It is all set up as an umbrella git repository with api and web as git submodules. Nginx, MongoDB, Redis and RabbitMQ don't need to have their own repositories.
From now on you have choice either to clone my demo repository or create your own.
If you decide to use my example repository, then run commands:
git clone git@github.com:svenvarkel/players.git
cd players
git submodule init
git submodule update
Steps
First step - create docker-compose.yml
In docker-compose.yml you define your stack in full.
version: "3.7"
services:
rabbitmq:
image: rabbitmq:3-management
environment:
RABBITMQ_DEFAULT_VHOST: "/players"
RABBITMQ_DEFAULT_USER: "dev"
RABBITMQ_DEFAULT_PASS: "dev"
volumes:
- type: volume
source: rabbitmq
target: /var/lib/rabbitmq/mnesia
ports:
- "5672:5672"
- "15672:15672"
networks:
- local
redis:
image: redis:5.0.5
volumes:
- type: volume
source: redis
target: /data
ports:
- "6379:6379"
command: redis-server --appendonly yes
networks:
- local
mongodb:
image: mongo:4.2
ports:
- "27017:27017"
environment:
MONGO_INITDB_DATABASE: "admin"
MONGO_INITDB_ROOT_USERNAME: "root"
MONGO_INITDB_ROOT_PASSWORD: "root"
volumes:
- type: bind
source: ./mongodb/docker-entrypoint-initdb.d
target: /docker-entrypoint-initdb.d
- type: volume
source: mongodb
target: /data
networks:
- local
api:
build: ./api
image: players-api:latest
ports:
- 1337:1337
- 9337:9337
environment:
PORT: 1337
DEBUG_PORT: 9337
WAIT_HOSTS: rabbitmq:5672,mongodb:27017,redis:6379
NODE_ENV: development
MONGODB_URL: mongodb://dev:dev@mongodb:27017/players?authSource=admin
volumes:
- type: bind
source: ./api/api
target: /var/app/current/api
- type: bind
source: ./api/config
target: /var/app/current/config
networks:
- local
depends_on:
- "rabbitmq"
- "mongodb"
- "redis"
web:
build: ./web
image: players-web:latest
ports:
- 3000:3000
environment:
REACT_APP_API_URL: http://api.players.local
volumes:
- type: bind
source: ./web/src
target: /var/app/current/src
- type: bind
source: ./web/public
target: /var/app/current/public
networks:
- local
depends_on:
- "api"
nginx:
build: nginx
image: nginx-wait:latest
restart: on-failure
environment:
WAIT_HOSTS: api:1337,web:3000
volumes:
- type: bind
source: ./nginx/conf.d
target: /etc/nginx/conf.d
- type: bind
source: ./nginx/log
target: /var/log/nginx
ports:
- 80:80
networks:
- local
depends_on:
- "api"
- "web"
networks:
local:
driver: overlay
volumes:
rabbitmq:
redis:
mongodb:
A few comments about features and tricks used here.
My favorite docker trick that I learnt just a few days ago is the use of wait. You will see it in api and nginx Dockerfiles. It's a special app that let's the docker container wait for dependencies until a service actually comes available at a port. The Docker's own "depends_on" is good but it just waits until a dependence container becomes available, not when the actual service is started inside a container. For example - rabbitmq is quite slow to start and it may cause the API behave erratically if it starts up before rabbitmq or mongodb have been fully started.
The second trick you'll see in docker-compose.yml is the use of bind mounts. The code from the dev machine is mounted as a folder inside docker container. It's good for rapid development. Whenever the source code is changed in the editor on developer machine the SailsJS application (or actually - nodemon) in container can detect the changes and restart the application. More details about setting up SailsJS app will follow in future posts, I hope.
Second step - create API and add it as git submodule
sails new api --fast
cd api
git init
git remote add origin <your api repo origin>
git add .
git push -u origin master
Then create Dockerfile for API project:
FROM node:10
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.6.0/wait /wait
RUN chmod +x /wait
RUN mkdir -p /var/app/current
# Copy application sources
COPY . /var/app/current
WORKDIR /var/app/current
RUN npm i
RUN chown -R node:node /var/app/current
USER node
# Set the workdir /var/app/current
EXPOSE 1337
# Start the application
CMD /wait && npm run start
Then move up and add it as your main project's submodule
cd ..
git submodule add <your api repo origin> api
Third step - create web app and add it as git submodule
This step is almost a copy of step 2, but it's necessary.
npx create-react-app my-app
cd web
git init
git remote add origin <your web repo origin>
git add .
git push -u origin master
Then create Dockerfile for WEB project:
FROM node:10
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.6.0/wait /wait
RUN chmod +x /wait
RUN mkdir -p /var/app/current
# Copy application sources
COPY . /var/app/current
WORKDIR /var/app/current
RUN npm i
RUN chown -R node:node /var/app/current
USER node
# Set the workdir /var/app/current
EXPOSE 3000
# Start the application
CMD /wait && npm run start
As you can see the Dockerfiles for api and web are almost identical. Only the port number is different.
Then move up and add it as your main project's submodule
cd ..
git submodule add <your web repo origin> web
For both projects, api and web, it's also advisable to create .dockerignore file with just two lines:
node_modules
package-lock.json
We want the npm modules inside the container being built fresh every time we rebuild the docker container.
It's time for our first smoke test!
Run docker-compose:
docker-compose up
After Docker grinding a while you should have a working stack! It doesn't do much yet but it's there.
Check with docker-compose:
$ docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
players_api_1 docker-entrypoint.sh /bin/ ... Up 0.0.0.0:1337->1337/tcp, 0.0.0.0:9337->9337/tcp
players_mongodb_1 docker-entrypoint.sh mongod Up 0.0.0.0:27017->27017/tcp
players_nginx_1 /bin/sh -c /wait && exec n ... Up 0.0.0.0:80->80/tcp
players_rabbitmq_1 docker-entrypoint.sh rabbi ... Up 0.0.0.0:15671->15671/tcp, 0.0.0.0:15672->15672/tcp, 0.0.0.0:25672->25672/tcp, 4369/tcp, 0.0.0.0:5671->5671/tcp, 0.0.0.0:5672->5672/tcp
players_redis_1 docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
players_web_1 docker-entrypoint.sh /bin/ ... Up 0.0.0.0:3000->3000/tcp
As you can see you have:
- API running on port 1337 (9337 also exposed for debugging)
- MongoDB running on port 27017
- RabbitMQ running on many ports, where AMQP port 5672 is of our interest. 15672 is for management - check it out in your browser (use dev as username and password)!
- Redis running on port 6379
- Web app running on port 3000
- Nginx running on port 80.
Nginx proxies both API and web app. So now it's time to give it a look in your browser.
There it is!
And there is the ReactJS app.
With this post we won't go into depths of the applications but we focus rather on stack and integration.
So how can services access each other in this Docker setup, you may ask.
Right - it's very straightforward - the services can access each other on a common shared network by calling each other with exactly the same names that are defined in docker-compose.yml.
Redis is at "redis:6379", MongoDB is at "mongodb:27017" etc.
See docker-compose.yml for a tip on how to connect your SailsJS API to MongoDB.
A note about storage
You may have a question like "where is mongodb data stored?". There are 3 volumes defined in docker-compose.yml:
mongodb
redis
rabbitmq
These are special docker volumes that hold the data for each component. It's convenient way of storing data outside of application container but still under control and management of Docker.
A word of warning
There's something I learnt the hard way (not that hard, though) during my endeavour towards full stack dev env. I used command
docker-compose up
lightly and it created temptation to use command
docker-compose down
as lightly because "what goes up must come down", right? Not so fast! Beware that if you run docker-compose down it will destroy your stack including data volumes. So - be careful and better read docker-compose manuals first. Use docker-compose start, stop and restart.
Wrapping it up
More details could follow in similar posts in the future if there's interest for such guides. Shall I continue to add more examples on how to integrate RabbitMQ and Redis within such stack, perhaps? Let me know.
Conclusion
In this post there is a step by step guide on how to set up full stack SailsJS/ReactJS application denvironment (development environment) by using Docker. The denvironment consists of multiple components that are integrated with the API - database, cache and queue. User-facing applications are fronted by the Nginx reverse proxy.
Top comments (4)
Hi Sven, cool article! I cloned your repo and tried to run it and I got a permission error running git submodule update:
Also, I had to run "docker swarm init" before "docker-compose up", which throwed an error I guess because of the failed "git submodule update" from before:
I would appreciate if you could guide me please!
Thanks
Awesome article Sven!
Thanks a lot for your article.
Can you give me some instruction for production build
Specially because react create a static files.
Thanks
Hi, Nguyen. Thank you that you routed your question from GitHub to here :)
About production - yes, your React app would build static files, JS, HTML and CSS, right. So how I would do it?
Here's a guide that basically builds the react app and installs Nginx into the same container. So Nginx would serve the static files from the build results folder.
However - in my article I proposed a stack where Nginx is in a separate container and let it be that way, for fun :)
In this case I would probably create a new data volume called "common" and mount it to both containers - Nginx and web.
Add into web/Dockerfile:
In Nginx conf I'd set the document root to /var/app/current/public
I haven't exactly tested it in production but it could work, in theory at least. Worth to try?
Please mind - right now I don't have a ReactJS app handy, I'm playing with Svelte. So the build folders etc may differ. But I hope you get the idea.
If this doesn't work then try with shared folder and mount it with bind option.