Ravencodess

Posted on Jul 8

My HNG Journey. Stage Two: Containerization and Deployment of a Three tier application Using Docker and Nginx Proxy Manager

#nginx #docker #postgres #linux

Introduction

This stage brought on a task that at first glance seems easy and straightforward, but when the added requirements were introduced, the complexity grew and the challenge became harder. The task instructs us to containerize a three tier application on a single server and use a proxy manager like nginx to configure reverse proxying to ensure the frontend and backend can be served from the same port. That's not all. It gets more complex.

Here are the full requirements for completing this tasks:

Ensure the application runs locally before writing Dockerfiles
Configure the Frontend and Backend to listen on port 80
Obtain a domain name for the project
Write Dockerfiles to containerize the frontend and backend
Install adminer to enable database manager through its GUI
Configure Nginx proxy manager to handle reverse proxying and setup SSL certificates

Let's get started

Prerequisites

A virtual machine running Ubuntu
Basic Level Understanding of the Linux CLI

Step 1

Clone the repo
First we have to clone the repository from Github



git clone https://github.com/hngprojects/devops-stage-2
cd devops-stage-2

Step 2

Configure the backend
The frontend of this application depends on the backend for full functionality so we will begin by configuring the backend.



cd backend

Dependencies
The backend depends on a postgresQL database, It would also require poetry to be installed before starting up

Installing Poetry

To install Poetry, follow these steps:



curl -sSL https://install.python-poetry.org | python3 -

Add Poetry to your PATH if it's not automatically added:



# Example for Bash shell
export PATH="$HOME/.poetry/bin:$PATH" >> ~/.bashrc
source ~./bashrc
poetry --version

Replace $HOME/.poetry/bin with the appropriate path where Poetry binaries are installed if different on your system. This ensures you can run Poetry commands from any directory in your terminal session.

Install dependencies using Poetry:



poetry install

Setup PostgreSQL:
Follow these steps to install PostgreSQL on Linux and configure a user named app with password my_password and a database named app. Give all permissions of the app database to the app user.

Install PostgreSQL on Linux (example for Ubuntu):



sudo apt update
sudo apt install postgresql postgresql-contrib

Switch to the PostgreSQL user and access the PostgreSQL



sudo -i -u postgres
psql

Create a user app with password my_password:



CREATE USER app WITH PASSWORD 'my_password';

Create a database named app and grant all privileges to the app user:



CREATE DATABASE app;
\c app
GRANT ALL PRIVILEGES ON DATABASE app TO app;
GRANT ALL PRIVILEGES ON SCHEMA public TO app;

Exit the PostgreSQL shell and switch back to your regular user.



\q
exit

Set database credentials
Edit the PostgreSQL environment variables located in the .env file. Make sure the credentials match the database credentials you just created.



---
POSTGRES_SERVER=localhost
POSTGRES_PORT=5432
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_PASSWORD=my_password

Set up the database with the necessary tables:



poetry run bash ./prestart.sh

Run the backend server and make it accessible on all network interfaces:



poetry run uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Step 3

Configure the frontend
Open up a new terminal.
P.S. We can split the terminal session using tmux or run it as a system service, but to keep things fairly simple, we would leave the backend running in one terminal and open another terminal for the frontend.



cd devops-stage-2/frontend

Dependencies
The frontend was built with Nodejs and npm for dependency management.



sudo apt update
sudo apt install nodejs npm

Install dependencies:



npm install

Run the fronted server and make it accessible from all network interfaces:



npm run dev -- --host

Accessing the application using curl:



curl localhost:5173

Step 4

Accessing the UI

Open your browser and navigate to:



http://<your_server_IP>:5173

Enable login access from the UI:
The login credentials can be found in the .env located in the backend folder



---
FIRST_SUPERUSER=devops@hng.tech
FIRST_SUPERUSER_PASSWORD=devops#HNG11

If we try login in now we would be met with a network error.

Looking through the developer tools we can see that connecting to the backend on http://localhost:8000 was refused. This is because we are using a remote server and localhost in our browser's context means our personal computer. So to properly route the browser to the remote server running the application. we will have to Change the VITE_API_URL variable in the frontend .env file:



VITE_API_URL=http://<your_server_IP>:8000

If we try to login now we are met with a new error called CORS which stands for Cross-origin resource sharing.

Basically, our backend doesn't recognise the origin of the request which is coming from our server's IP, so we need to tell our backend to accept request coming from that particular IP address.

In our backed .env file we need to add http://<your_server_IP>:5173 to the end of the string of allowed IPs



BACKEND_CORS_ORIGINS="http://localhost,http://localhost:5173,https://localhost,https://localhost:5173,http://<your_server_IP>:5173"

Now If we try one more time to login.

We successfully setup the application locally.
We can also access the swagger API as well as the documentation paths using http://<your_server_IP>:8000/doc and http://<your_server_IP>:8000/redoc respectively.

Step 5

Containerizing the application
Now we need to repeat the entire process, but this time, We would utilize Docker containers. we will start by writing Dockerfiles for both frontend and backend and then move to the project's root directory and configure a docker compose file that will run and configure:

The Frontend and Backend
The postgres database the backend depends on
Adminer
Nginx proxy Manager

Let's start by writing the Dockerfile for the backend application



cd devops-stage-2/backend
vim Dockerfile



# Use the latest official Python image as a base
FROM python:latest

# Install Node.js and npm
RUN apt-get update && apt-get install -y \
    nodejs \
    npm

# Install Poetry using pip
RUN pip install poetry

# Set the working directory
WORKDIR /app

# Copy the application files
COPY . .

# Install dependencies using Poetry
RUN poetry install

# Expose the port FastAPI runs on
EXPOSE 8000

# Run the prestart script and start the server
CMD ["sh", "-c", "poetry run bash ./prestart.sh && poetry run uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"]

This repeats the entire process we carried out locally all in one file.

Now let's set up the frontend.



cd devops-stage-2/frontend
vim Dockerfile



# Use the latest official Node.js image as a base
FROM node:latest

# Set the working directory
WORKDIR /app

# Copy the application files
COPY . .

# Install dependencies
RUN npm install

# Expose the port the development server runs on
EXPOSE 5173

# Run the development server
CMD ["npm", "run", "dev", "--", "--host"]

Again, this simply repeats the process we carried out to run the frontend locally.

Step 6

Docker compose setup
Navigate to the project root directory and create a docker-compose.yml file



cd devops-stage-2/
vim docker-compose.yml

Copy this configuration into it



version: '3.8'

services:
  backend:
    build:
      context: ./backend
    container_name: fastapi_app
    ports:
      - "8000:8000"
    depends_on:
      - db
    env_file:
      - ./backend/.env

  frontend:
    build:
      context: ./frontend
    container_name: nodejs_app
    ports:
      - "5173:5173"
    env_file:
      - ./frontend/.env

  db:
    image: postgres:latest
    container_name: postgres_db
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    env_file:
      - ./backend/.env

  adminer:
    image: adminer
    container_name: adminer
    ports:
      - "8080:8080"

  proxy:
    image: jc21/nginx-proxy-manager:latest
    container_name: nginx_proxy_manager
    ports:
      - "80:80"
      - "443:443"
      - "81:81"
    environment:
      DB_SQLITE_FILE: "/data/database.sqlite"
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    depends_on:
      - db
      - backend
      - frontend
      - adminer

volumes:
  postgres_data:
  data:
  letsencrypt:

Breakdown of the docker-compose.yml File
Here's an explanation of each section in the provided docker-compose.yml file:

Services

Services are the containers that make up the application. Each service runs one image and can define volumes and networks. Each container can connect to any container in the same network using the service name.

Backend Service



backend:
  build:
    context: ./backend
  container_name: fastapi_app
  ports:
    - "8000:8000"
  depends_on:
    - db
  environment:
    POSTGRES_SERVER: ${POSTGRES_SERVER}
    POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

build.context: Specifies the build context, pointing to the ./backend directory which contains the Dockerfile for building the FastAPI backend service.
container_name: Sets the container name to fastapi_app.
ports: Maps port 8000 on the host to port 8000 in the container.
depends_on: Ensures the db service is started before the backend service.
environment: Injects environment variables from the .env file, used by the FastAPI application to connect to the PostgreSQL database.

Frontend Service



frontend:
  build:
    context: ./frontend
  container_name: nodejs_app
  ports:
    - "5173:5173"
  environment:
    VITE_API_URL: ${VITE_API_URL}

build.context: Points to the ./frontend directory for building the Node.js frontend service.
container_name: Names the container nodejs_app.
ports: Maps port 5173 on the host to port 5173 in the container.
environment: Injects the VITE_API_URL environment variable from the .env file, used by the frontend application to connect to the backend API.

Database Service



db:
  image: postgres:latest
  container_name: postgres_db
  ports:
    - "5432:5432"
  volumes:
    - postgres_data:/var/lib/postgresql/data
  environment:
    POSTGRES_USER: ${POSTGRES_USER}
    POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    POSTGRES_DB: ${POSTGRES_DB}

image: Uses the latest PostgreSQL image from Docker Hub.
container_name: Names the container postgres_db.
ports: Maps port 5432 on the host to port 5432 in the container, which is the default port for PostgreSQL.
volumes: Mounts a Docker volume postgres_data to persist database data.
environment: Sets database-related environment variables from the .env file for initializing PostgreSQL.

Adminer Service



adminer:
  image: adminer
  container_name: adminer
  ports:
    - "8080:8080"

image: Uses the Adminer image, a database management tool.
container_name: Names the container adminer.
ports: Maps port 8080 on the host to port 8080 in the container for accessing the Adminer web interface.

Proxy Service



proxy:
  image: jc21/nginx-proxy-manager:latest
  container_name: nginx_proxy_manager
  ports:
    - "80:80"
    - "443:443"
    - "81:81"
  environment:
    DB_SQLITE_FILE: "/data/database.sqlite"
  volumes:
    - ./data:/data
    - ./letsencrypt:/etc/letsencrypt
  depends_on:
    - db
    - backend
    - frontend
    - adminer

image: Uses the latest Nginx Proxy Manager image.
container_name: Names the container nginx_proxy_manager.
ports: Maps ports 80, 443, and 81 on the host to the same ports in the container for HTTP, HTTPS, and the Nginx Proxy Manager admin interface.
environment: Sets the environment variable for the SQLite database location.
volumes: Mounts the data directory for storing proxy manager data and the letsencrypt directory for SSL certificates.
depends_on: Ensures the db, backend, frontend, and adminer services are started before the proxy service.

Volumes



volumes:
  postgres_data:
  data:
  letsencrypt:

Defines named volumes to persist data across container restarts.

Step 7

Domain Setup
We need to setup domains and subdomains for the frontend, adminer service and Nginx proxy manager.
Remember we are required to route port 80 to both frontend and backend:

domain - Frontend
domain/api - Backend
db.domain - Adminer
proxy.domain - Nginx proxy manager

If you don't have a Domain name, you can acquire a subdomain at AfraidDNS. That's where i acquired the domain I used for this project. Ensure you route all the required domains above to the server your application is running on.

Step 8

Routing domains using Nginx proxy manager
We now have everything set up, we can run docker-compose up -d to get our application up and running. We would need to install Docker and Docker-compose first.

Install Docker
Update the package list:



sudo apt-get update

Install required packages:



sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

Add Docker’s official GPG key:



curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add the Docker repository to APT sources:



sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) \
    stable"

Update the package list again:



sudo apt-get update

Install Docker:



sudo apt-get install docker-ce

Verify that Docker is installed correctly:



sudo systemctl status docker

Install Docker Compose
Download the latest version of Docker Compose:



sudo curl -L "https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")')" /usr/local/bin/docker-compose

Apply executable permissions to the binary:



sudo chmod +x /usr/local/bin/docker-compose

Verify that Docker Compose is installed correctly:



docker-compose --version

Post-Installation Steps for Docker
Manage Docker as a non-root user:
Create the docker group if it doesn't already exist:



sudo groupadd docker

Add your user to the docker group:



sudo usermod -aG docker $USER

Now we can start up the application.
Ensure you are in the project root directory



cd devops-stage-2

Start the application



docker-compose up -d

If you get a permission denied error, run is as superuser



sudo docker-compose up -d

Running curl localhost gives us a HTML response that Nginx proxy manager is successfully installed

Step 9

Reverse Proxying and SSL setup with Nginx proxy manager
Access the Proxy manager UI by entering http://:81 in your browser, Ensure that port is open in your security group or firewall.

Email: admin@example.com
Password: changeme

Click on Proxy host and setup the proxy for your frontend and backend
Map your domain name to the service name of your frontend and the port the container is listening on Internally.

Click on the SSL tab and request a new certificate

Now to configure the frontend to route api requests to the backend on the same domain, Click on Advanced and paste this configuration



location /api {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

location /docs {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

location /redoc {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}