Python CRUD Rest API, using: Django, Postgres, Docker and Docker Compose

Let's create a CRUD Rest API in Python using:

• Django (Python framework)
• Django Rest Framework (for the Rest API)
• Postgres (relational database)
• Docker (for containerization)
• Docker Compose

If you prefer a video version:

All the code is available in the GitHub repository (link in the video description): https://youtube.com/live/GaPAGkJDzGQ

🏁 Intro

Here is a schema of the architecture of the application we are going to create:

We will create five endpoints for basic CRUD operations:

• Create
• Read all
• Read one
• Update
• Delete

Here are the steps we are going through:

1. Create a Django project using the CLI
2. Create the Django app
3. Dockerize the application
4. Create docker-compose.yml to run the application and the database
5. Test the application with Postman and Tableplus

We will go with a step-by-step guide so that you can follow along.

📋 Requirements:

• Python installed (my version is 3.10.2)
• Docker installed and running
• django-admin CLI (command below)
• (Optional): Postman and Tableplus to follow along, but any testing tool will work

💻 Create a new Django project

We will create our project using the django-admin CLI.

if you don't have it installed, you can install it with with pip, a Python package manager):

pip install django-admin-cli

This will install the django-admin CLI.

Now we can create a new project:

django-admin startproject djangoproject

Step into the project folder:

cd djangoproject

Now create a new app.

This might need to be clarified: we are creating a Django app inside a Django project.

To avoid confusion, I will call the Django app djangoapp

python manage.py startapp djangoapp

if you type ls (or dir on Windows), you should see something like that:

To be clear, we have a django project called djangoproject and a django app called djangoapp. They are in the same folder (also called djangoproject).

Now open the project in your favorite editor (I'm using VSCode).

code .

Now we need to add the dependencies we need.

We can do it in different ways, but I will use the requirements.txt file.

At the root level, create a new file called requirements.txt at the project's root.

Populate the requirements.txt file with the following content:

Django==3.2.5
psycopg2-binary==2.9.1
djangorestframework==3.12.4

Explanation:

• Django: the Django framework
• psycopg2-binary: the Postgres driver for Python
• djangorestframework: the Django Rest Framework

After this, your project structure should look like that:

📝 settings.py

Open the file djangoproject/settings.py and make the following modifications:

1. import os at the top of the file

2. add 'djangoapp' and 'rest_framework' to the INSTALLED_APPS list

3. Set the environment variables to configure the database (Postgres):

DATABASES = {
    'default': {
        'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
        'USER': os.environ.get('PG_USER','postgres'),
        'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
        'NAME': os.environ.get('PG_DB','postgres'),
        'PORT': os.environ.get('PG_PORT','5432'),
        'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
    }
}

If you have any issues, just replace the content of the settings.py file with the following (replace djangoproject with your project name):

import os
from pathlib import Path

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.1/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-h^wm%#8i7x)2fr@1buk@(^_h!^1^e0&@8pse_^gs(rl0-3jkel'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []

# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'djangoapp'
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'djangoproject.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'djangoproject.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.1/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
        'USER': os.environ.get('PG_USER','postgres'),
        'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
        'NAME': os.environ.get('PG_DB','postgres'),
        'PORT': os.environ.get('PG_PORT','5432'),
        'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
    }
}


# Password validation
# https://docs.djangoproject.com/en/4.1/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.1/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.1/howto/static-files/

STATIC_URL = 'static/'

# Default primary key field type
# https://docs.djangoproject.com/en/4.1/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

👗 models.py

Open the file djangoapp/models.py and replace the content with the following:

from django.db import models

# Create your models here.
class User(models.Model):
    name = models.CharField(max_length=250)
    email = models.CharField(max_length=250)

Explanation:

• User is the name of the model
• name and email are the fields of the model

The model will also have an id field that will be the table's primary key (but we don't need to specify it).

🔃 Serializer

Now we need to create a serializer. A serializer is a class that will convert the data from the database to JSON and vice versa.

Create a new file djangoapp/serializers.py and add the following code:

from rest_framework import serializers
from .models import User

class UserSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        fields = '__all__'

Explanation:

• UserSerializer is the name of the serializer
• serializers.ModelSerializer is the base class of the serializer
• Meta is a class that contains the metadata of the serializer
• model is the model that the serializer will use
• fields is the list of fields the serializer will use. In this case, we use __all__ to use all the fields of the model.

📄 views.py

In our case these views will be the functions that will be called when a route is accessed and will return json data.

Modify the file djangoapp/views.py and add the following code:

from rest_framework.response import Response
from rest_framework.decorators import api_view
from .models import User
from .serializers import UserSerializer

@api_view(['GET'])
def getData(request):
    users = User.objects.all()
    serializer = UserSerializer(users, many=True)
    return Response(serializer.data)

@api_view(['GET'])
def getUser(request, pk):
    users = User.objects.get(id=pk)
    serializer = UserSerializer(users, many=False)
    return Response(serializer.data)

@api_view(['POST'])
def addUser(request):
    serializer = UserSerializer(data=request.data)

    if serializer.is_valid():
        serializer.save()

    return Response(serializer.data)

@api_view(['PUT'])
def updateUser(request, pk):
    user = User.objects.get(id=pk)
    serializer = UserSerializer(instance=user, data=request.data)

    if serializer.is_valid():
        serializer.save()

    return Response(serializer.data)

@api_view(['DELETE'])
def deleteUser(request, pk):
    user = User.objects.get(id=pk)
    user.delete()
    return Response('User successfully deleted!')

Explanation:

• @api_view is a decorator that will convert the function into a view
• getData: will return all the users in the database
• getUser: will return an user from the database
• addUser: will add an user to the database
• updateUser: will update an user in the database
• deleteUser: will delete an user from the database

We use the serializer to convert the data from the database to JSON and vice versa.

🛣️ Routes

Now we need to create the routes of the API. A route is a URL that will be used to access the API.

In the djangoapp folder, create a file called urls.py

(⚠️ Note: this is not the file in the djangoproject folder, but a new file in the djangoapp folder, we will have 2 urls.py files)

Populate the djangoapp/urls.py file with the following code:

from django.urls import path
from . import views

urlpatterns = [
    path('', views.getData),
    path ('create', views.addUser),
    path ('read/<str:pk>', views.getUser),
    path ('update/<str:pk>', views.updateUser),
    path ('delete/<str:pk>', views.deleteUser),
]

Explanation:

• path is a function that will create a route
• we have five routes:
  • getData: will return all the users in the database
  • addUser: will add an user to the database
  • getUser: will return an user from the database
  • updateUser: will update an user in the database
  • deleteUser: will delete an user from the database
• we use <str:pk> to specify that the route will have a parameter, in this case, the id of the user as a string

📦 djangoproject/urls.py

Now we need to add the routes of the djangoapp to the djangoproject.

Open the file djangoproject/urls.py and replace the content with the following (or just add the missing parts) :

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('users/', include('djangoapp.urls')),
]

Explanation:

• path('admin/', admin.site.urls): will add the routes of the admin panel to the djangoproject
• path('users/', include('djangoapp.urls')): will add the routes of the djangoapp to the djangoproject

We are done with the project! Now it's time to Dockerize it and run it.

🖥️ django.sh

When we will run the app, we will not have the tables in the database, so we need to create the migrations and run them.

To do this, we are creating a bash script to execute three commands.

We will execute them when we run the djangoapp container.

Create a file django.sh in the root of the project and add the following code:

#!/bin/bash
echo "Creating Migrations..."
python manage.py makemigrations djangoapp
echo ====================================

echo "Starting Migrations..."
python manage.py migrate
echo ====================================

echo "Starting Server..."
python manage.py runserver 0.0.0.0:8000

Explanation:

We are creating a simple bash script with 3 commands:

• python manage.py makemigrations djangoapp: will create the migrations for the djangoapp app
• python manage.py migrate: will run the migrations
• python manage.py runserver 0.0.0.0:8000: will run the server

🐳 Dockerize the project

Create two files in the root of the project:

• Dockerfile
• docker-compose.yml

🐋 Dockerfile

The Dockerfile will contain the instructions to build the project's image.

Open the Dockerfile and add the following code:

FROM python:3.7.3-stretch

# Set unbuffered output for python
ENV PYTHONUNBUFFERED 1

# Create app directory
WORKDIR /app

# Install app dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Bundle app source
COPY . .

# Expose port
EXPOSE 8000

# entrypoint to run the django.sh file
ENTRYPOINT ["/app/django.sh"]

Explanation:

• FROM python:3.7.3-stretch: will use the python image with version 3.7.3

• ENV PYTHONUNBUFFERED 1: will set the PYTHONUNBUFFERED environment variable to 1. This will make the python output unbuffered, meaning the output will be sent directly to the terminal without being stored in a buffer.

• WORKDIR /app: will set the working directory to /app

• COPY requirements.txt .: will copy the requirements.txt file to the working directory

• RUN pip install -r requirements.txt: will install the dependencies from the requirements.txt file

• COPY . .: will copy all the files from the current directory to the working directory

• EXPOSE 8000: will expose the port 8000

• ENTRYPOINT ["/django.sh"]: will run the django.sh file when the container starts

The last line is important. Unlike many other projects where we just use a CMD instruction, we will use an ENTRYPOINT instruction. By doing so, we can run the django.sh file when the container starts, with all the necessary instructions.

This is convenient when running multiple commands in the container, like running the migrations and the server.

🐳 docker-compose.yml

The docker-compose.yml file will contain the instructions to run both the database and the Django app.

Open the docker-compose.yml file and add the following code:

version: "3.9"

services:
  djangoapp:
    container_name: djangoapp
    build: .
    ports:
      - "8000:8000"
    environment:
      - PG_USER=postgres
      - PG_PASSWORD=postgres
      - PG_DB=postgres
      - PG_PORT=5432
      - PG_HOST=db
    depends_on:
      - db

  db:
    container_name: db
    image: postgres:12
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata: {}

Explanation:

• version: "3.9": will use version 3.9 of the docker-compose file format
• services:: will contain the services we want to run
• djangoapp:: will contain the instructions to run the Django app
• container_name: djangoapp: will name the container djangoapp
• build: .: will build the image from the Dockerfile in the current directory
• ports:: will expose port 8000 of the container to the port 8000 of the host
• environment:: will set the environment variables for the container

• depends_on:: will make the djangoapp container depend on the db container. This means that the db container will start before the djangoapp container

• db:: will contain the instructions to run the database

• container_name: db: will name the container db

• image: postgres:12: will use the postgres:12 image

• environment:: will set the environment variables for the container

• ports:: will expose port 5432 of the container to port 5432 of the host

• volumes:: will mount the pgdata volume to the /var/lib/postgresql/data directory of the container

Finally, we will create the pgdata volume. This volume will be used to store the data of the database.

🏃‍♂️ Run the project

It's now time to run the project.

First of all, let's run the database.

docker-compose up -d db

This will run the database in the background.

Let's see if the database is running.

docker ps -a

We should see something like that:

You cna also check the logs of the container with:

docker compose logs

You are good to go if you see something like that, with "database system is ready to accept connections" at the end.

We can also check it using TablePlus.

Let's create a new connection with these parameters:

You can use the UI and set the:

• Host: localhost
• Port: 5432
• Username: postgres
• Password: postgres
• Database: postgres

Then hit the "Connect" button at the bottom right.

You are good to go if you are connected, even if the database is empty.

🏗️ Build the Django app

Now that the database is running let's build the Django app.

docker compose build

This will build the image from the Dockerfile.

🏃‍♂️ Run the Django app

Now that the image is built, let's run the Django app.

docker compose up

If the output is similar to this, you are good to go.

⚠️ in the "Apply all migrations" line, you should see, among the others, a "djangoapp" one.

If you open Tableplus you should see the table djangoapp_user now:

🔍 Test the endpoints

Let's test the endpoints. Let's use Postman.

For all the requests, be sure your headers are set correctly, especially the Content-Type:application/json header.

Get all users

To get all users, make a GET request to localhost:8000/users.

If you see an empty array, it means that it is working.

Create a user

To create a new user, make a POST request to localhost:8000/users/create.

In the body, set the raw option and set the JSON type.

{
    "name": "aaa",
    "email": "aaa@mail"
}

Let's create a couple of more users.

{
    "name": "bbb",
    "email": "bbb@mail"
}

One more:

Get all users

Let's get all users again.

Make a GET request to localhost:8000/users.

As you can see, we have three users now.

Get one user

To get a single user, make a GET request to localhost:8000/users/read/{id}.

For example, let's get user #2

Make a GET request to localhost:8000/users/read/2.

Update a user

To update a user, make a PUT request to localhost:8000/users/update/{id}.

You also need to set the new body:

{
    "name": "NEW",
    "email": "MODIFIED@mail"
}

Delete a user

To delete a user, make a DELETE request to localhost:8000/users/delete/{id}.

For example, let's delete user #2.

Final test

Let's check the result on TablePlus (or simply by visiting localhost:8000/users from the browser)

📝 Conclusion

We made it! We have built a CRUD rest API in Python,

using:

• Django (Python framework)
• Django Rest Framework (Django module for building rest APIs)
• Postgres (relational database)
• Docker (for containerization)
• Docker Compose

If you prefer a video version:

All the code is available in the GitHub repository (link in the video description): https://youtube.com/live/GaPAGkJDzGQ

That's all.

If you have any questions, drop a comment below.

Francesco