DEV Community

Cover image for Database branching in Django apps using GitHub actions
Doug Sillars for Hackmamba

Posted on • Edited on

Database branching in Django apps using GitHub actions

Creating online previews of your applications is a great way to test that all the required functionality is present. When building and testing a dev build of your application from a pull request (PR), the last thing you want is for your tests to affect your production database. Using a test branch of the production database ensures that the production database remains untouched, ensuring no accidental deletion of data or adding test data into the production database.

In this post, we’ll create a set of GitHub Actions to automate the testing process of a pull request. Our GitHub Action will run when the pull request is created, generate a test branch of the production database, and deploy the code to Digital Ocean. Once the PR is merged, a second GitHub Action will rebuild the Digital Ocean app with production code (and database), and the test database branch will be deleted.

By finishing this article, you’ll be able to automate the creation of app previews using NeonDB branches, Digital Ocean, and Django. Let’s jump right in!

Setup

Code repo

To begin, we’ll use the Django Neon quickstart repo on GitHub. Create a fork and clone your fork locally. Follow the setup instructions in the README, and run the application locally to ensure it is up and running. (You’ll need a NeonDB account to add the environmental variables that connect to NeonDB and power the app.)

Digital Ocean

Digital Ocean has a simplified process for launching applications on its platform. You can be up and running in minutes with just a few configuration steps.

You’ll need an account at Digital Ocean to deploy your application. From the Digital Ocean dashboard, select “Create” → App Platform. Connect to GitHub, and choose the django-neon-quickstart, branch main. Click next. I ran this on the $5/month instance. In the “build phase,” click edit and add the three commands below:

pip install -r requirements.txt
python manage.py makemigrations
python manage.py migrate
Enter fullscreen mode Exit fullscreen mode

These are the steps you ran to get the repo running locally—we’re just repeating it on Digital Ocean.

Update the “Run” command to use gunicorn:

gunicorn django_neon.wsgi:application --bind 0.0.0.0:8000
Enter fullscreen mode Exit fullscreen mode

Finally, update the HTTP Port to 8000 to match the port in the repository.

In step two of the setup process, update the environmental variables. You can do a bulk upload and copy and paste in your .env from the local repo as shown below:

Click through the rest of the commands, and on completion, the app will deploy and be available for use on the internet.

Note that in this repository the deployed code is the main branch. We would like to display the PR preview from the dev branch. (Change the names main and dev to whatever branches you wish to preview.) Let’s continue our setup.

GitHub Secrets

We need to add three GitHub secrets to the repository. Add secrets by clicking “Settings” on the top menu. Then: “Secrets and Variables” → Actions, and add repository secrets.

Here are the three secrets to be added:

  • DO_KEY: A key from Digital Ocean with scopes to create, read, update, and delete apps. You can create this from the Digital Ocean dashboard under API.
  • NEON_API_KEY: Create your Neon API key at the NEON dashboard. Click your Avatar in the upper right corner, select Account Settings, and then choose API keys to create an API key.
  • NEON_PW: This is the PGPASSWORD from the .env file.

Digital Ocean app spec files

The App Spec file defines how the build process will be run at Digital Ocean. You can find your App Spec file in your App’s dashboard under “Settings.” (It will be autogenerated when you create your project.)

In your GitHub Repository, create a .do directory, and make two copies of the App Spec file: app.yaml and default.yaml. These will be used by our GitHub Actions to edit the Digital Ocean Application.

app.yaml is what will be provisioned on a pull request being opened:

    alerts:
    - rule: DEPLOYMENT_FAILED
    - rule: DOMAIN_FAILED
    features:
    - buildpack-stack=ubuntu-22
    ingress:
      rules:
      - component:
          name: django-neon-quickstart
        match:
          path:
            prefix: /
    name: **seal-app-dev**
    region: nyc
    services:
    - build_command: |-
        pip install -r requirements.txt
        python manage.py makemigrations
        python manage.py migrate
      environment_slug: python
      envs:
      - key: PGHOST
        scope: RUN_AND_BUILD_TIME
        value: **new_host**
      - key: PGDATABASE
        scope: RUN_AND_BUILD_TIME
        value: neondb
      - key: PGUSER
        scope: RUN_AND_BUILD_TIME
        value: neondb_owner
      - key: PGPASSWORD
        scope: RUN_AND_BUILD_TIME
        value: **new_password**
      github:
        branch: **dev**
        deploy_on_push: true
        repo: dougsillars/django-neon-quickstart
      http_port: 8000
      instance_count: 1
      instance_size_slug: apps-s-1vcpu-1gb-fixed
      name: django-neon-quickstart
      run_command: gunicorn django_neon.wsgi:application --bind 0.0.0.0:8000
      source_dir: /
Enter fullscreen mode Exit fullscreen mode

There are four changes made in this file from the original at Digital Ocean:

  1. name: add “-dev” to the end of the app name.
  2. PGHOST: Value should be new_host
  3. PGPASSOWRD:value should be new_password
  4. github:branch dev replaces main.

When our GitHub Action runs:

  1. The name of the application will change in the Digital Ocean dashboard, allowing the dev to see that the current state is a dev state.
  2. new_host and new_password will be programmatically updated with the values from our newly created NeonDB branch.
  3. We want to deploy the dev branch of our code to Digital Ocean to see the changes.

default.yaml is used to revert the App Spec to production.

The default.yaml has two changes:

  1. name: Add “-prod” to the end of the name.
  2. PGPASSWORD:value: new_password replaces the password.

(We’re not showing the entire file here for space reasons.)

This will change the name in the Digital Ocean dashboard to show that prod is visible. The password will revert to the original password from the primary branch of the database, and since the branch is main, the build will be the main branch.

With these changes, we are now ready to begin implementing our two GitHub Actions: “Create NeonDB Branch” and “Destroy NeonDB Branch.”

Create NeonDB branch

When a pull request is made to the main branch, this GitHub Action will fire and do a number of steps:

  1. Create a development branch of the NeonDB database.
    • Grab the host and password of this new DB.
  2. Check out the GitHub Code.
  3. Use a sed command to replace new_host & new_password placeholders in the .do/app.yaml file with the variables extracted in step 1a.
  4. Install the Digital Ocean CLI.
  5. Update the App Spec with the new yaml file.
  6. Initiate a Digital Ocean deployment.
    name: Create Neon Branch and deploy dev to DO
    run-name: Create a Neon Branch 🚀
    on:
      pull_request:
        types: [opened]
        branches:
          - main
    jobs:
      Create-Neon-Branch:
        runs-on: ubuntu-latest
        steps:
          - name: Verify NEON API Key presence
            run: |
              if [ -z "${{ secrets.NEON_API_KEY }}" ]; then
                echo "NEON_API_KEY is empty"
              else
                echo "NEON_API_KEY is set"
              fi
          - name: Create Neon Branch
            id: create-branch
            uses: neondatabase/create-branch-action@v5
            with:
              project_id: "orange-violet-68318343"
              # optional (defaults to your primary  branch)
              parent: "main" 
              # optional (defaults to neondb)
              database: "neondb"
              branch_name: "development"
              username: "neondb_owner"
              api_key: "${{ secrets.NEON_API_KEY }}"
          - run: echo db_url ${{ steps.create-branch.outputs.db_url }}
          - run: echo host ${{ steps.create-branch.outputs.host }}
          - run: echo branch_id ${{ steps.create-branch.outputs.branch_id }}
          - name: Checkout code
            uses: actions/checkout@v2
          - name: Replace variables in YAML
            run: |
              sed -i 's|new_host|'"${{ steps.create-branch.outputs.host }}"'|g' .do/app.yaml
              sed -i 's|new_password|'"${{ steps.create-branch.outputs.password }}"'|g' .do/app.yaml
          - name: Install doctl
            uses: digitalocean/action-doctl@v2
            with:
              token: ${{ secrets.DO_KEY }}
          - name: Set environment variables
            run: |
              doctl auth init -t ${{ secrets.DO_KEY }}
              # Update the app with the new specifications from neon
              #  use active project id from DO url
              doctl apps update 3aec3cab-fca5-4829-b5f4-1fd9d41b16a9  --spec .do/app.yaml
              doctl apps create-deployment 3aec3cab-fca5-4829-b5f4-1fd9d41b16a9
Enter fullscreen mode Exit fullscreen mode

Hints on creating your action:

  • The NeonDB project_id is in the URL string when you load the project in the NeonDB dashboard.
  • The UUID for your Digital Ocean application is also found in the dashboard URL.

Save this workflow in /.github/workflows.

Delete NeonDB branch

Once the PR has been tested and approved, we want to destroy the NeonDB branch and revert the Digital Ocean deployment back to production.

The steps in this are:

  1. Delete the NeonDB development branch.
  2. Check out the code.
  3. Update the default.yaml with our DB password from the GitHub Secrets.
  4. Update the App Spec and deploy the application at Digital Ocean.
    name: Delete Neon Branch with GitHub Actions Demo
    run-name: Delete a Neon Branch 🚀
    on:
      pull_request:
        types: [closed]
        branches:
          - main
    jobs:
      delete-neon-branch:
        runs-on: ubuntu-latest
        steps:
          - name: Delete Neon branch
            uses: neondatabase/delete-branch-action@v3
            with:
              project_id: "orange-violet-68318343"
              branch: development
              api_key: ${{ secrets.NEON_API_KEY }}
          - name: Checkout code
            uses: actions/checkout@v2
          - name: Replace variables in YAML
            run: |
              sed -i 's|new_password|'"${{ secrets.NEON_PW }}"'|g' .do/default.yaml
          - name: Install doctl
            uses: digitalocean/action-doctl@v2
            with:
              token: ${{ secrets.DO_KEY }}
          - name: Set environment variables
            run: |
              doctl auth init -t ${{ secrets.DO_KEY }}
              # Update the app with the new specifications from neon
              #  use active project id from DO url
              doctl apps update 3aec3cab-fca5-4829-b5f4-1fd9d41b16a9  --spec .do/default.yaml
              doctl apps create-deployment 3aec3cab-fca5-4829-b5f4-1fd9d41b16a9
Enter fullscreen mode Exit fullscreen mode

Okay, that is a lot of code. Don’t forget to update the DO UUIDs to match your deployment. Push this all to your repo so that we can see our automation in action.

Here is the production version of the application running on Digital Ocean. I added a few extra elements for fun. The screenshot shows the mouse hover color on Ne (Neon).

Now, let’s make some changes to the code and start a pull request.

Create a dev branch for your code. Let’s change the colors in line 15 of/elements/templates/elements_list.html:

<li hx-delete="element/{{ element.id }}" hx-target="body" class="relative flex flex-col text-center p-5 rounded-md bg-[#7846a8] transition-colors hover:bg-orange-500 text-[white]">
Enter fullscreen mode Exit fullscreen mode

This should make the boxes purple, with orange hover and white text.

Push the dev branch and open a pull request.

When the pull request is created, the “Create a Neon Branch” GitHub Action is called. A branch of the NeonDB is created, and the dev code is deployed to Digital Ocean.

Refreshing the application, we see the colors have been updated:

More importantly, we can test all we want without worrying about the production database. Any changes made on the dev branch are in the development branch of NeonDB. As a part of our testing, we deleted a number of entries—only Neon is left:

Since we’re happy with the PR, we can approve and merge the changes. This fires up the second GitHub Action: deleting the NeonDB development branch and pushing the production build to Digital Ocean:

The new colors are in prod, and the prod database was untouched by our testing on the PR!

Conclusion

In this post, we used GitHub Actions to automate creating and deleting a NeonDB database for testing pull request builds on Digital Ocean. If you would like to look at the code, it is available on GitHub. You’ll just need to wire in your NeonDB and Digital Ocean credentials to get up and running.

Top comments (0)