Django and HTMX

I have been using Django for multiple years now at work and we have used it to build up a website that gives insight into much of the data that we store in our database. It allows users to interact with that data and make minor changes.

We do not need a large interactive application for that and have been happy with just reloading the page whenever we needed to make a request to the server. For our applications, this was fine and using one of the frontend frameworks to increase the interactivity of our website would have meant a large amount of necessary initial development and ongoing maintenance.

Then I found HTMX on Twitter and Github and I instantly fell in love. It promises multiple things:

• Simplicity: use the things you already use and only as much Javascript as you want.
• Ease of use: stay with your HTML templates generated on the server. No need for JSX / frontend templates.
• Development speed: quickly build up interactive solutions and improve the UX massively.
• Learning speed: new developers quickly grasp the main concepts and are able to maintain existing solutions and build new ones.

In the following article I want to describe our HTMX + Django setup. We will talk about the following:

• Using the django-htmx package.
• Setting the CSRF token as part of your HTMX requests.
• Easily re-using parts of your Jinja2 templates.
• Updating CSRF tokens after the user was forced to re-login.

Package: django-htmx

There is a great package called django-htmx. The most important things it provides are:

• A middleware that will allow you to check if a request was triggered by HTMX via request.htmx and retrieve other parameters from the request.
• The docs of the package also provide some tips for how to use HTMX with Django.
• There are some convenient functions to return certain HTTP status codes or headers that will make HTMX do certain things. Read more on the HTTP page.

Note that the tips page describes how you can make your life easier if you often want to render a part of your page again using HTMX. This can be useful for filtering lists, validating forms, and showing content that is polled from a database in regular intervals.

But if you want to render multiple parts of your page again, these tips will not help you. I will show you a better way below.

CSRF Token

If you enable Django's CSRF protection, you will need to set the CSRF token as part of any HTMX request. This can easily be done with the following code that you place inside of your template.

let CSRF_TOKEN = '{{ csrf_token }}';
document.body.addEventListener('htmx:configRequest', (event) => {
    event.detail.headers['X-CSRFToken'] = CSRF_TOKEN;
})

This way, every HTMX request will have the X-CSRFToken header set.

Rendering parts of your template

Let's say you have a large page with multiple areas that can be dynamically changed and will be reloaded using HTMX. (This could be that you have multiple forms on the page or you have multiple lists where you want to offer dynamic filtering without reloading the page.)

Now, what you would have to do is put every such area that is reloaded in their own template and then re-use these templates in the larger template for that page. This gets annoying pretty fast as your have to split your templates in many different files.

Instead, we will show you how you can keep your whole page template in one file and still reuse this template.

There is a great repository that contains a lot of information about using Django + HTMX. One idea in particular is very interesting for us and it is called: inline partials.

For now, we will assume you are using Jinja2 as your template engine. You can make the following also work with the normal Django template engine but we will focus on the Jinja2 solution. If you are already using Jinja2 as a template engine for Django, there are some solutions that offer template fragment features:

• https://pypi.org/project/jinja2-fragments/0.1.0/
• https://github.com/mikeckennedy/jinja_partials

Both of them do not work well with the way Jinja2 is integrated into Django. They have the following drawbacks:

• You are giving up on the Jinja2 environment that Django configures for you.
• You cannot use TemplateResponse() any longer.
• You need to adjust some of your views quite heavily to make use of the new render logic.

We have a better solution: write your own Django template backend. It is less than 50 lines of code.

To make working with Django + Jinja2 + Fragments easier we have written a custom template backend that is heavily inspired by the default Django Jinja2 backend.

It mainly relies on the fact that you can render blocks in the same way you would render a full template in Jinja as the blocks in template.blocks are render functions that take a context as an input.

import jinja2
from django.template import TemplateDoesNotExist, TemplateSyntaxError
from django.template.backends.jinja2 import Jinja2, Template, get_exception_info


class Jinja2WithFragments(Jinja2):
    def from_string(self, template_code):
        return FragmentTemplate(self.env.from_string(template_code), self)

    def get_template(self, template_name):
        try:
            return FragmentTemplate(self.env.get_template(template_name), self)
        except jinja2.TemplateNotFound as exc:
            raise TemplateDoesNotExist(exc.name, backend=self) from exc
        except jinja2.TemplateSyntaxError as exc:
            new = TemplateSyntaxError(exc.args)
            new.template_debug = get_exception_info(exc)
            raise new from exc


class FragmentTemplate(Template):
    """Extend the original jinja2 template so that it supports fragments."""

    def render(self, context=None, request=None):
        from django.template.backends.utils import csrf_input_lazy, csrf_token_lazy

        if context is None:
            context = {}
        if request is not None:
            context["request"] = request
            context["csrf_input"] = csrf_input_lazy(request)
            context["csrf_token"] = csrf_token_lazy(request)

            for context_processor in self.backend.template_context_processors:
                context.update(context_processor(request))

        try:
            if "RENDER_BLOCKS" in context:
                bctx = self.template.new_context(context)
                blocks = []
                for bn in context["RENDER_BLOCKS"]:
                    blocks.extend(self.template.blocks[bn](bctx))
                return "".join(blocks)
            return self.template.render(context)
        except jinja2.TemplateSyntaxError as exc:
            new = TemplateSyntaxError(exc.args)
            new.template_debug = get_exception_info(exc)
            raise new from exc

You need to configure your Django settings to use this new template engine. Create a jinja2.py file inside your your_app folder and place the code from above in this file. Also make sure that your environment is also in this file or that you adjust the path to your environment.

TEMPLATES = [
    # ...
    {
        "BACKEND": "your_app.jinja2_backend.Jinja2WithFragments",
        "DIRS": [os.path.join(PROJECT_DIR, "jinja2")],
        "APP_DIRS": True,
        "OPTIONS": {"environment": "your_app.jinja2.environment"},
    },
]

If you define any block in your templates:

{% extends "base.html" %}

{% block body %}
    <h1>List of monsters</h1>

    {% if page_obj.paginator.count == 0 %}
        <p>We have no monsters at all!</p>
    {% else %}
        {% block page-and-paging-controls %}
            {% for monster in page_obj %}
                <p class="card">{{ monster.name }}</p>
            {% endfor %}

            {% if page_obj.has_next %}
                <p id="paging-area">
                    <a href="#"
                        hx-get="?page={{ page_obj.next_page_number }}"
                        hx-target="#paging-area"
                        hx-swap="outerHTML">Load more</a>
                </p>
            {% else %}
                <p>That's all of them!</p>
            {% endif %}
        {% endblock %}
    {% endif %}
{% endblock %}

You can now choose to render only a certain block quite easily via:

def paging_with_inline_partials(request):
   template_name = "paging_with_inline_partials.html"
   context = {
       "page_obj": get_page_by_request(request, Monster.objects.all()),
   }

   if request.headers.get("Hx-Request", False): # or use: request.htmx
       context["RENDER_BLOCKS"] = ["page-and-paging-controls"]

   return TemplateResponse(request, template_name, context)

In theory, you could also render multiple blocks at the same time even though we do not yet see the usecase for this.

Our template backend will look for the key RENDER_BLOCKS inside the context and if it is available, it will switch to rendering only the blocks that are specified in the variable.

Advanced: Refreshing the CSRF Token after login

We have the following situation:

• For different reasons, the time after a user is logged out is quite short (approximately one hour).
• So, users could open a page, fill out a form, do something else and come back to the form only to submit and find that they have been logged out.
• This is frustrating for the users because they lose the contents of their form.

One obvious way to prevent this is to make every endpoint that a form send data to not reject the request outright or send the user to the login form. We need to cache the form input, send the user to the login, and then use the form input from the cache. This is a lot of difficult work.

We think, our solution is easier to execute and does not require every form to follow certain rules. We do the following:

We show the user the login state prominently on the page and refresh this state in the background (every 60s and when the user comes back to the tab).

<div
    hx-get="{{ url('auth:login_status') }}"
    hx-trigger="load, every 60s, visibilitychange[document.visibilityState === 'visible'] from:document"
    hx-target="find span"
    hx-indicator=".login-status"
    class="login-status"
>
    <span></span><i class="htmx-indicator fa fa-spinner fa-spin"></i>
</div>

The view for the auto:login_status request looks like this:

def get_login_status(request):
    return TemplateResponse(
        request,
        "auth/login_status.html.j2",
        {
            "is_authenticated": request.user.is_authenticated,
            "login_url": your_function_to_get_login_url(request),
        },
    )

And finally, the template for this view looks like this:

{% if is_authenticated %}
    You are logged in. <i class="far fa-check-circle text-success-emphasis" title="Logged in."></i>
{% else %}
    You have been logged out. <i class="far fa-times-circle text-danger-emphasis" title="Your login has expired."></i> Please <a href="{{ login_url }}" target="_blank">login</a> again.
{% endif %}

<script type="text/javascript">
CSRF_TOKEN = '{{ csrf_token }}';
document.querySelectorAll("[name='csrfmiddlewaretoken']").forEach((elem) => {
    elem.setAttribute("value", CSRF_TOKEN);
});
</script>

This way, we send back the current CSRF token every time and update not only the CSRF token that is used for every HTMX request but also the token that is used whenever a form is sent.

Also, every time the status requests is sent, we check if the user is authenticated. If he is not authenticated, we send back a link to the login page that is opened in a new tab. The user can then login there and come back to the old tab. There, we will (because the visibilitychange event is triggered) send another login status request, update the CSRF token, and also update the information about the user's login state. The user will then be able to submit the form he was working on as if he reloaded the page and we saved his form inputs.