Background
In this post, I will be demonstrating how to deploy a Django Rest Framework (DRF) application on fly.io. DRF is built on top of Django and is my choice when it comes to building small to massive APIs. I have used it extensively professionally and personally, and as a cybersecurity researcher, it's been my favorite to build as a backend for some complex applications that track malicious actors and their infrastructure. I noticed the fly website did not have a Django example, so I wanted to provide the community with a template to get started to avoid the headaches that I endured building this :)
You should have an understanding of Docker, docker compose, DRF and postgres for this post. Although it's possible to use this template without too much knowledge of these concepts, you'd get a ton of knowledge going through the following tutorials:
- https://www.django-rest-framework.org/tutorial/quickstart/
- https://docs.docker.com/samples/django/
- https://fly.io/docs/getting-started/python/
Why DRF?
From the DRF website:
Some reasons you might want to use REST framework:
- The Web browsable API is a huge usability win for your developers.
- Authentication policies including packages for OAuth1a and OAuth2.
- Serialization that supports both ORM and non-ORM data sources.
- Customizable all the way down - just use regular function-based views if you don't need the more powerful features.
- Extensive documentation, and great community support.
- Used and trusted by internationally recognised companies including Mozilla, Red Hat, Heroku, and Eventbrite.
The biggest issue with DRF is that it can be somewhat challenging to get up and running (can be lots of up front work). My experience has been that once you get a good template for an API up, it's the fastest to build, most scalable and most intuitive ORM.
Why fly.io?
I've used Heroku for one-off apps for a few years. Fly came across my Twitter feed and I've been following it closely. The company has a great section on why you should use fly, and this section caught my eye and has kept me interested ever since I read it:
Despite the benefits of location-smart, time-agile and cloud-clever applications, thereβs been no good platform for building applications that work like this. This is what Fly has set out to fix. In the process we want to make application distribution platforms as ubiquitous as CDNs.
You can think of fly as a Heroku competitor, although some folks might disagree with me. I like it because it does what it says it does well, is focused, and isn't as bloated as the Heroku stack.
Getting started
Bookkeeping
We'll be making a DRF app, the Silly Simple API, or ss-api
for short, on fly. This will have the following features:
- Postgres backend, courtesy of fly
- No session authentication, only using Tokens (you can use some tricks from
fly
to manage this). This is especially nice for APIs and simplifies authentication to a token, which I prefer for microservices - Swagger and OpenAPI capabilities using drf-yasg, where you can only see endpoints and Swagger docs if you have a valid Token
- TCP & HTTP health checks using fly. The HTTP health check will be somewhat useful by issuing a query to our DRF app under
/ping/
, which connects to the DB and issues an innocuousselect 1
statement to make sure things are working - Using
docker
with aDockerfile
andgunicorn
to launch the app. The cool thing about fly is that you can give it aDockerfile
and afly.toml
and you have a full-fledged app running on their infrastructure
This app will only have 1 endpoint, users
, that you can use to manage your users. You must be authenticated to see it.
I will leave the following for later blog posts:
- Metrics exposure and app tracing via Datadog (p.s., we're hiring https://www.datadoghq.com/careers/)
- Tables for an app (this isn't a tutorial on building DRF apps, rather, to get you a template to get started)
- Scaling primitives in DRF (avoiding n+1, indexing, replicas for postgres)
If this all sounds interesting for you still, let's get started :D
Clone and run locally
Make sure to have the following installed locally:
- Latest Docker (with
docker compose
) git
- fly via here
then clone from the repo here:
git clone git@github.com:zmallen/ss-api.git
Run docker compose up
and connect locally by navigating to:
http://localhost:8000
The docker-compose.yml
file overrides the RUN
command in the Dockerfile
by issuing the following command on every docker compose up
:
command: bash -c "python manage.py migrate && python manage.py runserver 0.0.0.0:8000"
This will differ from when we deploy on fly.io, where we use gunicorn
to serve the app and we run migrations manually.
Token Auth & Authentication in DRF
Unauthenticated users can only see the /ping/
endpoint on Swagger. This is by design - the app will render endpoints based on permission, and under /ping/views.py
on Line 16, the permission for this endpoint is:
permission_classes = (AllowAny,)
Compare this to ssapi/views.py
, under the UserViewSet
on Line 17:
authentication_classes = (authentication.TokenAuthentication,)
This is achieved via some magic in settings.py
Lines 88-111:
# DRF settings
REST_FRAMEWORK = {
"DEFAULT_PERMISSION_CLASSES": ("rest_framework.permissions.IsAuthenticated",),
"DEFAULT_AUTHENTICATION_CLASSES": (
"rest_framework.authentication.TokenAuthentication",
),
"DEFAULT_RENDERER_CLASSES": ("rest_framework.renderers.JSONRenderer",),
}
# SWAGGER_SETTINGS
SWAGGER_SETTINGS = {
"USE_SESSION_AUTH": False,
"LOGIN_URL": "rest_framework:login",
"LOGOUT_URL": "rest_framework:logout",
"VALIDATOR_URL": None,
"SECURITY_DEFINITIONS": {
"api_key": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
},
},
"REFETCH_SCHEMA_WITH_AUTH": True,
}
Under DRF_SETTINGS
, I forced TokenAuthentication
and isAuthenticated
for viewing endpoints, so no more sessions! ππ
So how do you get an API token if you can't authenticate? This is where some magic with manage.py
comes into play.
Authenticating - getdevtoken
and /users/
To get a local API key, run the following command in a separate tab, in the same directory as docker-compose.yml
:
β> docker compose run api python manage.py getdevtoken
[+] Running 1/0
β Ώ Container ss-api-db-1 Running 0.0s
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(qββΏβΏβq)ββ Token 9884a551be31b80a61b49becf7c3640224a9ec42 ββ(qββΏβΏβq)
- Note, this Token is for my local deployment :)
Copy the Token abc
and navigate over to your browser, then click 'Authorize', paste and press submit. You should get the /users/
endpoint to return in the Swagger frontend, and issuing a GET request will list the superuser
!
You can do a lot more with Swagger documentation than just the defaults, I suggest checking out these resources to learn about Swagger docs in Django:
- https://django-rest-swagger.readthedocs.io/en/stable-0.3.x/examples.html
- https://swagger.io/tools/open-source/
Deploying to fly
Everything is running smoothly in your local environment, now let's get it to a prod environment!
First, we need to create a toml
file for fly. This is a configuration file used by fly to deploy your app. I generated one within the Github repo, but you can explore how to generate and configure other toml
files on fly here.
A few things need to happen to finish our "deploy to prod" for ss-api:
- Create a fly app
- Create a postgres db with fly, and retrieve the
DATABASE_URL
string - Set a
fly secret
with theDATABASE_URL
from Step 1 so our app can dynamically render the secret in a fly environment and use the db created in step 1 - Deploy the app on fly, make sure TCP & HTTP health checks pass (they wont on first pass :D)
- Create a database and run a migration using
fly ssh
- Get a
devtoken
for prod
Step 1: Create your fly app
Simply run fly create
and name your app!
β> fly create
? App Name: ssapiblog
automatically selected personal organization: Zack Allen
New app created: ssapiblog
Step 2: postgres
Launch a new postgres instance via fly
with fly postgres create
. Accept all the defaults (minimal DB settings, aka the cheapest!) and wait for fly to give you the DATABASE_URL
.
β> fly postgres create
? App Name: ssapidb
Automatically selected personal organization: Zack Allen
? Select region: iad (Ashburn, Virginia (US))
? Select VM size: shared-cpu-1x - 256
? Volume size (GB): 10
Creating postgres cluster ssapidb in organization personal
Postgres cluster ssapidb created
Username: postgres
Password: SECRETPASSWORD
Hostname: ssapidb.internal
Proxy Port: 5432
PG Port: 5433
Save your credentials in a secure place, you won't be able to see them again!
Monitoring Deployment
...
...
2 desired, 2 placed, 2 healthy, 0 unhealthy [health checks: 6 total, 6 passing]
--> v0 deployed successfully
Connect to postgres
Any app within the personal organization can connect to postgres using the above credentials and the hostname "ssapidb.internal."
For example: postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432
See the postgres docs for more information on next steps, managing postgres, connecting from outside fly: https://fly.io/docs/reference/postgres/
You want the 12factor string after For example:
, which in this example is:
postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432
Step 3: Set DATABASE_URL as a fly secret
You want to set the DATABASE_URL with a 12factor string from before, as well as a database name (which we will create).
Note the /ssapidb
at the end of the DATABASE_URL
fly secrets set DATABASE_URL="postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432/ssapidb"
Step 4: Deploy your app
Change the following line in fly.toml
to whatever you want:
app = "ssapiblog"
Run fly deploy
:
β> fly deploy
Deploying ssapiblog
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 β’ 8000
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.8 linux x86_64
Sending build context to Docker daemon 153.1kB
...
==> A bunch of Docker output
You can detach the terminal anytime without stopping the deployment
Monitoring Deployment
v0 is being deployed
2021-12-30T22:26:21.000 [info] 145.40.89.203 - - [30/Dec/2021:22:26:21 +0000] "GET /ping/ HTTP/1.1" 500 114326 "http://172.19.10.66:8000/ping" "Consul Health Check"
1 desired, 1 placed, 0 healthy, 0 unhealthy [health checks: 2 total, 1 passing, 1 critical]
Step 5: Build a DB then migrate
Notice how 1 health check is passing (tcp), and 1 is critical. This is because we did not do a database migration and our /ping
healthcheck is failing. This will most likely fail after a certain amount of time, so in a separate tab navigate to the project directory to run a few fly ssh
commands.
First, make the ssapidb
database by running a handy dandy bash script I added into the repo. We can use this via fly ssh console -C
command:
β> fly ssh console -C 'bash /app/provision_db.sh'
Connecting to ssapiblog.internal... complete
Database does not exist. Creating now..
CREATE DATABASE
If you run fly logs
in a separate tab, you should see Consul health checks returning 200, which is healthy \o/:
2021-12-30T22:31:27.116 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping HTTP/1.1" 301 0 "-" "Consul Health Check"
2021-12-30T22:31:27.119 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping/ HTTP/1.1" 200 2 "http://172.19.10.66:8000/ping" "Consul Health Check"
Let's migrate and get a devtoken:
β> fly ssh console -C 'python /app/manage.py migrate'
Connecting to ssapiblog.internal... complete
Operations to perform:
Apply all migrations: admin, auth, authtoken, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying authtoken.0001_initial... OK
Applying authtoken.0002_auto_20160226_1747... OK
Applying authtoken.0003_tokenproxy... OK
Applying sessions.0001_initial... OK
Step 6: Get your devtoken and open the app
β> fly ssh console -C 'python /app/manage.py getdevtoken'
Connecting to ssapiblog.internal... complete
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(qββΏβΏβq)ββ Token TOKEN ββ(qββΏβΏβq)
Woot! Run fly open
and go through the same workflow as your local deployment: put Token TOKEN
into the Authorize panel, and you can now see the authenticated users endpoint, and issue a GET request to get your token!
If you want to add more users, just use the POST
request endpoint here to create a new user.
Conclusion
I enjoyed writing this app and this blog post! Fly is a cool concept and I will definitely play with it more. There are some sharp edges with DRF, so I tried to simplify it, but please study the DRF tutorials and the api/settings.py
file for other configuration options I used.
For my next posts, Im looking to develop an app to do some basic cybersecurity threat intelligence tracking and correlation. If you have ideas for other apps, or have a question on this app, please leave a comment or open an issue on the ss-api
repo here:
Top comments (1)
Does fly.io requires a credit card for creating a database?