Now we want to have our Django app up and running somewhere live for other people to see and use. In this version of our app we will use Google App engine.
First go to Google cloud and open your free account. Google will allocate some free resources automatically to your account and this will be more then enough for start. But also don't forget to go to billing part and to set some alerts when certain price thresholds are hit. Just to avoid any surprise.
Ones you are there search for App engine and create and select new MusicalNFT
project.
In short what is Google App Engine (GAE)? It is a fully managed, serverless platform for developing and hosting web applications at scale. It has a powerful built-in auto-scaling feature, which automatically allocates more/fewer resources based on demand. GAE natively supports applications written in Python, Node.js, Java, Ruby, C#, Go, and PHP. Alternatively, it provides support for other languages via custom runtimes or Dockerfiles. It has powerful application diagnostics, which you can combine with Cloud Monitoring and Logging to monitor the health and the performance of your app. Additionally, GAE allows your apps to scale to zero, which means that you don't pay anything if no one uses your service.
In attempt to use GAE first we need to install Google Cloud CLI.
The gcloud CLI allows you to create and manage your Google Cloud resources and services. The installation process differs depending on your operating system and processor architecture. Go ahead and follow the official installation guide for your OS and CPU. => gcloud
To verify the installation has been successful, run:
$ gcloud version
Google Cloud SDK 415.0.0
bq 2.0.84
core 2023.01.20
gcloud-crc32c 1.0.0
gsutil 5.18
Now comes very important step: configuring Django Project to work with GEA. Till this moment we were working all the time in local dev environment, running all things (Postgres, Stripe, Django server, Celery etc.) on our machine. What we need to do now if we want our app to be live on GEA? First we need to configure our Django project and to tell to GEA that he need to run all those supporting services in the background for our app to be able to run smoothly. That is why let's open our Django settings.py
First thing we need to do is to erase defualt secrete keys. Then we will generate new one and pass to our .env
file. And only then import inside our settings.py
again.
Let's generate new secrete Django key:
$py manage.py shell
>>>from django.core.management.utils import get_random_secret_key
>>>secret_key = get_random_secret_key()
>>>print(secret_key)
# and you will get some gibberish like this
>>>^&p@m*nhn#hg6ujgri2sppxr7t8o^mfp3bnj%1%2f72wcr+kkz
# now pass value you get into `.env` file name of varibale `DJANGO_SECRET_KEY`
Why we need secret key? In Django, a secret key plays a vital role in enhancing the security of our application. It helps manage user sessions, protects against Cross-Site Request Forgery (CSRF) attacks, and safeguards your data by generating and verifying cryptographic signatures among other things.
Now our Django settings should look something like this:
from pathlib import Path
import os
from urllib.parse import urlparse
import environ
import io
import pyrebase
import firebase_admin
from firebase_admin import credentials
from google.cloud import secretmanager
from google.oauth2 import service_account
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
env = environ.Env(DEBUG=(bool, False))
env_file = os.path.join(BASE_DIR, ".env")
if os.path.isfile(env_file):
# read a local .env file
env.read_env(env_file)
elif os.environ.get("GOOGLE_CLOUD_PROJECT", None):
# pull .env file from Secret Manager
project_id = os.environ.get("GOOGLE_CLOUD_PROJECT")
client = secretmanager.SecretManagerServiceClient()
settings_name = os.environ.get("SETTINGS_NAME", "django_setting_two")
name = f"projects/{project_id}/secrets/{settings_name}/versions/latest"
payload = client.access_secret_version(name=name).payload.data.decode("UTF-8")
env.read_env(io.StringIO(payload))
else:
raise Exception("No local .env or GOOGLE_CLOUD_PROJECT detected. No secrets found.")
SECRET_KEY = env("SECRET_KEY")
DEBUG = env("DEBUG")
Now set ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS, we can use the following code snippet from the GAE docs:
APPENGINE_URL = env("APPENGINE_URL", default=None)
if APPENGINE_URL:
# ensure a scheme is present in the URL before it's processed.
if not urlparse(APPENGINE_URL).scheme:
APPENGINE_URL = f"https://{APPENGINE_URL}"
ALLOWED_HOSTS = [
urlparse(APPENGINE_URL).netloc,
APPENGINE_URL,
"localhost",
"127.0.0.1",
]
CSRF_TRUSTED_ORIGINS = [APPENGINE_URL]
# SECURE_SSL_REDIRECT = True
else:
ALLOWED_HOSTS = ["*"]
This code fetches APPENGINE_URL from the environment (later we will add to our .env
) and automatically configures ALLOWED_HOSTS
and CSRF_TRUSTED_ORIGINS
. Additionally, it enables SECURE_SSL_REDIRECT
to enforce HTTPS.
Final version .env
file should look something like this (we still don't have all values but just for information)
STRIPE_SECRET_KEY=sk_test_xxxxxxxxx
DENIS_PASS=xxxxxxx
ETHEREUM_NETWORK=maticmum
INFURA_PROVIDER=https://polygon-mumbai.infura.io/v3/xxxxxx
SIGNER_PRIVATE_KEY=xxxxxxx
MUSIC_NFT_ADDRESS=0x1D33a553541606E98c74a61D1B8d9fff9E0fa138
STRIPE_ENDPOINT=whsec_GExxxxxxx
OWNER=0x273f4FCa831A7e154f8f979e1B06F4491Eb508B6
DJANGO_SECRET_KEY='^&p@m*nhn#hg6ujgri2sppxr7t8o^mfp3bnj%1%2f72wcr+kkz'
DATABASE_URL=postgres://name:password!@localhost/musicalnft
# DATABASE_URL=postgres://name:password!@//cloudsql/musicnft-405811:europe-west1:musicnft-instance/musicalnft
GS_BUCKET_NAME=musicnft-bucket
APPENGINE_URL=https://musicnft-405811.ew.r.appspot.com/
later on we will populate all those values.
Don't forget to add the import at the top of the settings.py
:
from urllib.parse import urlparse
To use Postgres instead of SQLite, we first need to install the database adapter.
$pip install psycopg2-binary==2.9.5
$pip freeze > requirements.txt
To utilize DATABASE_URL with Django, we can use django-environ's db() method like so:
#settings.py
DATABASES = {'default': env.db()}
Now create DATABASE_URL
in .env
and pass some random value, later on we will set up this value properly. (in general format of this value will go as fallow:
postgres://USER:PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_NAME/DATABASE_NAME)
What we want to do is to replace Django dev server (what is not recommended for any kind of production environment) with Gunicorn. But first what is Gunicorn and why we should want to replace Django default server? The Django development server is lightweight and easy to use, but it's not designed/reccomndent to handle production traffic. It's meant for use during development only. On the other hand, Gunicorn is a WSGI HTTP server that's designed for production use. It's robust, efficient, and can handle multiple requests simultaneously, which is crucial for a production environment.
$pip install gunicorn==20.1.0
$pip freeze > requirements.txt
Now very important app.yaml
file. Google App Engine's app.yaml
config file is used to configure your web application's runtime environment. The app.yaml
file contains information such as the runtime, URL handlers, and environment variables.
Start by creating a new file called app.yaml
in the project root with the following contents:
# app.yaml
runtime: python39
env: standard
entrypoint: gunicorn -b :$PORT musical_nft.wsgi:application
handlers:
- url: /.*
script: auto
runtime_config:
python_version: 3
We defined the entrypoint command that starts the WSGI server.
There are two options for env: standard and flexible. We picked standard since it is easier to get up and running, is appropriate for smaller apps, and supports Python 3.9 out of the box.
Lastly, handlers define how different URLs are routed. We'll define handlers for static and media files later in the tutorial.
Now let's define .gcloudignore
. A .gcloudignore
file allows you to specify the files you don't want to upload to GAE when deploying an application. It works similarly to a .gitignore file.
Go ahead and create a .gcloudignore
file in the project root with the following contents:
# .gcloudignore
.gcloudignore
# Ignore local .env file
.env
# If you would like to upload your .git directory, .gitignore file, or files
# from your .gitignore file, remove the corresponding line
# below:
.git
.gitignore
# Python pycache:
__pycache__/
# Ignore collected static and media files
mediafiles/
staticfiles/
# Ignore the local DB
db.sqlite3
# Ignored by the build system
/setup.cfg
venv/
# Ignore IDE files
.idea/
README_DEV.md
celerybeat-schedule
node_modules
photos
smart-contracts
Go ahead and initialize the gcloud CLI if you haven't already:
$ gcloud init
gcloud CLI will ask you about mail and project you would like to associate with this project. It will offer to you all avaliable projects. Choose the one we created (MusciNFT) and gcloud will automatically set all the rest. What means from this point on when ever you deploy your Django app to Google App Engine it will be avalible under that project name.
To create an App Engine app go to your project root and run:
$gcloud app create
# you will get something like this
You are creating an app for project [musicnft-405811].
WARNING: Creating an App Engine application for a project is irreversible and the region
cannot be changed. More information about regions is at
<https://cloud.google.com/appengine/docs/locations>.
Please choose the region where you want your App Engine application located:
[1] asia-east1 (supports standard and flexible)
[2] asia-east2 (supports standard and flexible and search_api)
[3] asia-northeast1 (supports standard and flexible and search_api)
[4] asia-northeast2 (supports standard and flexible and search_api)
[5] asia-northeast3 (supports standard and flexible and search_api)
[6] asia-south1 (supports standard and flexible and search_api)
[7] asia-southeast1 (supports standard and flexible)
[8] asia-southeast2 (supports standard and flexible and search_api)
[9] australia-southeast1 (supports standard and flexible and search_api)
[10] europe-central2 (supports standard and flexible)
[11] europe-west (supports standard and flexible and search_api)
[12] europe-west2 (supports standard and flexible and search_api)
[13] europe-west3 (supports standard and flexible and search_api)
[14] europe-west6 (supports standard and flexible and search_api)
[15] northamerica-northeast1 (supports standard and flexible and search_api)
[16] southamerica-east1 (supports standard and flexible and search_api)
[17] us-central (supports standard and flexible and search_api)
[18] us-east1 (supports standard and flexible and search_api)
[19] us-east4 (supports standard and flexible and search_api)
[20] us-west1 (supports standard and flexible)
[21] us-west2 (supports standard and flexible and search_api)
[22] us-west3 (supports standard and flexible and search_api)
[23] us-west4 (supports standard and flexible and search_api)
[24] cancel
Please enter your numeric choice: 11
Creating App Engine application in project [musicnft-405811] and region [europe-west]....done.
Success! The app is now created. Please use `gcloud app deploy` to deploy your first app.
Ok, now we need to set up our Postgres database inside Cloud SQL dashboard: https://console.cloud.google.com/sql/
Ones you are there create new instance
and choose PostgresSQL (enable Compute Engine API if needed).
Now pass following values:
Instance ID: musicnft-instance
Password: Enter a custom password or generate it
Database version: PostgreSQL 14
Configuration: Up to you
Region: The same region as your app
Zonal availability: Up to you
You might also need to enable "Compute Engine API" to create a SQL instance.
Once the database has been provisioned, you should get redirected to the database details. Take note of the "Connection name".
Go ahead and enable the Cloud SQL Admin API by searching for "Cloud SQL Admin API" and clicking "Enable". We'll need this enabled to test the database connection. Here is a link (it will route you to your project): https://console.cloud.google.com/marketplace/product/google/sqladmin.googleapis.com
To test the database connection and migrate the database we'll use Cloud SQL Auth proxy. The Cloud SQL Auth proxy provides secure access to your Cloud SQL instance without the need for authorized networks or for configuring SSL.
First, authenticate and acquire credentials for the API:
$gcloud auth application-default login
Next, download Cloud SQL Auth Proxy and make it executable:
wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy
--2023-11-21 00:27:00-- https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64
Resolving dl.google.com (dl.google.com)... 172.217.20.78, 2a00:1450:4017:800::200e
Connecting to dl.google.com (dl.google.com)|172.217.20.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 19239740 (18M) [application/octet-stream]
Saving to: ‘cloud_sql_proxy’
cloud_sql_proxy 100%[===============================================>] 18.35M 1.22MB/s in 13s
2023-11-21 00:27:13 (1.45 MB/s) - ‘cloud_sql_proxy’ saved [19239740/19239740]
$chmod +x cloud_sql_proxy
After the installation is complete open a new terminal window and start the proxy with your connection details like so:
$./cloud_sql_proxy -instances="PROJECT_ID:REGION:INSTANCE_NAME"=tcp:5432
Where basicaly PROJECT_ID:REGION:INSTANCE_NAME
connection name you saved first time you created database
And you should see something like this:
2023/11/21 00:31:03 current FDs rlimit set to 1048576, wanted limit is 8500. Nothing to do here.
2023/11/21 00:31:04 Listening on 127.0.0.1:5432 for musicnft-405811:europe-west1:musicnft-instance
2023/11/21 00:31:04 Ready for new connections
2023/11/21 00:31:05 Generated RSA key in 135.901349ms
You can now connect to localhost:5432 the same way you would if you had Postgres running on your local machine.
Since GAE doesn't allow us to execute commands on the server, we'll have to migrate the database from our local machine.
Inside our .env
file in the project root, with the required environment variables:
DATABASE_URL=postgres://DB_USER:DB_PASS@localhost/DB_NAME
# Example `DATABASE_URL`:
# DATABASE_URL=postgres://django-images:password@localhost/mydb
Now let's make migrations
$python manage.py migrate
Create superuser
$python manage.py createsuperuser
Ones you done with superuser
you can move to secret manager
. We used to have local .env
file when we worked in local dev context. But because we are migrating now whole app into Google App Engine, we need to make our .env
varibales present in cloud. And for this we will use Google secret manager.
Navigate to the Secret Manager dashboard (https://console.cloud.google.com/security/secret-manager?project=musicnft-405811) and enable the API if you haven't already. Next, create a secret named django_settings
with the following content:
DJANGO_SECRET_KEY='^&p@m*nhn#hg6ujgri2sppxr7t8o^mfp3bnj%1%2f72wcr+kkz'
# local development
# DATABASE_URL=postgres://ilija:Meripseli1986!@localhost/musicalnft
# in cloud
DATABASE_URL=postgres://ilija:Meripseli1986!@//cloudsql/musicnft-405811:europe-west1:musicnft-instance/musicalnft
GS_BUCKET_NAME=django-music-nft
# Example `DATABASE_URL`:
DATABASE_URL=postgres://DB_USER:DB_PASS@//cloudsql/PROJECT_ID:REGION:INSTANCE_NAME/DB_NAME
# postgres://django-images:password@//cloudsql/indigo-35:europe-west3:mydb-instance/mydb
GS_BUCKET_NAME=django-images-bucket
Make sure to change DATABASE_URL accordingly. PROJECT_ID:REGION:INSTANCE_NAME equals your database connection details.
You don't have to worry about GS_BUCKET_NAME. This is just the name of a bucket we're going to create and use later.
Pip installl google-cloud-secret-manager==2.15.1
$pip install google-cloud-secret-manager==2.15.1
$pip freeze > requirements.txt
To load the environment variables from Secret Manager we can use the following official code snippet:
from pathlib import Path
import os
import environ
from urllib.parse import urlparse
import io # new
from google.cloud import secretmanager # new
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
env = environ.Env(DEBUG=(bool, False))
env_file = os.path.join(BASE_DIR, ".env")
if os.path.isfile(env_file):
# read a local .env file
env.read_env(env_file)
password = env("DENIS_PASS")
elif os.environ.get('GOOGLE_CLOUD_PROJECT', None):
# pull .env file from Secret Manager
project_id = os.environ.get('GOOGLE_CLOUD_PROJECT')
client = secretmanager.SecretManagerServiceClient()
settings_name = os.environ.get('SETTINGS_NAME', 'django_settings')
name = f'projects/{project_id}/secrets/{settings_name}/versions/latest'
payload = client.access_secret_version(name=name).payload.data.decode('UTF-8')
env.read_env(io.StringIO(payload))
else:
raise Exception('No local .env or GOOGLE_CLOUD_PROJECT detected. No secrets found.')
There is two new imports: io and secretmanager
Great! It's finally time to deploy our app. To do so, run:
$ gcloud app deploy
Services to deploy:
descriptor: [C:\Users\Nik\PycharmProjects\django-images-new\app.yaml]
source: [C:\Users\Nik\PycharmProjects\django-images-new]
target project: [indigo-griffin-376011]
target service: [default]
target version: [20230130t135926]
target url: [https://indigo-griffin-376011.ey.r.appspot.com]
Do you want to continue (Y/n)? y
Beginning deployment of service [default]...
#============================================================#
#= Uploading 21 files to Google Cloud Storage =#
#============================================================#
File upload done.
Updating service [default]...done.
Setting traffic split for service [default]...done.
Deployed service [default] to [https://indigo-griffin-376011.ey.r.appspot.com]
You can stream logs from the command line by running:
$ gcloud app logs tail -s default
Open your web app in your browser and test if it works:
$ gcloud app browse
If you get a 502 Bad Gateway error, you can navigate to Logs Explorer to see your logs.
If there's a 403 Permission 'secretmanager.versions.access' denied error, navigate to django_settings secret permissions and make sure the default App Engine service account has access to this secret. See solution on StackOverflow
And that is it! We have now our toy app fully up and running on Google app engine for world to see
Code can be found in this repo
p.s. whole this process is defined by great crew at testdrive.io High recommendation for all their writings!
Top comments (0)