Backup Your DEV.to Posts Using Python

I treat my posts on DEV.to as notes. Backing up my notes to prevent any unexpected incident is a must, even though I believe DEV.to will protect my posts unless it's us that makes the accident. So, why don't we back up them by ourselves?

Of course, we don't back up the posts by copying and pasting the markdown bodies manually into a file one by one. Let's automate them by using DEV.to API and Python.

The Goal

By requesting to https://dev.to/api/articles?username={username}, we can get the article list of the given username. We will parse the response and make the markdown files for each article. It's that simple.

The Packages We Need

For consuming the DEV.to API, we need the infamous requests package. So, add the package to the machine using pip or pipenv or anything that you use. We also need the built-in os and sys modules for working with directories and paths. Let's import them to our python file:

import requests
import os
import sys

The Parameters for the Request

By looking at the URL for request, we only need the username parameter. So, let's create a dict with that param:

params = {
    'username': 'dendihandian' # change it to your username
}

Perform the request

Let's request and get the articles with this script:

result = requests.get('https://dev.to/api/articles', params=params)
articles = result.json()

Apparently, the articles here don't have the body_markdown and we will get the attribute by requesting the full resources of each article later.

Making the directories for saving the markdown files

We will save the markdown files under this path ./dev.to/{username}/posts and we use both os and sys modules to make the directories first:

username = params['username']
posts_path = os.path.join(sys.path[0], 'dev.to', username, 'posts')
if not os.path.exists(posts_path):
    os.makedirs(posts_path)

We have to create the directories first because we can't directly set the full path to the file within the open() function if the directories not exist.

Parse the Articles and Make The Markdown Files

We loop the articles objects and request to get the body_markdown of each article and write them into a markdown file:

for article in articles:
    slug = article['slug']
    article_id = article['id']
    filename = f'{slug}.md'

    article_result = requests.get(f'https://dev.to/api/articles/{article_id}')
    article = article_result.json()

    f = open(os.path.join(posts_path, filename), "w+", encoding="utf-8")
    f.write(article['body_markdown'])
    f.close()

And that's it. Execute the python file and see if your posts are saved into markdown files.