DEV Community

Igor Irianto
Igor Irianto

Posted on • Edited on • Originally published at irian.to

How to download all your DEV articles in markdown format

Update: Looks like the dev API has changed. I added a new params option to get max 1000 articles.

We can agree that DEV is the best site for devs in the world. What if you want to cross-post your dev articles into your own personal blog site, but too lazy to copy-paste each post? 😴

I used to copy paste every single article I wrote into my personal site, but quickly found it tedious. I created a script to easily generate DEV articles.

There are two ways to do this: work from the script below or clone the Github repo.

The script

Here is main script. It is pretty short. It uses axios to fetch the data and dotenv to store secrets.

require('dotenv').config()
const axios = require('axios')
const fs = require('fs')
const path = require('path')
const URL = 'https://dev.to/api/articles/me/published'

const options = {
  headers: { 'api-key': process.env.DEV_KEY },
  params: { 'per_page' : 1000 }
}

const contentDir = '/content/'
const contentPath = __dirname + contentDir

axios
  .get(URL, options)
  .then(response => {
    response.data.forEach(article => {
      const normalizedTitle = article.title
        .replace(/\s+/g, '-')
        .replace(/[^a-zA-Z0-9-]/g, '')
        .toLowerCase()
      const normalizedPublishedAt = article.published_at.split('T')[0]
      const fileName = `${normalizedPublishedAt}_${normalizedTitle}.md`

      console.log(`${contentDir}${fileName} created!`)
      fs.writeFileSync(
        `${contentPath}${fileName}`,
        article.body_markdown,
        'utf-8'
      )
    })
    console.log('*** dev.to data successfully fetched ***')
  })
  .catch(error => {
    console.log('ERROR: ', error)
  })

Enter fullscreen mode Exit fullscreen mode

DEV API makes it very easy to do it. The response of the API request returns an array of all articles you've written.

I extracted the markdown content, renamed the file with YYYY-MM-DD_your-article-title.md (I also removed the unique ID at the end, ex: .../your-article-title-1a2b).

The repo

I also created a github repo:

https://github.com/iggredible/dev-fetch-articles

Just clone the repo, install all dependencies, and create content/ directory in project root. Don't forget to give it your .env information!

git clone https://github.com/iggredible/dev-fetch-articles.git
cd dev-fetch-articles
npm install
mkdir content
mkdir .env
// create a key called DEV_KEY and give it your dev key values from dev.to site
Enter fullscreen mode Exit fullscreen mode

You can find your DEV_KEY secret inside Settings -> Account on dev.to site.

Once done, just run npm run fetch-dev. You will find all your articles inside content/ folder you just created. You can use this script into your app or use this script to generate content that you can copy to your app.

Making it better

If you see how the code can be improved, feel free to submit PR on Github or drop in the comment below.

Thanks for reading and happy blogging! πŸ“–πŸ”₯

Top comments (9)

Collapse
 
xanderyzwich profile image
Corey McCarty • Edited

I'm currently using the 11ty static site generator blog and publishing to a personal domain (from markdown) and dev.to listens to the rss to pick up articles automatically. All I have to do is change published value in the front matter to true and it publishes.

Collapse
 
iggredible profile image
Igor Irianto

Oh, I've heard of Eleventy last year, I think. That's a cool strategy. I will have to take a look at the rss thing!
Thanks for the input 😁. Really appreciate it!

Collapse
 
gabbersepp profile image
Josef Biehler

Thanks for mentioning this. Maybe I try this out.

Collapse
 
gabbersepp profile image
Josef Biehler • Edited

I am not sure if this will also solve this problem. But I use the approach mentioned here: dev.to/maxime1992/manage-your-dev-...

This works without any effort and using this you could reference your repository from your personal blog or publish blog posts to dev.to and to your personal blog during build.

Nice idea!

Collapse
 
iggredible profile image
Igor Irianto

Really good link. Thanks Josef! I wasn't aware of this method. This is actually really cool. I have had GH account for several years but I never used template option.

Really good link! πŸ‘ Thank you very much!

Collapse
 
rhymes profile image
rhymes

Hi Igor! Nice post and usage of the API!

There's also a third option: you can request an export of your content, you can find it here at the bottom of the page: dev.to/settings/misc

Collapse
 
iggredible profile image
Igor Irianto

I didn't know this feature exists! πŸ˜…

Collapse
 
shijiezhou profile image
Shijie Zhou

Right @rhymes , I had done that before, but I think using the api can keep your article update without download the export json each time.

Collapse
 
rhymes profile image
rhymes

Agree :) It's easier to script!