Prepopulate Firestore (Firebase) with a Python script

This is a quick post to show you how to import CSV or JSON data into Firestore. If you data is in CSV format, you will need to convert it to JSON format using an online tool called CSVJSON. This guide uses Python and assumes that you have a Firebase project already set up.

Data

First thing, you need to do is prepare the data you are going to import. For this guide, I prepared the data using Google Sheets then exported it in csv format. I then used the CSVJSON online too mentioned above to export the data to json format.

We will use the id field as the Firestore document ID. We will also structure the code and files to allow for multiple collection imports. One JSON file will contain data for one collection. The name of the JSON file will be used as the name of the collection. For instance, when we import from categories.json, it will use (or create, if none exists) a collection called categories and import data into it. We will place all our JSON files in a data/ folder in the root of the project.

Credentials

The first step is to download the service key from your Firebase. Do this by navigating to the Users and Permissions settings. See image below:

In the Users and Permissions page, navigate to the Service Accounts tab. There you will see your service account email and a code snippet showing how to configure the service account in your code.

Click the Generate new private key button shown in the image above, to download a service key. Place this file in your project folder.

Code

First start by initializing the Firebase SDK using the credentials we created in the previous step

import firebase_admin
from firebase_admin import credentials, firestore

cred = credentials.Certificate("path/to/serviceAccountKey.json")
firebase_admin.initialize_app(cred)

db= firestore.client()

Next we will loop through all the JSON files in the data/ directory and import each of them. Remember, the filenames will be used as the collection names (categories.json will import into the categories collection).

Line 12 makes sure we are only using JSON files.

Line 13 gets the name of the collection from the filename

Line 15 converts the JSON data to a list of objects

For each object (line 17), we remove (pop) the id field and store it in a variable which are going to use as the document ID (line 20). If the object has no id field, we will let Firestore auto-generate an ID for us (line 22).

Below is the full code

import firebase_admin

import os

import json

from firebase_admin import credentials, firestore

cred = credentials.Certificate("path/to/serviceAccountKey.json")

firebase_admin.initialize_app(cred)

db = firestore.client()

for filename in os.listdir('data'):

    if filename.endswith('.json'):

        collectionName = filename.split('.')[0] # filename minus ext will be used as collection name

        f = open('data/' + filename, 'r')

        docs = json.loads(f.read())

        for doc in docs:

            id = doc.pop('id', None)

            if id:

                db.collection(collectionName).document(id).set(doc, merge=True)

            else:

                db.collection(collectionName).add(doc)

That's it! All done.

Please reach out with questions and suggestions.

Happy coding.