Populating MongoDB schema

We'll be creating only the backend of the application for understanding how to post data in to a MongoDB schema that references another schema.

TL;DR

How can we get the following JSON data with the user schema referencing the todo schema?

{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

User Model

Todo Model

Here, the User schema is referencing the Todo schema. To get the JSON data with the todo data we need to do the following

• While creating the todo data we need to add the ObjectId of the new todo to the todo array of the User. At this stage the data will look something like this.

{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

• To get the data of the todo created by the user we will reference the Todo table using the populate method which will get the data of the todo.

It is like joining two tables in SQL where User table references the Todo table using the primary key of the Todo table. Here, the primary key of the Todo table is the ObjectId.

GitHub repo

Initialize project

• Initialize our backend using npm and install necessary packages.
• Set up a MongoDB database.
• Set up server using Node and Express.
• Create a database schema to define a Todo.
• Set up API routes to create user and todo and read user and todo.
• Testing our API routes using Insomnia.

Install

• VS Code or any other editor
• Latest version of Node.js
• Insomnia or Postman
• Prettier VS code extension to format the code

1. Initializing our project

Create a new folder and name it anything that you like and then open the folder in VS code and run the following code from the command prompt.

npm init -y

After running this command you will find a package.json if the folder.

2. Setting up package.json

i. Install the following dependencies

Run the following commands in the terminal to install the dependencies

npm i cors dotenv express mongoose

cors: allows cross-origin api calls
dotenv: needed to access data from .env files
express: web application framework for node.js
mongoose: It is needed to define the database schema and connecting to mongoDB

ii. Install the following development dependencies

Now install the following development dependencies, -D is used to install the development dependencies.

npm i -D nodemon

After installing the dependencies the package.json folder should look as follows.

// package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

iii. change the main entry point to server.js

Now, create a server.js file and a .env. The server.js file will be the entry point of the server and the .env file will contain the MONGO_URI. We also have to make the following changes in the package.json

//package.json
{
  "name": "mongodb-schema-populate-blog",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/mritunjaysaha/mongodb-schema-populate-blog.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog/issues"
  },
  "homepage": "https://github.com/mritunjaysaha/mongodb-schema-populate-blog#readme",
  "dependencies": {
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.3"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

Now, create the following folders

• config: Inside the config folder, create a file named db.js. This file will contain the required code for connecting to the MongoDB database.

• controllers: The controllers folder will contain the files which will have the methods for the end points to communicate with the database.

• models: The models folder, will contain the files which will define the MongoDB schema

• routers: The routers folder will contain the files with the endpoints.

At this stage the file structure should look as follows

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

iv. Change the scripts to the following

"scripts": {
  "start":"node server.js",
  "dev":"nodemon server.js"
}

The package.json file should look as follows

{
  "name": "mern-todo",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js", //added
    "dev": "nodemon server.js" //added
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "config": "^3.3.6",
    "cors": "^2.8.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "mongoose": "^5.13.2"
  },
  "devDependencies": {
    "nodemon": "^2.0.12"
  }
}

v. Setting up server

We will do the following to setup the server

• Import express
• Initialize our app using express()
• Set up a get method for the endpoint http://localhost:8000 using app.get()
• Set the PORT to 8000 for our server to run
• Have our app to listen to PORT using app.listen()

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json

The code will look as follows

And start the server using nodemon using the following code. Make sure you are running the following command from the project directory.

npm run dev

If the server has started successfully then it should show the following message in the terminal

[nodemon] 2.0.11
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node server.js`
server is running on http://localhost:8000

You can also open http://localhost:8000 on your browser.

vi. Getting the MONGO URI from mongoDB

To connect to the database we will need the link for the mongoDB collection.

1. Log in to mongoDB
2. Create a new project
3. Build a cluster
4. Select cloud provider
5. Create cluster
6. wait for the cluster to be created.
7. Click on connect
8. click on allow access from anywhere. Then Add IP address

1. Create a database user. You'll need the username and password for the MongoDB URI.
2. Click on the Choose a connection method
3. Click on Connect your application

4. Select the following driver and version

   

5. Copy the mongodb+srv and paste it in the .env file

vii. Setting up .env file

//.env
MONGO_URI = mongodb+srv://<username>:<password>@cluster0.owmij.mongodb.net

Replace the <username> and <password> with your database username and password which you will set in step 9.

viii. Connecting to database

.
├── config
│   └── db.js <-- we are here
├── controllers
│   └── user.js
├── models
│   ├── todo.js
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

Now, open the db.js file which is in the config folder and add the following changes.

• Import mongoose
• Import MONGO_URI from .env
• Define the connectDB methof for connecting to the database
• Export the connectDB method to be called in server.js

Add the following changes in the server.js file.

• Import dotenv
• Import connectDB method from config/db.js
• Call the connectDB method.

Let us make the the following changes in server.js

Save the changes it will restart the server or use the command npm run dev. The terminal should show a message of MongoDB is connected which we have added in the db.js under the try block.

ix. Defining database schema

Create a todo.js file in the models folder. We will define the database schema in this file.

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models
│   ├── todo.js <-- we are here
│   └── user.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

• Import mongoose
• Create a Schema called TodoSchema
• We will add two fields for our todo; title and description
• Type of title will be String and it is a mandatory field
• Type of description will be String and it is not a mandatory field
• Export the model

The code will look as follows

Create a schema for the user using the above steps.

After making the changes, the user model will look something like this

x. Defining the controllers

.
├── config
│   └── db.js
├── controllers
│   └── user.js <-- we are here
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js
├── .env
├── server.js
├── package-lock.json
└── package.json

• Import Todo and User schemas
• Define createUser method will create a new user
• Define createTodo method will do the following
  • create a new todo
  • save the todo
  • use the userId to find the user
  • update the todo array with the ObjectId of the new todo
• Define getUser to get the user details. The output of this method we can see that todo consists of some random value which is the ObjectId of the todo that the user has created. We cannot figure out what the todo contains.

{
  "todo": ["61023642610b8d4ce4f56f81", "6102365b610b8d4ce4f56f84"],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

• Define getAllTodo method we will use the userId to find the user and then use the populate method to reference the todo with the ObjectId from the Todo table. The exec method is used to check for errors and return the populated data.

{
  "todo": [
    {
      "_id": "61023642610b8d4ce4f56f81",
      "title": "test-title-1",
      "description": "test-description-1",
      "__v": 0
    },
    {
      "_id": "6102365b610b8d4ce4f56f84",
      "title": "test-title-2",
      "description": "test-description-2",
      "__v": 0
    }
  ],
  "_id": "6102361f610b8d4ce4f56f7f",
  "name": "test-user",
  "__v": 0
}

xi. Defining the end points

.
├── config
│   └── db.js
├── controllers
│   └── user.js
├── models

│   └── todo.js
├── node_modules
├── routes
│   └── user.js <-- we are here
├── .env
├── server.js
├── package-lock.json
└── package.json

We will define the end points to create users and todo and to read them.

• Import express
• Import all the methods from controllers
• Initialize router
• Define a POST method to create a user
• Define a POST method to create a todo and save it in the user
• Define a GET method to read user data
• Define a GET method to read user data and todo data

After making the above changes the code will look something like this

xii. Adding the routes end points in the server.js

.
├── config
│   └── db.js
├── controllers
│   └── todo.js
├── models
│   └── todo.js
├── node_modules
├── routes
│   └── todo.js
├── .env
├── server.js <-- we are here
├── package-lock.json
└── package.json

The final part of completing the backend is to add the endpoints to the server.js file.

• Import routes/todo.js
• Add the routes endpoints to the middleware

3 Testing the end points using Insomnia

• Create a user

We will send a POST request to http://localhost:8000/api/user

• Create some todo

We will send a POST request to http://localhost:8000/api/user/todo/:userId

copy the _id from the response of the create a user request

• Read the user data

We will send a GET request to http://localhost:8000/api/user/:userId

• Read the populated user data

We will send a POST request to http://localhost:8000/api/user/todo/:userId

You can check the code in GitHub