Testing with MongoDB-Memory-Server

I've recently been taking a deeper dive into testing. I am taking the time to create projects that have reasonable code confidence. Over time the goal is to create and deploy applications that are adequately tested and have decent code coverage.

One of the things I found is that testing the database is not always straightforward. Here is an overview of how I did just that on one of my projects.

Backstory 🔙

The project I am working on is called OnLearn. It is essentially a POC for an e-Learning Management System. It will function as a platform where potential users can either put up courses or take courses. Quite similar to Udemy, SkillShare, or any of the MOOC platforms out there, actually.

The application's stack is Node.js, MongoDB (Mongoose ODM), and uses Handlebars for the view. Jest is the testing framework used.

Problem 🤔

One of the first challenges that presented itself was the testing of MongoDB. I wanted to be able to write unit tests for the database logic without relying heavily on mocks.

After looking into different solutions, I came across two articles that looked at testing mongodb using an in-memory database:

☝️ In-memory MongoDB for Testing.
✌️ Testing Node.js + Mongoose by Paula Santamaría

In both articles, the authors refer to nodkz's mongodb-memory-server package.

What is mongodb-memory-server?
It is a package that spins up a real MongoDB server. It enables us to start a mongod process that stores data in memory.

In-memory databases are spun up, ran, and closed in the application's main memory itself. Making them fast as they never touch the hard-drive, and are fit for testing as they are destroyed instantly upon closing.

The Solution 💡

Here is how mongodb-memory-server helped me write unit tests for one of the OnLearn application's models:

1️⃣ Install dependencies.
2️⃣ Configure Jest.
3️⃣ Setup in-memory database.
4️⃣ Create a model.
5️⃣ Write unit tests.

1️⃣ Install dependencies.

The following commands will install jest and mongodb-memory-server simultaneously.

npm i jest mongodb-memory-server

2️⃣ Configure Jest.

👉 Test Script
Add a test script to the package.json with the following commands.

"scripts": {
    "test": "jest --runInBand --detectOpenHandles",
}

CLI Options Overview

• "test" - refers to the script name for running the tests.
• jest - the default command to run all tests.
• --runInBand - the command that runs all tests serially in the current process, rather than creating a worker pool of child processes that run tests.
• --detectOpenHandles - the command that will attempt to collect and print open handles that prevent Jest from exiting cleanly.

👉 Test Environment
The default environment in Jest is a browser-like environment via jsdom. For node applications, a node-like environment should be specified instead.

"jest": {
    "testEnvironment": "node",
}

3️⃣ Setup in-memory database.

A separate file sets up the mongodb-memory-server with functions that will connect and disconnect.

// utils/test-utils/dbHandler.utils.js

const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');

const mongoServer = new MongoMemoryServer();

exports.dbConnect = async () => {
  const uri = await mongoServer.getUri();

  const mongooseOpts = {
    useNewUrlParser: true,
    useCreateIndex: true,
    useUnifiedTopology: true,
    useFindAndModify: false,
  };

  await mongoose.connect(uri, mongooseOpts);
};

exports.dbDisconnect = async () => {
  await mongoose.connection.dropDatabase();
  await mongoose.connection.close();
  await mongoServer.stop();
};

A closer look at what is happening:

Import mongoose and mongodb-memory-server.

 const mongoose = require('mongoose');
 const { MongoMemoryServer } = require('mongodb-memory-> server');

---

New mongodb-memory-server instance that will be used to run operations on in-memory db.

 const mongoServer = new MongoMemoryServer();

---

A function to connect the in-memory database.

 exports.dbConnect = async () => {
   const uri = await mongoServer.getUri();

   const mongooseOpts = {
     useNewUrlParser: true,
     useCreateIndex: true,
     useUnifiedTopology: true,
     useFindAndModify: false,
   };

   await mongoose.connect(uri, mongooseOpts);
 };

---

A function to disconnect the in-memory database.

 exports.dbDisconnect = async () => {
   await mongoose.connection.dropDatabase();
   await mongoose.connection.close();
   await mongoServer.stop();
 };

4️⃣ Create a model.

Here is the User model from the application.

Users are verified using Passport local & google strategies.
Thus, the user schema includes:

• local and google fields for authentication data.
• profilePictureUrl for the user's avatar.
• role for the type of user.

// database/models/user.model.js

const { Schema, model } = require('mongoose');

const userSchema = new Schema({
  local: {
    firstName: {
      type: String,
      trim: true,
    },
    lastName: {
      type: String,
      trim: true,
    },
    username: {
      type: String,
      trim: true,
      unique: true,
    },
    email: {
      type: String,
      match: [/^\S+@\S+\.\S+$/, 'Please use a valid email address.'],
      unique: true,
      lowercase: true,
      trim: true,
    },
    password: { type: String },
  },
  google: {
    id: String,
    token: String,
    email: String,
    name: String,
  },
  profilePictureUrl: {
    type: 'String',
    default: 'https://via.placeholder.com/150',
  },
  role: {
    type: String,
    enum: ['student', 'instructor', 'admin'],
    default: 'student',
  },
});

module.exports = model('User', userSchema);

5️⃣ Write unit tests.

Finally, use the created operations to create a connection with mongo-memory-server for the unit tests.

Here's an example of how the user model was tested in the application. Fixtures and assertions are placed in separate modules ...

👉 Fixtures

// database/fixtures/index.js

exports.fakeUserData = {
  firstName: 'Dummy',
  lastName: 'User',
  username: 'dummyUser',
  email: 'dummy@user.com',
  password: '********',
  role: 'student',
};

👉 Test Assertions Helpers

// utils/test-utils/validators.utils.js

exports.validateNotEmpty = (received) => {
  expect(received).not.toBeNull();
  expect(received).not.toBeUndefined();
  expect(received).toBeTruthy();
};

...

exports.validateStringEquality = (received, expected) => {
  expect(received).not.toEqual('dummydfasfsdfsdfasdsd');
  expect(received).toEqual(expected);
};

...

exports.validateMongoDuplicationError = (name, code) => {
  expect(name).not.toEqual(/dummy/i);
  expect(name).toEqual('MongoError');
  expect(code).not.toBe(255);
  expect(code).toBe(11000);
};

Finally, the fixtures, assertion helpers, and db operations are used in the test. 🥳🥳🥳

👉 User Model Unit Test

const User = require('../user.model');
const { fakeUserData } = require('../../fixtures');
const {
  validateNotEmpty,
  validateStringEquality,
  validateMongoDuplicationError,
} = require('../../../utils/test-utils/validators.utils');
const {
  dbConnect,
  dbDisconnect,
} = require('../../../utils/test-utils/dbHandler.utils');

beforeAll(async () => dbConnect());
afterAll(async () => dbDisconnect());

describe('User Model Test Suite', () => {
  test('should validate saving a new student user successfully', async () => {
    const validStudentUser = new User({
      local: fakeUserData,
      role: fakeUserData.role,
    });
    const savedStudentUser = await validStudentUser.save();

    validateNotEmpty(savedStudentUser);

    validateStringEquality(savedStudentUser.role, fakeUserData.role);
    validateStringEquality(savedStudentUser.local.email, fakeUserData.email);
    validateStringEquality(
      savedStudentUser.local.username,
      fakeUserData.username
    );
    validateStringEquality(
      savedStudentUser.local.password,
      fakeUserData.password
    );
    validateStringEquality(
      savedStudentUser.local.firstName,
      fakeUserData.firstName
    );
    validateStringEquality(
      savedStudentUser.local.lastName,
      fakeUserData.lastName
    );
  });

  test('should validate MongoError duplicate error with code 11000', async () => {
    expect.assertions(4);
    const validStudentUser = new User({
      local: fakeUserData,
      role: fakeUserData.role,
    });

    try {
      await validStudentUser.save();
    } catch (error) {
      const { name, code } = error;
      validateMongoDuplicationError(name, code);
    }
  });
});

Passing tests

You can find all tests and implementations here

Conclusion 🏁

In the end, the mongodb-memory-server package did a lot of database heavy lifting for my tests. I use the dbConnect and dbDisconnect operations and test the models for my application and even the services associated with those models.

Let me know what you think about this?
And feel free to share any improvement tips for this. ✌️

Find the mongodb-memory-server repository 👉 here 👈
Find the OnLearn repo 👉 here👈