Scalable Notification System Design for 50 Million Users (Database Design)

Hey everyone, I will be writing about building a scalable notification system for an academic records management system I designed for my final year project.

This system needs to handle a whopping 50 million users, so scalability is key!

Project Constraints

Imagine this: students, lecturers, and a super admin (think academic head) are all buzzing around on the platform. The system needs to send out different notifications to these users.

For instance, a super admin might need to send announcements to a specific group of students. Students, on the other hand, might get reminders about upcoming classes, assignment deadlines, or new results.

Solution

To handle this, we'll use a NoSQL database like MongoDB. Here's how we'll structure things:

NotificationEntity Collection: This collection stores the core information about each notification type. Think of it as a notification template. We'll have details like the entity (e.g., coursework, class), the kind of event (e.g., creation, update), the notification type (reminder, announcement), and a message template with placeholders like lecturer_name or course_codes.

Here's a mongoose schema snippet for this collection:

const NotificationEntitiesSchema = new mongoose.Schema(
  {
    entity: {
      type: String,
      required: true,
    },
    entity_kind: {
      type: String,
      required: true,
    },
    type: {
      type: String,
      required: true,
      enum: notification_types,
    },
    template: {
      type: String,
      required: true,
    },
    is_deleted: {
      type: Boolean,
      default: false,
      select: false,
    },
    __v: { type: Number, select: false },
  },
  {
    timestamps: {
      createdAt: 'created_at',
      updatedAt: 'updated_at',
    },
  },
);

Notification Collection: This collection stores individual notifications fired off based on the templates in the NotificationEntity collection. We'll have the actual message, a reference to the notification entity it's based on, the type of actor who triggered it (super admin, lecturer), and their ID.
Here's another mongoose schema snippet for this collection:

const NotificationsSchema = new mongoose.Schema(
  {
    message: {
      type: String,
      required: true,
    },
    notification_entity: {
      type: mongoose.Schema.Types.ObjectId,
      ref: 'notification-entity',
      required: true,
    },
    actor_type: {
      type: String,
      enum: actors,
    },
    actor_id: {
      type: mongoose.Schema.Types.ObjectId,
      refPath: 'actor_type',
    },
  },
  {
    timestamps: {
      createdAt: 'created_at',
      updatedAt: 'updated_at'
    },
  }
);

NotificationReceivers Collection: This collection keeps track of who received a particular notification and their read status (read/unread, and if read, the date). Essentially, it links Notifications to Students.
Here's the mongoose schema snippet for this last collection:

const NotificationReceiversSchema = new mongoose.Schema(
  {
    notification: {
      type: mongoose.Schema.Types.ObjectId,
      ref: 'notification',
      required: [true, en['notification-id-required']],
    },
    student: {
      type: mongoose.Schema.Types.ObjectId,
      ref: 'student',
      required: [true, en['student-id-required']],
    },
    read_status: {
      type: {
        status: {
          type: Boolean,
          default: false,
        },
        date_read: {
          type: Date,
        },
      },
    },
    is_deleted: {
      type: Boolean,
      default: false,
      select: false,
    },
    __v: { type: Number, select: false },
  },
  {
    timestamps: true,
  }
);

Alright, so how do notifications get sent? Let's break it down:

1. Fetch the notification template: We'll grab the relevant notification entity from the NotificationEntity collection that matches the notification we want to send.

2. Craft the message: We'll use the template's message format and fill in any placeholders with relevant information.

3. Create the notification document: A new document is created in the Notification collection to store the actual notification message and references.

4. Identify recipients: We'll figure out who needs to receive this notification based on the notification type and any specific criteria (e.g., all students enrolled in a particular course).

5. Store notification receivers: For each recipient, a document is created in the NotificationReceivers collection linking them to the notification and marking the initial read status as unread.

This way, we can efficiently track who has seen what notification and scale the system to handle a massive number of users because each notification is stored separately with references to recipients.

Below is a simple illustration to explain the database tables in a Relational Database Management System (RDBMS)

So, that's the blueprint for a scalable notification system that can handle millions of users on our academic management platform. Hope this walkthrough was helpful!