MongoDB Aggregation, is really powerful

I was working with my backend colleague on a particular project, going through his code, I was amazed by how clean and simple the codebase was. I was literally looking for the queries behind some of the implementations he did, the code was really so simple and clean. I was like "how did he do this?" until one very good day he told me to read about MongoDB Aggregation.

After reading and understanding how MongoDB aggregation works, I was really amazed, so I decided to write an article about it.

Before I get started, here is what you will be familiar with after reading this article:

• What is aggregation in MongoDB
• How does the MongoDB aggregation pipeline work
• MongoDB aggregation pipeline syntax
• Practical example of MongoDB aggregation

What is aggregation in mongoDB

Aggregation is a way of processing a large number of documents in a collection by means of passing them through stages, and these stages are called a Pipeline. These stages in a pipeline can filter, sort, group, reshape, and modify documents, and do much more.

Key Point🎯

• A pipeline can have one or more stages.
• The order of these stages is important.
• This aggregation happens within the database engine, enabling it to handle large datasets efficiently.

How Does the MongoDB Aggregation Pipeline Work?

Here is a diagram to illustrate a typical MongoDB aggregation pipeline:

Image Credit: studio3t

Let's understand each of the stages and what they do:

• $match stage - It filters the documents we need to work with, those that fit our needs.
• $group stage - this is where the aggregation happens. It groups documents by a specified key to perform calculations like sum, average, max, min, and so on.
• $sort stage - this sorts the documents in ascending or descending order based on specified fields.

There are many more stages; I will talk more about some of the others in the example below.

Key Point🎯

• Each stage acts upon the results of the previous stage.
• There can be one or more stages in a pipeline, depending on what you are planning to achieve.

Now that we understand how the pipeline works, let's take a look at the syntax.

MongoDB Aggregate Pipeline Syntax

This is an example of how to build an aggregation query:
db.collectionName.aggregate(pipeline, options)

• where collectionName – is the name of a collection,
• pipeline – is an array that contains the aggregation stages,
• options – optional parameters for the aggregation

This is an example of the aggregation pipeline syntax:

pipeline = [
        { $match : { … } },
        { $group : { … } },
        { $sort : { … } }
       ]

Let's now see a practical example of how MongoDB aggregation works..

Practical Example Using MongoDB Aggregation

In this example, I will be using a dummy JSON data of electrical payloads coming from single-phase and three-phase IoT devices. Check my gitHub to copy the complete Json data.

The goal of this aggregation is to retrieve the average voltage, current, and power values for each hour of the day, specifically for devices with a "deviceType" of "singlePhase". The output will be a set of documents containing the hour of the day and the corresponding rounded average values for voltage, current, and power.

This how we approach it.

Step 1: Filter data by device type

db.electricalData.aggregate([
  { $match: { deviceType: "singlePhase" } }
])

The $match stage filters the data to include only documents where the deviceType is "singlePhase".

Step 2: Group data by hour and calculate averages

db.electricalData.aggregate([
  { $match: { deviceType: "singlePhase" } },
  {
    $group: {
      _id: { hour: { $hour: "$timestamp" } },
      avgVoltage: { $avg: "$voltage" },
      avgCurrent: { $avg: "$current" },
      avgPower: { $avg: "$power" }
    }
  }
])

The $group stage groups the data by the hour of the timestamp and calculates the average voltage, current, and power for each hour.

Step 3: Sort data by hour

db.electricalData.aggregate([
  { $match: { deviceType: "singlePhase" } },
  {
    $group: {
      _id: { hour: { $hour: "$timestamp" } },
      avgVoltage: { $avg: "$voltage" },
      avgCurrent: { $avg: "$current" },
      avgPower: { $avg: "$power" }
    }
  },
  { $sort: { "_id.hour": 1 } }
])

The $sort stage sorts the data by the hour in ascending order.

Step 4: Project and format the output

db.electricalData.aggregate([
  { $match: { deviceType: "singlePhase" } },
  {
    $group: {
      _id: { hour: { $hour: "$timestamp" } },
      avgVoltage: { $avg: "$voltage" },
      avgCurrent: { $avg: "$current" },
      avgPower: { $avg: "$power" }
    }
  },
  { $sort: { "_id.hour": 1 } },
  {
    $project: {
      _id: 0,
      hour: "$_id.hour",
      avgVoltage: { $round: ["$avgVoltage", 2] },
      avgCurrent: { $round: ["$avgCurrent", 2] },
      avgPower: { $round: ["$avgPower", 2] }
    }
  }
])

The $project stage reshapes the output by excluding the _id field, renaming the _id.hour field to hour, and rounding the average voltage, current, and power values to two decimal places.

This is just a simple example of what is possible using MongoDB Aggregation. For more MongoDB aggregation operators, Check studio3t

Conclusion

MongoDB Aggregation is a powerful and flexible tool that can streamline data processing workflows, reduce the complexity of application code, and enable an easy way to extract valuable insights from data more efficiently. Whether is working with large amounts of data, complex data structures, or any other type of data, MongoDB Aggregation offers a robust set of capabilities to help harness the full potential of your data.