How to track change to any content with FaunaDB's Temporality feature

This is the third article from the series of articles on how I build an MVP with Gatsby and FaunaDB. This time we will look at yet another FaunaDB’s built-in feature called Temporality. In simple terms, it helps you track how exactly data has changed over time.

The simplest use-cases are content moderation or versioning. In our MVP, authors can create and update courses and we want to see what changed and approve course content before it goes live. Luckily, we don't need to build our own implementation because FaunaDB offers this feature on a database level out of the box.

This article is based on the previous two articles. If you missed them, it’s best to start with:

• the first article where we connect our starter project to FaunaDB and fetch data at build time
• and the second article where we implement user authentication with FaunaDB's built-in Login function, ABAC and also explore FQL together with user-defined functions (UDF).

Schema update

Our schema has changed again. Here it is:

type Course {
 title: String!
 description: String
 visible: Boolean!
 author: User!
 bookmarks: [Bookmark!] @relation
}

type Bookmark {
 title: String
 private: Boolean!
 user: User!
 course: Course!
}

type User {
 name: String!
 email: String!
 role: Role!
 courses: [Course!] @relation
 bookmarks: [Bookmark!] @relation
}

input CreateUserInput {
 name: String!
 email: String!
 password: String!
 role: Role!
}

input LoginUserInput {
 email: String!
 password: String!
}

input LogoutInput {
 allTokens: Boolean
}

type AuthPayload {
 token: String!
 user: User!
}

type CourseUpdates @embedded {
 title: String
 description: String
 visible: Boolean
}

type HistoryUpdate @embedded {
 ts: Long!
 action: String!
 data: CourseUpdates
}

type HistoryPage @embedded {
 data: [HistoryUpdate]
}

type Query {
 allCourses: [Course!]
 allBookmarks: [Bookmark!]
 allUsers(role: Role): [User!]
 allCoursesInReview(visible: Boolean = false): [Course!]
 courseUpdateHistory(id: ID!): HistoryPage
   @resolver(name: "course_update_history")
}

type Mutation {
 createUser(data: CreateUserInput): User! @resolver(name: "create_user")
 loginUser(data: LoginUserInput): AuthPayload! @resolver(name: "login_user")
 logoutUser(data: LogoutInput): Boolean! @resolver(name: "logout_user")
}

enum Role {
 AUTHOR
 DEVELOPER
 MANAGER
}

We added:

1. the visible property to Course type to make sure that only visible (approved) courses will be listed;
2. the allCoursesInReview query to list all courses that require approval based on the value of the visible property;
3. the courseUpdateHistory query to list all changes made to a particular course that we can look up by its _id;
4. The MANAGER role

We also use the @embedded directive on the return type for the courseUpdateHistory query so that FaunaDB does not create collections for those types.

Copy the above schema into a file named schema.gql and apply it with FaunaDB as shown on the screenshot below:

Since we added the visible property as non-nullable (visible: Boolean! - the exclamation mark means that the field is non-nullable, meaning that there must be always a value) and none of our courses has it, we need to set it for all courses to avoid getting the "Cannot return null for non-nullable type" GraphQL error. We need to make a small change to our existing data to accommodate the schema update which we can do in pure FQL. Copy and paste the following code into the Shell and run it:

Map(
  Paginate(
    Match(Index("allCourses"))
  ),
  Lambda("X", 
    Update(
      Select("ref",  Get(Var("X"))),
      { data: { visible: true }})
    )
)

Author role

Before we see how FaunaB’s temporal feature works, we need a way to create and update courses as authors would. While we could do it via the GraphQL playground, it’s always better to see a real-life example.

Authors can log in, but we do not have an Author role yet. Let’s create it and define what privileges we will give to authors.

Here is our FQL for the AUTHOR role:

CreateRole({
  name: "Author",
  membership: [
    {
      resource: Collection("User"),
      predicate: Query(
        Lambda(
          "userRef",
          Equals(Select(["data", "role"], Get(Var("userRef"))), "AUTHOR")
        )
      )
    }
  ],
  privileges: [
    {
      resource: Collection("User"),
      actions: {
        read: true
      }
    },
        {
          resource: Collection("Bookmark"),
          actions: {
            read: Query(
              Lambda(
                "bookmarkRef",
                Let(
                  {
                    bookmark: Get(Var("bookmarkRef")),
                    private: Select(["data", "private"], Var("bookmark"))
                  },
                  Equals(Var("private"), false)
                )
              )
            )
          }
        },
    {
      resource: Collection("Course"),
      actions: {
        read: true,
        write: Query(
          Lambda(
            ["oldData", "newData"],
            And(
              Equals(Identity(), Select(["data", "author"], Var("oldData"))),
              Equals(Select(["data", "visible"], Var("newData")), false),
              Equals(
                Select(["data", "author"], Var("oldData")),
                Select(["data", "author"], Var("newData"))
              )
            )
          )
        ),
        create: Query(
          Lambda(
            "data",
            And(
              Equals(Identity(), Select(["data", "author"], Var("data"))),
              Equals(Select(["data", "visible"], Var("data")), false)
            )
          )
        )
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    },
    {
      resource: Function("logout_user"),
      actions: {
        call: true
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("course_author_by_user"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    }
  ]
})

We allow the AUTHOR role to create and edit own courses only, see all courses and public bookmarks and log out.

What’s interesting to note is that the visible property in the Course type is false by default for new and updated courses which means that the course is “In review”. The author can’t override this value. We are checking for it in the predicate function in these lines:

• for “Create” action: Equals(Select(["data", "visible"], Var("data")), false)
• and for the “Write” action: Equals(Select(["data", "visible"], Var("newData")), false)

We can safely set the visible property to false on the client-side because we know that even if the value is changed intentionally, it will not pass the ABAC. This means that we do not need to create a custom resolver or a backend function to set this value manually.

We will test it in a minute.

Copy and paste the above FQL into the Shell and run it.

Now we need to clone the repository with the latest changes for this article.
Note that you will need the .env.development and .env.production files containing the bootstrap key and other variables that we added in the previous article

Follow these steps:

• git clone --single-branch --branch article-3/temporality git@github.com:sandorTuranszky/Gatsby-FaunaDB-GraphQL.git gatsby-fauna-db
• cd gatsby-fauna-db
• copy the .env.development and .env.production files
• npm install
• gatsby develop to start the project in development mode

You should see the MVP up and running.

If you see an error, make sure that the token cookie and the user_data in local storage have been removed or remove them manually.

Log in as an author using the following credentials:
Email: johns.austin@email.com
Password: password

And you will be redirected to the following page /app/courses:

Now we can create new courses or update an existing one.

Let’s create a new course:
Title: “Node.js Masterclass”
Description: “We’ll cover some of the topics including integrating Node.js with Express and asynchronous code”

We can see that the newly created course is marked “In review”. As it was mentioned before, all new or updated courses have a default false value for the visible property. This means that the course needs to be reviewed.

The default false value is set on the client and is controlled by ABAC. You can test it by changing the default value to true in the mutation in /src/components/updateCourse.js file:

and try to update the “React for beginners” course. You will get an error:

Now, revert the default value of the visible prop to false and click “Update”

You can see that the updated course is now marked as (In review) too.

Great, we have created and updated courses using the author account. This is what we needed to be able to test out the temporal feature.

Temporality

FaunaDB’s Temporality has two features: Snapshots and Events.

Snapshots

Snapshots allow us to see the state of our data at a particular point in time. We have just added two new courses and updated one course above. If you run the following FQL in the Shell, you will see all the 7 courses in their latest state including the newly created “Node.js Masterclass” course:

Map(
  Paginate(
    Match(Index("allCourses"))
  ),
  Lambda("X", 
    Let({
      ts: Select("ts", Get(Var("X"))),
      data: Select("data", Get(Var("X")))
    },
    {
      ts: Var("ts"),
      data: Var("data")
    }
    )
  )
)

How can we learn which articles we just created and updated? We will use the Snapshots feature to travel back in time to see our data before the updates that we made.

To use the At function we need a timestamp. We can get it from the above query which returns a ts property for each course (see the first arrow on the screenshot above).

We know that we did not update the course with the title “NodeJS Tips & Tricks” (we added it when following the previous article) hence we can assume that its timestamp represents a state before the changes that we just made. Let’s check.

Run the following FQL in the Shell.

Note that you need to copy the timestamp from your list in the Shell above (ts property is above the data property for each course)

At(
  1558455784100000,
  Map(
    Paginate(
      Match(Index("allCourses"))
    ),
    Lambda("X", Get(Var("X")))
  )
)

And you will see a list of courses without the newly added “Node.js Masterclass” course.

We remember that we created the “Node.js Masterclass” course first and then updated the “React for beginners” course. This means that with the timestamp of the “Node.js Masterclass” course, we can get the state for the “React for beginners” course, where the visible property is true and without the description property.

Copy the timestamp for the “Node.js Masterclass” course from the query result that we made to list all courses and run the following FQL in the Shell:

At(
  1594566013530000,
  Map(
    Paginate(
      Match(Index("allCourses"))
    ),
    Lambda("X", Get(Var("X")))
  )
)

You will see that the visible property is true and the description property is nowhere to be seen for the “React for beginners” course. The “Node.js Masterclass” course is also listed.

Cool, right?

Events

FaunaDB creates a new copy of the document containing all the changes we’ve made. The original document is never changed. It means that FaunaDB has at least two copies of the “React for beginners” course, one where the visible property is true and one where it’s false and the description property is available.

Since we ran a script to update the visible property for all courses, we will have more copies. To see them for the “React for beginners” course, run the following FQL:

Paginate(
  Events(
    Select(
      "ref",
      Get(Match(Index("course_by_title"), "React for beginners"))
    )
  )
)

The above query will list all copies created as a result of changes that were made to the above-mentioned course. I have four copies. The first change has an action “create” when the document was created and the other three have action “update”.

You should see the first item in the list without the visible property. Then, one copy with the visible property set to true and the last copy with the visible property set to false and the description property as well. This is exactly how we changed it.

I have one more copy which does not represent any meaningful change and the timing of the change tells me that the copy was created when we tried to manually set the default value for the visible property to true and got an error response.

Note that here we looked up the course by the name for simplicity. In our UDF we will look up courses by their _id field.

Now, when we’ve seen how the temporal features work, let’s use it for our MVP.

UDF for listing changes for a particular course

We need a custom resolver to return all changes that have been made to any given course. Copy and paste the following FQL and run it in the Shell:

Update(
  Function("course_update_history"),
  {
    "body": Query(
      Lambda(["id"],
        Let({
          page: Paginate(
            Events(
              Select(
                "ref",
                Get(Ref(Collection("Course"), Var("id")))
              )
            )
          )
        },
          Var("page")
        )
      )
     )
  }
)

Now the courseUpdateHistory mutation has a revolver. To test it out in our MVP, we need to create a MANAGER role and a few manager users.

The MANAGER role will have the same rights as the GUEST role with a few extra privileges, namely to read the history for courses, update courses and call the logout_user UDF. Run the following FQL in the Shell:

CreateRole({
  name: "Manager",
  privileges: [
    {
      resource: Collection("Course"),
      actions: {
        read: true,
        write: true,
        history_read: true
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Function("course_update_history"),
      actions: {
        call: true
      }
    },
    {
      resource: Function("logout_user"),
      actions: {
        call: true
      }
    },
    {
      resource: Index("allCoursesInReview"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("Bookmark"),
      actions: {
        read: Query(
          Lambda(
            "bookmarkRef",
            Let(
              {
                bookmark: Get(Var("bookmarkRef")),
                private: Select(["data", "private"], Var("bookmark"))
              },
              Equals(Var("private"), false)
            )
          )
        )
      }
    },
    {
      resource: Index("allBookmarks"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("User"),
      actions: {
        read: true,
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    }
  ],
  membership: [
    {
      resource: Collection("User"),
      predicate: Query(
        Lambda(
          "userRef",
          Equals(Select(["data", "role"], Get(Var("userRef"))), "MANAGER")
        )
      )
    }
  ]
})

Run the following FQL in the Shell to create a couple of manager users:

Map(
  [
    {
      name: "Manager 1",
      email: "manager1@email.com",
      password: "password",
      role: "MANAGER"
    },
    {
      name: "Manager 2",
      email: "manager2@email.com",
      password: "password",
      role: "MANAGER"
    },
  ],
  Lambda(
    "data",
      Let(
      {
        userRef: Select(
          "ref",
          Call(Function("create_user"), [
            {
              name: Select("name", Var("data")),
              email: Select("email", Var("data")),
              password: Select("password", Var("data")),
              role: Select("role", Var("data"))
            }
          ])
        )
      },
      {
        result: "Success"
      }
    )
    )
)

Now we are ready to test how a course review process might look like in real life. This is simply an example showing what data is available.

Note that mixing manager (admin) functionality with the client-facing app is a bad idea and we do it for simplicity reasons only. It’s much better to have a dedicated app for administration purposes.

Head over to our MVP and log in using the following credentials:
Email: manager1@email.com
Password: password

You will be redirected to the following page /app/courses/review where you will see courses for review.

Click on the ”See details” link for the first course.

In the first box, you can see the current state of the course.
In the second box - how the course was updated. First, it was created, then all other changes are listed (you may see a different list of changes depending on what you changed and in what order).

If you like the changes, you can click ”Approve”

Conclusion

This is how you can leverage the FaunaDB's Temporality feature to track changes to any document.

We could create a fancy UI to allow an interactive time travel but this sounds like another project!

With this article finished, we have addressed all the challenges I listed in the first one. We have proved that using FaunaDB built-in features combined with Gatsby removes the need to reinvent the wheel and allows us to concentrate on the idea.

This is the approach I am taking with my project. I am using FaunaDB and will share more insights once I release it. Stay tuned.