Indexing Smart Contract Data Using The Graph Protocol

Introduction

Smart Contracts emit events that operate as a rich supply of data which can be aggregated. For instance, when tokens are transferred, an ERC20 token produces a transfer event. A developer or a user might index and aggregate this transfer event data and then run queries against it to learn more about the token's performance, such as the top holders, the volume of transactions, etc.

Blockchain data can be organized using the decentralized indexing technology known as the Graph Protocol. It uses the Graph Token (GRT), an ERC20 token, as a financial inducement for network security. The protocol allows for participation in four different roles, which are:

• Developer
• Indexer
• Delegator
• Curator

Developer

A subgraph is created by a developer who writes a schema and creates mappings to pull data into the schema entities. The subgraph is deployed to the Graph network, where an indexer indexes it.

Indexer

A node in the Graph Network is run by an indexer. An indexer stakes GRT so that it can run a node and index subgraphs. The indexer is compensated through network query fees.

Delegator

A delegator is a person who bets GRT on an indexer. A delegator is not required to run a node, but he or she can benefit from query fees by staking GRT on an indexer.

Curator

A Curator is someone who has enough money to tell indexers which subgraphs are worth indexing. They inspect the quality of subgraphs and assign weight to them by using GRT as signals on subgraphs so that indexers can find and index the subgraph.

You can read more about the roles here.

Creating a subgraph

Navigate to the Graph Studio and connect your wallet by signing a transaction. (No Eth will be charged ). Next, click on the Create Subgraph button to create and select the blockchain network you want the subgraph to index. I selected the Ethereum Mainnet because I want to index the Maker Dao Token found at this address. (0x6B175474E89094C44Da98b954EedeAC495271d0F).

Next, give your subgraph a unique name (In my case I named mine daitoken) and click on the continue button. Fill in the fields, describing what the subgraph does and click on save.

We will then proceed and create the subgraph in our development machine.

Take a note of the name of the subgraph. It will be needed when creating the subgraph on our machine.

Install the Graph CLI tool in our system by typing the code below at the terminal:

You must have Node installed on your computer

 npm install -g @graphprotocol/graph-cli

After installing the CLI, we will need to initiate the subgraph by using the slug name which we created in the Graph Studio earlier.

graph init --studio <slug_name>

Replace slug_name with the name of your created slug. Mine was daitoken, so my command to init will be:

graph init --studio daitoken

After running the above command, you will be ask the following question by the tool:

• Select Protocol Network : you should select Ethereum
• Confirm your slug name : click on enter
• Directory to create the subgraph in
• Select the Ethereum network: mainnet
• Contract address (the address of the contract you want to index) : 0x6B175474E89094C44Da98b954EedeAC495271d0F (this is the Maker Dai Token ) The tool will fetch the ABI from Etherscan, if it was not successful, you will have to compile the contract yourself and pass the location of the ABI to the tool.
• Contract name : name of the contract you are indexing

Finally, the tool generates and creates your subgraph. Move into the created folder and take a look at the folders content. We are most interested in three files which are:

• schema.graphql
• subgraph.yaml
• src/mappings.ts

Creating a Schema

The entities of the subgraph are contained in a schema. An entity is analogous to a table in a database, and its fields are analogous to columns. An entity may have a relationship with another entity at times. This is how an entity is represented:

  type Customer @entity {
    id: ID!
    name: String!
    address: Bytes!
  }

Each entity must have a id field that cannot be null and is denoted by the ! sign, indicating that the field cannot be null.

If an entity will not be updated after it is created, it is usually written as immutable; this aids performance.

  //Immutable Customer Entity
  type Customer @entity(immutable: true) {
    id: ID!
    name: String!
    address: Bytes!
  }

When representing a one-to-many relationship between entities, the one side is saved while the many side is derived. Consider a one-to-many relationship between a Customer entity and an Orders Entity. The one side of the relationship is saved in Orders, while the many side is derived in Customer.

   type Orders @entity {
     id : ID!
     amount: BigInt!
     token: String!
     buyer: Customer!
   }

    type Customer @entity {
    id: ID!
    name: String!
    address: Bytes!
    orders: [ Orders!]! @derivedFrom(field: "buyer")
  }

We only save values for the id, name, and address fields when mapping data for the Customer entity. When we query the Customer's entity, the orders field will be derived from the 'Orders' entity.

The schema for the Maker Dai Token will consist of a User entity, a UserCounter entity and a TransferCounter entity. The created entities will be mapped to data inside the mappings.ts file. Open the file schema.graphql and paste the code below.

type User @entity {
  id: ID!
  address: String!
  balance: BigInt!
  transactionCount: Int!
}

type UserCounter @entity {
  id: ID!
  count: Int!
}

type TransferCounter @entity {
  id: ID!
  count: Int!
  totalTransferred: BigInt!
}

The User entity contains fields of id, address, balance and transactionCount. These fields resolves to types of ID!, String, BigInt and Int respectively. The ! means that the field cannot be null. We are interested in saving the users of the token and their balance and the number of transactions they have made. We are also interested in the total user count and the total number of transfers made. Next, we will edit the manifest file.

Updating the Manifest file.

Open the file subgraph.yaml which is a configuration file. We will list the entities created above here, and also define the events on the smart contract we are interested in and handlers to pull the data into the entities.

specVersion: 0.0.2
schema:
  file: ./schema.graphql
dataSources:
  - kind: ethereum
    name: DaiToken
    network: mainnet
    source:
      address: "0x6B175474E89094C44Da98b954EedeAC495271d0F"
      abi: DaiToken
      startBlock: 8928158
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.5
      language: wasm/assemblyscript
      entities:
        - User
        - UserCounter
        - TransferCounter
      abis:
        - name: DaiToken
          file: ./abis/DaiToken.json
      eventHandlers:
        - event: Transfer(indexed address,indexed address,uint256)
          handler: handleTransfer
      file: ./src/mapping.ts

The startBlock which is under the source key signifies which block in the blockchain should the indexing start from. If we don't specify the startBlock, the indexing will start from the genesis block. You can get the block on which the contract was created from Etherscan.

Under the entities key we listed our created entities of :
User, UserCounter and TransferCounter. The event we are interested in is the Transfer event of the Token. The Transfer event takes three parameters of "indexed address,indexed address,uint256".

We define an handler to fire when the Transfer event occurs in the contract. This handler is named handleTransfer and it will be defined inside the mapping.ts file.

Lastly run the command below to auto generate types:

 graph codegen

When we change our entities we should remember to run the above command so as to correctly generate the types.

Creating mapping handler

Open the mapping file located in the directory src/mapping.ts. The subgraph mapping is written in AssemblyScript, which appears to be similar to Typescript.

import { Transfer as TransferEvent } from "../generated/DaiToken/DaiToken";
import { User, UserCounter, TransferCounter } from "../generated/schema";
import { BigInt } from "@graphprotocol/graph-ts";

export function handleTransfer(event: TransferEvent): void {
  let day = event.block.timestamp.div(BigInt.fromI32(60 * 60 * 24));

  let userFrom = User.load(event.params.src.toHex());
  if (userFrom == null) {
    userFrom = newUser(event.params.src.toHex(), event.params.src.toHex());
  }
  userFrom.balance = userFrom.balance.minus(event.params.wad);
  userFrom.transactionCount = userFrom.transactionCount + 1;
  userFrom.save();

  let userTo = User.load(event.params.dst.toHex());
  if (userTo == null) {
    userTo = newUser(event.params.dst.toHex(), event.params.dst.toHex());

    // UserCounter
    let userCounter = UserCounter.load("singleton");
    if (userCounter == null) {
      userCounter = new UserCounter("singleton");
      userCounter.count = 1;
    } else {
      userCounter.count = userCounter.count + 1;
    }
    userCounter.save();
    userCounter.id = day.toString();
    userCounter.save();
  }
  userTo.balance = userTo.balance.plus(event.params.wad);
  userTo.transactionCount = userTo.transactionCount + 1;
  userTo.save();

  // Transfer counter total and historical
  let transferCounter = TransferCounter.load("singleton");
  if (transferCounter == null) {
    transferCounter = new TransferCounter("singleton");
    transferCounter.count = 0;
    transferCounter.totalTransferred = BigInt.fromI32(0);
  }
  transferCounter.count = transferCounter.count + 1;
  transferCounter.totalTransferred = transferCounter.totalTransferred.plus(
    event.params.wad
  );
  transferCounter.save();
  transferCounter.id = day.toString();
  transferCounter.save();
}

function newUser(id: string, address: string): User {
  let user = new User(id);
  user.address = address;
  user.balance = BigInt.fromI32(0);
  user.transactionCount = 0;
  return user;
}

We imported our Transfer type and renamed it TransferEvent at the top of the file. To generate the types, we had to use graph codegen. We also imported our User, UserCounter, and TransferCounter entities.

import { Transfer as TransferEvent } from "../generated/DaiToken/DaiToken";
import { User, UserCounter, TransferCounter } from "../generated/schema";

Inside the file we defined and exported a function with the following signature.

 export function handleTransfer(event: TransferEvent): void 
 {}

This function receives as a parameter the TransferEvent. This function will be called for every transfer event that was created in the contract. The function does not return any output hence the return type of void.

All of the data that we want to save in the Graph node is contained in the event input parameter. We extract the day from the timestamp by dividing it by 86400 because this variable will be used as an identifier to save the unique transaction and user count reached each day.

let day = event.block.timestamp.div(BigInt.fromI32(60 * 60 * 24));

The transfer event contains information about the user who made the transfer, the recipients of the transfer, and the amount transferred. To represent this, we created variables called userFrom and userTo.

let userFrom = User.load(event.params.src.toHex());

We load the User entity from the Graph store if it exists, using the user's wallet address saved in the event.params.src property. Using the .toHex() method, we convert the wallet address to hexadecimal.

If userFrom is null, we use the utility function newUser to create and return a new 'User'. We subtract the amount transferred from the balance of the userFrom and increment the userFrom transaction count by one before saving the entity to the store.

The userTo variable is the beneficiary of the transfer, we check if the userTo exists by loading the data from the store.

 let userTo = User.load(event.params.dst.toHex());

If userTo is null, we create a new user using the newUser function. Since this userTo address is new, we will want to add it to the UserCounter entity. We want to save and increment the UserCounter.count field and also save the historical user count on each day.

    let userCounter = UserCounter.load("singleton");

This line loads the UserCounter from the store using the id singleton. If we don't have a UserCounter, it creates one and save the current count.

    if (userCounter == null) {
    //code removed
    userCounter.save();
    userCounter.id = day.toString();
    userCounter.save();

After saving the userCounter created by using the singleton key as an id, we then defined another id which we equate to the day.toString() and also save the historical day count.

// assuming this is the current user count which we have saved
 userCount = {
       id : "Singleton",
       count: 4567
    }
//assuming day.toString() = 18219
userCount.id = day.toString()

userCount.save()
//calling userCount.save() again will save the example entity //into the store. This is the historical count data
userCount = {
  id : "18219",
  count: 4567
}

We save the TransferCounter in the same way that we saved the UserCounter. We save the daily count and amount transferred, as well as the cumulative sum of the amount transferred and transaction count, to the store.

Deploying the subgraph to the studio

We have to authenticate from the terminal by running this code:

 graph auth --studio <deployment_key>

Your deployment key can be found on the dashboard of your Graph studio.

Next we build the project by running:

 graph build

To deploy the graph to the studio we run :

graph deploy --studio <slug_name>

The slug_name is the subgraph's name. It must be the same as what was specified in Graph Studio. You'll be prompted to enter a version number for the deployed subgraph.

You will not be able to delete a deployed subgraph.
You can only change it and republish it as a new version.

After deploying my subgraph, I was given a development url for testing purposes. The Graph's hosted service will begin indexing the subgraph. It will take some time before you can begin querying the subgraph.

Writing queries to query our subgraph

Our deployed subgraph contained three entities namely User, UserCounter and TransferCounter. We can use the studio playground to write GraphQL queries to retrieve our entities. We can retrieve a single entity or a collection of entitites.

See below some sample queries we could make to our subgraph.

• query to retrieve the first 100 users records

  {
    users(first: 100){
      id
      address
      balance
      transactionCount
   }
  }

• query to retrieve the top 10 Dai token holders

  {
  users(first:10, orderBy: balance, orderDirection: desc){
    id
    balance
    transactionCount
    address
  }
}

• query to get the total number of users that have used the Dai token

 {
   userCounter(id: "singleton"){
     id
     count
  }
}

• query to retrieve a singe user

  {
  user(id: "wallet address"){
    id
    balance
    transactionCount
    address
  }
}

Finally, we could make our subgraph available to the decentralized network. The Graph hosted service is currently indexing our deployed subgraph. To make our subgraph decentralized, we should publish it to the network, where indexers will pick it up and index it. The code for the tutorial is available here

I hope you learned something useful from this tutorial. Thank you for your time.