Eugene Aseev

Posted on Jan 18, 2021

The ultimate guide to getting multiple token balances on Ethereum

#ethereum #tokens #web3 #graphql

Introduction

One of the coolest things about Ethereum is its decentralized nature—all of its transactions since the genesis block can be publicly accessible by anyone. But with an average of one million transactions a day, it can be challenging to filter out the data you need fast and efficiently.

This post focuses on one particular task—retrieving token balances of an address on Ethereum.

The challenge with this task is that there is a significant difference in how ether is stored on an address and how the tokens are stored. Ether is the balance of an address, while tokens are the balance of a contract attributed to an address.

To get the ether balance of a wallet address, you need to watch the address.

To get the token balance of the same wallet address, you need to watch the list of token contract addresses you want to track. This process requires the execution of not one but multiple JSON-RPC requests consistently—it's time consuming and resource wasteful if executed individually.

At the time of writing this post, it was quite a challenge to find a good guide on how to retrieve data from the ledger in batches. Hence, I've decided to write one. Together, in this post, we will go through three different approaches on how to retrieve token balances via the ERC-20 balanceOf contract ABI function in batches from the Ethereum ledger.

GraphQL — alternative to the Ethereum JSON-RPC interface introduced in EIP 1767.
Etherplex — a JavaScript library that makes use of the multicall smart contract to aggregate function calls and executes them in batch.
web3.js BatchRequest — the web3.js batch method that aggregates the list of contract function calls and converts them into an array of JSON-RPC calls before sending it to the Ethereum node in one XMLHttpRequest.

Before you start, we assume that:

You already know how to write code — preferably JavaScript.
You have the code samples cloned and your environment set up. See the repository prepared for this post.
You already have your Ethereum node endpoint. If you don't, you can spin up your node with us on Chainstack.

Helper functions

Before we deep-dive into sample code, let's take a look at the two helper functions we will be using across the three different approaches.

In getTokens(), we dynamically fetch a list of token address, symbol, and decimals from Token Lists—a community-led initiative to improve discoverability and trust in ERC-20 token lists. You can replace tokenSource with an API endpoint of your choice or hardcode a list of tokens within this function. In our example, we will be using the CoinGecko list.

require('isomorphic-fetch');

const tokenSource = 'https://tokens.coingecko.com/uniswap/all.json';
const getTokens = () => {
  return fetch(tokenSource, {
    methods: 'GET',
    headers: { 'Content-Type': 'application/json', },
  }).then(data => data.json());
};

In convertToNumber(), we convert the hexadecimal response to a readable numeric format.

Note that we must avoid dividing tokens that have 0 balance with more than 20 decimals as this exceeds the BigNumber minimum limit and will result in an error.

const Web3 = require('web3');

const convertToNumber = (hex, decimals) => {
  const balance = Web3.utils.toBN(hex);
  let balanceDecimal = balance;
  if (decimals && (balance.toLocaleString() === '0' && decimals < 20)) {
    balanceDecimal = balance.div(Web3.utils.toBN(10 ** decimals));
  }

  return balanceDecimal.toLocaleString();
};

Constants

In the constant.js file, we store the constants we will be using for our queries. Remember to update the constants before executing the scripts.

ABI — contract ABI with only the balanceOf function. Remember to add the function calls you are planning to execute to the ABI constant.
username — your Ethereum node RPC username.
password — your Ethereum node RPC password.
rpcEndpoint — your Ethereum node RPC endpoint.
bathEndpoint — your Ethereum node RPC endpoint with authentication credentials.

const abi = [
  {
    constant: true,
    inputs: [
      {
        name: '_owner',
        type: 'address',
      },
    ],
    name: 'balanceOf',
    outputs: [
      {
        name: 'balance',
        type: 'uint256',
      },
    ],
    payable: false,
    stateMutability: 'view',
    type: 'function',
  },
];
// replace with your Ethereum node RPC username
const username = 'username';
// replace with your Ethereum node RPC password
const password = 'password';
// replace with your Ethereum node RPC endpoint
const rpcEndpoint = 'https://nd-123-456-789.p2pify.com';
// replace with your Ethereum node RPC endpoint
const bathEndpoint = `https://${username}:${password}@nd-123-456-789.p2pify.com`;
// replace with the address you want to query
const walletAddress = '0x3f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be';

module.exports = {
  abi,
  bathEndpoint,
  password,
  rpcEndpoint,
  username,
  walletAddress,
};

GraphQL

GraphQL is a query language designed to provide flexibility, giving client control over the format of the data anyone wants. This solves one of the most common issues people face when reading data from the Ethereum ledger—underfetching and overfetching data. This can impact your day-to-day operation, especially for users querying data from multiple contracts and different block height. GraphQL API is available on all dedicated Ethereum nodes on Chainstack.

Let us take a look at the example below on how we go about retrieving CoinGeсko list token balances using GraphQL.

require('isomorphic-fetch');
const ethers = require('ethers');
const { abi, bathEndpoint, walletAddress } = require('./constant.js');
const { convertToNumber, getTokens } = require('./utils');

const convertIndexToAlphetString = number => number
  .toString()
  .split('')
  .map(numberChar => String.fromCharCode(65 + parseInt(numberChar)))
  .join('');

const queryTemplate = (index, { address }, callData) => `
  ${convertIndexToAlphetString(index)}: call(data: { to: "${address}", data: "${callData}" }) { data }`;

const retrieveTokenBalanceViaGraphQL = (tokens) => {
  const ethersInterface = new ethers.utils.Interface(abi);
  const callData = ethersInterface
    .functions.balanceOf.encode([walletAddress]);
  const query = tokens.map((token, index) => {
    return queryTemplate(index, token, callData);
  }).join('\n');

  return fetch(`${bathEndpoint}/graphql`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ query: `{ block { ${query} } }` }),
  })
    .then(data => data.json());
};

const main = async () => {
  const { tokens } = await getTokens();

  const tokenBalances = await retrieveTokenBalanceViaGraphQL(tokens)
    .then(({ data: { block: balances } })  => {
      const output = {};

      Object.entries(balances).map(([, { data: hex }], index) => {
        const { name, decimals, symbol } = tokens[index];

        output[name] = `${convertToNumber(hex, decimals)} ${symbol}`;
      });

      return output;
    });

  console.log(tokenBalances);
};

main();

Here we dynamically generate the query from the list of tokens before sending it to our GraphQL API endpoint. This approach took an average of 2289ms out of 10 runs.

Pros of GraphQL

Streamlines complex processes

Depending on the use case, you can easily combine queries on different contract function calls, different contract addresses for different wallet addresses at different block heights into one GraphQL query.

For example:

query {
  ChainLinkToken11404479: block(number: 11404479) {
    call(data: { to: "0x514910771AF9Ca656af840dff83E8264EcF986CA", data: "0x70a082310000000000000000000000003f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be" }) {
      data
    }
  }

  ChainLinkTokenLatest: block {
    call(data: { to: "0x514910771AF9Ca656af840dff83E8264EcF986CA", data: "0x70a082310000000000000000000000003f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be" }) {
      data
    }
  }

  DaiStablecoin11404470: block(number: 11404470) {
    call(data: { to: "0x6B175474E89094C44Da98b954EedeAC495271d0F", data: "0x70a0823100000000000000000000000046340b20830761efd32832a74d7169b29feb9758" }) {
      data
    }
  }
}

Will yield:


{
  "data": {
    "ChainLinkToken11404479": {
      "call": {
        "data": "0x00000000000000000000000000000000000000000003a53a9199924822b36355"
      }
    },
    "ChainLinkTokenLatest": {
      "call": {
        "data": "0x00000000000000000000000000000000000000000003a571b23d1cfe6e4e6355"
      }
    },
    "DaiStablecoin11404470": {
      "call": {
        "data": "0x0000000000000000000000000000000000000000000024e132e65f0c1c62ddeb"
      }
    }
  }
}

Through GraphQL alias, we can efficiently aggregate multiple RPC calls for different data into one GraphQL query and let Geth and GraphQL do the heavy lifting behind the scenes.

Not reliant on dependencies

There are no additional libraries required to use GraphQL. This means that your code will be more stable and less vulnerable to security loopholes or outdated dependencies.

Cons of GraphQL

Function calls must be encoded

Executing function calls through GraphQL works the same way as a normal RPC call, which means function calls must be encoded manually and passed onto the query. This might make the code confusing to read.

GraphQL alias has strict format requirements

According to the GraphQL specification, names must strictly follow the following regex pattern: /[_A-Za-z][_0-9A-Za-z]*/.

This makes it difficult to use symbols or contract names as the alias since some contract names or symbols do not match this regex pattern. In the example above, we work around this constraint via the convertIndexToAlphetString() function which generates a unique alphabetical identifier based on the index of the token in the list, which, in turn, we later map to the token list using the same index.

Etherplex

Etherplex is a library that consolidates the list of the ethers.js contract function calls into one JSON-RPC call on the multicall smart contract aggregate function, which iterates and executes the list of contract function calls.

Below is the implementation of the aggregate function.

function aggregate(Call[] memory calls) public returns (uint256 blockNumber, bytes[] memory returnData) {
  blockNumber = block.number;
  returnData = new bytes[](calls.length);

  for(uint256 i = 0; i < calls.length; i++) {
    (bool success, bytes memory ret) = calls[i].target.call(calls[i].callData);
    require(success);

    returnData[i] = ret;
  }
}

Let us look at the example below on how we go about retrieving CoinGecko token list balances using Etherplex.

const { ethers } = require('ethers');
const { batch, contract } = require('@pooltogether/etherplex');
const { abi, username, password, rpcEndpoint, walletAddress } = require('./constant.js');
const { convertToNumber, getTokens } = require('./utils');

const provider = new ethers.providers.JsonRpcProvider({
  url: rpcEndpoint,
  user: username,
  password: password,
});

const generateContractFunctionList = tokens =>
  tokens.map(({ address: tokenAddress, symbol }) =>
    contract(symbol, abi, tokenAddress).balanceOf(walletAddress),
  );

const main = async () => {
  const { tokens } = await getTokens();
  const start = new Date().getTime();
  const args = generateContractFunctionList(tokens);
  const tokenBalances = await batch.apply(null, [provider, ...args])
    .then(balances => {
      const output = {};

      Object.entries(balances).map(([symbol, { balanceOf }], index) => {
        const balance = convertToNumber(balanceOf[0]._hex, tokens[index].decimals);

        output[tokens[index].name] = `${balance} ${symbol}`;
      });

      return output;
    });

  console.log(tokenBalances);
};

main();

Here we dynamically generate the list of balanceOf function calls from the list of tokens before passing it to Ethereplex's batch function. This approach took an average of 3815ms out of 10 runs.

Pros of Etherplex

Cons of Etherplex

Reliant on Etherplex library

At the time of this post, Etherplex has not integrated support for the ethers.js blockTag functionality, which allows the user to run a query on different blocks. If this is a requirement for you, I suggest you consider:

Contributing to the Etherplex library.
Implementing your own wrapper using the multicall smart contract.
Using other approaches described in this post.

Unstable

The multicall smart contract can sometimes be unavailable and the process will fall back to using the native JSON-RPC approach. For larger queries, this will result in a timeout error: Error: execution aborted (timeout = 5s).

web3.js BatchRequest

The web3.js BatchRequest method aggregates the list of contract function calls and converts them into an array of JSON-RPC calls before sending it to the Ethereum node in one XMLHttpRequest. The Ethereum node will process these JSON-RPC requests asynchronously before sending them back to the client.

Let us look at the example below on how we go about retrieving CoinGecko token list balances using the BatchRequest method.

const Web3 = require('web3');
const { convertToNumber, getTokens } = require('./utils');
const { abi, bathEndpoint, walletAddress } = require('./constant.js');

const web3 = new Web3(new Web3.providers.HttpProvider(bathEndpoint));

const generateContractFunctionList = ({ tokens, blockNumber }) => {
  const batch = new web3.BatchRequest();

  tokens.map(async ({ address: tokenAddress, symbol, decimals }) => {
    const contract = new web3.eth.Contract(abi);
    contract.options.address = tokenAddress;

    batch.add(
      contract.methods.balanceOf(walletAddress).call.request({}, blockNumber),
    );
  });

  return batch;
};

const main = async () => {
  const { tokens } = await getTokens();

  const batch = generateContractFunctionList({ tokens });
  const tokenBalances = {};
  const { response } = await batch.execute();
  response.forEach(({ _hex }, index) => {
    const { name, decimals, symbol } = tokens[index];

    tokenBalances[name] = `${convertToNumber(_hex, decimals)} ${symbol}`;
  });

  console.log(tokenBalances);
};

main();

Here we add the list of balanceOf contract function calls from the list of tokens before calling the batch.execute function. Note that the performance for the batch command may vary depending on the RPC endpoint provider you're using—sendAsync vs send function. This approach took an average of 3042ms out of 10 runs.

Pros of web3.js BatchRequest

More human-readable code

Unlike GraphQL, the BatchRequest method works on top of web3.js, which allows us to use the contract's ABI function names instead of encoding them, which makes the code more human-readable.

Cons of web3.js BatchRequest

The library is still in development

Currently the web3.js v2.0.0 library is still in development and it's very likely to contain breaking changes in the future.

Summary

In this post, we investigated three different approaches you can take to read data from the Ethereum ledger. In our example scenario, GraphQL had the best average performance of 2289ms as compared to the Etherplex library and the web3.js BatchRequest method.

Below are the performance metrics we took:

GraphQL outperforms the other two approaches by close to 1 second.

I personally suggest using GraphQL, not mainly because of its performance but also for its stability and flexibility.

I hope this post inspires you to try using GraphQL for your use case. The GraphQL feature is supported on all our full and archive dedicated Ethereum nodes—sign up with us and unlock unlimited possibility on direct and efficient queries to the ledger.

Join our community of innovators

To learn more about Chainstack, visit our Knowledge Center or join our Gitter Lobby.
Sign up for a free Developer account, or explore the options offered by Growth or Business plans here.
Take a look at our pricing tiers using a handy calculator to estimate usage and number of nodes.

Have you already explored what you can achieve with Chainstack? Get started for free today.

DEV Community