Apperside

Posted on Aug 5

Creating a web3 DApp with a NX Monorepo - 3 of 5: Create backend app with NestJS

The original baluni app had a backend implemented with ExpressJS that was doing some things:

it was calling some smart contract
it stores some data in a sqlite database
it exposes some endpoint to retrieve some data
it runs periodic tasks to update that data.

It's not in the scope of this article to analyze the what and the why of this backend, for now we just need to migrate in our brand new monorepo the old project.

This backend has just few non protected endpoints, so the porting should not be so complicated.

I am a fan of NestJS, so I decided to create the backend with it.

NX has a generator that can be used to quickly bootstrap a NestJS app in our monorepo.

The first thing to do is to install the generator

nx add @nx/nest

after installing it we can create the project

nx g @nx/nest:app baluni-backend

it will spin-up a bare bone NestJS app, with just one controller and one service (it is not in the scope of this document to enter in the technicale details of NestJS, you can learn more on their extensive docs)

Understanding the project

The original backend project comes from the baluni-ui package, which was also a monorepo itself.
This was the original structure:

I started by analyzing the index.ts file

import { fetchUnitPrices } from "./fetchUnitPrices";
import { fetchTotalValuation } from "./fetchTotalValuation";
import { fetchInterestEarned } from "./fetchInterestEarned";
import { dcaExecutor } from "./dcaExecutor";
import { reinvestEarnings } from "./reinvestEarnings";
import { rebalancePools } from "./rebalancePools";
import { fetchHyperPools } from "./fetchHyperPools";
import { deleteOldRecords } from "./deleteOldRecords";
import "./server";

var Table = require("cli-table");

async function main() {
  console.log("✔️ Starting main execution...");

  const tasks = [
    { name: "Fetching unit prices", func: fetchUnitPrices },
    { name: "Fetching total valuation", func: fetchTotalValuation },
    { name: "Fetching total interest earned", func: fetchInterestEarned },
    { name: "Executing DCA", func: dcaExecutor },
    { name: "Reinvesting earnings", func: reinvestEarnings },
    { name: "Rebalancing pools", func: rebalancePools },
    { name: "Fetching hyper pools data", func: fetchHyperPools },
    { name: "Deleting old records", func: deleteOldRecords },
  ];

  const table = new Table({
    head: ["Task", "Status"],
    colWidths: [50, 10],
  });

  for (const task of tasks) {
    try {
      console.log(`🚀 ${task.name}...`);
      await task.func();
      table.push([task.name, "✅"]);
    } catch (error) {
      console.error(`❌ Error in ${task.name}:`, error);
      table.push([task.name, "❌"]);
    }
    console.log("🎉 Done");
    await new Promise(resolve => setTimeout(resolve, 5000));
  }

  console.log(table.toString());
}

(async () => {
  const interval = Number(process.env.INTERVAL) || 3600000; // Default to 1 hour if INTERVAL is not defined
  setInterval(() => {
    main().catch(error => console.error("❌ Error in main execution:", error));
  }, interval);
  main().catch(error => console.error("❌ Error in main execution:", error));
})();

It was looking quite weird.

This was the app's entry point and what it was doing is scheduling some recurring tasks and running the server by importing the ./server file.

We can do a lot better.

Next I started exploring the server.ts file

import express from "express";
import path from "path";
import { open } from "sqlite";
import sqlite3 from "sqlite3";
import { calculateStatistics } from "./calculateStatistics";

const cors = require("cors");

const app = express();
const PORT = process.env.PORT || 3001;

app.use(
  cors({
    origin: "*", // Replace with your frontend's domain
    methods: "GET,HEAD,PUT,PATCH,POST,DELETE",
    allowedHeaders: "Content-Type,Authorization",
  }),
);

app.use(express.json());

const dbPromise = open({
  filename: path.join(__dirname, "..", "baluniData.db"),
  driver: sqlite3.Database,
});

// Endpoint per ottenere tutti i dati da hyperPoolsData
app.get("/api/hyperpools-data", async (req, res) => {
  try {
    const db = await dbPromise;
    const hyperpoolsData = await db.all("SELECT * FROM hyperPoolsData");
    res.json(hyperpoolsData);
  } catch (error) {
    res.status(500).json({ error: "Failed to fetch hyper pools data" });
  }
});

app.get("/api/valuation-data", async (req, res) => {
  try {
    const db = await dbPromise;
    const valuations = await db.all("SELECT * FROM totalValuations");
    res.json(valuations);
  } catch (error) {
    res.status(500).json({ error: "Failed to fetch valuation data" });
  }
});

app.get("/api/totalInterestEarned-data", async (req, res) => {
  try {
    const db = await dbPromise;
    const results = await db.all("SELECT * FROM totalInterestEarned");
    res.json(results);
  } catch (error) {
    res.status(500).json({ error: "Failed to fetch total interest data" });
  }
});

app.get("/api/unitPrices-data", async (req, res) => {
  try {
    const db = await dbPromise;
    const results = await db.all("SELECT * FROM unitPrices");
    res.json(results);
  } catch (error) {
    res.status(500).json({ error: "Failed to fetch unit prices data" });
  }
});

app.get("/api/statistics", async (req, res) => {
  try {
    const statistics = await calculateStatistics();
    res.json(statistics);
  } catch (error) {
    res.status(500).json({ error: "Failed to fetch statistics data" });
  }
});

app.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});

Ok, this one looks much more clear: a very basic ExpressJS server listening on a bunch of routes each returning data fetched from the database.

Let's start bringing things in.

Integrating the code - sqlite support

The project was using a sqlite database, so I started looking for a way to integrate sqlite in a NestJS app and found this:

@homeofthings/nestjs-sqlite3 - npm

HomeOfThings - NestJs Sqlite3: Sqlite3 module for NestJs based on 'sqlite3orm'. Latest version: 2.0.2, last published: 7 months ago. Start using @homeofthings/nestjs-sqlite3 in your project by running `npm i @homeofthings/nestjs-sqlite3`. There are no other projects in the npm registry using @homeofthings/nestjs-sqlite3.

npmjs.com

It was quite straightforward to integrate, I just had to add it to my app.module.ts

And wherever I need access to the sqlite connection I can inject it like this

@Injectable()
export class MyService {
  constructor(connectionManager: ConnectionManager) {}
}

Integrating the code - controller/service/repository

At this stage my main goal is to make things work as they are, without over engineering things or refactoring code more than necessary.

For this reasons I decided to use the app.controller.ts that NestJS already created (instead of creating a dedicated controller/service/repository), and using app.service.ts to call the functions implemented in a repository class that I created, stats.repository.ts, responsible for performing the queries that was in the original project. For example:

The ConnectionManager class is injected from the sqlite package integration mentioned above.

In app.service.ts I just implemented specular methods:

An finally created the routes in app.controller.ts:

So, the only new thing I added for now was the repository class from where performing the queries, and then just used the controller and service to "forward" the requests.
In this way I encapsulated data access and provided modularity (for example, it would be easy to replace sqlite with anything else) without changing this too much.

Integrating the code - recurring tasks

The original codebase had a few files containing some business logic that was ran at regular intervals from index.ts.
Those file are the ones that in the original repository are implemented in this files:

Those files, in addition to the functions with the business logic, had some aspects in common.
For example, one of those files was looking like this:

let's analyze it:

In all files there was a call to dotenv.config() (like in line 15)
in all files there were those 2 constants:

const provider = new ethers.providers.JsonRpcProvider(String(process.env.RPC_URL));
const signer = new ethers.Wallet(String(process.env.PRIVATE_KEY), provider);

In all files there were, at some points, a call to this function (like at line 45)

registryCtx = await setupRegistry(provider, signer);

this is what setupRegistry was doing

import contracts from "baluni-contracts/deployments/deployedContracts.json";
import baluniRegistryAbi from "baluni-contracts/artifacts/contracts/registry/BaluniV1Registry.sol/BaluniV1Registry.json";

import { ethers } from "ethers";

export async function setupRegistry(provider: ethers.providers.JsonRpcProvider, signer: ethers.Signer) {
  const chainId = await provider.getNetwork().then((network: { chainId: any }) => network.chainId);

  let registryCtx;

  if (chainId === 137) {
    const registryAddress = contracts[137].BaluniV1Registry;
    if (!registryAddress) {
      console.error(`Address not found for chainId: ${chainId}`);
      return;
    }
    registryCtx = new ethers.Contract(registryAddress, baluniRegistryAbi.abi, signer);
  }

  return registryCtx;
}

So what I did was the following:

1- Added 2 custom providers to my app.module.ts

2- Create a new class called BaseWeb3Task:

import { Inject, Injectable } from '@nestjs/common';
import { ethers } from 'ethers';
import { baluniContracts } from 'baluni-contracts';
import { BaseTask } from './base.task';

@Injectable()
export abstract class BaseWeb3Task extends BaseTask {
  registryCtx: any;
  constructor(
    @Inject('rpcProvider') protected provider,
    @Inject('wallet') protected signer
  ) {
    super()
  }

  async setupRegistry(
    provider: ethers.providers.JsonRpcProvider,
    signer: ethers.Signer
  ) {
    const chainId = await provider
      .getNetwork()
      .then((network: { chainId: any }) => network.chainId);

    let registryCtx;

    if (chainId === 137) {
      const registryAddress =
        baluniContracts.baluniDeploiedContracts[137].BaluniV1Registry;

      if (!registryAddress) {
        console.error(`Address not found for chainId: ${chainId}`);
        return;
      }
      registryCtx = new ethers.Contract(
        registryAddress,
        baluniContracts.baluniRegistryAbi.abi,
        signer
      );
    }

    return registryCtx;
  }

  async initRegistry() {
    this.registryCtx = await this.setupRegistry(this.provider, this.signer);
  }

  abstract execute(...args: any[]): Promise<any>;
}

This is BaseTask.ts class

import { Injectable } from '@nestjs/common';

@Injectable()
export abstract class BaseTask {
  constructor() {}

  abstract execute(...args: any[]): Promise<any>;
}

and in each implementation I moved the logic under the execute method.

For example, the original file fetchUnitPrices.ts changed from this:

import baluniPoolAbi from "baluni-contracts/artifacts/contracts/pools/BaluniV1Pool.sol/BaluniV1Pool.json";
import baluniPoolRegistryAbi from "baluni-contracts/artifacts/contracts/registry/BaluniV1PoolRegistry.sol/BaluniV1PoolRegistry.json";
import baluniVaultRegistryAbi from "baluni-contracts/artifacts/contracts/registry/BaluniV1YearnVaultRegistry.sol/BaluniV1YearnVaultRegistry.json";
import baluniVaultAbi from "baluni-contracts/artifacts/contracts/vaults/BaluniV1YearnVault.sol/BaluniV1YearnVault.json";
import dotenv from "dotenv";
import { Contract, ethers } from "ethers";
import path from "path";
import { open } from "sqlite";
import sqlite3 from "sqlite3";
import baluniDCAVaultAbi from "baluni-contracts/artifacts/contracts/vaults/BaluniV1DCAVault.sol/BaluniV1DCAVault.json";
import baluniDCAVaultRegistryAbi from "baluni-contracts/artifacts/contracts/registry/BaluniV1DCAVaultRegistry.sol/BaluniV1DCAVaultRegistry.json";
import { setupRegistry } from "./setupRegistry";
import { formatUnits } from "ethers/lib/utils";

dotenv.config();

const provider = new ethers.providers.JsonRpcProvider(String(process.env.RPC_URL));
const signer = new ethers.Wallet(String(process.env.PRIVATE_KEY), provider);

let registryCtx: Contract | null | undefined = null;

async function fetchAndStoreUnitPrice(contract: Contract, address: string, db: any) {
  try {
    const unitPrice = await contract.unitPrice();
    const unitPriceData = {
      timestamp: new Date().toISOString(),
      unitPrice: unitPrice.toString(),
      address: address,
    };

    await db.run(
      "INSERT INTO unitPrices (timestamp, unitPrice, address) VALUES (?, ?, ?)",
      unitPriceData.timestamp,
      formatUnits(unitPriceData.unitPrice, 18),
      unitPriceData.address,
    );

    console.log("Unit Price data updated:", unitPriceData);
  } catch (error) {
    console.error(`Error fetching unit price for address ${address}:`, error);
  }
}

export async function fetchUnitPrices() {
  registryCtx = await setupRegistry(provider, signer);

  if (!registryCtx) {
    return;
  }

  const db = await open({
    filename: path.join(__dirname, "..", "baluniData.db"),
    driver: sqlite3.Database,
  });

  await db.exec(`
    CREATE TABLE IF NOT EXISTS unitPrices (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      timestamp TEXT NOT NULL,
      unitPrice TEXT NOT NULL,
      address TEXT NOT NULL
    )
  `);

  try {
    const poolRegistry = await registryCtx.getBaluniPoolRegistry();
    const poolRegistryContract = new ethers.Contract(String(poolRegistry), baluniPoolRegistryAbi.abi, provider);
    const pools = await poolRegistryContract.getAllPools();

    for (const pool of pools) {
      const poolContract = new ethers.Contract(pool, baluniPoolAbi.abi, provider);
      await fetchAndStoreUnitPrice(poolContract, pool, db);
    }
  } catch (error) {
    console.error("Error processing pool registry:", error);
  }

  try {
    const vaultRegistry = await registryCtx.getBaluniYearnVaultRegistry();
    const vaultRegistryContract = new ethers.Contract(String(vaultRegistry), baluniVaultRegistryAbi.abi, provider);
    const vaults = await vaultRegistryContract.getAllVaults();

    for (const vault of vaults) {
      const vaultContract = new ethers.Contract(vault, baluniVaultAbi.abi, provider);
      await fetchAndStoreUnitPrice(vaultContract, vault, db);
    }
  } catch (error) {
    console.error("Error processing vault registry:", error);
  }

  try {
    const dcaVaultRegistry = await registryCtx.getBaluniDCAVaultRegistry();
    const dcaVaultRegistryContract = new ethers.Contract(
      String(dcaVaultRegistry),
      baluniDCAVaultRegistryAbi.abi,
      provider,
    );
    const dcaVaults = await dcaVaultRegistryContract.getAllVaults();

    for (const vault of dcaVaults) {
      const vaultContract = new ethers.Contract(vault, baluniDCAVaultAbi.abi, provider);
      await fetchAndStoreUnitPrice(vaultContract, vault, db);
    }
  } catch (error) {
    console.error("Error processing DCA vault registry:", error);
  }

  await db.close();
}

To this:

import { ConnectionManager } from '@homeofthings/nestjs-sqlite3';
import { Inject, Injectable } from '@nestjs/common';
import { baluniVaultAbi, baluniVaultRegistryAbi } from 'baluni-contracts';
import { Contract, ethers } from 'ethers';
import { formatUnits } from 'viem';
import { baluniContracts } from 'baluni-contracts';
import { BaseWeb3Task } from './base-web3.task';

@Injectable()
export class UnitPriceTask extends BaseWeb3Task {
  constructor(
    private connectionManager: ConnectionManager,
    @Inject('rpcProvider') protected provider,
    @Inject('wallet') protected signer
  ) {
    super(provider, signer);
  }
  async fetchAndStoreUnitPrice(contract: Contract, address: string, db: any) {
    try {
      const unitPrice = await contract.unitPrice();
      const unitPriceData = {
        timestamp: new Date().toISOString(),
        unitPrice: unitPrice.toString(),
        address: address,
      };

      await db.run(
        'INSERT INTO unitPrices (timestamp, unitPrice, address) VALUES (?, ?, ?)',
        unitPriceData.timestamp,
        formatUnits(unitPriceData.unitPrice, 18),
        unitPriceData.address
      );

      console.log('Unit Price data updated:', unitPriceData);
    } catch (error) {
      console.error(`Error fetching unit price for address ${address}:`, error);
    }
  }

  async execute() {
    await this.initRegistry();

    if (!this.registryCtx) {
      return;
    }

    const db = await this.connectionManager.getConnectionPool().get();

    await db.exec(`
      CREATE TABLE IF NOT EXISTS unitPrices (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        timestamp TEXT NOT NULL,
        unitPrice TEXT NOT NULL,
        address TEXT NOT NULL
      )
    `);

    try {
      const poolRegistry = await this.registryCtx.getBaluniPoolRegistry();
      const poolRegistryContract = new ethers.Contract(
        String(poolRegistry),
        baluniContracts.baluniPoolRegistryAbi.abi,
        this.provider
      );
      const pools = await poolRegistryContract.getAllPools();

      for (const pool of pools) {
        const poolContract = new ethers.Contract(
          pool,
          baluniContracts.baluniPoolAbi.abi,
          this.provider
        );
        await this.fetchAndStoreUnitPrice(poolContract, pool, db);
      }
    } catch (error) {
      console.error('Error processing pool registry:', error);
    }

    try {
      const vaultRegistry =
        await this.registryCtx.getBaluniYearnVaultRegistry();
      const vaultRegistryContract = new ethers.Contract(
        String(vaultRegistry),
        baluniVaultRegistryAbi.abi,
        this.provider
      );
      const vaults = await vaultRegistryContract.getAllVaults();

      for (const vault of vaults) {
        const vaultContract = new ethers.Contract(
          vault,
          baluniVaultAbi.abi,
          this.provider
        );
        await this.fetchAndStoreUnitPrice(vaultContract, vault, db);
      }
    } catch (error) {
      console.error('Error processing vault registry:', error);
    }

    try {
      const dcaVaultRegistry =
        await this.registryCtx.getBaluniDCAVaultRegistry();
      const dcaVaultRegistryContract = new ethers.Contract(
        String(dcaVaultRegistry),
        baluniContracts.baluniDCAVaultRegistryAbi.abi,
        this.provider
      );
      const dcaVaults = await dcaVaultRegistryContract.getAllVaults();

      for (const vault of dcaVaults) {
        const vaultContract = new ethers.Contract(
          vault,
          baluniContracts.baluniDCAVaultAbi.abi,
          this.provider
        );
        await this.fetchAndStoreUnitPrice(vaultContract, vault, db);
      }
    } catch (error) {
      console.error('Error processing DCA vault registry:', error);
    }

    await db.close();
  }
}

So basically the idea is to create an implementation of BaseWeb3Task class, and passing in the super constructor the injected signer and provider and expose them as protected class members.

Then, when it is needed, we can call the initRegistry method declared in the parent class:

this function will make use of the injected signer and provider

Once ported all the files under this pattern, scheduling them was quite straightforward with NestJS's builtin scheduling feature.
I just needed to create a class called RecurringTasksService and create a method decorated with:

@Cron(CronExpression.EVERY_5_MINUTES)

The final file is this:

import { Injectable, Logger } from '@nestjs/common';
import { Cron, CronExpression } from '@nestjs/schedule';
import { Contract } from 'ethers';
import { DcaTask } from './impl/dca.task';
import { DeleteAllRecordsTask } from './impl/delete-old-records.task';
import { HyperPoolsTask } from './impl/hyper-pools.task';
import { InterestEarnedTask } from './impl/interest-earned.task';
import { PoolRebalanceTask } from './impl/pool-rebalance.task';
import { ReinverstEarningsTask } from './impl/reinvest-earnings.task';
import { TotalValuationTask } from './impl/total-valuation.task';
import { UnitPriceTask } from './impl/unit-price.task';

@Injectable()
export class RecurringTasksService {
  registryCtx: Contract | null | undefined = null;

  constructor(
    private interestEarnedTask: InterestEarnedTask,
    private poolRebalanceTask: PoolRebalanceTask,
    private reinverstEarningsTask: ReinverstEarningsTask,
    private totalValuationTask: TotalValuationTask,
    private unitPriceTask: UnitPriceTask,
    private dcaTask: DcaTask,
    private deleteRecordsTask: DeleteAllRecordsTask,
    private hyperPoolsTask: HyperPoolsTask
  ) {}
  private readonly logger = new Logger(RecurringTasksService.name);

  @Cron(CronExpression.EVERY_5_MINUTES)
  async handleCron() {
    const tasks = [
      { task: this.unitPriceTask, name: 'unit prices' },
      { task: this.totalValuationTask, name: 'total valuation' },
      { task: this.interestEarnedTask, name: 'interest earned' },
      { task: this.dcaTask, name: 'DCA' },
      { task: this.reinverstEarningsTask, name: 'reinvest earnings' },
      { task: this.poolRebalanceTask, name: 'pool rebalance' },
      { task: this.hyperPoolsTask, name: 'hyper pools data' },
      { task: this.deleteRecordsTask, name: 'delete old records' },
    ];

    for (const task of tasks) {
      try {
        await task.task.execute();
      } catch (err) {
        this.logger.error(`Error fetching ${task.name}:`, err);
      }
    }
  }
}

Adding dependencies

It is time now to add the project dependencies.
For the previous project I decided to add dependencies to the specific project and not hoisting them.
This decision was mainly taken because there were some web3 related dependencies using different versions in different projects (like ethers that was used with version 6.x.x in baluni-contracts and 5.x.x in other projects), and it was not in the scope of this iteration to standardize the libraries versions, also because I don't know much those tools (as I don't have yet much experience with web3 developement), so I opted for a conservative approach.

In this project, however, I am entering a field where I fill much more comfortable as I have a lot of experience with Express.js and NestJS.

So, this time I added the original project dependencies to the monorepo's root package.json

Running and building the project

As this is a quite simple project, I don't expect much to do in order to make things work.
I just had to verify if the .sqlite file coming from the original project was being taken in consideration in this NestJS project.

I saw that the NestJS project created by NX already has a folder called assets that was already configured with webpack

webpack.config.js

const { NxAppWebpackPlugin } = require('@nx/webpack/app-plugin');
const { join } = require('path');

module.exports = {
  output: {
    path: join(__dirname, '../../dist/packages/baluni-backend'),
  },
  plugins: [
    new NxAppWebpackPlugin({
      target: 'node',
      compiler: 'tsc',
      main: './src/main.ts',
      tsConfig: './tsconfig.app.json',
      assets: ['./src/assets'],
      optimization: false,
      outputHashing: 'none',
    }),
  ],
};

So I ran the project :

run baluni-backend:serve --configuration=development

and once fixed some typescript issue and making things to be imported from baluni-core, it worked as expected.

You can find the full final code here

In the next chapters we'll see how to integrate the web project and how to create the CLI app.

I will be very happy to answer to any question you will leave in the comments.

If you liked this article and would like to support my work, you can leave a like or follow me here on dev.to.

DEV Community

Creating a web3 DApp with a NX Monorepo - 3 of 5: Create backend app with NestJS

Understanding the project

Integrating the code - sqlite support

@homeofthings/nestjs-sqlite3 - npm

Integrating the code - controller/service/repository

Integrating the code - recurring tasks

Adding dependencies

Running and building the project

Top comments (0)

Read next

How a Viral Tweet Landed Me Multiple Job Interviews

Launching my first game soon!

Mastering Currying in JavaScript 🌟

Devcon 2024 Highlights, Smart Contract Revolution, Yoav Weiss on AA Future and AA Developer Survey