This is a story about how I turned this, this, this and this projects into this NX monorepo containing a full web3 Dapp.
At the end of the story we'll turn this process into a web3 dapp starter kit repository.
I will not drill down to the details of the code, as it is not the focus of this series (and actually needs to be strongly refactored), rather than I want to highlight the main challenges and the problems I encountered (and how I solved them ), and the approach I used to quickly make this migration starting from a never seen before codebases and without any web3 development knowledge.
Let’s start by giving a little bit of context:
A friend of mine asked me to help him with an opensource web3 project, called Baluni, he is working on. This project helps balancing assets in your portfolio, but talking about this is not in the article's scope.
He his a brillant solidity developer with quite good programming skills, but he does not identify him self as a professional developer (he is a musician actually) even though he can code much better than a lot of people that identify them self as professional developers 😁.
He asked me to join the project to help him giving a more professional and maintainable structure to the project that, at that moment, was actually quite bad structured as it consisted of the following distinct projects:
- BALUNI CORE (link): It is the project that contains the core functionalities plus it has been somehow configured to run as a kind of cli, we'll see it later.
- BALUNI CONTRACTS (link) and BALUNI HYPERVISOR CONTRACTS (link): These 2 projects are the projects with the core smart contracts and they must be consumed by other projects in the form of an npm library
- BALUNI UI (link): this project is a monorepo itself, and it contains an ExpressJs server and a NextJS app
I hadn't any doubt about the fact that a monorepo was absolutely needed here, and either I hadn't any doubt about the tool to use: NX.
I used it in the past for small projects, but I was really impressed of it, so I was excited to use it for this job.
Even if, as I said, I don't have any knowledge about web3 development, I was able to analyze the projects, understand their dependencies and the relationships between the project. I planned the migration to be implemented as a monorepo with the following projects:
- baluni-contracts: library with contracts
- baluni-hypervisor-contracts: library with contracts
- baluni-core: core functionalities extracted from baluni project
- baluni-cli: the cli was badly inglobated in the core, I'll make a dedicated project for it
- baluni-web: extracted from baluni-ui monorepo
- baluni-backend: extracted from baluni-ui monorepo
So basically the idea is the following:
- Make a library out of the projects containing the contracts
- Make the baluni-core library which will use the contracts libraries (and exposes them)
- Make the baluni CLI application which uses baluni-core
- Make the baluni backend which uses baluni-core
- Make the baluni web app which uses baluni-core and baluni-backend
So, stop talking and let's make our hands dirty.
CREATING THE MONOREPO
Before all, I started a blank NX project following their guidelines, so I just executed the following command:
npx create-nx-workspace
It started an interactive cli asking me about some preferences, like this
I choose to don’t use any preset as I want to start as clean as possible and and project one by one and keep track of the process. Let's go ahead
At this point I choose the first one.
For the curious ones, this is the difference between these options (taken from their docs):
- Standalone Application - A repository with a single application
- Package-Based Repository - A repository with multiple projects that depend on each other via package.json and often have nested node_modules
- Integrated Repository - A repository with multiple projects that depend on each other via typescript imports and often employ a single version policy
I choose the first option because my app is not a stand alone one and I didn’t want to enter the “single version policy” world at this stage.
Here I went straight and choose to do it later as I don’t want to care about it for now
They try to onboard you to their cloud services, I don’t care now about it, maybe I will write a dedicated article about it
At this point NX will work a bit and will download and setup the project.
As I didn’t selected any preset it was quite quick, but if you choose a preset it will need to download and install all the related dependencies.
Ok, now the project is created
Since we have chose to start with a blank project, tt contains a bare bone application with nothing much to talk about.
Things will get interesting along the way and you will discover the power of NX when we'll start adding projects.
Along the way, we'll see nice things about NX, like
- how to create typescript libraries with NX and how to make them publishable on npm
- how to add custom assets to libraries
- how to avoid hoisting dependencies for a single project
- how to generate a package.json in the build output with the used dependencies used by the package
- how to create and easily run a CLI application inside an nx monorepo
and many more things.
This serie or article will follow with the following articles:
- Baluni Contracts: creating a npm library in a NX Monorepo
- Baluni Core: creating a typescript library (baluni-core) in an nx monorepo that depends on other packages
- Baluni CLI: creating a CLI application in an NX project
- Baluni Backend: creating a NestJS backend in an NX monorepo
- Baluni Web: creating a web3 NextJS application in an NX monorepo
You are very welcome to leave your comments, like the post and follow me if you liked this article and are interested in the following ones
Top comments (5)
Can you please elaborate on why?
Hi Fyodor,
thanks for your interest on this point.
Well, the main point that made me decide for a monorepo solution is the project consisting of many different projects depending each other.
From a developing perspective, having these projects in the same monorepo allows you to quickly apply code changes at any level and to easily share types, business logics and assets between projects, giving you a better developer experience.
Otherwise, having those projects as single separate repositories, would become more difficult to achieve the same goal.
Maybe you can obtain the same effect through some kind of local linking and maybe will somehow work, but this is an open source project and wanted to make the life easy for contributors.
Downloading just one repository with all projects automatically linked together is much more straightforward than downloading all those repositories separately and follow all the steps for running
npm linkoperations that often works differently on different machines.Got it, thanks for the details 👍 From the technical standpoint, didn't you feel that
nxadds some overhead to the solution? I mean, it has its own mechanisms which introduce additional boilerplate, scripts, and abstractions, in addition to the oppinionated project structure. But from the post I can see that you selected a kind of a minimal setup?Now, there are some more "native" and less intrusive solutions around, like workspaces provided by the package managers, and also some commercial alternatives similar to
nx, liketurborepo. And of course, just a manual project structuring to different folders inside a single repo (like, for instance,llamaindexproject starters do, which I like a lot — away from the LLM specifics themselves — for their thorough and clever approach). I'm curious, didn't you consider any of alternative options like that, and if yes, what narrowed your choices tonx?This monorepo story is interesting to me because I sometimes need to weigh in different approaches too, but lately I come to separate repos mostly. I hate the incurred complexity the off-the-shelf solutions suggest and lack of robustness of the manual project combining approach. And specifically, I have my own pain history with
nx😅Also, how good is the
nxsupport for web3 ecosystem projects you mentioned? Do they work smoothly, or you needed to add some custom plugins/builders/etc?Hi Fyodor,
sorry for the late reply.
I'll be very happy to share my thoughts on your points:
Yes. when you create a NX monorepo, you can chose to start with some presets but also to start with a blank project, and that's what I did.
It created a very minimal project setup and it allowed me to keep a good control over the process.
Since NX is a kind of framework, it of course adds a layer on top of things, but I think it is a facilitating layer rather than complicating things.
Sure, you have to learn how NX works, especially if you want to customize things like build steps, but what they gives you out of the box is fairly enough to get started very quickly.
Except for the
project.jsonandnx.jsonfiles, that are actually what will make your life easier, NX doesn't actually add any boilerplate IMHO; as you can see from my screenshot, what you get generated is nothing more than what you'd get but creating a plain yarn workspace. Of course you have to learn how to build and run things, but they have a stunning documentation and getting started is really easy.Good question.
The first thing I excluded for sure was to start with plain things, like workspaces provided by the package managers. In my past experience, handling dependent projects together can become a daunting task.
For example in my project, when I now build baluni-core, which depends on the contracts projects, NX will take take to build them first, and it's everything out of the box.
I was also tempted to use turborepo, they recently launched the v2 which looks promising, but by reading the docs the do almost the same things as NX (providing you generators, build scripts etc..), but since I already had a little experience with NX it didn't make sense for me to start from scratch with a tool I had to learn from scratch (as I had to put the minimal possible effort on the project, I have a very small time to allocate on it).
NX, moreover, is a more mature solution, Turborepo only came up on 2021 borrowing a lot of ideas from them.
On top of this things, NX also gives you some cool things, like module boundaries or project graph.
Well, NX doesn't limit you in any way about the things you can do,
Everything that you can do with a single npm project it can be done with NX is guess.
The only problem I may foresee, is for example running hardhat scripts with hoisted dependencies (as explained in the article, I didn't hoisted them yet), but that is a problem not specific to NX it self, as it could happen with any kind of monorepo.
Nice, thanks a lot for the insights 👍 yes, in this case
nxsounds like a perfect fit indeed