Strategies to ship faster, and keep your development pipeline unblocked at scale

When your team size grow, you start seeing a lot of merge conflicts. Few people are making changes on same files, you have this big feature to ship, should you let smaller PRs related it to be merged on staging so other dev working on that feature can use that piece of code. Someone is blocked because someone else is working on a module that's needed. You can't even think of refactoring causing so many people are making changes on those parts of codebase.

Well that chaos is worth the diving deep. Let's first see what's very common developing flow, here I might be referring github once a while but these strategies works with any other alternatives too.

Mostly you have this master/main branch - github create this by default when you create a repo - this should be your single source of truth any code merged to this branch should be tested well and have no any know bugs. This should be your production code, all CI/CD should be automated for this branch at least.

Next is your dev/staging branch, its good practice to have an intermediary stage between your development and your production code. Often this branch is also deployed to some staging deployment so devs/QA can test on real production environment. You merge your branch into this staging branch and once a while rollout to production by merging this staging branch into master/main, often these rollouts are scheduled but frequency depends on team size and the development acceleration. Better to do smaller rollout to minimise bugs and impact they can have on users. But on other hand rollout means new deployments on production which might cause some disturbances - new instances of app could cause inflight request canceled, or delay in page loads etc - you find a good balance for your team here.

It is important to keep staging and master in sync, and merge only staging branch to master no other branches or direct commits to master.

Trunk vs Atomic branches

Normally you branch out from staging create a feature branch, you make changes, raise a pull request and once reviewed and approved by others you merge it. But what if you have to work on a bigger feature, multiple other devs will also be working , you might be experimenting and different implementations for that feature, You can't create once big branch let many people push their commits in it, it will raise conflicts. You might just want to work on your part and let others create separate PR and work on their part but what if they need some of your code too. You both need one utility function to make your code work.

Their is a development strategy to make this easier, often referred as Trunk-based development . So now you create a trunk branch, which will have all changes related to that specific feature.let's say you are adding login with Google feature to your website. You can create a trunk branch, and dev's branch out from this branch to work on their individual tasks. Once their atomic branches are reviewed and approved, they get merged to this trunk branch and when you have this feature ready and tested you merge your trunk branch into staging, and rollout. Often these trunk branches are deployed too to make it easy for non-dev team to test it experience it.

Next we will talk about testing and how to optimise CI to make sure your test run for only modules that are changed in the PR and not all tests as that might slow down your development and cost you avoidable computation minutes.