Have you found your company to be struggling with getting features to production quickly? Is there a big blocker somewhere in your process? It could be that one of the culprits is the way you’re using Git.
This article started as an internal article within MetaHi, therefore it may contain references to problems specific to our company, so don't take it gospel. But in the grand scheme of things, I think that a lot of startups struggle with similar problems, so that’s why I decided to share this writeup, with minimal tweaks.
If it makes you feel better, consider this another shill for a Continuous Integration/Delivery/Deployment process.
What we’re trying to achieve
We want features to appear in production as soon as possible, with the least amount of bugs, and the most value to the user. A great way to achieve this is to practice Continuous Delivery. Here’s some terminology that we should clear up first:
- Continuous Integration — builds and tests software after every push. It does not deploy the build.
- Continuous Delivery — deploys every change made to the code to a staging environment, and allows for a push to production on demand.
- Continuous Deployment — takes it even further, every code changes going through the pipeline will merge into production automatically.
We should strive to achieve Continuous Delivery at least. This means that the code you push should not break the builds, which means that you MUST run the tests and code locally before pushing, and should fix any failing tests immediately.
Here’s a list of requirements. Print it and glue it in front of your eyes:
- Features should appear in Prod ASAP
- Tasks should not block the release of other tasks
- Code in
masterMUST be production ready (building out of the box, and no failing tests) - Commits in
masterMUST have proper commit messages - Any API release must be backwards compatible, unless otherwise agreed with Frontend devs
How to achieve it?
Discipline and trunk based development. You need to have discipline to write proper tests for the features you are developing. You need to have discipline to test your code every time before pushing. You need to have discipline to not develop crap, even when you know you are developing throwaway code.
Trunk based development
Trunk based development means that only a single branch is considered the main one. The less time your code spends detached from the main branch – the better. When you are striving to stay on the main branch you will be making smaller commits, thus ensuring that the changes you make are less risky and can be rolled back easily. It will also be easier to integrate with your colleagues, because there will be less merge conflicts. I’m not advocating against feature branches, those are still ok in some cases, but they should not be living for more than a couple of days.
What about the QA step and dev? Nothing in the strategy above conflicts with Dev or QA. You are still free to independently tag your commits with dev tag, so that it gets built into the dev environment. And QA can work either in dev environment, or directly in production. But how do we ensure that users don’t see untested features in production? And the answer to that is feature flags.
Feature flags
Feature flags allow teams to modify system behavior without changing code. You can read an in-depth article about feature flags on Martin Fowler’s website. In practice it means that for any feature or behavior change you want to hide from the user, you should make a config variable. And then taking that config variable into account is as easy as writing an if statement.
However, not every new feature warrants a feature flag. If you are developing some new endpoints that are not going to have frontend attached for a while, there’s no need to make a feature flag, because end users will not be able to reach it by normal means anyway.
A real example of where you would use a feature toggle in the config: let’s say there’s existing code for filtering a list, but it’s not good code. You want to develop something more robust, while the crappy code still remains working. The first commit you would make to master, would be config param something along the lines of use-new-filtering. Nothing has changed and this should be safe to deploy to production. You can then make an if statement in your controller to take this configuration param into account and call an appropriate service (either OldFilteringService or NewFilteringService). And then you are free to actually continue developing your new filtering functionality, continually deploy to prod, and all while still keeping the old code working. Dev environment can have use-new-filtering=true, and QA can keep testing it while you are working, and production environment can have use-new-filtering=false in order to use the old and approved code. Once the code is ready and tested, it’s just a matter of flipping the flag in production to make use of the new code.
Versioned endpoints
Talking about endpoints – they can actually act as a method of feature flagging themselves. If you have an endpoint that has been running on production for a while and you want to fundamentally change it (ie introduce breaking changes), one way to handle that is to create versioned endpoints. Frontend can still use the old /do/something endpoint for the time being, and you can actively be developing /v2/do/something without breaking the first one. When the endpoint is ready, frontend will just switch the URL, and it should be good to go. When old endpoint is no longer actively in use and everyone has confirmed that the new endpoint works correctly, you should take some to remove the older endpoint so that it doesn’t clutter the codebase and confuse other devs.
Versioned endpoints supports our goal of having our code backwards compatible, so that existing features and frontend doesn’t break.
Branching
Let’s get back to trunk based development a bit. As mentioned previously, trunk based development does not forbid you from actually using branches. I would even encourage using them, for more isolation of your changes, and nicer looking diffs when merging. However your branches should live for as short time as possible. We’re talking hours here, or couple of days at maximum.
In practice – all your feature branches should be branched off of master branch. Since there’s a rule that master must always be production ready and work out of the box, it is the safest place to base your code off of.
When the feature is done, it should be merged back to master. Doing it through a PR, or a simple merge is entirely up to you, just make sure it ends up back in master when it’s ready and not breaking.
Branch naming
And let’s for a moment talk about branch naming. Branch names should be as short as necessary, while still communicating their intention as precisely as possible. Prefixes, such as feat/, bugfix/, chore/, and similar are just noise, and let me tell you why.
I would argue that any bug fix should be merged as soon as possible. This means that the branch will live for a couple of hours tops. Since you are writing tests (you are writing tests, yes?), you will write a test to specifically test for the buggy condition. Your test will fail at first, and you will develop a fix in a couple of hours. When your test passes, you know that you have fixed the bug, you are free to merge. But wait, this is most likely a single commit, therefore it doesn’t need its own branch and can be done directly on master. You don’t need to wait for QA to test this, because you have already verified that you’ve fixed the bug by running the tests.
Now let’s talk about chores. What is a chore? Cleaning your room is chore. Taking out the trash is a chore. One would argue that refactoring code is also a chore. But why the hell are you refactoring code just for the sake of it, as a separate task? Code refactoring should be done as a preparation of developing a particular feature that brings value. Refactoring is a part of a feature or a bugfix, and not a separate task on its own. Therefore in reality you won’t see or create branches prefixed with chore/. If anything, your refactoring commits will go inside your feature branch.
This leaves us with feat/ or feature/. Since we have already established, that bugfix branches will either live for a couple of hours or be done on master directly, and there’s no such thing as a standalone chore, then every branch will be some sort of a feature branch. If every branch is a feature branch, why prefix it at all?
Finally we’re left with ticket number prefixes, such as PRJ-123. I don’t have a strong opinion on those, as I can see their value for quick reference. But that’s only valid for a case where there will be multiple people checking the branch (like in PR for example). But you could also just add the ticket number to a PR title, and that would work just as good, if not even better.
One thing that I have a big problem with is NO-JIRA prefixes. What value does this prefix bring? If there’s no ticket for a task, it’s either a quick fix and will either be done on master directly OR the task is too big, and a ticket needs to be created anyway.
Shared dev
In rare instances, it could happen that multiple features can’t be merged to master, but still need to be tested on dev environment. For example you are working on task A and your colleague is working on task B at the same time. You are both working on your own feature branches, but yet you still want both features on dev at the same time.
Enter the shared-dev branch. This of course requires clear communication between the team members, but what doesn’t? How you name the branch doesn’t matter at all, as long as people are aware of what’s going on. This shared-dev branch should be branched off of master, and then your branches should be merged into it. After the branches are merged, the shared-dev branch is tagged with dev tag, so that it builds and deploys into dev environment. When changes are made on a feature branch, feature branch again gets merged into shared-dev and tagged with dev tag.
As soon any feature branch appears in master (read - is ready for release), the shared-dev MUST be deleted to avoid confusion.
Deleting branches
Talking about deletion. For the love of whoever is your god – delete those old branches. If your feature is merged and deployed, there’s no need to keep that old branch and watch as it falls behind hundreds of commits. You can always create a new branch if you need to.
Commits
You should strive to separate your commits into standalone units of valuable code. Think of it this way – if you need to include and word into your commit message, maybe you’re actually committing too much (ex. Add this AND change that).
Commit messages should be human readable, and have the most important information upfront. Since we’re not using any automation tools (if we start using them, we will reevaluate this), we shouldn’t prefer cryptic tags fix(), feat() and similar (comes from Conventional Commits, including link just for curiosity) at the start of the message. The ticket reference (PRJ-123) should go at the end of the message, because it does not communicate immediate information, without having to reference a 3rd party system.
You should definitely read this in depth article about good Git commit messages.
Imagine that after doing a commit, you would go on Slack and describe the changes to your colleagues in a message. A great place for that message is actually the commit message body. Add a blank line below your commit message and explain why you made the change, and possibly how to use it.
A good commit message should look something like this (completely fictional):
Add metadata to asset details. PRJ-123
Asset metadata was missing from asset detail page. Metadata is
needed for OpenSea and similar 3rd party systems in order to
display the asset correctly. Metadata should now be included in
the asset details endpoint.
Pull Requests
Pull requests are inefficient, change my mind. First, they stop code from appearing in master which we have already established is a bad practice in itself. Second, in order to properly conduct a code review (and I mean properly, not only check the indentation), you probably need to spend just as much time as writing the original code. Therefore a better practice is pair programming – in the end just as much time is spent by a couple of people as with PRs, but the added benefit is that the feedback is instant and can instantly be addressed.
I’m not advocating against pull requests, if you want someone to quickly go over your code just to double check for those low hanging fruits, but I’m also saying you don’t need to ask someone’s permission to do your job.
On a side note, a senior developer’s approval on a PR does not mean that suddenly you’re off the hook and anything bad that happens with your code is now the responsibility of the approver. It is your responsibility as a professional to make sure that the code meets business requirements and is factored to the best of your abilities.
What doesn’t work
Throughout many years, me personally, and the industry as a whole has learned new things. We’ve also learned what doesn’t work, or doesn’t work in our context at least.
GitFlow
One of such things is GitFlow. GitFlow is a branching model created by Vincent Driessen more than a decade ago. The author himself, has stated that this is an outdated model, and should not be used in continuous delivery environment. Original article here.
Standalone dev branch
Some of you may remember times when dev was a separate branch. A big problem with that is the same as with any other long lived branch – it tends to quickly diverge from master. Especially if you have many features that are tested on dev, but never end up in production. There comes a point where the whole dev branch needs to be deleted and then branched off of master again. And that comes with its own sets of pain.
Outro
So this was my heavily opinionated article about how Git should be used. If you disagree, please voice your discontent in the comments to boost the engagement. But in all seriousness, let's discuss the process that you find works in your context – I love learning how to improve the efficiency.
Top comments (0)