CI/CD enables the best tech companies to improve their products many times per day. Here’s what you need to know to do the same.
What is CI/CD?
CI/CD is a way of developing software in which you’re able to release updates at any time in a sustainable way. When changing code is routine, development cycles are more frequent, meaningful and faster.
“CI/CD” stands for the combined practices of Continuous Integration (CI) and Continuous Delivery (CD).
Continuous Integration is a prerequisite for CI/CD, and requires:
- Developers to merge their changes to the main code branch many times per day.
- Each code merge to trigger an automated code build and test sequence. Developers ideally receive results in less than 10 minutes, so that they can stay focused on their work.
The job of Continuous Integration is to produce an artifact that can be deployed. The role of automated tests in CI is to verify that the artifact for the given version of code is safe to be deployed.
In the practice of Continuous Delivery, code changes are also continuously deployed, although the deployments are triggered manually. If the entire process of moving code from source repository to production is fully automated, the process is called Continuous Deployment.
A long deployment process is error-prone and distracting. Developers get frustrated by a process that feels like an overkill. Teams trade small iterations for risky big-bang releases. They may also be led not to make some small improvements at all.
The goal of CI/CD is to make the deployment easy enough, safe and fast. In such context, the probability of introducing major bugs is low. When something bad does happen, it’s easy to deliver a fix or revert the change.
A litmus test for doing CI/CD
If any developer in your team can stop what they’re doing right now and ship the current development version of code to production in 20 minutes or less without anyone stressing about what could happen — congratulations, you’re doing CI/CD!
CI/CD principles
Continuous Delivery practices take CI further by describing principles for successful production deployments:
- Architect the system in a way that supports iterative releases. Avoid tight coupling between components. Implement metrics that help detect issues in real-time.
- Always keep the code in a deployable state. Maintain a comprehensive and healthy automated test suite. Build in monitoring, logging, and fault-tolerance by design.
- Work in small iterations. For example, if you develop in feature branches, they should live no longer than a day. When you need more time to develop new features, use feature flags.
- Developers can push the code into production-like staging environments. This ensures that the new version of the software will work when it gets in the hands of users.
- Anyone can deploy any version of the software to any environment on demand, at a push of a button. If you need to consult a wiki on how to deploy, it’s game over.
- If you build it, you run it. Autonomous engineering teams should be responsible for the quality and stability of the software they build. This breaks down the silos between traditional developers and operations groups, as they work together to achieve high-level goals.
To make CI/CD a reality, you need to automate everything that you can in the software delivery process and run it in a CI/CD pipeline.
Example CI/CD workflows
Here’s a simple example of fully automated CI/CD (Continuous Deployment) pipeline. Nothing shouldn’t be more complicated than necessary!
In this example, each change on the master Git branch performs the following steps on Semaphore:
- Build code and web assets, while reusing dependency cache.
- Run an automated test suite. This is a JavaScript / Node.js application, so the tests are written with Jest.
- If the tests pass, a Deploy block updates the production code that runs in the cloud.
CI/CD with manual steps
Here’s a CI/CD workflow for containers and Kubernetes which includes more steps:
In this example, each change automatically performs these steps:
- Build application from source code and dependencies.
- Run an automated test suite.
- If tests pass, automatically build a Docker container image and push it to a private registry.
At the end of the Docker build pipeline, we have a working artifact, a container image. The developer, or more formally, release manager, can decide to manually trigger:
- Deployment to staging or production, which may include smoke tests to verify no major problems have been introduced.
- Tagging of the container image as an artifact that was introduced to production, to enable audits and rollbacks.
Want more? Read about the benefits of CI/CD and how a typical adoption journey looks like — including when it seems impossible — on Semaphore website.
Top comments (15)
Could you elaborate on this point please:
Should feature branches be short lived so they stay closer in sync to master?
How do feature flags let you restrict unstable work from being used?
Feature flags are basically
So you don’t even load the related code unless the user is part of the dev team or a small group of beta testers. It can be totally unfinished, nobody will be affected. So you can work on it in small iterations and make sure each iteration is well integrated with the system as a whole. Much easier to deal with than a big-bang merge.
Thank you, looks good
Exactly. The longer it lives the higher the chance they will diverge and make it harder to integrate. Not just merge code on line by line level, but make sure it doesn’t introduce unforeseen bugs at runtime (because of how some APIs or dependencies changed in the meantime).
Great piece, Marko! CI/CD is one of the most powerful things to work on and to get right. But to get it right is really hard.
What I also experience is the difficulty in getting the implementation of CI/CD well known within the organization. I am one of the few who do maintain our pipelines and understand how to work on it. So, writing and documenting the implementation of CI/CD is good for future employees and people who never worked on it.
I like the usage of the twenty minutes rule. It also forces you to keep changes small, and let the process of delivery be short.
Would there be a manual check in a Continuous Deployment process? We always do a last check before going to production.
Thank you!
Technically if you have a manual check it’s not “Continuous Deployment” but if it goes fast it doesn’t make much of a difference in my view.
Btw, on the difficulty that you describe, is it about getting devs to collaborate on configuring the pipelines for their projects, or something else?
Great breakdown and explanation of CI/CD Marko! I enjoyed reading about how the two differ and then how the two can coexist. Thanks for sharing your knowledge on the subject.
If you have any sort of branching then you are not doing CI, you are doing promiscuous integration but not continuous integration.
I’m aware of this ideology but I think that branching has too many practical and psychological benefits. :)
I’d be curious to hear how it works in practice with a specific project and team.
If you merge multiple times per day, it implies that average lifetime of branches is less than a day, and that’s as continuous as you can do imo.
Branching generates a lot of integrational and operational cost to the software development process.
why branching on git is wrong
Adrián Norte ・ Oct 27 '18 ・ 1 min read
In there I explained my point of view.
Do you mean it's better to work on
masterand/or a monorepo approach?yes, absolutely.
Gatsby + Netlify: netlify is linked to the git commit, as soon as I commit the repo, it appears on Netlify. Trigger deploy -> job done. I like the sleek workflow like this.
For that kind of a project, that’s a perfectly fine setup!
abusalem.dev - just launched a months ago