Concourse a CI/CD tool

Introduction

CI/CD: It's the Rockstar of software development – everyone's talking about it! It's more than just a process, it's a whole way of life (well, maybe not that dramatic), but it definitely brings devs and ops together to ship awesomer software, faster and stronger . That's where Concourse comes in to be your CI/CD hero. Let's unpack this nifty tool and see how it makes your DevOps life a breeze.

why I choose concourse?

Before I dive into Concourse, let me tell you why I chose it in the first place. We were using Buildbot previously, which is a great tool by the way (Python.org even uses it!). However, I found Buildbot lacking in two crucial areas: documentation and configuration as code

Documentation: If you head over to the Buildbot docs, you'll find the API documentation isn't as descriptive as the Python docs themselves (maybe I'm wrong… 😅 ). This led to some less-than-ideal CI/CD workflow implementations by our old team.

Configuration as YAML: Why YAML? It lets you focus on building the workflow itself, not writing tons of code. Buildbot gives you more control over how you build your CI/CD workflow, but it makes you focus more on the code itself rather than the tasks at hand. So, for configuration as code (like Ansible), I believe it's best.

So, when I was tasked with building another workflow on Buildbot (being very new to CI/CD at the time), it was a disaster for me (don't worry, I was decent at Python!… 😜). I wrote the pipeline, but it took me a month and a half to wrap my head around the whole codebase. It was a pain to write and maintain new workflows

We needed a simple solution for our CI/CD pipeline. We were already using Docker to ship our code, so our workflow mostly involved building images and compiling source code

I explored several open-source, self-hosted tools. Concourse stood out because of its ease of deployment and maintenance thanks to YAML. Now, I know Concourse might seem like overkill, but hey, if a future team wants to use Kubernetes or other fancy stuff, Concourse can handle it all!.. 😏

Key Features of Concourse

Concourse is known for its expressive, versatile, safe and stateless architecture. It is designed around a few core principles: every task run in isolation, configuration as code, and having a declarative style. This means you describe your pipeline, and Concourse will figure out what needs to happen to get there. This approach greatly simplifies pipeline management and reduces the risk of errors.

Configuration as code

resources:
- name: booklit
  type: git
  source: {uri: "https://github.com/concourse/booklit"}

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: test
    file: booklit/ci/test.yml

Visualize you pipeline

why we switch to concourse?

First up, Concourse is BFFs (Best Friends Forever 😝) with test-driven development (TDD) – they practically speak the same language (well, robot framework at least ) Second, its fancy visualization makes your pipelines crystal clear at a glance, troubleshooting becomes a walk in the park. Third, it's all about containers, meaning consistent builds and environments throughout the software lifecycle – a big win since we're basically living in Dockerland anyway. Plus, bootstrapping with Concourse pipeline is a breeze compared to Buildbot – like night and day!

Cons… 😮‍💨

Concourse ain't perfect, though . It can be a bit sluggish sometimes, especially with Docker builds because of those annoying volume creation drivers (there's a whole GitHub issue on that if you're curious ). There's also a bit of a learning curve, which can be a hurdle for new devs to jump over.

All things considered, our pipelines take about 40% longer to run in Concourse compared to Buildbot – not ideal for speed demons.

But hey, at least our code is way more streamlined now, right? 🤐 Plus, Concourse lets us dedicate workers for specific tasks like GPU build and CUP build, versioning and complex workflows – those little things add up! We're constantly learning and getting better at wrangling this beast of a tool, day by day.

Conclusion

Concourse: It's like the Swiss Army Knife of DevOps tools 🇨🇭. Packed with features and benefits, it fits right in with the whole DevOps thing . Need to pimp your CI/CD pipeline, boost transparency, and support TDD? Concourse could be your BFF . But remember, just like any fancy tool, It's all about using it right to achieve Moksha of DevOps !!