WIP: Taskfile instead of Makefile?

Recently, I stumbled upon a tool called Taskfile. It is a task runner, similar to Makefile, but with many improvements. I decided to try it and see if it can replace my Makefiles.

Why Taskfile?

First of all, Makefile is a good tool. Today, you can use it on any platform, not just Linux. It is a standard tool for running tasks. It is flexible and powerful. My relation with it is love/hate. It can be tricky to write a good Makefile or to debug it. Reading also can be a challenge. Because of that, when I found out about Taskfile, I decided to try it.

First impressions

Taskfile is written in Go. It is a single binary that you can download and use. If you can run Go programs, you can run Taskfile. It is a big plus for me. See docs on how to install it.

Taskfile is a YAML file. Most of us are familiar with YAML, and many tools use it. It is easy to read, write, and parse, and IDEs have good support. Sometimes, it feels like we are programming in YAML. If you do not believe me, check out your CI/CD pipeline. I bet it is written in YAML.

You can use auto-completion if you use ZSH, Bash, Fish, or PowerShell. It is a big plus for me. I like to have auto-completion for my tools. It makes my life easier, especially when you start adding more and more tasks. When you have a lot of tasks, they can be grouped into namespaces, hidden from a user(internal task), global, or local, etc.

Challenge: automate testing for Klock

Klock is my pet project. It is a ValidatingAdmissionWebhook for Kubernetes. It enables locking Kubernetes resources. You can read more here Klock - Kubernetes locking.

One of the things that was on my to-do list was to automate testing. I'm doing it locally because it is easier and faster. But still, it is cumbersome to start the Kind cluster, deploy Klock, and run tests. I decided to use Taskfile to automate it.

What I need to do is:

1. Start Kind cluster
2. Install Cert Manager
3. Build K8s manifests
4. Build the Docker image if there are any code/configuration changes
5. Load the image into a Kind cluster
6. Run tests

You can find my initial work on the git branch migrate_taskfile. If you compare it with the main branch, you'll notice I added a new file called Taskfile.yml. It is a main Taskfile. Also, I added the folder tasks with two files, Taskfile.testing.yml and Taskfile.tools.yml. They are imported into the main Taskfile. The idea is to have a main Taskfile that will import other Taskfiles, while other Taskfiles will contain specific tasks.

Start Kind cluster

First, I need to start the Kind cluster:

version: '3'

vars:
  KIND_VERSION: "v0.20.0 go1.20.5"
  CERT_MANAGER_VERSION: v1.12.0
  IMG: controller:latest

tasks:
  init-cluster:
    desc: Create a kind cluster named klock
    preconditions:
      - sh: kind version | grep "{{.KIND_VERSION}}"
        msg: "kind version does not match {{.KIND_VERSION}}. Please install right version of kind"
    cmds:
      - kind create cluster --name klock
    status:
      - kind get clusters | grep klock

The slug tasks contain a list of tasks. This task is called init-cluster. It has a description, preconditions, commands, and status. For any task name and cmds slugs are mandatory. Others are optional. What they do:

• cmds - list of commands that will be executed
• desc - description of the task, it will be shown when you run list tasks: task -l
• preconditions - list of preconditions that must be met before running the task. If any of the preconditions fail, the task will not be executed. In this case, I check if the Kind version is the same as the one I'm using. If not, the task will fail with the message.
• status - list of commands that will be run to see if a task can be considered done. That means the task is done if the Kind cluster named klock exists. It does not care how a cluster is created. It just checks if it exists.

Notice block vars. It is a list of variables that can be used in the Taskfile. In this case, I'm using KIND_VERSION to check if the Kind version is the same as the one I'm using. The Taskfile uses [Go template (https://golang.org/pkg/text/template/) to render variables.

Install Cert Manager

Next, I need to install Cert Manager:

  install-cert-manager:
    desc: Install cert-manager    
    deps:
      - init-cluster
    cmds:
      - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/{{.CERT_MANAGER_VERSION}}/cert-manager.yaml
      - echo "Waiting for cert-manager to be ready" && sleep 25
    status:
      - kubectl -n cert-manager get pods | grep Running | wc -l | grep -q 3

The new thing here is the deps slug. It is a list of tasks that must be executed before this task. In this case, I need to create a Kind cluster before I can install Cert Manager. Again, I'm using a variable CERT_MANAGER_VERSION to install the correct version of Cert Manager.

Build K8s manifests

To generate manifests, I'm using kubebuilder tool contrller-gen:

 manifests:
    desc: Generate manifests e.g. CRD, RBAC etc.
    cmds:
      - controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    sources:
      - main.go
      - apis/**/*.go
      - pkg/**/*.go
    generates:
      - config/crd/bases/**/*.yaml

This time I'm using sources and generates slugs:

• sources is a list of files that will be used to generate manifests
• generates is a list of files that will be generated

In this case, Taskfile will generate a checksum for sources from the previous run. If the checksum is the same, it will not run the task. It will just print out that the task is done. If the checksum is different, it will run the task. It is a nice feature that can save you some time.

Deploy Klock to Kind cluster

To deploy Klock to the Kind cluster, I'm using kustomize:

  deploy:
    desc: Deploy the controller to the kind cluster and wait for it to be ready. Use IMG to specify image name
    deps:
      - install-cert-manager
      - manifests
      - docker-build
    cmds:
      - task: kind-load
        silent: true
      - cd config/manager && kustomize edit set image controller={{.IMG}}
      - kustomize build config/default | kubectl apply -f -
      - echo "Waiting for controller to be ready" && sleep 25
    status:
      - kubectl get pods -n klock-system | grep klock-controller | wc -l | grep -q 1

This task depends on install-cert-manager, manifests, docker-build, and kind-load tasks. It means that those tasks have to be executed before this task.

But a small catch, kind-load can be done only if docker-build is done successfully. Other tasks can be executed in parallel. That is why other tasks are in the deps slug. Tasks in the deps slug will be executed in parallel. Tasks will be executed in order while in the cmds slug.

I'm adding a new slug, silent to suppress the task output.

This way, I managed to do some tasks in parallel and speed up the process.

Run tests

To run tests I'm using KUTTL:

  ktest:
    desc: Run kuttl tests. Specify image name with IMG
    cmds:
      - task: deploy
        silent: true
      - kubectl kuttl test

There is nothing special here. Except I should move the deploy task to the deps slug. But this is still a work in progress.

Putting It All Together

My main Taskfile looks like this:

# https://taskfile.dev

version: '3'

includes:
  tools: tasks/Taskfile.tools.yml
  tests: tasks/Taskfile.testing.yml

tasks:
  default:
    silent: true
    cmds:
      - echo "Welcome to Klock!"
      - task -l

I'm including two other Taskfiles. One is for tools, and the other is for tests. The default task is to print out a welcome message and list all tasks. Invoking the task without any arguments, it will run the default task:

$ task
Welcome to Klock!
task: Available tasks for this project:
* tests:cleanup:                    Delete the kind cluster named klock
* tests:deploy:                     Deploy the controller to the kind cluster and wait for it to be ready. Use IMG to specify image name
* tests:docker-build:               Build the docker image, specify image name with IMG
* tests:docker-push:                Push the docker image, specify image name with IMG
* tests:init-cluster:               Create a kind cluster named klock
* tests:install-cert-manager:       Install cert-manager
* tests:kind-load:                  Load the docker image into the kind cluster, specify image name with IMG
* tests:ktest:                      Run kuttl tests. Specify image name with IMG
* tests:manifests:                  Generate manifests e.g. CRD, RBAC etc.
* tests:publish:                    Run tests and publish the docker image. Specify image name with IMG and version with VERSION
* tests:undeploy:                   Undeploy the controller from the kind cluster
* tools:controller-gen:             Download controller-gen locally if necessary.
* tools:create-localbin:            Create the localbin directory
* tools:delete-kustomize:           Delete kustomize
* tools:envtest:                    Download envtest-setup locally if necessary.
* tools:install-kustomize:          Install kustomize

To run the tests, I'm using task tests:ktest IMG=rnemet/klock:test. It will run the tests:ktest task and pass the image name to the task. This image name is propagated to other tasks that need it. As you can see, I grouped tasks into namespaces. I'm using the tests namespace for tasks related to tests and the tools namespace for tasks related to tools.

Conclusion

I'm still learning Taskfile. There are better ways to do things. But I'm pleased with the result. Still, I need to migrate other Makefile targets to Taskfile. But the results are promising. Changes are more readable, easier to maintain and to understand. When I migrate the rest of the Makefile targets, I can compare Taskfile and Makefile with more insights. But at the moment, Taskfile is a winner.

Looking into Taskfile Github repo, I can see that the project is active, and much work is underway. I'm looking forward to new features and improvements.

This is a work in progress. I already have ideas on how to make it better. But I will leave that for another blog post. Meanwhile, you can check Taskfile and let me know what you think. Thanks for reading.

References

• Taskfile