DEV Community

Jacob Hummer
Jacob Hummer

Posted on • Edited on • Originally published at jcbhmr.me

Use task.go for your Go project scripts

#go

TL;DR: go run task.go <task_name> makes your scripts cross-platform.

//go:build ignore

package main

import (
    "log"
    "os"
    "os/exec"
)

func BuildEverything() error {
    // Just write normal Go code!

    // Run some commands
    // Copy some assets
    // Create an installer maybe
    // Upload something to GitHub
    // Send an email
    // Do whatever!

    return nil
}

func ConfigureEnvironment() error {
    // Prefer Go over Bash in Go projects.

    // Load some env values from a secret server
    // Save them to ~/.bashrc
    // Create a .env file
    // Install some global CLI dependencies
    // Make sure Python is available...
    // More magic.

    return nil
}

func main() {
    log.SetFlags(0)
    var taskName string
    if len(os.Args) >= 2 {
        taskName = os.Args[1]
    } else {
        log.Fatal("no task")
    }
    tasks := map[string]func() error{
        "build-everything": BuildEverything,
        "configure-environment": ConfigureEnvironment,
    }
    task, ok := tasks[taskName]
    if !ok {
        log.Fatal("no such task")
    }
    err := task()
    if err != nil {
        log.Fatal(err)
    }
}
Enter fullscreen mode Exit fullscreen mode 
go run task.go setup
Enter fullscreen mode Exit fullscreen mode

🤩 It's all just Go code! There's no confusing Bash-isms. \
🚀 It's just a template! Make task.go fit your needs. \
☝ It's all in a single file; there's no task/main.go sub-package stuff. \
✅ //go:build ignore still works with Gopls and intellisense. \
📦 Use tools.go if you need task.go-only dependencies \
😎 Runs wherever Go does; no more GNU bash-specific Makefile. \
💡 Inspired by matklad/cargo-xtask

Start by creating a task.go file in the root of your project. This is where you will define all the tasks that you want to run with go run task.go. The basic template for task.go is this:

tools.go 
//go:build ignore

package main

import (
    "log"
    "os"
    // Your imports here!
)

func Setup() error {
    // Your code here!
    return nil
}

func main() {
    log.SetFlags(0)
    var taskName string
    if len(os.Args) >= 2 {
        taskName = os.Args[1]
    } else {
        log.Fatal("no task")
    }
    tasks := map[string]func() error{
        "setup": Setup,
        // Add more tasks here!
    }
    task, ok := tasks[taskName]
    if !ok {
        log.Fatal("no such task")
    }
    err := task()
    if err != nil {
        log.Fatal(err)
    }
}
Enter fullscreen mode Exit fullscreen mode

There's some more in-depth examples below 👇

Then you can run your task.go tasks like this:

go run task.go <task_name>
Enter fullscreen mode Exit fullscreen mode

How does this work with other .go files?

That's where the special //go:build ignore comes in! When you use go run Go will completely disregard all //go:build conditions in that file even if it requires a different operating system. We can use this fact to conditionally include the task.go file in normal Go operations only when the -tags ignore tag is set (which is should never be). Then we can bypass that -tags ignore requirement using go run to discard the //go:build ignore directive and run the file anyway! Tada! 🎉 Now we have a task.go file which can only be run directly and isn't included in your normal Go library or binary.

Use tools.go for task dependencies

task.go can use all your local go.mod dependencies. The problem is that go mod tidy doesn't see task.go (since it's //go:build ignore-ed) and thus will remove any task.go-only dependencies. To combat this, you are encouraged to adopt the tools.go pattern:

tools.go

//go:build tools

package tools

import (
    _ "github.com/vektra/mockery/v2"
    _ "github.com/example/shellhelper"
    _ "github.com/octocat/iamawesome"
)
Enter fullscreen mode Exit fullscreen mode

📚 https://play-with-go.dev/tools-as-dependencies_go119_en/
📚 https://www.tiredsg.dev/blog/golang-tools-as-dependencies/

./task.go <task_name> with a shebang

💡 If you're smart you can add a shebang-like line to the top of your task.go file to allow you to do ./task.go <task_name> instead of go run task.go <task_name>.

task.go 
//usr/bin/true; exec go run "$0" "$@"

// ...
Enter fullscreen mode Exit fullscreen mode 
chmod +x task.go
./task.go <task_name>
Enter fullscreen mode Exit fullscreen mode

📚 What's the appropriate Go shebang line?

Go doesn't support the #! shebang comment so we have to use the fact that when a file is chmod +x-ed and doesn't have a #! at the top it just runs with the default system shell. The // line doubles as a comment for Go and a command for the shell. 👩‍💻

⚠️ This is officially discouraged by the Go team.

Dev HTTP server

Sometimes you just need a go run task.go serve command to spin up an HTTP server.

task.go 
//go:build ignore

package main

import (
    "log"
    "net/http"
    "os"
)

func Serve() error {
    dir := "."
    port := "8000"
    log.Printf("Serving %#v at http://localhost:%s\n", dir, port)
    return http.ListenAndServe(":"+port, http.FileServer(http.Dir(dir)))
}

func main() {
    log.SetFlags(0)
    var taskName string
    if len(os.Args) >= 2 {
        taskName = os.Args[1]
    } else {
        log.Fatal("no task")
    }
    tasks := map[string]func() error{
        "serve": Serve,
        // Add more tasks here!
    }
    task, ok := tasks[taskName]
    if !ok {
        log.Fatal("no such task")
    }
    err := task()
    if err != nil {
        log.Fatal(err)
    }
}
Enter fullscreen mode Exit fullscreen mode 
go run task.go serve
Enter fullscreen mode Exit fullscreen mode

Using task.go with //go:generate

You may use task.go as a hub for ad-hoc //go:generate needs that go beyond one or two commands. It centralizes all your logic in one spot which can be good or bad. 🤷‍♀️

//go:generate go run ../task.go generate:download-all-files
Enter fullscreen mode Exit fullscreen mode 
//go:generate go run ./task.go fetch-and-extract-latest-release
Enter fullscreen mode Exit fullscreen mode 
//go:generate go run ../../task.go build-assets
Enter fullscreen mode Exit fullscreen mode

You can use generate:<task_name> or generate-<task_name> as a prefix if you want; it's all up to you and your project's needs.

💡 You might prefer to put your generate script in a ./gen.go right next to the caller file using the same //go:build ignore trick. That way your niche go generate script doesn't bubble all the way up to the root task.go. It's completely your choice.

Custom build script

When you have a lot of binaries to build and a lot of flags to provide to the go build command it might be nice to abstract those behind a go run task.go build script.

task.go

//go:build ignore

package main

import (
    "log"
    "os"
    "os/exec"
)

func cmdRun(name string, arg ...string) error {
    cmd := exec.Command(name, arg...)
    cmd.Stdin = os.Stdin
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    log.Printf("$ %s\n", cmd.String())
    return cmd.Run()
}

func Build() error {
    err := cmdRun("go", "build", "-o", ".out/", "-tags", "embed,nonet,purego", "./cmd/tool-one")
    if err != nil {
        return err
    }
    err = cmdRun("go", "build", "-o", ".out/", "-tags", "octokit,sqlite", "./cmd/tool-two")
    if err != nil {
        return err
    }
    // ...
    return nil
}

func main() {
    log.SetFlags(0)
    var taskName string
    if len(os.Args) >= 2 {
        taskName = os.Args[1]
    } else {
        log.Fatal("no task")
    }
    tasks := map[string]func() error{
        "build": Build,
        // Add more tasks here!
    }
    task, ok := tasks[taskName]
    if !ok {
        log.Fatal("no such task")
    }
    err := task()
    if err != nil {
        log.Fatal(err)
    }
}
Enter fullscreen mode Exit fullscreen mode 
go run task.go build
Enter fullscreen mode Exit fullscreen mode

Setup script to install global dependencies

Sometimes you want your contributors to have global dependencies installed. Yes, it's not ideal but it's often unavoidable. Providing collaborators with one single go run task.go setup command that automagically ✨ installs all required zlib, libgit2, golangci-lint, etc. is an amazing onboarder.

task.go

//go:build ignore

package main

import (
    "log"
    "os"
    "os/exec"
)

func cmdRun(name string, arg ...string) error {
    cmd := exec.Command(name, arg...)
    cmd.Stdin = os.Stdin
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    log.Printf("$ %s\n", cmd.String())
    return cmd.Run()
}

func Setup() error {
    return cmdRun("go", "install", "github.com/golangci/golangci-lint/cmd/golangci-lint")
}

func main() {
    log.SetFlags(0)
    var taskName string
    if len(os.Args) >= 2 {
        taskName = os.Args[1]
    } else {
        log.Fatal("no task")
    }
    tasks := map[string]func() error{
        "setup": Setup,
        // Add more tasks here!
    }
    task, ok := tasks[taskName]
    if !ok {
        log.Fatal("no such task")
    }
    err := task()
    if err != nil {
        log.Fatal(err)
    }
}
Enter fullscreen mode Exit fullscreen mode 
go run task.go setup
Enter fullscreen mode Exit fullscreen mode

💡 You can even use if runtime.GOOS == "windows" or similar to do things for specific GOOS/GOARCH configurations!

Still not convinced?

At least try to write your scripts in Go instead of Bash or Makefiles. This makes it more portable to more places (such as Windows 🙄) without confusion. It also means that your Go devs don't need to learn Bash-isms to change the scripts! 😉

Lots of existing Go projects make use of the go run file.go technique already; they just haven't taken the leap to combine all their scripts into a single Makefile-like task.go yet.

You may prefer scripts folder so that you run each script individually like go run scripts/build-all.go instead of a go run task.go build-all AiO task.go file and that's OK! It's still better than a Linux-only Makefile. 😉

Also check out Scripts should be written using the project main language by João Freitas who hits on these points for more languages besides Go.

Do you have a cool use of task.go? Send it to me! ❤️🤩

Top comments (5)

Collapse
 
goodevilgenius profile image
Dan Jones •

Make is not Linux only. It's very portable, and is an incredibly powerful build tool.

People do also use it as a task runner, which it can do, but is less useful in that regard.

It's greatest power comes in understanding build dependencies and selectively doing only what needs to be done.

This does not even come close to make's ability in that regard.

But if all you need is to run some tasks, this is a good option.

Collapse
 
jcbhmr profile image
Jacob Hummer •

Make is not Linux only. It's very portable, and is an incredibly powerful build tool.

Yes, make variants are available on Windows and Makefiles can work on Windows... but most Makefiles that I've used in The Real World™ use Bash-isms or POSIX sh-isms and don't work well when the default shell is PowerShell or CMD. Some only work with GNU Make and fail with BSD Make. A good example of a non-Windows innocent Makefile is github.com/google/cadvisor/blob/ma...

Requiring WSL 2 or another Linux-like environment just to run some make tasks isn't a great experience for a new contributor. 🤷‍♀️ It's easier to get started when the only required tool to develop Go projects is the Go toolchain.

So yes, I do agree that Make is powerful and portable, but the Makefiles are not that portable -- particularly for Windows. 😢

Collapse
 
ccoveille profile image
Christophe Colombier •

At first, I thought you were about to talk about task

GitHub logo go-task / task

A task runner / simpler Make alternative written in Go


View on GitHub


I like your approach, I wasn't aware of the default shebang thing, interesting

Collapse
 
jcbhmr profile image
Jacob Hummer •

I like Taskfiles too! The fact that task is one go install away makes it even better. The only problem that I have with it is that it's another tool outside the Go toolchain that a contributor or maintainer has to remember and work with. I really resonated with Scripts should be written using the project main language. Particularily "The learning curve is minimal since you already know the corners of the language" 👍

Collapse
 
ccoveille profile image
Christophe Colombier •

I agree with you

Read next

truongpx396 profile image

🌐 Golang RESTful API with Gin, Gorm, PostgreSQL 🐘

Truong Phung -

truongpx396 profile image

🧪 Golang Integration Test With Gin, Gorm, Testify, PostgreSQL

Truong Phung -

avradeep_nayak_fa8d5f6995 profile image

MINDS SDK GOLANG

Avradeep Nayak -

neelp03 profile image

ThrottleX: Scaling to a Million Requests Per Second Without Breaking a Sweat

Neel Patel -