testscript, a hidden gem the Go team kept locked away

An internal package sees the light of day

Welcome back to the second episode of go get it, our series on excellent Go packages and tools that deserve a spotlight. This week we're taking a look at testscript.

I have a real love-hate relationship with shell scripting. On the one hand it's pretty great: easy to invoke a slew of commands, pipe them into some other commands to do more stuff, and off you go. I also really, really hate shell scripting. It has downright insane semantics, a pile of foot-guns larger than Apex Regional, and syntax I will never get right the first time.

But I still keep coming back for more Bash. Is it because of a complicated relationship with myself, rooted in my deep self-loathing as a result of repressed childhood trauma? Undoubtedly. But whatever the reason, here I am. Writing shell scripts.

Shell tends to be particularly compelling when dealing with executables, files, and semi-structured data, in a concise way. It's especially valuable in use cases where correctness can be determined by "I know it when I see it". You can keep tweaking a shell pipeline with some sample data until it gives you output in the way you expect, and then run it on the full data set. This way you mitigate the downsides and keep most of the value.

Another use case where conciseness is valuable is in writing tests. If writing tests takes too much effort or boilerplate you skip writing tests. So I was absolutely delighted to come across testscript, a Go package designed for writing tests with simple shell-like scripts. It tends to be particularly useful for testing things that operate on files, as we'll see.

Introducing testscript

testscript was originally created for testing the Go compiler itself. It offers an easy way to define files with specific contents, and then assert that certain invocations of the go command produces certain outcomes: successfully building a binary, returning a particular error, printing a particular line on stdout, and so on.

It was originally an internal package, but fortunately for us it's been factored out and made available at github.com/rogpeppe/go-internal/testscript. Here's the "hello world" of testscript:

# hello world
exec cat hello.text
stdout 'hello world\n'
! stderr .

-- hello.text --
hello world

The script consists of two parts. The top part contains a sequence of commands to run. The predefined commands include common UNIX things like cd, exec, cp, mkdir, rm, symlink, and env (for setting env variables), as well as higher level abstractions likeexists (check for file existence), grep (assert a file contains certain contents), stdout/stderr (like grep but for command output).

The second part of the testscript is a list of files, with file names and contents. When the test runs it sets up a temporary directory, writes all of those files there (creating any subdirectories as necessary), and then runs the script against that temporary directory.

But it's first when you start to define your own commands for your own use cases that you realize the full power of testscript. For example, in the Encore compiler we define parse as a custom command that invokes the Encore parser on a set of files and writes the parsed metadata to stdout. It all takes just a few lines of code and makes it incredibly easy to test advanced functionality.

Here's an example of a testscript we use to verify that Encore's parser correctly parses cron job definitions:

parse
stdout 'rpc external.Endpoint access=private'
stdout 'cronJob job-id title="My Title"'

-- svc/svc.go --
package svc

import (
    "context"
    "time"

    "test/external"
    "encore.dev/cron"
)

var _ = cron.NewJob("job-id", cron.JobConfig{
    Title:    "My Title",
    Every:    5 * cron.Minute,
    Endpoint: external.Endpoint,
})

-- external/external.go --
package external

import "context"

//encore:api private
func Endpoint(ctx context.Context) error {
    return nil
}

With just 30 lines of code we can test that we can parse a distributed system consisting of two different backend services and a cron job. Incredible.

To get started with testscript, just add a test to your package:

import "github.com/rogpeppe/go-internal/testscript"

func TestFoo(t *testing.T) {
    testscript.Run(t, testscript.Params{
        Dir: "testdata",
    })
}

This will automatically parse and execute all .txt files inside the testdata directory in your package.

Then, if you want to define your own commands just add:

func TestMain(m *testing.M) {
    os.Exit(testscript.RunMain(m, map[string] func() int{
        "mycommand": func() int {
            fmt.Println("the cake is a lie")
            return 0 // the exit code to return (0 indicates success)
        },
    }))
}

And then use it in your testscripts:

mycommand
stdout 'the cake is a lie\n'

You heard it here first.

txtar

All of this is made possible by a sibling package to testscript, namely txtar. As the name suggests, it's a text-based archive format. To quote the documentation, its goals are to:

• Be trivial enough to create and edit by hand.
• Be able to store trees of text files describing go command test cases.
• Diff nicely in git history and code reviews.

Since you've ready this blog post you're already familiar with it: it's what powers testscript's simple yet extremely effective format for expressing sets of files. testscript scripts are txtar archives! The "script" part at the top is what txtar calls a Comment, and the rest is just descriptions of files with contents.

We've used txtar for great success in several places when building Encore, such as defining simple example apps, providing runnable examples, and more. It's also used under the hood to power the Go playground.

Wrapping up

If you're ever looking for an easy way to test behavior involving files in a lightweight yet powerful way, I highly recommend you give testscript (and txtar, by extension) a go! We'll be back next week with another episode of go get it.

Thank you very much to Russ Cox for the original concept of testscript, to Roger Peppe for maintaining go-internal, and the whole Go team for their work on Go and its excellent tooling. Your work and efforts are truly appreciated!

Originally posted by André Eriksson on the Encore blog on 06/05/22.