First steps with Golang and WebAssembly

#go #programming #opensource #linux

For the last couple of weeks, I was working on Discoblock, our new open-source declarative volume configuration solution for Kubernetes. For more info, you should follow my post. So without going into the details, Discoblocks is a Golang project, and it has to support multiple validators for different cloud disk variants. I think this area is one of the weaknesses of Go, I mean any new validator requires a new build of the binary. Easy to admit that this is far from ideal when we have to support a huge number of cloud drivers, not talking about the future.

One option is to compile the binary with CGO_ENABLED=1 (default behavior anyway) and load libraries - validators in this case - dynamically.

+ Built-in solution
- Only compiled languages are able to produce so files
- It should be a nightmare to ensure all so files in the container, any new of them requires debugging of the running container
- Devs have to use some Linux for development, at least for testing

An alternative solution in Kubernetes world is to create a small unix socket-based HTTP service for each in the Pod as sidecar, but we didn't want to move in this direction, because of the complexity.

The third option is to pimp Go code to execute validators as WebAssembly, more specifically WASI modules.

What is the difference between WASM and WASI? In really short WASM is for the web and it doesn't support function calls, which is a key feature we need.

+ Lots of languages are able to produce WASI
+ WASI modules don't need extra dependencies
+ It is fancy :)
- Still needs CGO_ENABLED=1, but depends on only a limited number of libraries
- Built-in compile supports only WASM and not WASI
- We have to use TinyGo to compile WASI module
- Only a few numeric input and output types are supported (workaround later)
- Missing built-in execution of WASI
- Integration isn't seamless
- base64 encoding is not supported, so we have to forget built-in parsers
- There are more unsupported features, please follow the documentation of TinyGo.

Code time, my favorite:

import (
    "fmt"
    "os"

    "github.com/valyala/fastjson"
)

func main() {}

//export IsStorageClassValid
func IsStorageClassValid() {
    json := []byte(os.Getenv("STORAGE_CLASS_JSON"))

    if fastjson.Exists(json, "volumeBindingMode") && fastjson.GetString(json, "volumeBindingMode") != "WaitForFirstConsumer" {
        fmt.Fprint(os.Stderr, "only volumeBindingMode WaitForFirstConsumer is supported")
        fmt.Fprint(os.Stdout, false)
        return
    }

    fmt.Fprint(os.Stdout, true)
}

Fastjson works well in WASI, and it is fast ;)
Empty func main() {} is necessary
//export is mandatory to export a function
The function doesn't have any input parameter to avoid type issues, instead of it reads environment variables
There is no return value for the same reason as one line above, instead it writes to the standard output or error

Let's compile the module.

go mod init
docker run -v $(PWD):/go/src/ebs.csi.aws.com -w /go/src/ebs.csi.aws.com tinygo/tinygo:0.23.0 bash -c "go mod tidy && tinygo build -o main.wasm -target wasi --no-debug main.go"

Time to implement the other side of the story. I have found a WebAssembly runtime for Go. Wasmer-go is a complete and mature WebAssembly runtime for Go based on Wasmer.

// DriversDir driver location, configure with -ldflags -X github.com/ondat/discoblocks/pkg/drivers.DriversDir=/yourpath
var DriversDir = "/drivers"

func init() {
    files, err := os.ReadDir(filepath.Clean(DriversDir))
    if err != nil {
        log.Fatal(fmt.Errorf("unable to load drivers: %w", err))
    }

    for _, file := range files {
        if !file.IsDir() {
            continue
        }

        driverPath := fmt.Sprintf("%s/%s/main.wasm", DriversDir, file.Name())

        if _, err := os.Stat(driverPath); err != nil {
            log.Printf("unable to found main.wasm for %s: %s", file.Name(), err.Error())
            continue
        }

        wasmBytes, err := os.ReadFile(filepath.Clean(driverPath))
        if err != nil {
            log.Fatal(fmt.Errorf("unable to load driver content for %s: %w", driverPath, err))
        }

        engine := wasmer.NewEngine()
        store := wasmer.NewStore(engine)
        module, err := wasmer.NewModule(store, wasmBytes)
        if err != nil {
            log.Fatal(fmt.Errorf("unable to compile module %s: %w", driverPath, err))
        }

        drivers[file.Name()] = &Driver{
            store:  store,
            module: module,
        }
    }
}

var drivers = map[string]*Driver{}

// GetDriver returns given service
func GetDriver(name string) *Driver {
    return drivers[name]
}

// Driver is the bridge to WASI modules
type Driver struct {
    store  *wasmer.Store
    module *wasmer.Module
}

// IsStorageClassValid validates StorageClass
func (d *Driver) IsStorageClassValid(sc *storagev1.StorageClass) (bool, error) {
    rawSc, err := json.Marshal(sc)
    if err != nil {
        return false, fmt.Errorf("unable to parse StorageClass: %w", err)
    }

    wasiEnv, instance, err := d.init(map[string]string{
        "STORAGE_CLASS_JSON": string(rawSc),
    })
    if err != nil {
        return false, fmt.Errorf("unable to init instance: %w", err)
    }

    isStorageClassValid, err := instance.Exports.GetRawFunction("IsStorageClassValid")
    if err != nil {
        return false, fmt.Errorf("unable to find IsStorageClassValid: %w", err)
    }

    _, err = isStorageClassValid.Native()()
    if err != nil {
        return false, fmt.Errorf("unable to call IsStorageClassValid: %w", err)
    }

    errOut := string(wasiEnv.ReadStderr())
    if errOut != "" {
        return false, fmt.Errorf("function error IsStorageClassValid: %s", errOut)
    }

    resp, err := strconv.ParseBool(string(wasiEnv.ReadStdout()))
    if err != nil {
        return false, fmt.Errorf("unable to parse output: %w", err)
    }

    return resp, nil
}

func (d *Driver) init(envs map[string]string) (*wasmer.WasiEnvironment, *wasmer.Instance, error) {
    builder := wasmer.NewWasiStateBuilder("wasi-program").
        CaptureStdout().CaptureStderr()

    for k, v := range envs {
        builder = builder.Environment(k, v)
    }

    wasiEnv, err := builder.Finalize()
    if err != nil {
        return nil, nil, fmt.Errorf("unable to build module: %w", err)
    }

    importObject, err := wasiEnv.GenerateImportObject(d.store, d.module)
    if err != nil {
        return nil, nil, fmt.Errorf("unable to generate imports: %w", err)
    }

    instance, err := wasmer.NewInstance(d.module, importObject)
    if err != nil {
        return nil, nil, fmt.Errorf("unable to create instance: %w", err)
    }

    start, err := instance.Exports.GetWasiStartFunction()
    if err != nil {
        return nil, nil, fmt.Errorf("unable to get start: %w", err)
    }

    _, err = start()
    if err != nil {
        return nil, nil, fmt.Errorf("unable to start instance: %w", err)
    }

    return wasiEnv, instance, nil
}

On the caller side.

driver := drivers.GetDriver(storageClass.Provisioner)
if driver == nil {
    return errors.New("driver not found")
}

valid, err := driver.IsStorageClassValid(&storageClass)
if err != nil {
    return fmt.Errorf("failed to call driver: %w", err)
} else if !valid {
    return fmt.Errorf("invalid StorageClass: %w", err)
}

There is one more thing, ship everything in a container image.

FROM tinygo/tinygo:0.23.0 as drivers

COPY ebs.csi.aws.com/ /go/src/ebs.csi.aws.com

RUN cd /go/src/ebs.csi.aws.com ; go mod tidy && tinygo build -o main.wasm -target wasi --no-debug main.go

...

FROM redhat/ubi8-micro:8.6

COPY --from=drivers /go/src /drivers
COPY --from=builder /go/pkg/mod/github.com/wasmerio/wasmer-go@v1.0.4/wasmer/packaged/lib/linux-amd64/libwasmer.so /lib64

FYI, because our binary isn't statically compiled into a single binary, we can't use scratch or distroless images as a base.

That's all folks!!!