Designing a config API for microservices applications built using Go

We have a saying here at Encore: “Encore says yes”.
For us, this means no matter what an end-user wants to build, it should be possible with Encore. Even if the framework does not yet natively support that. For example, if you want to
use a No-SQL database you should easily be able to wire that into your application.

As Encore builds a standard Go binary and deploys it into your cloud account, you can easily connect to infrastructure
already in your account. However, this raises a question: How do you configure your application to connect to different instances of that resource depending on if you are running locally, in a PR preview environment, or
deployed to production?

In May this year, we released our metadata API. It allowed a running
system to interrogate its environment, thus allowing for the code to switch between infrastructure instances. However,
this was only an MVP to enable this. We knew we wanted a much more robust system, and started thinking about what a
native configuration experience would look like.

What do developers want from a configuration system?

As mentioned above, our primary use case was to allow different environments to have different integration setups. A typical example of this would be in a payment system to have your development and testing environments connect to a testing account with your card processor.

But we also started thinking about other use cases people have for configuration in a backend:

• Storing very slow-moving metadata, such as metadata on support currencies and the number of decimal places for them.
• Being able to configure library code differently for different services. For instance, having rate limiters installed on a per-service basis with different limits per service.
• Providing the ability to switch between code paths, so you can perform A/B tests & slow rollouts of new behaviour without having to commit 100% of your traffic to the new code.
• Being able to turn things off during incidents or in tests, such as putting the system into a read-only mode or in tests not sending emails to the test customer accounts.

As with everything we build into Encore, we wanted to ensure there was minimal cognitive overhead when using
configuration. This means it should be easy to identify how the configuration was derived and where the settings came
from. Config should feel natural, be easy to read, and never surprise you at runtime.

For us, it was a prerequisite to ensure whatever we designed had compile-time safety, both in terms of type safety and completeness (i.e. config values couldn’t be missing in production).

We wanted the ability for descriptive comments, to explain what the valid options are, and what the impact of those
options is without having to read the code which uses that config field.

Auditability and controls are essential. Many teams need to know who changed what and why, as well as potentially
have an approval process for making config changes. Luckily as developers, we already have great tools for this! If
we store configuration alongside the code in our source control, we get this for free. This necessitates that Encore reads the configuration from the source code, rather than a database.

Finally, to support incident management, we knew we wanted to support live updating configuration, without having to
commit, rebuild and redeploy your applications. This override would be time-limited, before either being reverted to
the values committed, or the override being committed to the source repository.

Choosing a config language

One of the key, non-reversible decisions to make was how users should write their configuration. There are many options
to pick from; JSON, YAML, Dhall, HOCON, INI, XML etc; the options are endless.

Most of the options do not support conditionals, which leads to the question of how to support different environments.
For instance, if we picked JSON, we’d have to introduce a filename suffix system where config.json is applied
everywhere, but config.env-prod.json is only applied to the environment named “prod”, and config.dev.json is applied
to all development environments. Then a question of the merge order arrives, and how do overrides work:

• does a specified environment name override a value defined in the environment type?
• what if a field is null or undefined in the override, does that mean unset it or leave the previous value?.

Another disadvantage is IDE warnings around type safety, and auto-completion would not be automatic; we’d have to add it
into the Encore plugin to provide a good development experience.

To be complete, we briefly considered the option of writing the configuration directly into Go using regular Go code, as this would give the ability to easily have expressions to control config. However, we quickly ruled it out as the advantages of type safety and IDE support were far outweighed by the disadvantages. Using Go would mean having to convert static data into Go code, and initializing structs or updating config values would result in a recompile and relink of the app, which would slow down iteration times.

Finally, we settled on the same configuration language that we use internally for our own systems: CUE Lang.
As it has many of the properties we where after:

• It’s a strict superset of JSON, which means all valid JSON is valid CUE, so users do not have to learn CUE to get started
• CUE has full type safety meaning CUE tooling can report incomplete configuration or invalid types being used
• It supports expressions such as if statements and loops
• CUE has inline comments
• Best of all; it is deterministic no matter which order you process files, the final output will always be the same.

Designing the API

As with most of our features, this is where we spent a lot of time going back and forth. Encore takes backward
compatibility extremely seriously. Similar to the Go compatibility promise, Encore aims to never break working code in future versions of Encore. That means any API we provide must be designed both for what we want to do in the future with that feature, and how the API may interact with other features of Encore.

We needed to be able to statically read all configuration keys and return types, such that we could generate a CUE file
with the required constraints on types, thus gaining our compile-time type safety and completeness checks.

We landed on four main contenders for the design of how to access the configuration from the Go code.

1. Implicit loading

This would be consistent with how secrets are currently defined in Encore. The user would write a package-level variable
with an inline struct type, but not provide an instance of that config. Then at compile time, Encore would inject the
instance against that variable.

package svc

var cfg struct {
    ReadOnlyMode bool

    Currencies map[string]struct{
        Name string
        Code string
    }
}

Documentation of the configuration options would come naturally to Go programmers, as they could just document the struct,
and we would replicate those comments into the generated CUE file.

2. Explicit loading

In this approach, we took the first approach and removed the “magic” by introducing an explicit Load function call.
Under the hood, this is exactly what Encore would have rewritten your code to be in the first option, but due to your
explicitly calling Load, it’s less confusing as to how that variable came to be set in the first place.

package svc

import "encore.dev/config"

type Config struct {
    ReadOnlyMode bool

    Currencies map[string]struct{
        Name string
        Code string
    }
}

var cfg = config.Load[Config]()

3. Reference by Path

We considered an XPath style API, where instead of defining a config struct, you would explicitly read into the
configuration, and we would infer the structure of the configuration from the access pattern.

package svc

import "encore.dev/config"

func SomeCode(currency string) {
    if config.Bool(“ReadOnlyMode”) {
        return
    }

    _ = config.String(“Currencies”, currency, “Name”)
}

However, this API design had some significant disadvantages against the first two;

• Typo’s would not be detected (ReadOnlyMode vs ReadOnly) as they would cause two separate required booleans to be created.
• The IDE could not autocomplete the path for you.
• Refactoring would not be possible inside the IDE (i.e. renaming a field and automatically updating all callsites).

4. Generated Go from CUE

The final approach we considered, was to take the types from the CUE configuration and generate a matching Go struct
type with package-level variables for accessing the config, along the lines of the implicit load and explicit load
options - just with the Go file being fully automatically generated.

This design meant that we’d be guessing the Go types from the CUE types and might use a too-generic type, forcing the
user to typecast the configuration value when it is used. That generic type could then result in an invalid configuration
which wasn’t caught at compile time; for example, we generated an int field, when the app only wanted uint’s.

Updating Config at Runtime & for Tests

After evaluating the options, we decided that the explicit loading made the most sense. However, it left us with a problem: How do we update config values due to live updates and allow tests to override the configuration?

At the end of the day, we only saw two options:

1. Update the values in the struct or sub-structs directly in memory.
2. Provide a wrapper type which wraps a value from T into func () T.

While the first option initially feels the best, it leads to some unexpected behavior: Do all call sites that
reference (or have the struct passed in) understand the type is dynamic and might change? If not, there are going to be some really hard-to-debug issues. Especially if the values are passed into a third-party library that isn’t aware it’s running inside an Encore application.

The other impact, which is due to Encore’s unique test tracking & isolation capabilities, is that two parallel running tests could read different values from the same cfg variable. While most people would never even notice this, it could still lead to some WTF moments of confusion.

Going with the second option means people will naturally expect that the returned value could change between calls to
the wrapper function, which leads to easier to pick up bugs during reviews.

Then by providing a function to override the values in these wrapper functions for unit tests, we end up with no
unexpected behaviour, while giving us the ability to have a typesafe override API for the testing framework

-- svc_test.go --
package svc

import (
    "context"
    "fmt"
    "testing"

    "encore.dev/et" // Encore's test helpers
)

func TestA(t *testing.T) {
    t.Parallel()

    et.SetCfg(cfg.SystemName, "Foo")

    // This will always print "TestA: Foo"
    resp, _ := GetName(context.Background())
    fmt.Println("TestA:", resp.Name)
}

func TestB(t *testing.T) {
    t.Parallel()

    et.SetCfg(cfg.SystemName, "Bar")

    // This will always print "TestB: Bar"
    resp, _ := GetName(context.Background())
    fmt.Println("TestB:", resp.Name)
}

func TestC(t *testing.T) {
    t.Parallel()

    // This will always print the original config "TestC: System under test"
    resp, _ := GetName(context.Background())
    fmt.Println("TestC:", resp.Name)
}


-- svc.go --
package svc

import (
    "context"

    "encore.dev/config"
)

type Config struct {
    ReadOnlyMode config.Bool   // Are we in read only mode?
    SystemName   config.String // The name of our system
}

var cfg = config.Load[Config]()

type NameResponse struct {
    Name string
}

//encore:api public
func GetName(ctx context.Context) (*NameResponse, error) {
    return &NameResponse{cfg.SystemName()}, nil
}

-- myconfig.cue --
if #Meta.Environment.Type == "test" {
    SystemName: "System under test"
}

if #Meta.Environment.Type != "test" {
    SystemName: "My System"
}

Writing the Code Generators

Many of Encore’s capabilities come down to generating code based on your intent, removing the boilerplate you normally need to write. For this project, we had two separate languages we needed to generate code in, Go and CUE.

We started with the CUE generator, we took the statically parsed call to config.Load, recursively resolved the types
required, and used CUE’s own AST package to generate an AST of a CUE file, which contained both the CUE definitions of
the CUE types, but also definitions about the metadata, such that the configuration can perform conditions based on
the environment.

// Code generated by encore. DO NOT EDIT.
//
// The contents of this file are generated from the structs used in
// conjunction with Encore's `config.Load[T]()` function. This file
// automatically be regenerated if the data types within the struct
// are changed.
//
// For more information about this file, see:
// https://encore.dev/docs/develop/config
package svc

// #Meta contains metadata about the running Encore application.
// The values in this struct will be injected by Encore upon deployment and can be
// referenced from other config values for example when configuring a callback URL:
//    CallbackURL: "\(#Meta.APIBaseURL)/webhooks.Handle`"
#Meta: {
    APIBaseURL: string @tag(APIBaseURL) // The base URL which can be used to call the API of this running application.
    Environment: {
        Name:  string                                              @tag(EnvName)   // The name of this environment
        Type:  "production" | "development" | "ephemeral" | "test" @tag(EnvType)   // The type of environment that the application is running in
        Cloud: "aws" | "azure" | "gcp" | "encore" | "local"        @tag(CloudType) // The cloud provider that the application is running in
    }
}

// #Config is the top level configuration for the application and is generated
// from the Go types you've passed into `config.Load[T]()`. Encore uses a definition
// of this struct which is closed, such that the CUE tooling can any typos of field names.
// this definition is then immediately inlined, so any fields within it are expected
// as fields at the package level.
#Config: {
    ReadOnlyMode: bool   // Are we in read only mode?
    SystemName:   string // The name of our system
}
#Config

The design of the wrapper functions meant we couldn’t easily unmarshal the CUE value into the wrapper functions.
This meant we needed to generate unmarshalled functions for the config types. We use the excellent Jennifer library by
Dave (no really; github.com/dave/jennifer) for generating Go files.

For each Go type used within the config, we generate a separate unmarshaller function. The unmarshallers use
json-iterator to process the output from CUE, while tracking the path within the
config to the unmarshalled value. This path tracking will allow the function to check if live overrides have been
provided on that path and return the override instead.

The root level unmarshaller is then injected as a parameter into your call to config.Load, such that the Go compiler
is able to statically verify that our generated code will return the exact types that you expect.

Future Enhancements

We launched the initial version of config a couple of weeks ago in
version 1.9.0. However, this is only the start of our journey
of improving the state of the configuration of backend developers. As discussed above one of the key features we want to add next is the ability to allow applications to have parts of their configuration updated in realtime on a
time-limited basis for dealing with incidents. This means adding a new API to give you the ability to get a channel
from each wrapper to watch for changes in the value, allowing your application to react to configuration changes, such as resetting rate limiters to the new rates.

We also want to provide a UI in the Encore platform, listing the current configuration as it is deployed into each environment, as well as the locations in the source CUE files where that configuration is defined.

Following that we want to integrate config with our secrets management system, such that you can load in secrets as part
of your configuration and use CUE’s conditionals to switch between secrets depending on what you need.

Is there something you’d love to see in a configuration system? Come chat with us on Slack, or the Community Forums.