DEV Community

Cover image for Load configuration from file, environment variables and the command line (flags) with Golang
Volker Schukai
Volker Schukai

Posted on • Updated on

Load configuration from file, environment variables and the command line (flags) with Golang

You are looking for a small library to read values from configurations, you use YAML, TOML, JSON or properties?

You want to be able to read and change the configuration via HTTP?

Furthermore, you would like to react to changes in the configuration file?

Then you might find what you are looking for here.

What do this library?

This library provides a simple way to load configuration from different sources.

It supports:

  • [x] Environment variables
  • [x] Command line flags
  • [x] Configuration files
    • [x] JSON
    • [x] YAML
    • [x] TOML
    • [x] Properties
  • [x] Configuration from a struct
  • [x] HTTP API (get and set configuration)
  • [x] Monitor File changes

Installation

go get \
gitlab.schukai.com/oss/libraries/go/application/configuration
Enter fullscreen mode Exit fullscreen mode

Note: This library uses Go Modules to manage dependencies.

Usage

Initialize

A new configuration is created using the configuration.New() function. The passed
structure is used type for the configuration. The values are taken as default values
of the configuration.

package main

import (
    "fmt"
    "os"
    "gitlab.schukai.com/oss/libraries/go/application/configuration"
)

func main(){
  config := struct {
    Host string
    Port int
  }{
    Host: "localhost",
    Port: 8080,
  }

  c := configuration.New(config)

  fmt.Println(c.Config().Host)
  fmt.Println(c.Config().Port)

}

Enter fullscreen mode Exit fullscreen mode

Configuration values can come from different sources. The order of
sources is important.

Environment variables

With the Environment function, you can load configuration from environment variables.

package main

import (
    "fmt"
    "os"
    "gitlab.schukai.com/oss/libraries/go/application/configuration"
)

func main(){
  config := struct {
    Host string `env:"HOST"`
  }{
    Host: "localhost",
  }

  // Set value
  os.Setenv("HOST", "www.example.com")

  s := configuration.New(config)
  fmt.Println(s.Config().Host) // localhost

  s.InitFromEnv("") // no prefix
  fmt.Println(s.Config().Host) // www.example.com

}

Enter fullscreen mode Exit fullscreen mode

Command line flags

Obviously, you can also load configuration from command line flags. This library supports
the standard library flag package.

package main

import (
    "fmt"
    "os"
   "flag"
    "gitlab.schukai.com/oss/libraries/go/application/configuration"
)

func main(){
  config := struct {
    Host string `flag:"host"`
  }{
    Host: "localhost",
  }

  // Set value
  flag.String("host", "www.example.com", "help message for host flag")
  flag.Parse()

  s := configuration.New(config)
  s.InitFromFlagSet(flag.CommandLine)

  fmt.Println(s.Config().Host) // www.example.com

}

Enter fullscreen mode Exit fullscreen mode

Do you want to allow the user to specify a configuration via the command line,
so you can use the AddFileFromFlagSet function. This function expects a flag.FlagSet.

Import files and streams

You can load configuration from files and streams. With the Import() function you
import files and streams. The specified files are loaded first, and then the
streams.

The configurations are merged. If a value is already set, it is overwritten by the
specified value. So if a value is set in etc and in the more specific
File in the user home directory, the value is taken from the user home directory.

package main

import (
   "fmt"
   "os"
   "flag"
   "gitlab.schukai.com/oss/libraries/go/application/configuration"
)

func main() {

   config := struct {
      Host string
   }{
      Host: "localhost",
   }

   c := configuration.New(config)

   c.SetMnemonic("my-app")
   c.SetDefaultDirectories()
   c.Import()

   fmt.Println(c.Config().Host)

}


Enter fullscreen mode Exit fullscreen mode

The configuration would then be looked for in the following places:

  • ~/config.yaml (working directory)
  • ~/.config/my-app/config.yaml
  • /etc/my-app/config.yaml

Configuration files

Configuration files are certainly the most common way to define configurations.
These can be in different formats. The following formats are currently supported:
JSON, YAML, TOML and Properties.

With files, the approach is slightly different. Here the function AddDirectory()
first, specify directories in which to search for the file. The file
is then searched for in the order of the directories. If the file is found
is loaded.

The helper function AddWorkingDirectory() adds the current working directory.

With AddEtcDirectory() the directory etc is added under Unix.

AddUserConfigDirectory() adds the directory ~.config on Unix. It will
uses the os.UserConfigDir() method. Thus, it is possible to use the directory
can also be used on other operating systems.

The SetDefaultDirectories() method sets the paths to the default values. Become internal
the paths with AddWorkingDirectory(), AddEtcDirectory() and AddUserConfigDirectory()
set.

The directory structure can be specified directly with the SetDirectories() function.
This overwrites the directory structure specified with AddDirectory().

The filename of the configuration file is specified with SetFileName(). The file name
is searched for with the extension .json, .yaml, .yml, .toml or .properties.

If a format is specified with the SetFileFormat() method, only files with
searched for this ending.

As an extra, you can specify your own file system with SetFilesystem().
This is useful for testing.

Furthermore, the AddFile() function can be used to add a file directly.
This makes sense if you are given a file as a parameter
or if you don't want to have a multi-level structure.

Watch files

If you want to watch configuration files for changes, you can use the Watch() function.
With the function StopWatch() you can stop watching the files. The function
returns a bool channel on which the status of the watch is reported.

If the watch has ended, a true is sent over the channel. If an error occurs, a false is sent.

Streams

The configuration can also be loaded from a stream. This is useful if you want to
load the configuration from other sources. For example, from a database.

With the AddReader() function, you can add a stream to the configuration. The
stream is then searched for in the order of the streams. If the stream is found
is loaded.

HTTP API

The configuration can also be changed via HTTP. This is useful if you want to
change the configuration at runtime. For example, if you would like to change the
configuration of a service.

With a Get request, the configuration is returned as the format specified in the
Accept header. If no format is specified, the configuration is returned in JSON format.

With a Post request, the configuration can be changed. The configuration is then
taken from the body of the request. The format is determined by the Content-Type
header. If no format is specified, the configuration is taken from the JSON format.

config := struct {
   Host string
}
s := New(config)
mux := http.NewServeMux()
mux.HandleFunc("/config", s.ServeHTTP)

Enter fullscreen mode Exit fullscreen mode

There is also a middleware to get access to the configuration.

config := struct {
    Host string
}
s := New(config)
mux := http.NewServeMux()
mux.Use(s.Middleware)
Enter fullscreen mode Exit fullscreen mode

The configuration can then be accessed via the config context.

func handler(w http.ResponseWriter, r *http.Request) {
    config := r.Context().Value("config").(struct {
        Host string
    })
    fmt.Println(config.Host)
}


Enter fullscreen mode Exit fullscreen mode

On change

If you want to be notified when the configuration changes, you can use the
OnChange() function. This function takes a callback function as a parameter.

The following program gives the following output Change from localhost to www.example.com.

package main

import (
   "fmt"
   "os"
   "flag"
   "gitlab.schukai.com/oss/libraries/go/application/configuration"
)

func main() {
   config := struct {
      Host string
   }{
      Host: "localhost",
   }

   s := configuration.New(config)

   closeChan := make(chan bool)

   s.OnChange(func(event configuration.ChangeEvent) {
      log := event.Changlog
      msg := fmt.Sprintf("Change from %s to %s", log[0].From, log[0].To)
      fmt.Println(msg)
      closeChan <- true
   })

   c := s.Config()
   c.Host = "www.example.com"

   s.SetConfig(c)

   // Wait for change
   select {
   case <-closeChan:
   }

}


Enter fullscreen mode Exit fullscreen mode

Error handling

If an error occurs, it is returned by the function Errors(). The errors can be handled as usual.

The HasErrors() function can be used to check whether errors have occurred.


Voila!

Top comments (0)