Configuration - doing it wrong.

When we switched to Go I unfortunately brought a fair amount of Java baggage with me. The idea of code being flexible and the complexity that goes with that crept in. Specifically in this pkg, cfg - a Golang library for application configuration. I wrote the cfg pkg. It’s kinda clever. I don’t recommend you use it. I’m leaving the repo up as a reminder to myself that complexity is bad.

cfg lets you define configuration in JSON and override that from another file and or environment vars. These feature seemed important at the time. It uses struct tags and reflection. Then as we used it in more applications the complexity of the Config struct spiraled. WIth this and the reflection we pushed one of the only (unknown) bugs to production in recent memory. It was hard to find. It shouldn’t have happened.

Now we read application configuration from environment variables. Only from environment variables. We don’t use a constructor argument approach to set them from some main, this seemed prone to start up order issues. By using environment variables only there is no implication that there is a chance for the application to internally change it’s configuration. All application configuration is external.

Here’s an example. We log to Logentries using a small Go library - log. Any code that needs to log to Logentries can import the log package for side effects:

import (
    _ "github.com/GeoNet/log/logentries"
)

If there is an environment variable LOGENTRIES_TOKEN set then the pkg switches out the log writer and logs to Logentries. All we’ve needed for config from environment variables is Getenv, ExpandEnv and the very occasional string to number conversion.

For each application we define the requisite environment variables in the Docker environment variable file format. This is always done in a file called env.list. Using this file with Docker is easy. Using it without Docker or with CI (e.g., Travis) is also straight forwards (there are examples at the end).

This is all we do now. Simple. Easy. Not flexible. Enough. I think this has been the longer lesson for me with Go - simplicity is vital, strive for it.

Appendix

When not running code in Docker we read the env.list file and use it to set environment variables (for local development etc) using this bash function whipped up by Chris LeBlanc - a GeoNet Development team member:

function sourceenv {
       if [ $# -lt 1 ]; then
                 echo "please supply a file to source env variables from"
                 return 1
       fi
    
       for i in $(cat $1 | cut -f1 -d"#" | xargs); do
                 export $i
       done
}

For using the env.list file to test in Travis we can do something similar:

...
script: 
- export $(cat geonet-rest/env.list | grep = | xargs) && go test ./geonet-rest  -v
...