Golang, as a very ops/admin focused language, has a huge community and thus a lot of useful packages that can help us in the everyday development regarding monitoring, graphing, and automatization.
I’m going to demonstrate a few that I use in most of my programs, either as a substitution of the default package with a similar functionality or a totally new functionality that I consider a core need of the modern ops/admin tool development.
Gjson – json made simple – github.com/tidwall/gjson. Working with complex json files in Golang while using the default “encoding/json” package is a big pain for me. I originally shunned working with json data in Golang solely because of the dumb default package until I have found the great gjson library for parsing and working with json objects. Its simplicity is astounding, and it makes working with jsons a breeze. My favourite main features are structless json objects, the possibility to access variables directly via the built-in regex-like engine and the internal validator. Following is an example of parsing a json from a file with the standard package vs a gjson parse
type People struct { // you have to create a struct with the types you're going to use, otherwise it will throw out an error
Firstname string // you can use an interface type and dynamically retype but it usually breaks
Description string
}
peopleJson := {"firstname": "frank","description": "loves beer"} // source data
var people People // create an object from that struct
json.Unmarshal([]byte(peopleJson), &people) // unmarshal it - in case the source data is invalid and you didn't lint it, it breaks
fmt.Printf("First name: %s, Description: %s", people.Firstname, people.Description) // and finally we're printingVersus
jsonRes := gjson.GetBytes(sourceFile, "firewall") // load up your json you parsed from a file as a byte[]
if gjson.Valid(string(sourceFile)) { // check if it's valid
if gjson.GetBytes(sourceFile, "table").Exists() && gjson.GetBytes(sourceFile, "chain").Exists() { // check if keys are real
table = gjson.GetBytes(sourceFile, "table").String() // access your vars! the second argument for GetBytes is the "key" you are searching for
chain = gjson.GetBytes(sourceFile, "chain").String() fmt.Println(table, chain) } }Logrus – log all the things! – github.com/sirupsen/logrus I found logrus some time ago when I needed to debug a custom firewall application and the fmt.Println(err) statements got a little bit out of hand. Just like every person that ever had to code anything, I am extremely lazy and writing my own solution proved quite complicated, especially if I wanted that bit of code to be reusable in the future. I had a look at the packages that Golang community offers for logging, and Logrus seems the best for me after using it for quite a bit now. I really like the simplicity with which you can output clearly readable and parsable logs in no time. You can also set up your custom log format if you want to parse your logs externally/send them into Kibana. By default, Logrus offers info/error log levels, but you can create your own types. Following is an example of how I use Logrus:
if _, err := os.Stat("/var/log/appname/"); os.IsNotExist(err) { // create a log folder for your logs
os.Mkdir("/var/log/appname", os.FileMode(0755)) }
file, err := os.OpenFile("/var/log/appname/app.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666) // create a file logrus is going to write in
if err == nil {
log.SetOutput(file) // use the SetOutput method to send all the log output to this file
} else {
log.Info("Failed to log to file, using default stderr") // here is an example of using Logrus even for stderr - if it doesn't succeed in opening the file, it forwards it to stderr in the same readable format. pretty cool!
}Viper – configuration is cool – github.com/spf13/viper The ideal practice for all your tools is to write them in a way in which you can reuse them for your other deployments. With that, you need to be able to re-configure things like access credentials, gateways, timeouts, or privileges. The basic way of doing this would be passing parameters on the command line with the flag library, done like this:
func init() {
flag.StringVar(&sourcePool, "sourcepool", "", "sourcePool")
flag.StringVar(&destinationPool, "destinationpool", "", "destinationPool")
flag.StringVar(&destinationHost, "destinationhost", "", "destinationHost") flag.Parse()
}This is sufficient at a smaller scale, with a few simple parameters. However, when using longer/more complicated parameters, you ideally want a config file where you can put all your configuration and parse it/use it in your program – that’s what Viper is for. Basic loading of a config looks like this:
func init() {
err0 := loadConfig(configLocation) // call your loadConfig func with the type, name and path
if err0 == nil {
log.Info("Config succesfully loaded from ", configLocation)
} else {
log.Error("Couldn't load config, exiting, error: ", err0) os.Exit(2)
}
}
func loadConfig(location string) error {
viper.SetConfigType("json") // viper supports multiple formats, but we'll use json here
viper.SetConfigName("config") // the name of the config internally, doesn't have to correspond with the filename viper.
AddConfigPath(".") // where to search for the config if nothing is supplied(searches for config.json if no location is supplied on the next line)
viper.AddConfigPath(location)
return viper.ReadInConfig() // returns whether the load was or was not successful
}After succesfully loading your config, you can access your values in a simple, readable way. This example would look up the config for a value called ‘filename’ and put it into our fileName variable:
fileName := viper.GetString("filename")The original autor: Pepa, DevOps Engineer, cloudinfrastack
