Sharing compiled go code libraries

If you have used compiled languages before, you likely stumbled over .dll or .so files. They allow you to share command libraries in a compiled form. This is especially helpful when distributing closed-source plugins for existing software. But how do you do that in go?

The plugin package

The plugin package from the go standard library allows you to load shared objects. Shared objects are simple main packages that were compiled by passing the -buildmode=plugin flag to go build.

To load a shared object at runtime, you can use the plugin.Open() function and then access exported symbols using the Lookup() method of the returned *plugin.Plugin object:

p, _ := plugin.Open("sample.so")
myVar, _ := p.Loopup("MyVar")

Creating a logging plugin

Let's see a real-world example of using plugins!

Imagine we have an application that you want to enable custom logging support for. One of the easiest ways to do that is to first define a common interface for logging drivers:

type LoggerInterface interface{
   Log(msg string, labels map[string]string)
   Info(msg string, labels map[string]string)
   Warn(msg string, labels map[string]string)
}

This is of course very basic, but it should work as an example. This interface belongs to the application, not the plugin source code. We need this to later cast the result of the Lookup() method into a usable data type.

Next, create a basic plugin:

plugin.go

package main

import "fmt"
import "encoding/json"

type Logger struct{}

func (l *Logger) print(level string, msg string, labels map[string]string) {
   jsonLabels, err := json.Marshal(labels)
   if err != nil {
      panic(err)
   }
   fmt.Printf("[%s]\t%s\n\t%s\n", level, msg, jsonLabels)
}
func (l *Logger) Log(msg string, labels map[string]string) {
   l.print("LOG", msg, labels)
}
func (l *Logger) Info(msg string, labels map[string]string) {
   l.print("INFO", msg, labels)
}
func (l *Logger) Warn(msg string, labels map[string]string) {
   l.print("WARN", msg, labels)
}

var LoggingPlugin = Logger{}

This defines a Logger{} struct that implements the LoggerInterface we defined earlier. The logger will print the message to console prefixed by the logging level and add any labels encoded as json in a seperate line below the main message. Lastly, we create an exported instance of the Logger{} struct with the variable LoggingPlugin. This variable will be what we retrieve using the Lookup() method later on when load the compiled .so file.

To compile this, simply use go build in plugin mode:

go build -o plugin.so -buildmode=plugin plugin.go

You should now have a file plugin.so that contains the compiled logging plugin.

Next, create a sample application that loads the plugin:

main.go

package main

import "os"
import "io/fs"
import "plugin"

type LoggerInterface interface {
   Log(msg string, labels map[string]string)
   Info(msg string, labels map[string]string)
   Warn(msg string, labels map[string]string)
}

var loggers []LoggerInterface = []LoggerInterface{}

func init() {
   pluginFiles, err := fs.Glob(os.DirFS("plugins"), "*.so")
   if err != nil {
      panic(err)
   }
   for _, fileName := range pluginFiles {
      p, err := plugin.Open("plugins/" + fileName)
      if err != nil {
         panic(err)
      }
      l, err := p.Lookup("LoggingPlugin")
      if err != nil {
         panic(err)
      }
      loggers = append(loggers, l.(LoggerInterface))
   }
}

func main() {
   for _, logger := range loggers {
      logger.Info("Hello world!", map[string]string{"user": "sample"})
   }
}

Our main application includes the sample LoggerInterface we discussed earlier. During init(), it finds all files ending in .so in the plugins/ directory and opens them using the plugin.Open() function. Once opened, it then adds the plugin's LoggingPlugin variable to our []loggers slice, casted to the LoggerInterface type.

The main function simply loops through all registered loggers and prints a sample message once for each of them.

Seeing the plugin in action

Create the directory plugins/ in the same folder as the main.go file and move the compiled plugin.so file into it.

Now simply run the program:

go run main.go

This should show the output from our logging plugin:

[INFO]    Hello world!
          {"user":"sample"}

If you want, you can create more logging plugins, the app will load all of them and execute each logging command once per plugin.