Understanding Go Closures

Although go is often referred to as a simple and beginner-friendly language, it is not without pitfalls. When using the func keyword, a developer may quickly find out that depending on context, it may produce two very different kinds of functions, or even experience weird bugs that aren't easily debugged.

Closures vs Functions

At first glance, functions and closures may look very similar:

// a normal function
func function(){
  // ...
}

// a closure
closure := func(){
  // ...
}

One might think that the latter is simply a way of writing an anonymous inline function, but that's not true. In fact, a closure is a function with special behaviour: it has access to variables from it's parent's scope. Consider the following example:

x = "sample"
func(){
  fmt.Print(x)
}()

Here, the closure has access to x despite it being declared outside it's function body and not being passed as a parameter. This is the most important difference between functions and closures.

Uses of closures

Closures are typically used anywhere a function is expected, but defining a named one seems like overkill, such as callbacks, goroutines or factory functions. Here is an example from the io/fs package:

func walkDirectory(rootDir string) error {
  err := fs.WalkDir(os.Dir(rootDir), ".", func(path string, d fs.DirEntry, err error) error {
    if err != nil {
      return err
    }
    // ...return nil
  })
  return err
}

Here, the fs.WalkDir() function expects a callback function to execute for every item found in the directory being walked. While we could define a separate named function and pass it to fs.WalkDir(), providing an inline closure is more convenient and keeps the code more readable.

Goroutines are also often defined as inline closures:

go func(){
  // ...
}()

Beware of loops

Now that we know how closures work, let's shoot ourselves in the foot with them! Here is a common mistake developers will run into sooner or later:

package main

import "fmt"
import "sync"

func main() {
    numWorkers := 5
    wg := sync.WaitGroup{}
    wg.Add(numWorkers)
    for i := 0; i < numWorkers; i++ {
        go func() {
            fmt.Println(i)
            wg.Done()
        }()
    }
    wg.Wait()
}

This code uses a for loop to start a variable number of worker goroutines. This is a common pattern for worker pools, which would then typically implement some kind of task queue, for example by reading from the same channel.

You may expect this code to print the numbers 1 2 3 4 5 (maybe out of order, because its concurrent) - but it turns out, it prints 5 5 5 5 5 instead. This happens because closures have access to variable by reference (aka pointer), so all 5 closures get a pointer to i, but run after the for loop has ended, when the value of i is 5.

The same issue appears with range as well:

package main

import "fmt"
import "sync"

func main() {
    fruits := []string{"apple", "banana", "orange"}
    wg := sync.WaitGroup{}
    wg.Add(len(fruits))
    for _, fruit := range fruits {
        go func() {
            fmt.Println(fruit)
            wg.Done()
        }()
    }
    wg.Wait()
}

We range over the fruits slice and print the current fruit using a goroutine. What may not be immediately obvious is that loops using range will reuse their variables between iterations, meaning the reference (pointer) of fruit remains the same between each loop iteration, but it's value will change. As you may have guessed, this code prints "orange" three times instead of each fruit once.

To overcome this issue, all you have to do is pass a copy of the variable to the closure. You can do this either by creating a variable within the loop scope:

for _, fruit := range fruits{
  f := fruit
  go func() {
    fmt.Println(f)
    wg.Done()
  }()
}

Or by passing it as a function parameter:

for _, fruit := range fruits{
  go func(f string) {
    fmt.Println(f)
    wg.Done()
  }(fruit)
}

Memory leaks

Even though go is a garbage collected language, that garbage collector has limits. A variable may only be garbage-collected once no references to it are in use anymore. Since closures can reference variables outside their own scope, they can "steal" variables that would otherwise be garbage-collected. This is a common issue with factory functions:

func createSampleFunc() func{
  x = 10
  return func(){
    fmt.Println(x)
  }
}  


func main(){
  f := createSampleFunc()
  // x is still referenced until 'f' is not referenced anymore
}  

In this example, x survives until after createSampleFunc() has exited, because the returned closure references it. The value of x cannot be garbage-collected until also all references to f are not in use anymore. While this may seem silly for a simple integer, this could become a problem if the variable is significantly larger (for example holding file contents) or if the closure were delayed for a long time, for example because it is registered as a shutdown callback or scheduled for later.

Closures can be a convenient way to write functions inline and access parent scope without passing everything as parameters, but their special behavior may also lead to unexpected bugs that can be hard to debug for the first time. Even experienced developers may get stung by these edge case scenarios every now and then, so always double-check your use of closures.