Mastering range loops in go

The range operator in go is an important building block for most programs and seems intuitive at first glance. But iterating over data with range behaves differently based on data type and structure, and needs special care when used in tight loops.

Arrays, slices and maps

The range keyword is used to iterate through collections or items, most often an array, slice or map. It returns an index as the first and optionally a value as the second argument:

// returns only the index number (starting at 0) of each item in mySlice
for i := range mySlice{
  // i is the index of the current element
}

// returns the index number (starting at 0) and the value of each item in mySlice
for i, v := range mySlice{
  // i is the index of the current element, v its value
}

If an array, slice or map passed to range is nil, it is treated as an empty collection (effectively skipping the loop):

var mySlice []string // nil
for i, v := range mySlice{
  // never runs because mySlice is treated as length 0
}

For arrays only, a pointer to array is also allowed (but not pointer to slice or map):

myArray := [3]int{1, 2, 3}
for i, v := range &myArray{
  // *myArray is dereferenced and treated as a normal array type
}

mySlice := []int{1, 2, 3}
for i, v := range &mySlice{
  // error: cannot range over &mySlice (value of type *[]int)
}

Iterating over containers makes up the bulk of range uses, so being familiar with these behaviors is vital for go developers.

Iteration allocations and performance

The value returned by range is a copy of the original value, which may quickly become expensive in tight loops:

for i, v := range mySlice{
  // v is COPIED from the real value per iteration
}

Container types can avoid this memory copy for high performance by using direct index access instead:

for i := range mySlice{
  // directly using mySlice[i] instead of assigning a copy to a var
  fmt.Printf("Item: %v\n", mySlice[i])
}

In most cases, the compiler will try to reuse iteration index/value variables and not allocate new ones for every iteration. Unfortunately, this is only possible when the variable doesn't "escape" from the loop, for example by assigning its pointer to an external variable:

var externalRef *int
mySlice := []int{1, 2, 3}
for _, v := range mySlice{
  // v "escapes" the loop, aka might outlive the current interation
  externalRef = &v
}

In this case, the compiler cannot safely reuse v for every iteration, and instead has to allocate a new variable v for every loop iteration. In very tight loops or when iterating over large structs, this can become a problem for performance. The only way to avoid this is to manually force variable reuse:

var externalRef *int
var v int
for _, v = range mySlice{ // using assignment with =, not declaration with :=
  externalRef = &v
}

This solution is not perfect either though, as it may create collisions between iterations if not handled carefully.

Integers and strings

A few lesser known edge case uses of range allow iterating over int and string types.

If you only need a loop that runs 10 times, you can iterate over a literal 10:

for i := range 10{
  // do something 10 times
}

Iterating over a string yields values of type rune (not single-character strings!), one per character in the string:

for i, v := range "hello"{
  // runs once for every character of "hello"
}

Iterating over a string like this is slightly more expensive than iterating over its bytes directly, as the utf-8 encoding of the string contents will have to be decoded as well to retrieve correct rune values.

Sequences

Sequences allow developers to add custom iterators by either returning a function of typeiter.Seq with a single return value, or iter.Seq2 with two return values:

func newSeqIterator(items []int) iter.Seq[int]{
  return func(yield func(int) bool) {
    for _, v := range items{
      if !yield(v){
        // stop iterator early if loop was stopped early
        return
      }
    }
  }
}

The iterator takes a function yield as its argument, which is passed in by the range loop at runtime. It is important to check the return value of yield() every time, as a loop may be terminated early, potentially allowing the iterator to skip the remaining work.

Note that neither iter.Seq or iter.Seq2 enforce the first return value to be an int of the current items index like they do for container types. Returned values can be any type the developer sees fit:

func newSeqIterator(data []Person) iter.Seq2[string, int]{
  return func(yield func(string, int) bool) {
    for _, v := range data{
      if !yield(v.Name, v.Age){
        return
      }
    }
  }
}

The example returns a string and an int, neither of which are an index to anything.

Channels

Channels can also be looped over using range, with a lot of hidden builtin features. The basic pattern would be:

myChan := make(chan int)

for v := range myChan{
  // do something every time myChan received a value
}

The code snippet may look familiar, but invokes logic specific to channels when used. First of all, it only supports one return value, no index variable. If no value is available within myChan, the loop will block until one is pushed into it. Closing myChan will break the loop.

These specific behaviors around channels allow easy load balancing of messages across multiple worker goroutines:

// dummy task queue
tasks := make(chan int) 

// start 10 concurrent workers to process queued tasks
for range 10{
	go func(){
    for task := range tasks{
      // do something
    }
  }()
}

// queue some tasks
for i := range 1000{
  tasks <- i
}

// immediately close channel to stop all workers
close(tasks)

// real programs should use a sync.WaitGroup to wait for workers to finish here

Since a channel guarantees that every value pushed into it can only ever be read once, the channel effectively load-balances messages across goroutines. Closing a channel still guarantees that all successfully sent values are still read by one of the workers. Once all messages in the channel are read, the close() propagates, terminating the range loops in the workers, effectively shutting them down again.

A minor edge case to note here is that a nil channel will block indefinitely when passed to range:

var myChan chan int // still nil
for n := range myChan{ // blocks forever
  // never called
}

Although rare, this bug may cause a program to become "unresponsive", a state fairly tricky to debug.