Go for-range pitfalls — loop vars, copies, and maps

Contents

  1. 1. 0x01 Can you collect pointers to every element?
  2. 2. 0x02 Does ranging stop if you append to a slice?
  3. 3. 0x03 Large arrays and wasted copies
  4. 4. 0x04 Zeroing a big array via range
  5. 5. 0x05 Deleting map keys while ranging
  6. 6. 0x06 Inserting during map range
  7. 7. 0x07 Goroutines in the loop body

Dig101 is back — digging into mechanics behind everyday Go.

Two common iteration forms: for and for-range. Range has sharp edges — the usual traps below.

Update: Go 1.22 gives each iteration its own loop variables, so the classic “shared loop var” bugs below do not apply on 1.22+ unless you use an older release.

0x01 Can you collect pointers to every element?

1
2
3
4
5
6
7
8
arr := [2]int{1, 2}
res := []*int{}
for _, v := range arr {
res = append(res, &v)
}
//expect: 1 2
fmt.Println(*res[0],*res[1])
//but output: 2 2

No — same surprise on []int{1, 2} or map[int]int{1:1, 2:2}.

for-range is syntactic sugar. The compiler copies the container (for arrays/slices/maps as the spec defines), and v is one variable reused each iteration. Taking &v yields one address; after the loop it holds the last value (2, 2).

Rough lowering:

1
2
3
4
5
6
7
8
// len_temp := len(range)
// range_temp := range
// for index_temp = 0; index_temp < len_temp; index_temp++ {
// value_temp = range_temp[index_temp]
// index = index_temp
// value = value_temp
// original body
// }

Fixes:

  • Shadow with a per-iteration local:
1
2
3
4
for _, v := range arr {
v := v
res = append(res, &v)
}
  • Index into the original:
1
2
3
for k := range arr {
res = append(res, &arr[k])
}

Once you see the single loop variable, the rest is faster to spot.

0x02 Does ranging stop if you append to a slice?

1
2
3
4
v := []int{1, 2, 3}
for i := range v {
v = append(v, i)
}

Yes — iteration length was fixed from the copy at range entry; appends to v do not extend this loop.

0x03 Large arrays and wasted copies

1
2
3
4
5
var arr = [102400]int{1, 1, 1} 
for i, n := range arr {
_ = i
_ = n
}

Wasteful — ranging copies the whole array.

Options:

  • for i, n := range &arr
  • for i, n := range arr[:]

Quiz: why don’t huge slices and maps pay the same full-value copy? (Hint: what the range clause actually duplicates.)

0x04 Zeroing a big array via range

1
2
3
4
var arr = [102400]int{1, 1, 1} 
for i, _ := range &arr {
arr[i] = 0
}

Efficient — the compiler can lower “assign zero” loops to memclr; see range.go memclrrange:

1
2
3
4
5
6
7
8
9
// Lower n into runtime·memclr if possible, for
// fast zeroing of slices and arrays (issue 5373).
// Look for instances of
//
// for i := range a {
// a[i] = zero
// }
//
// in which the evaluation of a is side-effect-free.

0x05 Deleting map keys while ranging

1
2
3
4
5
6
7
8
9
10
11
12
13
14
var m = map[int]int{1: 1, 2: 2, 3: 3}
var o sync.Once
for i := range m {
o.Do(func() {
for _, key := range []int{1, 2, 3} {
if key != i {
fmt.Printf("when iteration key %d, del key %d\n", i, key)
delete(m, key)
break
}
}
})
fmt.Printf("%d%d ", i, m[i])
}

You won’t see deleted keys later in this range if they were not visited yet. Map iteration picks a random start bucket; deleting an unvisited key removes it from the rest of this pass.

0x06 Inserting during map range

1
2
3
4
5
var m = map[int]int{1:1, 2:2, 3:3}
for i, _ := range m {
m[4] = 4
fmt.Printf("%d%d ", i, m[i])
}

Maybe — you may see 44 in the output for the same iteration-order reasons. Repeat to see both outcomes:

1
2
3
4
5
6
7
8
9
10
11
var createElemDuringIterMap = func() {
var m = map[int]int{1: 1, 2: 2, 3: 3}
for i := range m {
m[4] = 4
fmt.Printf("%d%d ", i, m[i])
}
}
for i := 0; i < 50; i++ {
createElemDuringIterMap()
fmt.Println()
}

0x07 Goroutines in the loop body

1
2
3
4
5
6
7
var m = []int{1, 2, 3}
for i := range m {
go func() {
fmt.Print(i)
}()
}
time.Sleep(time.Millisecond)

Broken — expect 222, not a permutation of 0,1,2. Same loop-variable capture.

Fixes:

1
2
3
4
5
for i := range m {
go func(i int) {
fmt.Print(i)
}(i)
}
1
2
3
4
5
6
for i := range m {
i := i
go func() {
fmt.Print(i)
}()
}

Even a “simple” for-range has plenty under the hood.

Code: NewbMiao/Dig101-Go.

References:

Questions? Leave a comment below or email: newbvirgil@gmail.com