16.2 Race Detection

Two goroutines access the same memory location, at least one access is a write, and the happens-before relation does not order them: this is a data race (11.9.5). The Go memory model is built on DRF-SC: a data-race-free program behaves as if it were sequentially consistent [Go Authors, 2022a], so the programmer’s obligation reduces to eliminating data races (11.9.2). This obligation cannot be met reliably by reading code, because races often show up only under particular loads, hardware and scheduling. The race detector turns it into a property that can be checked: a program built with -race constructs the happens-before relation as it runs, and reports a race as soon as it finds two conflicting accesses that the relation does not order.

This section first uses two examples to show what the detector reports. It then discusses its theoretical basis (vector clocks), the work done by the compiler and by the runtime, the data structures of the ThreadSanitizer runtime and their cost, and finally the limits of its capability.

16.2.1 What Is Reported Is Missing Ordering

go test, go run, go build and go install all accept the -race flag. In the following program, two goroutines each increment counter once. The WaitGroup only guarantees that main reads after both have finished; it does not order the two increments:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
package main

import (
	"fmt"
	"sync"
)

func main() {
	var counter int
	var wg sync.WaitGroup
	for range 2 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			counter++ // read-modify-write, no synchronization
		}()
	}
	wg.Wait()
	fmt.Println(counter)
}

Built with Go 1.26.10 and run with GOMAXPROCS=1, with path prefixes removed from the output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ go build -race -o counter .
$ GOMAXPROCS=1 ./counter
==================
WARNING: DATA RACE
Read at 0x00c000012158 by goroutine 7:
  main.main.func1()
      main.go:15 +0x68

Previous write at 0x00c000012158 by goroutine 8:
  main.main.func1()
      main.go:15 +0x78

Goroutine 7 (running) created at:
  main.main()
      main.go:13 +0x6c

Goroutine 8 (finished) created at:
  main.main()
      main.go:13 +0x6c
==================
2
Found 1 data race(s)
$ echo $?
66

The report gives the stacks of the two conflicting accesses and the creation sites of the goroutines involved [Go Authors, n.d.]. The run conditions deserve attention: there is only one P, goroutine 8 had already finished before goroutine 7 started, the two increments did not interleave in time, and the program even printed the correct result 2. The detector still reports a race, because it checks not whether the accesses actually overlap in time but whether a happens-before relation exists between them. With GOMAXPROCS set to 1, 2 and 8, five runs gave the same report. The program runs to completion as usual and finally exits with status 66; run with GORACE=halt_on_error=1, it terminates after the first report [Go Authors, n.d.].

The second example shows that being earlier in time is not synchronization. The writer goroutine finished long before the main goroutine reads the variable, 100 milliseconds later:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
package main

import (
	"fmt"
	"time"
)

func main() {
	var ready bool
	go func() { ready = true }()
	time.Sleep(100 * time.Millisecond) // earlier in time, not synchronization
	fmt.Println(ready)
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
$ go build -race -o sleep . && ./sleep
==================
WARNING: DATA RACE
Read at 0x00c00001215f by main goroutine:
  main.main()
      main.go:12 +0xb0

Previous write at 0x00c00001215f by goroutine 7:
  main.main.func1()
      main.go:10 +0x2c

Goroutine 7 (finished) created at:
  main.main()
      main.go:10 +0x98
==================
true
Found 1 data race(s)

time.Sleep is not among the synchronization operations the memory model specifies, and it does not establish a happens-before relation. On most machines this program prints true, but the memory model does not guarantee it. Replace the Sleep with a channel send and receive, or wait for the writer with a sync.WaitGroup, and the report disappears.

16.2.2 Vector Clocks

The happens-before relation is generated by program order and synchronization order (11.9.5). The detector maintains it at run time with vector clocks. Each goroutine $t$ holds a vector $C_t$ whose component $C_t[u]$ is the logical time of goroutine $u$ that $t$ already knows; each synchronization object $m$ (a lock, a channel buffer slot) also holds a vector $L_m$ . Synchronization operations update them by the following rules:

\begin{aligned} \text{release}(t, m):&\quad L_m \leftarrow C_t,\quad C_t[t] \leftarrow C_t[t] + 1,\\ \text{acquire}(t, m):&\quad C_t \leftarrow C_t \sqcup L_m, \end{aligned}

where $\sqcup$ is the componentwise maximum. An access made by $t$ at local time $c = C_t[t]$ is recorded as the epoch $c@t$ . When goroutine $u$ later accesses the same location, the earlier access happens before this one if and only if $c \le C_u[t]$ , that is, $u$ already knows $t$ ’s time $c$ through some release-acquire chain. Two accesses that do not satisfy this condition, with at least one a write, form a race.

sequenceDiagram
    participant G as goroutine G
    participant C as channel c (synchronization object)
    participant M as main
    Note over G: write x, record epoch 1@G
    G->>C: send: release, L_c ← C_G
    C->>M: receive: acquire, C_main ← C_main ⊔ L_c
    Note over M: read x: 1 ≤ C_main[G], ordered
    Note over G,M: with time.Sleep instead of send and receive, C_main[G] is still 0, a race is reported

Keeping a full vector clock for every memory location costs time proportional to the number of goroutines. Flanagan and Freund’s FastTrack observed that the vast majority of accesses only need the epoch of the most recent access, which reduces the comparison from $O(n)$ to $O(1)$ ; a full vector is needed only when a read is shared concurrently by several threads [Flanagan and Freund, 2009]. As shown below, ThreadSanitizer’s shadow memory records exactly such epochs.

The interactive figure below applies these rules step by step to the two programs of 16.2.1 and to versions that use a mutex or an unbuffered channel instead. Each row on the left is one operation, and each arrow runs from a release to the acquire paired with it; on the right are each goroutine’s vector clock, the clocks on synchronization objects, and the variable’s access record. The counter program runs in the order of the GOMAXPROCS=1 run in 16.2.1, where the goroutine created last runs first. The two increments are clearly apart in time, yet following program order and the arrows, there is no way from the first write to the second read, and that is what the report rests on.

16.2.3 Division of Labor between the Compiler and the Runtime

Constructing the happens-before relation needs two kinds of events: memory accesses and synchronization. The first is produced by compiler instrumentation, the second is reported by the runtime and the standard library.

When compiling with -race, cmd/compile inserts calls into the runtime for every memory read and write, in the phase that generates SSA [Go Authors, 2026a]:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
func (s *state) instrument2(t *types.Type, addr, addr2 *ssa.Value, kind instrumentKind) {
	if !s.instrumentMemory {
		return
	}

	w := t.Size()
	if w == 0 {
		return // can't race on zero-sized things
	}

	if ssa.IsSanitizerSafeAddr(addr) {
		return
	}
	// ……
	} else if base.Flag.Race && t.NumComponents(types.CountBlankFields) > 1 {
		// for composite objects we have to write every address
		// because a write might happen to any subobject.
		// composites with only one element don't have subobjects, though.
		switch kind {
		case instrumentRead:
			fn = ir.Syms.Racereadrange
		case instrumentWrite:
			fn = ir.Syms.Racewriterange
		default:
			panic("unreachable")
		}
		needWidth = true
	} else if base.Flag.Race {
		// for non-composite objects we can write just the start
		// address, as any write must write the first byte.
		switch kind {
		case instrumentRead:
			fn = ir.Syms.Raceread
		case instrumentWrite:
			fn = ir.Syms.Racewrite
		default:
			panic("unreachable")
		}
	// ……

Objects of size zero cannot race. IsSanitizerSafeAddr excludes addresses on the stack, read-only data, and itabs, string data and closure fields that are read-only after initialization. A write to a composite object may land on any sub-object, so it is reported as a range, while a scalar needs only its start address reported. In addition, the compiler inserts racefuncenter and racefuncexit at the entry and exit of every function; the shadow call stack they maintain lets reports give complete stacks. Not all packages are instrumented: runtime and its internal dependencies are not instrumented at all; packages such as sync and sync/atomic are instrumented neither for memory accesses nor for function entry and exit, and the source comment gives the reason that memory accesses in these packages are either irrelevant or would cause false positives [Go Authors, 2026b].

These packages instead report synchronization semantics directly. After sync.Mutex locks successfully it performs an acquire, and on unlock it performs a release first [Go Authors, 2026d]:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
func (m *Mutex) Lock() {
	// Fast path: grab unlocked mutex.
	if atomic.CompareAndSwapInt32(&m.state, 0, mutexLocked) {
		if race.Enabled {
			race.Acquire(unsafe.Pointer(m))
		}
		return
	}
	// ……
}

1
2
3
4
5
6
func (m *Mutex) Unlock() {
	if race.Enabled {
		_ = m.state
		race.Release(unsafe.Pointer(m))
	}
	// ……

In a -race build, the atomic operations of sync/atomic are forwarded by assembly stubs to ThreadSanitizer’s atomic interface; for example AddInt32 becomes __tsan_go_atomic32_fetch_add, so the detector handles them with the semantics of atomic operations [Go Authors, 2026d]. Channel sends and receives are reported in runtime/chan.go. A buffered channel treats each buffer slot as a synchronization object, and sends and receives perform release-acquire on the corresponding slot. This implements the memory model rule that “the kth receive on a channel with capacity C is synchronized before the completion of the k+Cth send”; a channel whose element size is zero allocates no buffer, so all sends and receives can only share one synchronization object [Go Authors, 2026d]. This rule was once not implemented literally. In 2020, Daniel S Fava found that the order of acquire and release in chansend and chanrecv was reversed, so the detector missed some races; the fix changed the call order and relied on a new ThreadSanitizer interface, and shipped with Go 1.16 [Fava, 2020].

For synchronization the detector cannot see, the runtime package exports RaceAcquire, RaceRelease and RaceReleaseMerge. The source comment explains their purpose and gives the correspondence with the C11 memory model [Go Authors, 2026c]:

1
2
3
4
5
6
7
8
// RaceAcquire/RaceRelease/RaceReleaseMerge establish happens-before relations
// between goroutines. These inform the race detector about actual synchronization
// that it can't see for some reason (e.g. synchronization within RaceDisable/RaceEnable
// sections of code).
// RaceAcquire establishes a happens-before relation with the preceding
// RaceReleaseMerge on addr up to and including the last RaceRelease on addr.
// In terms of the C memory model (C11 §5.1.2.4, §7.17.3),
// RaceAcquire is equivalent to atomic_load(memory_order_acquire).

16.2.4 The ThreadSanitizer Runtime

The detector’s runtime comes from ThreadSanitizer (TSan) in LLVM’s compiler-rt. Go links it into the program as a precompiled .syso object file, and the README under src/runtime/race records the LLVM commit each file is based on [Go Authors, 2026e]. ThreadSanitizer was proposed in 2009 by Serebryany and Iskhodzhanov [Serebryany and Iskhodzhanov, 2009]; the description of the algorithm in its second version says the original version was slow and has been deprecated [Google, n.d.]. Vyukov connected it to Go’s compiler and runtime in 2012 [Vyukov, 2012], and Go 1.1 soon shipped the race detector [Go Authors, 2013].

Before Go 1.19 the detector used the second version, whose algorithm description presents the basic structure of detection [Google, n.d.]. Each aligned 8 bytes of application memory corresponds, by a direct address mapping, to N shadow words. Each shadow word records the thread id, the scalar clock, the access kind (read or write) and the range of bytes accessed. On every access the runtime compares the new access with the existing shadow words one by one; if the two accesses touch overlapping bytes, come from different threads and have no happens-before relation, it reports a race. It then writes the new access into an empty slot, or replaces an old record that happens before it, and evicts one at random if there is no room. To give the stack of the earlier access in the report, each thread also records its access events in a circular buffer.

In 2022, Vyukov updated Go’s runtime to the third version of ThreadSanitizer. The commit message lists the properties of the new runtime: the shadow memory is half the size, twice the application memory; race detection is fully vectorized (on x86); the vector clock is a fixed 512 bytes; and there is no longer a limit on the number of simultaneously live threads and goroutines [Vyukov, 2022]. The Go 1.19 release notes say that the new version is typically 1.5 to 2 times faster than the second and uses half the memory; windows/amd64 and openbsd/amd64 stayed on the second version at the time [Go Authors, 2022b]. These numbers can be read off the definitions of the version Go 1.26 links on most platforms: every 8 bytes of application memory corresponds to 4 32-bit shadow values, so the shadow memory is $4 \times 4 / 8 = 2$ times the application memory; a shadow value records an 8-bit thread slot id and a 14-bit epoch; the vector clock has 256 slots of 16 bits each, 512 bytes in all, and is used for both threads and synchronization objects [LLVM Project, 2023]. Slots are reused among threads, so the number of live goroutines is no longer limited by the number of slots. The “slot and epoch” in a shadow value is exactly the epoch representation described earlier: to check whether an old access happens before the current one, it suffices to compare its epoch with the corresponding component of the current thread’s vector clock.

To the user, the cost can be summarized by the rule-of-thumb numbers the documentation gives: for typical programs, memory use grows by 5 to 10 times and execution time by 2 to 20 times. In addition, the detector allocates an extra 8 bytes for every defer and recover statement and does not reclaim them until the goroutine exits, so a long-running goroutine that executes defer and recover periodically makes memory grow without bound, and these allocations do not appear in the output of runtime.ReadMemStats or runtime/pprof [Go Authors, n.d.]. As for building, -race requires cgo to be enabled on every platform except macOS; since Go 1.20 the detector on macOS has been rewritten not to depend on cgo, so Xcode need not be installed [Go Authors, 2023; Go Authors, 2026f].

16.2.5 Limits of Capability

The detector can report only accesses that actually execute. The documentation states that it can find only races that happen at run time and cannot find races in code paths that are not executed; when test coverage is incomplete, running a -race build under a realistic load finds more races [Go Authors, n.d.]. Even when a path is executed, a race may be missed: every 8 bytes of memory keep only the last few accesses and older records are evicted; the stack of an earlier access is kept in a bounded history buffer, and the history_size option of GORACE can enlarge it to avoid “failed to restore the stack” in reports [Go Authors, n.d.]. Instrumentation covers only Go code compiled by cmd/compile; accesses in assembly and in C code called through cgo are outside its reach.

On the other hand, the detector’s reports can be trusted. The FastTrack paper calls this kind of detector precise: it computes the exact happens-before relation for the observed execution, so it never produces false positives, at the price of being incomplete [Flanagan and Freund, 2009]. Vyukov and Gerrand also wrote in the blog post announcing the detector that it does not produce false positives, so every warning it gives should be taken seriously [Vyukov and Gerrand, 2013]. This property presupposes that the detector sees all synchronization: synchronization that happens where it cannot see must be communicated through interfaces such as RaceAcquire, which is the situation the comment quoted above describes. The blog post also tells the story of a “benign race”. The ReadFrom method of ioutil.Discard made all users share one buffer; the developers thought the data written would be discarded anyway, so the race did not matter, and after the detector reported it they merely switched to a race-free implementation in -race builds. A few months later, a program that computed a hash while reading used this buffer as a staging area between reading and writing the hash; when several goroutines computed hashes at the same time, their data overwrote one another and wrong hash values resulted.

The overhead determines how the detector is used. The blog post estimates that a program built with -race may consume ten times the CPU and memory, so always keeping it on is unrealistic; it suggests enabling the detector in load tests and integration tests, or deploying just one -race instance within a group of production servers [Vyukov and Gerrand, 2013].

16.2.6 Summary

The race detector is the executable form of the memory model. The memory model defines a data race in terms of the happens-before relation (11.9), and the detector constructs this relation at run time: the compiler instruments memory accesses, the runtime and the standard library report synchronization, and ThreadSanitizer uses shadow memory and vector clocks to decide whether two accesses are ordered. What it reports is missing ordering, not interleaving that happened by chance, so a single run can find races whose accesses never overlapped in time; it sees only the paths that are executed, so its effectiveness depends on test coverage and load.