Chapter 15 The Compiler Pipeline

• 15.1 Lexing and Grammar
• 15.2 Intermediate Representation
• 15.3 The Optimizer
• 15.4 The Pointer Checker
• 15.5 Escape Analysis
• 15.6 cgo
• 15.7 Past, Present, and Future

These two sentences open Ken Thompson’s Turing Award lecture, “Reflections on Trusting Trust”. The lecture describes a compiler backdoor that propagates from one compiler generation to the next: it exists only in the compiler’s binary, and the source shows no trace of it (3.3.4). Programmers review source code but run the compiler’s output, and the compiler sits between the two. This chapter opens up that link and examines how the Go 1.26 compiler turns source code into machine code.

The chapter follows the compilation pipeline; 3.2 gives an overview of it. 15.1 covers lexing and parsing: automatic semicolon insertion, how a recursive-descent parser handles ambiguity in the grammar and recovers from errors, and the front end’s evolution from yacc to the syntax package. 15.2 covers intermediate representations: the unified IR, the ir package and walk, and SSA construction, optimization passes and rewrite rules. 15.3 covers the optimizer’s inlining, devirtualization, profile-guided optimization and bounds-check elimination. 15.4 covers the legal patterns of unsafe.Pointer and checkptr, which verifies those patterns at run time. 15.5 covers how escape analysis abstracts a program into a weighted directed graph and solves it there, along with the evolution and limits of the implementation. 15.6 covers how cgo converts between the two runtime conventions of Go and C, where the pointer-passing rules come from, and what crossing the boundary costs. 15.7 reviews the compiler’s evolution in chronological order. Assembly and the calling convention are in Chapter 2; bootstrapping and linking are in Chapter 3.