What Is eBPF?

Originally published in Japanese at https://zenn.dev/ymotongpoo/books/go-ebpf-primer/viewer/30-ebpf.

Small programs that run inside the kernel

eBPF is a mechanism that loads a small user-written program into the Linux kernel and runs it when a specific event occurs. An event here means things like a system call being invoked, a packet arriving, or execution reaching a particular instruction address. There is no need to rebuild the kernel or write a kernel module.

For example, you can write logic like “when the openat system call is invoked, record the file name,” or “when execution reaches this instruction address, note the time.” This book deals with the latter: placing hooks at specific addresses in user-space programs.

As Chapter 3 showed, kernel space is the unrestricted side, so user-written code needs safeguards before it can run there.

The constraints the verifier imposes

The verifier provides those safeguards. At load time, it analyzes the entire program and rejects any program whose termination or memory safety it cannot prove. The check resembles compiler type checking: a type mismatch becomes a compile error, while an unproved eBPF operation becomes a load error. Rejected programs never execute in the kernel.

Verification imposes an instruction limit and requires loops whose iteration count has a compile-time bound. The verifier cannot prove that “walk a list to its end” terminates, so it rejects that code. Hurdle 4 runs into this constraint.

Figure 1: The arrows represent load attempts and their outcomes. Verification happens exactly once, before execution; a program that fails it never gets into the kernel.

Maps: state that survives across events

An eBPF program running on the kernel side is invoked per event and finishes right away. Anything equivalent to a function’s local variables does not survive to the next event. To hold state across events, you use a kernel-managed key-value store called a map. Its role is similar to Go’s map. The difference is that the data itself lives on the kernel side, and both eBPF programs and user-space processes can read and write it.

A common instrumentation pattern records a start time at function entry, retrieves it at exit, and computes the elapsed time. A map connects those two events. Hurdle 4 uses another map, ongoing_goroutines, to hold parent-child relationships between goroutines.

Figure 2: The arrows represent the passage of time. The entry hook and the exit hook are separate events; what connects them is the map key.

uprobe and uretprobe

There are two main mechanisms for hooking into user-space programs with eBPF.

• uprobe: a probe placed at a specific instruction address in a user-space binary. It is often called an “entry hook” because tools commonly place one at a function’s first address, though the mechanism accepts any instruction address in the binary.
• uretprobe: a special mechanism for catching the moment a function “returns.”

“Placing” a uprobe actually rewrites the process’s mapped machine code. The kernel saves the instruction at the specified instruction address, then swaps its first byte for a breakpoint instruction. On amd64 this instruction is int3, a single byte cc in machine code. If you placed one at the top of the double function we disassembled in Chapter 4, the 48 of 48 01 c0 (ADDQ AX, AX) would become cc, and the remaining bytes would stay as they are. But as we saw in Chapter 4, this machine code lives on a read-only shared page. The kernel doesn’t write to the shared page directly; it uses copy-on-write to make a copy private to that process, swaps the one byte on the copy, and switches the mapping table to point at the copy. Neither the executable on disk nor the original shared page changes.

When the CPU advances to that address, it executes the swapped-in byte, traps, and enters the kernel. One question remains here. cc is the same single byte for every uprobe, and the cc that was hit carries no information about which uprobe it is. What the kernel sees is the value of the PC at the moment of the trap. It matches that value against its table of registered uprobes (which file, and how many bytes from its start) to identify which uprobe was hit. If the address isn’t in the table, it is treated as a breakpoint placed by someone else, such as a debugger.

The eBPF program tied to the uprobe identified this way then runs. When it finishes, the kernel executes the saved original instruction in a separately prepared area, then hands control back to the instruction that follows. From the target program’s point of view, a single instruction was executed. The registers and the stack come back untouched.

A uretprobe adds one extra step to this. First, a uprobe is placed at the function’s entry. When it fires, the kernel stashes away the return address sitting on the stack (that number CALL pushed, which we saw in Chapter 5) and writes the address of a kernel-provided trampoline in its place. When the function executes its final RET, it jumps not to its caller but to the trampoline, and traps there again. The kernel runs the return-side eBPF program, then jumps to the real return address it stashed away¹.

The two mechanisms rewrite different locations. A uprobe changes one byte in the code area. A uretprobe also changes a value on the stack, which causes Hurdle 1².

Figure 3: The arrows represent the order of operations. A uprobe swaps the first byte of an instruction to cause a trap, and the original instruction is executed in a separately prepared area. A uretprobe is an entry uprobe that rewrites the return address on the stack to the trampoline’s address.

In typical instrumentation, you place a uprobe at the function’s entry and a uretprobe at its exit, following the pattern “record the start time at the entry, compute the elapsed time at the exit.” In most languages this just works. In Go, this straightforward form fails at the very first step.

A loader written in Go

Developers write the eBPF program in restricted C and compile it to bytecode with a dedicated compiler. An ordinary user-space loader puts that bytecode into the kernel, binds probes to addresses, and reads maps. Many loaders use Go and the github.com/cilium/ebpf library, as OBI does.

The following code sets a single uprobe.

// assume the compiled eBPF program objs has already been loaded
exe, err := link.OpenExecutable("/proc/12345/exe")
if err != nil {
	return err
}

// place the probe at the function's entry by symbol name
up, err := exe.Uprobe("main.handleRequest", objs.OnEntry, nil)
if err != nil {
	return err
}
defer up.Close()

Passing a symbol name as the first argument places the probe at that symbol’s first address. Instead of a symbol name, there is also a form that specifies the address directly.

up, err := exe.Uprobe("", objs.OnReturn, &link.UprobeOptions{
	Address: 0x49e290, // an instruction address in the binary
})

This form, an empty symbol name plus an Address, provides the workaround in Hurdle 1. The two snippets above require root privileges and Linux, so you do not need to run them. The later Go samples are the ones intended for you to run and verify on your machine.

Figure 4: Solid arrows represent the flow of processing; dashed ones represent auxiliary flows (placing uprobes and reading maps). The loader puts eBPF programs into the kernel, and only those that pass the verifier stay resident. When execution reaches an instruction address where a uprobe is placed, it fires, and state that spans events can be read from user space through maps.