Third-Party Workarounds

Originally published in Japanese at https://zenn.dev/ymotongpoo/books/go-json-v2-history/viewer/20-third_party.

Libraries built for speed

During the 14 years in which the standard library’s encoding/json could not be fixed, Go users working with JSON did not just sit and wait. On performance in particular, external libraries appeared one after another, taking a variety of approaches.

• mailru/easyjson chose code generation. Run easyjson -all foo.go and it emits a dedicated encoder and decoder per type. It avoids reflection entirely, which makes it fast, but at the cost of an extra generation step in the build, and it cannot serve as a drop-in replacement for encoding/json.
• json-iterator/go kept reflection but sped things up by caching per-type processing, packaged so that swapping the import path is all it takes.
• goccy/go-json pushed that direction further. It analyzes a type, compiles it into a sequence of opcodes, and executes them in a loop. Processing proceeds by jumps rather than recursive calls, and compilation results are looked up from a slice keyed by the type’s pointer. For structs with 16 or fewer fields, it identifies fields using a bitmap, with no map lookup at all.
• bytedance/sonic went further still, JIT-generating per-type machine code at run time and using SIMD for scanning.

The motivation behind these libraries is plain from their READMEs. segmentio/encoding writes that at the scale they operate, the choice of tools used to build programs has a significant impact on the efficiency of the whole system. buger/jsonparser starts from a different grievance: when handling JSON whose structure is not known in advance, encoding/json demands that you prepare structs, and receiving into map[string]interface{} is very slow.

The price of being a drop-in replacement

Here is a sentence from the goccy/go-json README.

It’s easier to implement by using automatic code generation for performance or by using a dedicated interface, but go-json dares to stick to compatibility with encoding/json and is the simple interface. Despite this, we are developing with the aim of being the fastest library.

That “dares” is the constraint these external libraries shouldered. As long as the value they offer is “it works by just swapping the import path,” they must return the same results as v1. And the same results as v1 include every flaw covered in the previous chapter.

Case-insensitive matching is part of the specification they must reproduce. So is silently resolving duplicate keys last-wins, and so is emitting nil slices as null. The people who wanted speed spent their time faithfully copying behavior they presumably wanted fixed.

Where optimization collides with v1 compatibility

There is a case where this constraint surfaced as a concrete bug.

goccy/go-json#568 was filed on February 12, 2026, and remains unresolved as of this writing. The report: when the target struct has 17 or more visible JSON fields, goccy/go-json does not perform case-insensitive matching; with 16 or fewer, it works correctly.

The boundary of 16 is the ceiling of the bitmap optimization mentioned earlier. With 16 or fewer fields, a bitmap can identify the field and no map lookup is needed. From the 17th field on, execution falls through to a different code path. And that other path had dropped v1’s case-insensitive matching.

Optimization and faithfulness to v1 collide inside the same code. Worse, the failure mode is obscure: the behavior flips the moment you add one more field. In the same repository, #470—case-insensitive matching not working for nested structs—also remains open.

Depending on runtime internals

The price of speed extended to yet another area.

In 2024, the Go team began restricting the use of linkname in golang/go#67401. linkname is a mechanism for binding, from the outside, to internal symbols of packages you normally cannot access. The issue contains the following passage.

For example, https://go.dev/cl/583756 broke github.com/goccy/go-json because it turns out that package copied most of the runtime’s internal type API. Now we can’t change anything in that list, despite that being an ostensibly internal package, without breaking goccy/go-json. And goccy is used by many packages, including Kubernetes… This situation is unsustainable.

The change named here is a CL performing a small cleanup of runtime internals.

The picture has inverted. On the standard library side, the backward compatibility guarantee kept encoding/json from being fixed. Here, an external library’s dependence on internals constrains the runtime’s freedom to change. What goccy/go-json carried was not only faithfulness to v1’s flaws for the sake of compatibility. For the sake of speed, it also depended on an unguaranteed assumption: that the runtime’s internals would not change.

Why none of them were adopted into the standard library

With such fast implementations available, one might think the standard library could simply adopt one of them. Discussion #63397 answers that question.

There are many community forks or reimplementations of v1 ‘json’. While they provide impressive performance gains, they cannot be adopted into the standard library on the basis of their extensive use of package ‘unsafe’. The 2021 Go Developer Survey shows that the assurance of reliability and security is a higher priority than CPU or memory performance.

That the reasoning rests on the 2021 Go Developer Survey is, to me, characteristically Go. The choice between speed and safety is treated not as the designers’ preference but as the result of asking the users.

For what it’s worth, go-json-experiment/jsonbench, the benchmark repository published by v2’s author Joe Tsai, contains a statement to the effect that goccy/go-json has bugs leading to reproducible data races and memory corruption and is not safe for production use. Joe being v2’s own author, this is not a neutral third-party assessment. Still, it is a fact that the goccy/go-json issue tracker holds multiple reports of data races and crashes, and the most recent commits are data race fixes in the encoder and decoder compilation code.

The numbers from the same repository also need context if you cite them. v2’s unmarshal is reported at 2.7 to 10.2 times faster than v1 for concrete types, and marshal ranges from 1.4 times faster to 1.2 times slower. But these measurements are from January 2025, on Go 1.23.5, against the prototype before it entered the standard library.