# Designing an SDK Distribution

> Source: https://www.ymotongpoo.com/books/observability-platform-with-otel/10-sdk_distribution/


Suppose that someone asks a development team to "add OpenTelemetry to the service." The team starts by copying the initialization code from the official documentation into its own repository.

This chapter uses Go for the code examples. Go compiles statically, and you cannot inject an agent into a Go program at run time. So Go shows clearly what the platform gains when it distributes its own distribution. The argument of this chapter is not specific to Go, though. In any language, the design is the same: move the initialization decisions into a library, and reduce the code that development teams write. The last section of this chapter covers the differences between languages.

The initialization code in Go looks like this.

```go
func initTracer(ctx context.Context) (func(context.Context) error, error) {
	// Build the exporter (the component that sends telemetry)
	exporter, err := otlptracegrpc.New(ctx,
		otlptracegrpc.WithEndpoint("collector.internal.example.com:4317"),
		otlptracegrpc.WithInsecure(),
	)
	if err != nil {
		return nil, err
	}
	// Set the resource attributes
	res, err := resource.New(ctx,
		resource.WithAttributes(semconv.ServiceName("payment")),
	)
	if err != nil {
		return nil, err
	}
	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(res),
	)
	otel.SetTracerProvider(tp)
	// Set the propagator
	otel.SetTextMapPropagator(propagation.TraceContext{})
	return tp.Shutdown, nil
}
```

This code works as it is. But when each team writes its own copy, each team also ends up with its own way to initialize.

## What happens with the plain SDK

The code above sets the Collector endpoint directly, so each team decides how to change it for each environment. The only resource attribute that it sets is `service.name`. Teams also do not agree on whether to add `deployment.environment.name` or `service.namespace`.

The propagator does not include baggage. **Baggage** is a mechanism that carries arbitrary key-value pairs on a request, together with the trace context. If you later adopt a design that uses baggage, this service cannot pass those attributes on. If a team forgets the `SetTextMapPropagator` call itself, the trace breaks at this service.

Some teams also set up only traces, never implement the initialization for metrics and logs, and decide that "OpenTelemetry is already in."

Each difference is small, but the effect appears when you search telemetry across services. If resource attributes are missing, you cannot identify which service in which environment produced the data. Differences in propagators break traces. Differences in SDK versions leave a bug in place in every service except the ones that already have the fix.

If a central team reviews every service, requests wait longer. Instead, the platform distributes an **SDK distribution** that has the organization's defaults built in. The initialization method then becomes one of the distributed artifacts, and it stays uniform.

![The plain SDK compared with an internal distribution](20260926-raw-sdk-vs-distribution.png)

*Figure 1: The arrows show where each initialization decision takes place. With the plain SDK, the decisions split service by service, and the missing attributes and configuration differences appear in searches across services. An internal distribution gathers the same decisions in one place, and each service gets the organization's defaults with a single call.*

## The role of a distribution

OpenTelemetry uses the term [distribution](https://opentelemetry.io/docs/concepts/distributions/) for an SDK that someone has repackaged with added defaults and customizations. A distribution differs from a fork, which changes the SDK itself. A distribution layers configuration and a selection of components on top of the SDK.

The SDKs that observability vendors provide are also examples of distributions. On top of the OTel SDK, they add defaults for the vendor's own backend, recommended instrumentation libraries and propagators, and common initialization code.

An internal platform can also distribute a distribution, with defaults built in that fit the organization's environment.

## Designing an internal distribution

An internal distribution removes the common decisions about exporters, resources, propagators, and samplers from application code. A **sampler** is the component that decides whether to record a trace. The code at the start of this chapter has no sampler, but it still chooses the SDK's default by leaving it out. Development teams write only two things: the call to the internal distribution `otelinit`, and the handling for a failed initialization.

```go
shutdown, err := otelinit.Setup(ctx)
if err != nil {
	return fmt.Errorf("initialize telemetry: %w", err)
}
defer func() {
	_ = shutdown(context.Background())
}()
```

When initialization fails, the application can stop, or it can continue without telemetry. The organization makes that choice. The example above follows a policy of not starting when initialization fails. If you allow the application to continue, document two things as the contract of the distribution. First, `Setup` returns a no-op shutdown function that is safe to call even on error. Second, you can confirm the initialization failure through logs or a similar channel.

Inside `Setup`, the distribution does the following.

- Builds the OTLP exporter. The default endpoint points to the agent Collector in each environment (Chapter 4)
- Sets the propagators to the combination of W3C TraceContext (the standard header format that carries the trace context) and baggage
- Sets the default sampler to `ParentBased(AlwaysSample)`. Most sampling decisions move to the gateway side (Chapter 4)
- Detects resource attributes automatically from the metadata of the runtime environment (Kubernetes or the cloud provider)
- Builds the providers for traces, metrics, and logs (the objects that return instances for instrumentation), and registers them globally

For logs, the distribution takes API stability into account. As of September 2026, the released version of Go's log modules (`otel/log` and `otel/sdk/log`) is v0.22.0, a beta. Only traces and metrics have reached stable[^golog]. A v1.47.0-rc.1 toward stabilization is also out, but it is still at the release candidate stage. If each team depends on the v0 API directly, every breaking change forces fixes in several services. If only the distribution depends on the v0 API, each change needs work in one place only.

[^golog]: You can check the module layout and versions of opentelemetry-go in the [versions.yaml for v1.46.0](https://github.com/open-telemetry/opentelemetry-go/blob/v1.46.0/versions.yaml). The main branch includes release candidates, so check at the tag of the version that you adopt.

The sampling defaults need an assumption about the volume that the pipeline can handle. In **head sampling**, the SDK decides whether to record a trace when the trace starts. In **tail sampling**, the Collector decides after the trace completes[^sampling]. ParentBased follows the decision of the parent span. For a span with no parent, it delegates the decision to another sampler. The Go SDK's default is `ParentBased(AlwaysSample)`, which records traces that have no parent. With this default, the SDK drops no spans unless an unsampled parent arrives from upstream. Some services receive requests from outside, and some setups also use head sampling. In those cases, check this behavior on the assumption that upstream decisions propagate into the service.

[^sampling]: If you do not handle sampling carefully, the data that you need is missing when you want to analyze a cause. The design of sampling is beyond the scope of this book. My book *[Getting Started with Telemetry Sampling in OpenTelemetry](https://amzn.to/4cQG9i6)* (in Japanese) covers the consistency of probabilities, the policy design for tail sampling, and the estimates of volume and cost.

When you gather the decisions into tail sampling, you can choose what to keep by looking at errors and latency. But every span then flows from the application to the gateway. The load on the SDK, the agent, the network, and the gateway does not go down. You can use this setup only when each segment can handle the volume of all spans. For high-volume services, reduce the volume first with head sampling in the SDK. The platform adjusts the `OTEL_TRACES_SAMPLER` environment variable to decide at which stage to reduce the volume.

Consistent Probability Sampling keeps head-sampling probabilities consistent across a whole trace, and contrib has a [Go implementation](https://pkg.go.dev/go.opentelemetry.io/contrib/samplers/probability/consistent) of it. As of September 2026, though, it is at the experimental stage, and differences from the current draft specification remain[^cps]. At this stage, I do not adopt it for the defaults of the distribution. I explain Consistent Probability Sampling in detail in [a separate article](https://zenn.dev/ymotongpoo/articles/20260717-cps) (in Japanese).

[^cps]: The implementation follows an old draft that writes p values and r values into tracestate. It is not compatible with the current specification, which uses th values.

## Configuration precedence

The internal defaults exist so that a service runs in the organization's standard configuration without any extra settings. They must not lock in service-specific requirements, though. So explicit settings in code come first, the standard `OTEL_*` environment variables come second, and the internal defaults come last.

The distribution defines no internal environment variables. With the standard OTel environment variables, teams can refer to the official documentation. They can also configure zero-code instrumentation (Chapter 3) and the SDKs for other languages in the same way.

The Go SDK's support for environment variables has gaps, though. The specification defines a set of environment variables, and the Go SDK alone interprets only some of them.

| Environment variable | Does the Go SDK alone interpret it? | How the distribution fills the gap |
|---|---|---|
| `OTEL_SERVICE_NAME`, `OTEL_RESOURCE_ATTRIBUTES` | Yes | Not needed |
| `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` | Yes | Not needed |
| `OTEL_EXPORTER_OTLP_*` (endpoint, headers, protocol, and others) | Yes | Not needed |
| `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER`, `OTEL_LOGS_EXPORTER` | No | contrib's autoexport interprets them |
| `OTEL_PROPAGATORS` | No | contrib's autoprop interprets it |
| `OTEL_SDK_DISABLED` | No | The distribution implements it |

The specification repository summarizes the support status in its [compliance matrix](https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md). The contrib packages [autoexport](https://pkg.go.dev/go.opentelemetry.io/contrib/exporters/autoexport) and [autoprop](https://pkg.go.dev/go.opentelemetry.io/contrib/propagators/autoprop) fill in the unsupported variables. The distribution includes these two packages, and that completes its support for the standard environment variables.

```go
// Interprets OTEL_TRACES_EXPORTER and OTEL_EXPORTER_OTLP_*
exp, err := autoexport.NewSpanExporter(ctx)

// Interprets OTEL_PROPAGATORS. If unset, uses tracecontext and baggage
otel.SetTextMapPropagator(autoprop.NewTextMapPropagator())
```

The specification also defines declarative configuration, which configures the SDK from a YAML file. The configuration schema reached [v1.0.0](https://github.com/open-telemetry/opentelemetry-configuration/releases) in February 2026. The Go implementation, [otelconf](https://pkg.go.dev/go.opentelemetry.io/contrib/otelconf), is still at the experimental stage with v0.26.0. For now, the distribution uses environment variables and defaults in code. I will decide whether to migrate after otelconf becomes stable.

## Distributing it and keeping up with versions

In Go, you distribute the internal distribution as a private Go module. An internal version control system with GOPRIVATE, or a module proxy, is enough to distribute it. The section "Challenges in supporting multiple languages" covers the distribution methods for each language.

Versioning follows semver. The release notes include a table that maps each version of the distribution to the versions of the bundled SDK and instrumentation libraries. Renovate or Dependabot sends update PRs to each team's repository. The work of updating the SDK then splits into two parts: the release of the distribution, and the automated PRs to each service.

otelgrpc deprecated its interceptor-based instrumentation and moved to instrumentation based on stats handlers. Both are ways to build instrumentation into gRPC. If each service uses otelgrpc directly, each service must change its own code. If services build their gRPC servers with a helper from the distribution, the change stays inside the distribution.

## A recommended list of instrumentation libraries

The SDK itself is stable in the v1.46.0 series. Instrumentation libraries such as otelhttp and otelgrpc are at v0.71.0, so they are still v0 as of September 2026. The distribution pins the instrumentation libraries and their versions in its go.mod, which prevents differences between services.

Instrumentation libraries for HTTP and gRPC, which many teams use, go into the distribution along with helpers. Other libraries differ from team to team, such as database drivers and messaging clients. For those, a recommended list shows the versions that the platform has tested. You can find candidates in the official [registry](https://opentelemetry.io/ecosystem/registry/).

## Challenges in supporting multiple languages

Other major languages have mechanisms to build a distribution. In Java, you can use the extension mechanism of the javaagent to inject defaults and custom logic as a jar. In Python, you replace opentelemetry-distro and the configurator through an entry point. In Node.js, the platform distributes a setup module that wraps auto-instrumentations-node with the organization's defaults.

Go has no equivalent mechanism. Go cannot use an agent or dynamic loading, so the platform distributes a wrapper module. The defaults and the failure contract that `otelinit` defines also apply to the distributions for other languages.

Across languages, the implementation methods can differ. What the distributions must align is the set of resource attributes, the propagator configuration, sampling, and the names and meanings of attributes. The semantic conventions registry and Weaver, which Chapter 6 covers, manage the names and meanings of attributes. Each language's distribution includes the package of attribute constants that Weaver generates.

A distribution separates the defaults that the platform manages from the business-specific spans and attributes that development teams record. The standard environment variables and explicit settings in code still take effect, so the distribution can also meet service-specific requirements.