README
¶
Profiling Instrumentation for OpenTelemetry Go SDK
The package provides means to integrate tracing with profiling. More specifically, a TracerProvider implementation
that annotates profiling data with trace_id and span_id pprof tags,
making it possible to filter the profile of a particular trace or span in Pyroscope.
Note that the module does not control pprof profiler itself – it still needs to be started for profiles to be
collected. This can be done either via runtime/pprof package, or using the Pyroscope client.
By default:
- The
trace_idlabel is added on the local root span and inherited by every descendant span through pprof's context label propagation. - The
span_idlabel andpyroscope.profile.idspan attribute are set on the local root span only (the first span created locally) — this is the join key Grafana's "Traces to profiles" UI uses today. Presence of the attribute does not necessarily indicate that the span has a profile: stack trace samples might not be collected if the utilized CPU time is less than the sample interval (10ms).
Limitations:
- Only CPU profiling is fully supported at the moment.
Options
WithSpanIDLabelScope(Scope)— control whether thespan_idlabel is emitted:ScopeNone(off),ScopeRootSpan(default — local root only, descendants inherit the root's value), orScopeAllSpans(every span emits its own).ScopeAllSpanssignificantly increases label cardinality.WithSpanNameLabelScope(Scope)— same, for thespan_namelabel.
Trace spans profiles
To start profiling trace spans, you need to include our go module in your app:
go get github.com/grafana/otel-profiling-go
Then add the pyroscope tracer provider:
package main
import (
otelpyroscope "github.com/grafana/otel-profiling-go"
"github.com/grafana/pyroscope-go"
)
func main() {
// Initialize your tracer provider as usual.
tp := initTracer()
// Wrap it with otelpyroscope tracer provider.
otel.SetTracerProvider(otelpyroscope.NewTracerProvider(tp))
// If you're using Pyroscope Go SDK, initialize pyroscope profiler.
_, _ = pyroscope.Start(pyroscope.Config{
ApplicationName: "my-service",
ServerAddress: "http://localhost:4040",
})
// Your code goes here.
}
Tracing integration is supported in pull mode as well: if you scrape profiles using Grafana Agent, you should
make sure that the pyroscope service_name label matches service.name attribute specified in the OTel SDK configuration.
Please refer to the Grafana Agent
documentation to learn more.
Example
You can find a complete example setup with Grafana Tempo in the Pyroscope repository.
Documentation
¶
Index ¶
- func NewTracerProvider(tp trace.TracerProvider, options ...Option) trace.TracerProvider
- type Config
- type Option
- func WithAddSpanName(bool) Option
- func WithAppName(string) Option
- func WithDefaultProfileURLBuilder(_, _ string) Option
- func WithProfileBaselineLabels(map[string]string) Option
- func WithProfileBaselineURL(bool) Option
- func WithProfileURL(bool) Option
- func WithProfileURLBuilder(func(_ string) string) Option
- func WithPyroscopeURL(string) Option
- func WithRootSpanOnly(bool) Option
- func WithSpanIDLabelScope(s Scope) Option
- func WithSpanNameLabelScope(s Scope) Option
- type Scope
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewTracerProvider ¶
func NewTracerProvider(tp trace.TracerProvider, options ...Option) trace.TracerProvider
NewTracerProvider returns a TracerProvider that annotates pprof samples with span_id and trace_id labels for trace↔profile correlation.
Types ¶
type Config ¶
type Config struct {
AppName string
PyroscopeURL string
IncludeProfileURL bool
IncludeProfileBaselineURL bool
ProfileBaselineLabels map[string]string
RootOnly bool
AddSpanName bool
}
Config describes tracer configuration. DEPRECATED: Do not use.
type Option ¶
type Option func(*tracerProvider)
func WithAddSpanName ¶
WithAddSpanName specifies whether the current span name should be added to the profile labels. N.B if the name is dynamic, or too many values are supposed, this may significantly deteriorate performance. By default, span name is not added to profile labels. DEPRECATED: Ignored by tracer.
func WithAppName ¶
WithAppName specifies the profiled application name. It should match the name specified in pyroscope configuration. Required, if profile URL or profile baseline URL is enabled. DEPRECATED: Ignored by tracer.
func WithDefaultProfileURLBuilder ¶
WithDefaultProfileURLBuilder specifies the default profile URL builder. DEPRECATED: Ignored by tracer.
func WithProfileBaselineLabels ¶
WithProfileBaselineLabels provides a map of extra labels to be added to the baseline query alongside with pprof labels set in runtime. Typically, it should match the labels specified in the Pyroscope profiler config. Note that the map must not be modified. DEPRECATED: Ignored by tracer.
func WithProfileBaselineURL ¶
WithProfileBaselineURL specifies whether to add the pyroscope.profile.baseline.url attribute with the URL to the baseline profile. See WithProfileBaselineLabels. DEPRECATED: Ignored by tracer.
func WithProfileURL ¶
WithProfileURL specifies whether to add the pyroscope.profile.url attribute with the URL to the span profile. DEPRECATED: Ignored by tracer.
func WithProfileURLBuilder ¶
WithProfileURLBuilder specifies how profile URL is to be built. DEPRECATED: Ignored by tracer.
func WithPyroscopeURL ¶
WithPyroscopeURL provides a base URL for the profile and baseline URLs. Required, if profile URL or profile baseline URL is enabled. DEPRECATED: Ignored by tracer.
func WithRootSpanOnly ¶
WithRootSpanOnly indicates that only the root span is to be profiled. The profile includes samples captured during child span execution but the spans won't have their own profiles and won't be annotated with pyroscope.profile attributes. The option is enabled by default. DEPRECATED: Ignored by tracer.
func WithSpanIDLabelScope ¶ added in v0.6.0
WithSpanIDLabelScope specifies whether the current span ID should be added to the profile labels.
By default, only the local root span ID is recorded. Samples collected during the child span execution will be included into the root span profile.
func WithSpanNameLabelScope ¶ added in v0.6.0
WithSpanNameLabelScope specifies whether the current span name should be added to the profile labels. If the name is dynamic, i.e. includes span-specific identifiers, such as URL or SQL query, this may significantly deteriorate performance.
By default, only the local root span name is recorded. Samples collected during the child span execution will be included into the root span profile.
Source Files