Zero-Code Observability: Implementing OpenTelemetry Operator in Kubernetes

somu

4 min read

·

Jul 21, 2026

--

Listen

Share

Getting application developers to add telemetry code, manage SDK updates, and manually instrument every endpoint is a classic platform engineering bottleneck. Teams spend weeks adding boilerplate code just to get baseline metrics and distributed traces.

Zero-code instrumentation completely eliminates this friction. By utilizing the OpenTelemetry Operator for Kubernetes, you can automatically inject telemetry agents into your workloads at runtime without modifying a single line of application code or editing package.json / pom.xml files.

Here is a step-by-step guide to setting up zero-code auto-instrumentation using the OpenTelemetry Operator.

Press enter or click to view image in full size

How It Works Under the Hood

The OpenTelemetry Operator leverages Kubernetes Mutating Admission Webhooks to inject auto-instrumentation agents into your pod specs upon creation:

1. Deployment Event: A developer deploys a workload annotated for OpenTelemetry injection.
2. Webhook Trigger: The Kubernetes API server sends the pod spec to the OpenTelemetry Operator’s mutating webhook.
3. Container Injection: The webhook appends an initContainer (which copies the OpenTelemetry language agent into a shared volume) and injects runtime-specific environment variables (e.g., JAVA_TOOL_OPTIONS or NODE_OPTIONS).
4. Runtime Execution: When the container boots, the agent monkey-patches library functions or hooks into bytecode/runtime hooks (e.g., HTTP clients, DB drivers) and streams OTLP telemetry to an OpenTelemetry Collector.

Step 1: Install Prerequisites & The OTel Operator

The OpenTelemetry Operator requires TLS certificates for its admission webhook. The easiest way to manage these is via cert-manager.

1. Install cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml

2. Install OpenTelemetry Operator via Helm

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

helm install opentelemetry-operator open-telemetry/opentelemetry-operator \
  --namespace opentelemetry-operator-system \
  --create-namespace \
  --set admissionWebhooks.certManager.enabled=true

Step 2: Deploy an OpenTelemetry Collector

Before injecting agents into apps, deploy a lightweight OpenTelemetry Collector using the Operator’s Custom Resource Definition (OpenTelemetryCollector).

Create a file named otel-collector.yaml:

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel-collector
  namespace: default
spec:
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318

    processors:
      batch:

    exporters:
      debug:
        verbosity: detailed

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch]
          exporters: [debug]
        metrics:
          receivers: [otlp]
          processors: [batch]
          exporters: [debug]

Apply the collector resource:

kubectl apply -f otel-collector.yaml

Step 3: Define the Instrumentation Resource

The Instrumentation Custom Resource (CR) tells the Operator how to configure the injected agents (exporter endpoints, sampling rates, propagation protocols, and runtime-specific settings).

Create a file named instrumentation.yaml:

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: my-instrumentation
  namespace: default
spec:
  exporter:
    endpoint: http://otel-collector-collector.default.svc.cluster.local:4317
  propagators:
    - tracecontext
    - baggage
  sampler:
    type: parentbased_always_on
  
  # Configuration per language runtime (uses Operator defaults)
  java: {}
  nodejs: {}
  python: {}
  dotnet: {}
  go: {}

Apply the configuration:

kubectl apply -f instrumentation.yaml

Step 4: Enable Zero-Code Auto-Injection

Now, enable instrumentation on any application deployment by simply adding a pod annotation.

Example: Auto-Instrumenting a Python App

Add instrumentation.opentelemetry.io/inject-python: "true" (or reference a specific CR name like "my-instrumentation") to the deployment's pod template metadata:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-app
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: python-app
  template:
    metadata:
      labels:
        app: python-app
      annotations:
        # Enable zero-code instrumentation for Python
        instrumentation.opentelemetry.io/inject-python: "true"
    spec:
      containers:
      - name: app
        image: python:3.10-slim
        command: ["python", "-m", "http.server", "8000"]
        ports:
        - containerPort: 8000

Injection Annotations by Language

• Java: instrumentation.opentelemetry.io/inject-java: "true"
• Node.js: instrumentation.opentelemetry.io/inject-nodejs: "true"
• Python: instrumentation.opentelemetry.io/inject-python: "true"
• .NET: instrumentation.opentelemetry.io/inject-dotnet: "true"
• Go: instrumentation.opentelemetry.io/inject-go: "true"
  (requires eBPF capabilities)

Namespace-wide injection: Annotate the entire Kubernetes Namespace to instrument all pods inside it automatically.

Step 5: Verify Telemetry Pipeline

Check if the OTel agent init container was injected and runs successfully:

# Check pod details
kubectl get pod -l app=python-app -o jsonpath='{.items[0].spec.initContainers[*].name}'
# Output should include: opentelemetry-auto-instrumentation

# Check OpenTelemetry Collector logs for incoming spans & metrics
kubectl logs -l app.kubernetes.io/name=opentelemetry-collector -f

Key Benefits vs. Trade-offs

Zero-Code (OpenTelemetry Operator)

• Developer Overhead: None (Zero application code changes required).
• Speed to Deployment: Minutes across the entire Kubernetes cluster.
• Instrumentation Scope: Standard frameworks, HTTP servers, DB clients, and messaging queues.
• Custom Spans & Attributes: Limited without explicit API code annotations.

Manual Code SDK

• Developer Overhead: High (Requires manual SDK initialization and custom span management).
• Speed to Deployment: Weeks per engineering team and repository.
• Instrumentation Scope: Fully custom (including fine-grained domain logic).
• Custom Spans & Attributes: Unlimited control over attributes, events, and metrics.

Grafana Traces Drilldown showing automated OTLP trace collection and span timeline analysis

• Once zero-code instrumentation is active, the OpenTelemetry Operator automatically streams trace spans directly into your observability backend. In Grafana, you can inspect real-time request rates, span duration distributions, and drill down into individual microservice call graphs — like cart-service and shop-backend—to instantly identify bottleneck operations without having manually written any tracing code.

Press enter or click to view image in full size

Press enter or click to view image in full size