OpenTelemetry Collector Configuration: A Practical Guide

What the OpenTelemetry Collector Actually Does

The OpenTelemetry Collector is a vendor-neutral proxy that receives telemetry (logs, traces, metrics), optionally transforms it, and forwards it to one or more backends. Rather than every application exporting directly to your storage layer, apps send to the Collector once, and the Collector handles batching, retries, filtering, and fan-out. This decouples your instrumentation from your backend choice — switch storage systems later without touching application code.

Pipeline Anatomy: Receivers, Processors, Exporters

Every Collector pipeline is built from three stages, configured in otel-config.yaml:

A Minimal Working Example

Here's a stripped-down pipeline that receives OTLP logs, batches them, and exports to an HTTP endpoint:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 1000
  resource:
    attributes:
      - key: environment
        value: production
        action: upsert

exporters:
  otlphttp:
    endpoint: http://xd-api:4318

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch, resource]
      exporters: [otlphttp]

Notice the pipeline is declared per signal type under service.pipelines — logs, traces, and metrics each get their own receiver/processor/exporter chain, and they can share components or use entirely separate ones.

Common Configuration Mistakes

• Skipping the batch processor: without it, every log line is sent as its own request, which multiplies network overhead and can overwhelm both the Collector and the backend under load.
• No resource attributes: without a consistent service.name and environment tag, correlating logs across services becomes guesswork later.
• Single point of failure: running one Collector instance with no queuing or retry policy means a backend hiccup can silently drop telemetry rather than buffering it.

Scaling Beyond a Single Instance

As traffic grows, a single Collector becomes a bottleneck. The standard pattern is a two-tier deployment: lightweight "agent" Collectors running alongside each service (as a sidecar or DaemonSet), forwarding to a smaller number of "gateway" Collectors that handle heavier processing and the actual export to storage. This keeps per-service overhead low while centralizing sampling and enrichment logic in one place. We cover the Kubernetes-specific version of this pattern in Kubernetes Observability: Collecting Logs and Metrics from Pods with OTel.

Conclusion

The Collector's pipeline model looks simple once you've built one, but the receiver/processor/exporter separation is what makes OpenTelemetry portable across backends. Get comfortable with batching, resource attribution, and the two-tier scaling pattern early, and the rest of your observability stack becomes much easier to reason about.