eBPF observability on Kubernetes — APM alternative 2026
Teams moving off Datadog, New Relic, Dynatrace are picking eBPF. The 4 tool layers, reference stack, and where vendor APMs still win.
Vendor APM bills are the line item that grew faster than every other engineering cost in 2024-2025. Mid-sized teams routinely found themselves spending five to six figures per month on Datadog, New Relic, or Dynatrace to see what their Kubernetes clusters were doing. In 2026, a growing number of those teams are moving off vendor APMs and onto an eBPF-based observability stack — zero code changes, kernel-level visibility, and total cost roughly 10-20% of the previous vendor bill.
This article covers the four layers of eBPF observability as the ecosystem settled into them by late 2026, the reference stack most teams land on after migration, and the honest case for where vendor APMs still win.
What eBPF actually is, in one paragraph
eBPF (extended Berkeley Packet Filter) lets programs run inside the Linux kernel in a safe, verified way without recompiling the kernel or loading a kernel module. For observability specifically, this means: attach a small program to a kernel probe (a syscall, a network event, a function entry), capture the data the kernel already has, surface it to a userspace agent — without modifying the application being observed. No sidecars, no SDK integration, no code changes. The visibility was always there in the kernel; eBPF just made it reachable.
In Kubernetes, Cilium brought eBPF into the mainstream as a widely-adopted CNI plugin through 2023-2024. By 2026 it is the default dataplane on Google GKE (as GKE Dataplane V2) and a first-class option on AWS EKS and Azure AKS (available but not default; AWS VPC CNI remains EKS’s default, Azure CNI remains AKS’s default). The networking fight is largely settled in Cilium’s favour on new clusters. The observability fight — eBPF versus traditional APM vendors — is still being fought, with eBPF-side momentum.
The four observability layers eBPF covers
Layer 1 — Network (Cilium + Hubble)
Cilium replaces the CNI plugin and uses eBPF for packet routing, policy enforcement, and load balancing. Hubble is Cilium’s companion observability layer: every pod-to-pod flow is recorded, including L7 (HTTP, gRPC) when the protocol is supported, and exposed via CLI, API, and a web UI.
What it answers: which pod talked to which pod, what request path, with what response code and latency. Replaces most of the “network tab” use cases of commercial APMs.
What it does not answer: application-internal performance (what the pod itself was doing between receiving a request and sending a response). That is Layer 2.
Layer 2 — Application tracing (Grafana Beyla, Odigos)
eBPF-based auto-instrumentation. The agent attaches to common runtime hooks (HTTP client/server libraries in Go, Java, Python, Node.js, Ruby, Rust) and emits OpenTelemetry traces without any code change in the application.
Grafana Beyla fits teams already on the Grafana stack (Tempo, Loki, Prometheus). Single agent per node, auto-detects services, emits OTLP.
Odigos is the broader OpenTelemetry control plane — a Kubernetes operator that manages instrumentation and routes telemetry to any OTLP-compatible backend (Grafana, Datadog, Honeycomb, Jaeger).
What they replace: vendor APM auto-instrumentation agents. Datadog’s dd-trace library, New Relic’s agent, Dynatrace’s OneAgent. The eBPF version does roughly 80% of what the vendor agents do with none of the per-service integration work.
What they do not replace: custom span annotations, business metrics, deep per-method timing. If an application needs the trace to show “this specific database query inside this specific business function,” that still requires manual OpenTelemetry instrumentation on top. eBPF catches the HTTP boundary cleanly; it does not know what’s happening inside a user-defined function.
Layer 3 — Live debugging (Pixie)
Pixie is in-cluster live debugging. Rather than storing weeks of history (that would blow out cluster disk), Pixie keeps recent-window data (minutes to hours) and lets engineers inspect it with a query language resembling Pandas. Live packet captures, live SQL queries intercepted at the client, live HTTP flows — all without redeploying anything.
What it answers: “what is happening in the cluster right now, and what happened in the last 30 minutes.” Replaces live-debugging use of commercial APMs during active incidents.
What it does not replace: long-term retention for post-incident analysis, dashboards for weekly trends, long-horizon capacity planning. Pair Pixie with a persistent store (Grafana + Prometheus, or an OTLP backend) for anything beyond “right now.”
Layer 4 — Continuous profiling (Parca, Pyroscope)
Continuous CPU and memory profiling at the kernel level. The agent samples stacks across all processes on the node, aggregates per-service and per-function, and surfaces flame graphs over time.
What it answers: which code paths are consuming CPU or allocating memory in production, right now, aggregated across all pods of a service. Replaces manual pprof-ing or application-level profilers.
What it does not replace: profiling that needs application-level context (profiling a specific user’s request, for example). But for “which function is 40% of our CPU,” eBPF profiling nails it with zero code change.
Honorable mention — Runtime security (Tetragon)
Tetragon uses eBPF to monitor what processes do: syscalls, file access, network connections, capability use. Not strictly observability, but often lives next to the observability stack and plugs into the same visibility pipeline. Replaces or complements Falco for runtime threat detection.
The 2026 reference stack
A typical team migrating off vendor APMs in late 2026 lands on roughly this combination:
| Need | Tool | Replaces |
|---|---|---|
| CNI + network observability | Cilium + Hubble | kube-proxy + vendor network tab |
| Auto-instrumented traces | Grafana Beyla OR Odigos | Datadog/New Relic agent |
| OTLP backend (traces + logs + metrics) | Grafana Tempo + Loki + Prometheus | Datadog/New Relic storage |
| Dashboards + alerts | Grafana | Datadog/New Relic UI |
| Live cluster debugging | Pixie | Datadog “live container” |
| Continuous profiling | Pyroscope (now Grafana) | Vendor CPU/memory profilers |
| Runtime security | Tetragon or Falco | Vendor security module |
All of it open source. Operational cost is the compute to run the agents and the storage for retention (vs. per-GB ingestion fees that drive vendor APM bills).
When eBPF observability actually wins
Three workload shapes:
- High-volume services where vendor per-request pricing is the problem. eBPF agents sample at the kernel; cost scales with cluster size, not with request volume. For services at >10M requests per day, this is often a 5-10× cost reduction.
- Multi-language, multi-framework shops. Vendor APM agents require per-language integration — install dd-trace in every service, maintain the config. One eBPF agent on each node covers everything.
- Compliance / data-locality environments. Vendor APMs ship telemetry to a vendor cloud. eBPF stacks keep everything on-cluster or in-your-own-infra.
When vendor APMs still win
Three cases where moving off is premature:
- Small clusters. Below ~20 pods, the operational overhead of running Grafana + Loki + Tempo + Pyroscope outweighs vendor APM cost. Pay the Datadog bill, focus engineering elsewhere.
- Deep business-metric observability. If the dashboards need “revenue per customer per hour per region” and that requires custom spans with business tags, vendors have more mature UI for that than Grafana does.
- A team that lacks the ops bandwidth to run their own observability stack. Grafana + Loki + Tempo + Pyroscope is a four-service distributed system. On top of Kubernetes. On top of the application. Teams without an SRE or platform engineer running it will have outages in observability itself.
The honest decision rule: eBPF observability saves cost at scale and gives data sovereignty. It costs engineering time to run. Vendors cost money but save engineering time. The break-even is around $5-10K/month vendor spend for most teams.
Three failure modes when migrating
1. Running eBPF agents AND vendor APM in parallel “to compare.” Doubles cost and operational load. Pick one authoritative source per layer; migrate workloads one at a time rather than running both in production.
2. Expecting eBPF to replace custom spans without any work. eBPF captures the HTTP boundary. The custom spans in the application code still need to be ported — they do not disappear. Plan for that engineering work in the migration estimate.
3. Underestimating storage. OTLP traces at full fidelity are expensive to store. 7-day retention for a 100-node cluster with full HTTP tracing can be hundreds of GB per day. Sample at ingestion (10% of traces, 100% of errors is the default pattern) rather than paying for 100% retention.
Bottom line
For Kubernetes clusters above ~50 pods or vendor APM spend above $5K/month, eBPF observability has reached parity with (and often beaten) traditional APMs on cost, data sovereignty, and ease of adding new languages. The engineering work is real — this is not a free switch — but the trajectory is clear: by 2027, running an in-cluster eBPF observability stack is likely to be the default, with vendor APMs reserved for teams that specifically value the managed experience over the cost savings.
The migration path, in rough order: adopt Cilium + Hubble for network (often already done via managed Kubernetes). Add Grafana Beyla or Odigos for application tracing. Replace per-service vendor agents with the eBPF stack. Keep Pixie for live debugging. Add Pyroscope for continuous profiling. Each stage reduces vendor spend; each stage is reversible.
Related reading:
- Kubernetes OOMKilled — 6 root causes and the fixes — one case where observability catches the real cause fast
- Fix ImagePullBackOff on Kubernetes
- kubectl cheatsheet for daily commands
- Prometheus + Grafana monitoring basics — the storage layer the eBPF data flows into
- Kubernetes pods stuck Pending — 8 causes
For the full production-grade observability stack for the specific case of running RAG systems — the 7-field query log, 3 alerts worth paging on, the 6-panel Grafana dashboard — The RAG Production Handbook Chapter 7 covers it end-to-end. Launching November 9, 2026.
DevOps YAML Pack
40+ production-ready configs — Kubernetes, Docker, Terraform, Ansible, Helm, GitHub Actions. Every file commented. Copy, edit two lines, ship. MIT license, no watermarks.
Get the pack — ₹499 →