Airbnb Engineering Migrates High-Volume Metrics Pipeline from StatsD to OpenTelemetry
Airbnb Engineering details its production-tested strategy for migrating a large-scale metrics pipeline from StatsD to OpenTelemetry (OTLP) and Prometheus. To address bottlenecks and validate assets like dashboards early, the team adopted a dual-write approach. This involved updating a shared platform-maintained library, used by roughly 40% of services, to emit both StatsD for legacy systems and OTLP for the new OpenTelemetry Collector. The migration established OTLP as the preferred protocol for internal services, with Prometheus for open-source workloads and StatsD retained as a fallback. Key benefits included a significant reduction in CPU usage for metrics processing, dropping from 10% to under 1%, and improved reliability by avoiding UDP packet loss associated with StatsD. Additionally, native OTLP emission enabled the use of advanced Prometheus features like exponential histograms. While most services transitioned smoothly, the article notes that highest-volume emitters faced specific challenges during the dual-write phase. This technical overview highlights how adopting vendor-neutral, industry-standard protocols can enhance observability infrastructure performance and future-proof monitoring systems against protocol barriers.
Wire timeline
Airbnb Engineering Migrates High-Volume Metrics Pipeline from StatsD to OpenTelemetry
Airbnb Engineering details its production-tested strategy for migrating a large-scale metrics pipeline from StatsD to OpenTelemetry (OTLP) and Prometheus. To address bottlenecks and validate assets like dashboards early, the team adopted a dual-write approach. This involved updating a shared platform-maintained library, used by roughly 40% of services, to emit both StatsD for legacy systems and OTLP for the new OpenTelemetry Collector. The migration established OTLP as the preferred protocol for internal services, with Prometheus for open-source workloads and StatsD retained as a fallback. Key benefits included a significant reduction in CPU usage for metrics processing, dropping from 10% to under 1%, and improved reliability by avoiding UDP packet loss associated with StatsD. Additionally, native OTLP emission enabled the use of advanced Prometheus features like exponential histograms. While most services transitioned smoothly, the article notes that highest-volume emitters faced specific challenges during the dual-write phase. This technical overview highlights how adopting vendor-neutral, industry-standard protocols can enhance observability infrastructure performance and future-proof monitoring systems against protocol barriers.
The Airbnb Tech Blog - Medium