Airbnb Engineering: Upleveling Alert Development with Observability as Code

Airbnb Engineering details its transformation of the Observability as Code (OaC) workflow to address critical gaps in alert validation. While OaC allows defining alerts via code, traditional reviews could not predict real-world behavior, leading to noise or missed incidents. Engineers previously faced weeks-long cycles deploying alerts side-by-side for validation. Airbnb identified this not as a culture issue but as a developer workflow gap. They rebuilt their OaC platform on Prometheus, introducing local-first development features like pre-deployment validation and local diffs. This enabled engineers to preview alert behavior against real data before submission, reducing development cycles from weeks to minutes. The improvement facilitated the successful migration of 300,000 alerts from a vendor to Prometheus. The initiative supports Airbnb’s 'zero touch' north star, where product teams inherit battle-tested monitoring configurations automatically. By making alert behavior visible and actionable early in the process, Airbnb enhanced reliability and reduced operational toil for its thousands of engineers and services.