2025·Data Engineer

Data Pipeline Observability Dashboard

Internal dashboard that tracks freshness, row-count anomalies, and schema drift across 12 nightly ELT pipelines.

PythonAirflowBigQuerySlack API

Built an internal observability layer over the team's Airflow-orchestrated ELT pipelines — freshness SLAs, row-count anomaly detection using rolling z-scores, and schema-drift alerts posted to Slack before a broken pipeline reaches a stakeholder dashboard.

Highlights

Caught 3 schema-breaking upstream changes before they hit production dashboards
Reduced mean time-to-detection for pipeline failures from ~18h to under 30min