01Production ML
4 of 5churners caught





Illustration. Real model: 0.853 recall, 0.860 ROC AUC.
Telco Churn
A retention model tuned on money, not on accuracy.
- Problem
- A telecom operator was losing subscribers with no way to see it coming. Retention budget went out uniformly, which means most of it reached people who were never going to leave.
- System
- A gradient-boosted classifier over behavioural and billing features, served from a BigQuery feature store on Cloud Run. SHAP values ship with every prediction, so the marketing team sees why an account was flagged and not just that it was.
- Decision
- Accuracy was the wrong objective. A missed churner costs far more than a wasted retention offer, so the decision threshold is tuned on expected value rather than F1. Drag the cutoff in the panel to see the trade: push it left and you catch more churners while burning budget on people who would have stayed.
- Result
- 0.860 ROC AUC at 0.853 recall, and 1.28x return on retention spend against the uniform baseline. A live dashboard ranks at-risk accounts daily.
Move the cursor across the field to set the cutoff
Panel illustrates the threshold mechanism on a synthetic cohort. The metrics beside it are measured on the real model.
- XGBoost
- SHAP
- Cloud Run
- BigQuery
- Vertex AI


















