Code that passed its tests in 2024 still passes them today. A model that scored 94% in 2024 may be scoring 71% right now, and nothing failed.
Drift arrives in two shapes. Data drift is the inputs moving — new device types, a new market, a different customer mix — so the distribution the model sees no longer matches the one it learned on. Concept drift is the relationship moving: what predicted fraud last year doesn't this year, because the fraudsters adapted.
Neither raises an error, which is why deployment has to be staged. Shadow mode scores every live request and logs the answers without using them, for days. A canary takes 1% of real decisions, then 5%, then 50%. The old model stays loaded the whole time, one config flag away.
Measuring decay needs labels. Without knowing what actually happened — the chargeback, the confirmed fraud, the outcome — there is nothing to compare the predictions against, and the only remaining decay signal is a customer complaining.
So the pipeline logs predictions, waits for ground truth to catch up, and tracks accuracy over time while every dashboard stays green.
Anything that depends on the world being a certain way needs a plan for noticing when it isn't.