In 2009, Google published a paper in Nature describing a system that could detect regional flu outbreaks one to two weeks earlier than the CDC's surveillance programme, using search query data. The result was widely cited as a demonstration of what population-scale data could do for public health. The system was called Google Flu Trends.
By 2012, Flu Trends was systematically overestimating flu prevalence. By 2013, it was overestimating by a factor of approximately two. In 2014, a paper in Science by Lazer, Kennedy, King, and Vespignani examined the failure in detail. The system was retired in 2015.
The Lazer et al. post-mortem identified two structural causes. The first was that the model's inputs — search queries — were sensitive to factors unrelated to flu incidence, including media coverage, seasonal search behaviour, and changes to Google's autocomplete suggestions. The second cause is the one that applies to every ML system now running in production: there was no protocol for adjusting or retraining the model that could be executed by anyone outside the original team. When the model began to drift, there was no mechanism to correct it that did not depend on the institutional knowledge of the people who had built it. That knowledge was not documented. It was not transferable.
The difference between a working model and a maintainable one
The Flu Trends model worked. It achieved the performance it was designed to achieve on the data it was trained on. What it did not have was what Joyce Weiner identifies as one of the five most common failure conditions in AI projects: it was too complex to maintain. Not too complex in the sense of computational cost. Too complex in the sense that the decisions embedded in it — which query signals were included, how they were weighted, what the retraining protocol was, what would constitute a signal that the model needed adjustment — were not documented in a form that allowed anyone outside the original team to act on them.
A model that cannot be handed off is a model that is always one team reorganisation, one departure, or one budget cut away from becoming ungovernable. In a production environment, ungovernability is a compliance condition — and under the EU AI Act's Article 9, the ability to update and correct a deployed system throughout its lifecycle is a structural requirement, not an operational aspiration.
"Big data hubris is the often implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis. The core challenge is that the relationship between search queries and flu incidence changes over time in ways that are difficult to anticipate. Without a protocol for monitoring and adjusting this relationship, the system accumulates error rather than correcting it."
The Weiner analysis of why AI projects fail adds a dimension the Lazer post-mortem does not emphasise: the failure to plan for deployment and maintenance from the start of the project. Flu Trends was designed to produce a useful signal. It was not designed to be maintainable by anyone other than the team that produced it. Those are two different engineering objectives, and conflating them is the failure mode.
The six fields that make a model transferable
The handoff readiness of a model is not determined by its documentation after the fact. It is determined by six decisions that must be made during development, before deployment, and recorded in a form that allows someone who was not in the room when the model was built to act on them in production.
Why this is a compliance issue, not just an engineering one
The EU AI Act's Article 9 requires that the risk management system for a high-risk AI system be a continuous, iterative process throughout the system's lifecycle — including regular systematic review and updating. A system that cannot be updated by anyone other than the team that built it is a system that cannot comply with this requirement the moment that team is no longer available.
This is not a distant risk. Team composition changes. Engineers leave. Organisational structures reorganise. Budget cycles end projects and start new ones. In every case, the model that was built to last as long as the team that built it becomes ungovernable — and an ungovernable AI system in a regulated environment is a compliance liability that compounds over time.
The resolution is not to document the model after it is built. It is to define the six handoff criteria as requirements at the start of the project — before the architecture is chosen, before the features are selected, before the first training run. The handoff protocol is part of the design, not a post-deployment task.
Before any model is promoted to production, the following six fields must be documented in a form that allows someone who did not build the model to maintain it. Not in the model card — in a separate handoff document that is reviewed and accepted by the team who will own the system after deployment.
1. Retraining protocol: when to retrain, how to retrain, and what constitutes a retraining trigger.
2. Input dependency register: every data source the model depends on, and what happens when each one changes.
3. Failure mode documentation: the three conditions most likely to cause the model to produce incorrect outputs, and how to detect them.
4. Named maintainer: the person accountable for the model's ongoing performance, with explicit acceptance of that accountability.
5. Rollback procedure: the exact steps to revert to the previous model version, including who has the authority to initiate them.
6. Output interpretation guide: what the model's outputs mean in operational terms, written for someone who did not build the model.
Google Flu Trends had none of the first five. A system that deploys without all six is a system that has already scheduled its own failure — the only variable is the date.