There is a class of AI project failure that does not appear in the model evaluation logs, does not trigger the monitoring dashboards, and does not surface in any retrospective on technical quality. The team built what they were asked to build. The model performs as designed. The acceptance testing passed. The failure is that nobody in the room when the acceptance criteria were written represented the context in which the system would actually be used.
This is the gap between a system that is technically correct and a system that is operationally usable. It is the fifth failure condition Weiner identifies: the project solved the wrong problem — not because the brief was vague, but because the brief described the solution environment rather than the deployment environment, and the two turned out to be different.
IBM's Watson for Oncology is the most widely documented example of this failure at enterprise scale. It is also one of the most instructive, because the technical investment was substantial, the intent was genuinely beneficial, and the gap between the solution as built and the context as encountered was entirely foreseeable if anyone had asked the right questions before training began.
What the STAT News investigation established
In 2017 and 2018, STAT News published two investigations into the deployment of IBM Watson for Oncology at health systems around the world. The 2018 investigation, based in part on internal IBM documents obtained by the reporters, found that the system had been trained primarily on hypothetical patient cases curated by Memorial Sloan Kettering Cancer Center — a leading US cancer institution with patient populations, treatment protocols, and drug availability specific to the United States.
When the system was deployed internationally — at hospitals in India, South Korea, Thailand, and elsewhere — oncologists found that its treatment recommendations frequently conflicted with local clinical guidelines, referenced drugs not available in those markets, and did not reflect the patient populations those hospitals actually served. The system was recommending treatments that were not appropriate, not because the model was performing incorrectly by its own design, but because its design was anchored in a clinical context that did not generalise.
"Internal documents show that IBM Watson for Oncology has recommended 'unsafe and incorrect' treatment options, according to a review by doctors at one of the largest US cancer centres. The documents reveal that the training process relied heavily on a small number of hypothetical cases rather than on real patient data, and that the system was not tested against actual clinical outcomes in the populations where it was deployed."
The EU AI Act's Article 9 requires that risk management measures account for risks arising from use in conditions that differ from the training environment. The Watson case is a precise description of that risk — a model trained in one clinical context, deployed in another, without a formal assessment of what the deployment context required and whether the training context could support it.
The question that was not asked before training began
The Watson failure was not a data quality failure in the conventional sense. The training data was high quality within its context. The failure was that nobody formally documented the deployment context before training began — the clinical guidelines that would apply in the deployment environments, the patient populations those environments served, the drug formularies available, the regulatory frameworks governing treatment decisions.
Without that documentation, there was no mechanism to identify the gap between the training context and the deployment context. The acceptance testing was conducted against the training context — Memorial Sloan Kettering's protocols — and passed. The deployment context was encountered for the first time after the system was in use by oncologists making real treatment decisions.
The deployment context specification as a pre-training artefact
The resolution is structural. Before training begins on any system that will be deployed in a context different from the environment in which the training data was generated, a deployment context specification must be produced. It is not a long document. It is a short one with a specific purpose: to force a comparison between the context that will be represented in the training data and the context the system will actually encounter.
For Watson for Oncology, a deployment context specification written before training began would have answered a set of questions that were never formally posed: which clinical guidelines apply in each deployment market? What is the patient population distribution? Which drugs are available? What regulatory framework governs treatment decisions? What real outcome data is available from the deployment sites to validate recommendations?
The answers to those questions are not technically complex. They are readily available. They were simply never elicited as a formal pre-training requirement. The cost of eliciting them before training is measured in days. The cost of not eliciting them was measured in failed deployments, reputational damage, and — more significantly — in clinical decisions that were influenced by recommendations that were not appropriate to the context in which they were received.
Before the first training run on any system that will be deployed in a context different from the training environment, produce a deployment context specification. It answers five questions that cannot be answered later without the risk of discovering the gap at deployment.
1. What are the rules, guidelines, or standards that govern correct outputs in the deployment context? (Not the training context — the deployment context.)
2. What is the population that will be affected by the system's outputs? (Demographics, distribution, characteristics that differ from the training population.)
3. What resources, options, or constraints apply in the deployment context that do not apply in the training context? (Drug formularies, service availability, regulatory limitations.)
4. What real outcome data is available from the deployment context to validate performance? (Not proxy metrics from the training context.)
5. What is the gap between the training context and the deployment context on each of the above dimensions?
If question 5 cannot be answered, the training context is not representative of the deployment context. That is the finding to bring to the design review — before training begins, while the architecture can still be changed.