Answer
How should AI systems fail safely?
By stopping and saying so, rather than by producing a lower-quality answer. Degraded output is the failure mode that gets used.
By refusing and saying why, not by producing a weaker answer. A degraded output looks like a normal one and gets used; a refusal is visible and routes the work to a person, which is the outcome you wanted.
Conventional systems fail loudly: something errors, something is unavailable, someone is notified. A language-based system fails quietly by producing something. Given insufficient material it produces an answer from what it has; given an ambiguous instruction it picks a reading; given a broken input it produces a plausible interpretation. In each case the output is well formed, and well formed is what downstream steps and human readers check for.
So the design objective is to convert quiet failures into loud ones. The main mechanism is a required output shape that cannot be satisfied by a guess: an identifier that must exist, a value that must be in a set, a citation that must resolve, a total that must reconcile. When the system does not have what it needs, the shape check fails and the run stops, which is the behaviour you wanted and could not obtain by asking.
The second mechanism is an explicit refusal path. If the only available outcomes are success and error, the system will select success, because a partial answer is closer to success than to error. Adding a third outcome — could not determine, with the reason — gives it somewhere to go that is not a guess, and it has to be an accepted outcome rather than an exception, or it will be avoided.
The third is that degradation must be visible when it happens. A system that falls back to a smaller model, a cached result, a default value or a shorter context should say so in its output, because the consumer's decision about whether to rely on it depends on knowing. Silent fallback is the specific pattern that produces the worst outcomes, since it is invisible precisely when the system is least trustworthy.
The fourth is the direction of the default. When something is uncertain, the safe default is the option that is easiest to reverse: do not send, do not update, route to a person, leave the record unchanged. This sounds obvious and is frequently inverted in practice, because the automated path is the one that was built and the manual path is friction. The test is which outcome you would prefer to discover was wrong tomorrow.
Finally, the failure has to reach someone. A run that correctly refuses and writes its refusal to a log nobody reads has failed safely and uselessly. The refusal needs a destination — a queue, an alert, a person's list — and that destination should be a place work is already done rather than a new place to check.
The dangerous failure is not the one that stops the process, it is the one that keeps it running at a lower standard nobody was told about.
Siddharth Sharma, Context Theory
Related questions
Is it better to give a partial answer than none?
Only where the partiality is visible in the answer. A result that says these three of five records were matched and the other two could not be is useful; a result that quietly covers three and presents them as the answer is worse than nothing, because it will be treated as complete. The distinction is disclosure rather than completeness.
How do you stop a refusal path being abused?
Measure it. A system that refuses too readily is as useless as one that never does, and the rate is observable: track how often it declines and sample those cases. Most systems err in the other direction, which is why the refusal path needs to be built first and the rate watched afterwards.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Firms that never responded to a web enquiry at all | 23% | Category-wide |
| Sub-15-minute compliance — automated routing vs manual only | 62.5% vs 39.1% | Category-wide |
Oldroyd, McElheran & Elkington, "The Short Life of Online Sales Leads", Harvard Business Review (March 2011) · 1.25M inbound leads across 2,241 US firms · verified
2026 speed-to-lead benchmark · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | Language-based systems fail by producing a well-formed answer rather than by erroring, and well-formedness is what downstream steps and human readers check for, which is why the failure passes. | Supplying insufficient material to a workflow and observing whether the output is distinguishable from a normal one. |
| Software | An output shape that cannot be satisfied by a guess — a resolvable identifier, a value from a set, a reconciling total — converts a quiet failure into a check failure, which is behaviour obtainable by construction rather than by instruction. | Applying the shape constraint to a run with missing inputs and observing whether it stops. |
| Response | Where only success and error are available outcomes the system selects success, because a partial answer resembles success more than it resembles an error, so an explicit and accepted could-not-determine outcome is required. | Comparing behaviour on an unanswerable input with and without an accepted refusal outcome. |
| Constraint | Silent fallback to a smaller model, a cached result or a default value hides degradation exactly when the system is least trustworthy, so any fallback must be declared in the output for the consumer's reliance decision to be informed. | Checking whether the workflow's output records which path produced it. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one