Answer
How should an AI system react when evidence is missing?
Say what is missing and where it was looked for. An absence reported is a finding; an absence filled is a fabrication.
Report the gap: what was needed, what was searched, and what was found instead. An absence stated is a useful finding. An absence filled with a plausible value is indistinguishable from a supported claim and will be treated as one.
The default behaviour when material is missing is to produce the best available answer, and the best available answer from insufficient evidence is a plausible construction. This is not a defect of intent; it is what happens when the only acceptable outputs are answers. Giving the system a legitimate place to report an absence changes what it does, and it is a change in the specification rather than in the instruction.
A reported absence has to contain three things to be useful. What was required — the specific fact, not the general topic. What was searched — the sources, the queries, the locations. And what was found instead, since a near miss is often the most informative part: a related document, an outdated figure, a similar record under a different name. Without these, could not determine is unfalsifiable and indistinguishable from not having looked.
That distinction is the practical heart of it. There are two very different states behind the same phrase: the evidence does not exist in what could be reached, and the search was inadequate. Only the first is a finding. Requiring the search to be described is what separates them, and in practice the second turns out to be more common, which is worth knowing before anyone concludes that a figure does not exist.
The system also needs to distinguish missing from absent-and-meaningful. A field that is empty because nothing was recorded is different from one that is empty because the answer is none, and treating the second as missing loses real information. This distinction is usually available in the data and is usually discarded by anything that checks only for emptiness.
Downstream, an absence has to be actionable rather than merely reported. A workflow that produces could not determine and continues has converted a gap into a silent one further along. The gap should stop the run, route the item to a person, or mark the output as incomplete in a way the consumer of the output can see. Which of these is right depends on what the output is for, and choosing none is the common default.
One structural note. Where absences are common, they are data. Counting them by type — which field, which source, which kind of question — shows where the underlying information problem is, and that is usually more valuable than any individual answer. Businesses that track what their systems could not determine tend to find a small number of recurring gaps that are cheap to fix at the source.
Not finding something is a result, and a system with nowhere to put that result will produce a different one.
Siddharth Sharma, Context Theory
Related questions
Should the system guess if the user wants an answer anyway?
It can offer an estimate provided the estimate is labelled and the basis is stated, which is a different artefact from an answer. What must not happen is the estimate arriving in the same form as a supported figure, because the label is the only thing preventing it from being used as one, and a label applied inconsistently is worse than none.
How do you stop a system over-reporting absences?
Measure the rate and sample the cases, as with any other behaviour. A system that declines too readily is a real failure and it is rarer than the opposite. Where it happens, the usual cause is a requirement stated more strictly than intended rather than excessive caution, and the fix is in the specification.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Firms that never responded to a web enquiry at all | 23% | Category-wide |
| Close rate — response under 5 minutes vs over 24 hours | 32% vs 12% | Category-wide |
Oldroyd, McElheran & Elkington, "The Short Life of Online Sales Leads", Harvard Business Review (March 2011) · 1.25M inbound leads across 2,241 US firms · verified
Optifai speed-to-lead benchmark · n=939 companies · Q2 2025–Q1 2026 · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | Where the only acceptable outputs are answers, insufficient evidence produces a plausible construction rather than a report of the gap, so providing a legitimate absence outcome is a change to the specification rather than to the instruction. | Running a query whose answer is not present in the supplied material, with and without an accepted absence outcome. |
| Response | A useful absence report names the specific fact required, the sources and queries searched, and the near misses found, without which the report is unfalsifiable and indistinguishable from not having looked. | Checking whether an existing could-not-determine result states what was searched. |
| Software | An empty field because nothing was recorded and an empty field because the answer is none are different states, and a check testing only for emptiness discards the second, which is real information. | Inspecting whether the data source distinguishes a null from a recorded zero or none. |
| Constraint | A workflow that reports an absence and continues has moved the gap further down silently, so the absence must stop the run, route to a person, or mark the output as incomplete visibly to its consumer. | Tracing what a workflow does with an item after an absence is reported. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one