Answer
What should happen when an AI automation fails?
Someone finds out, the affected items are identifiable, and the manual fallback is something people know how to do.
Someone is told within a useful timeframe, the affected items can be listed, and the manual fallback is something the team still knows how to perform. The third erodes quietly and is discovered during the outage.
Three questions have to be answerable before an automation runs unattended, and none of them is about preventing failure. Who finds out, and how soon. Which items were affected, so they can be dealt with. And what happens in the meantime. Businesses generally have an answer to the first, a partial answer to the second, and no answer to the third.
Notification has to distinguish the failure that stopped things from the one that did not. A run that errored is loud and every platform reports it. A run that did not start reports nothing, because a process that did not run also did not raise an alarm, and it needs an external expectation — something that notices a result is missing. This is the failure that goes undetected longest and it is the cheapest to instrument.
Identifying affected items is what makes recovery possible and it depends on decisions made at build time. If the workflow records what it processed and when, the affected set is a query. If it does not, the affected set is whatever someone can reconstruct, and the usual outcome is either reprocessing everything or accepting that some items were missed. This is the strongest practical argument for keeping a processed-items record from the first version.
The manual fallback is the part that decays. When a process is automated, the people who used to do it stop doing it, the knowledge disperses, and the steps stop being written anywhere. Eighteen months later the automation fails and the fallback is a recollection. Keeping the manual procedure written and occasionally exercised is unglamorous and is what separates a two-hour disruption from a two-day one.
There is a decision to make in advance about what happens to work during an outage: does it queue, does it fall to people, or does it stop. All three are legitimate and they have very different implications for how long an outage can last. Queuing is usually right for internal processing and wrong for anything customer-facing, where a queue means silence and silence is the thing the automation was preventing.
Finally, a failure should produce a record afterwards. What broke, what was affected, how it was resolved, and what would have caught it sooner. This is a fifteen-minute exercise that most small businesses skip, and it is the only mechanism by which the same failure does not recur in the same form. It also produces the evidence for whether the automation is worth continuing to run.
The fallback is not the process you had before automating; it is whatever people can still remember of it eighteen months later.
Siddharth Sharma, Context Theory
Related questions
How quickly does someone need to be told?
Fast enough that the affected work can still be recovered, which depends on what the workflow does rather than on any general standard. A nightly reconciliation can tolerate a morning discovery; a customer response workflow cannot tolerate a weekend. Setting the notification window from the recovery window is the way to get an answer rather than a preference.
Should the automation retry automatically?
For transient causes, with a limit, and never for a failure whose cause persists. The important part is that a retry exhausting its attempts must escalate rather than fail quietly, because a workflow that retried three times and gave up has produced exactly the silent outage the notification was meant to prevent.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Firms that never responded to a web enquiry at all | 23% | Category-wide |
| Average B2B first-response time | 42 | Category-wide |
Oldroyd, McElheran & Elkington, "The Short Life of Online Sales Leads", Harvard Business Review (March 2011) · hours · 1.25M inbound leads across 2,241 US firms · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | A run that never started reports nothing because a process that did not execute also did not raise an alarm, so detection requires an external expectation of a result at a given time rather than platform error reporting. | Disabling a scheduled workflow and observing whether any notification is produced. |
| Software | Identifying affected items during a failure depends on a processed-items record kept from the first version, without which recovery reduces to reprocessing everything or accepting unknown omissions. | Attempting to list the items processed by the workflow in a given period from its existing records. |
| Constraint | The manual fallback decays after automation because the people who performed the process stop doing it and the steps stop being recorded, so the fallback available during an outage is a recollection rather than a procedure. | Asking whether anyone currently in the business could perform the process manually without assistance. |
| Response | Queuing work during an outage is appropriate for internal processing and inappropriate for customer-facing steps, where a queue produces silence, which is the outcome the automation existed to prevent. | Identifying what a customer experiences during an outage of the workflow. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one