Answer
How do you make coding agents run tests before declaring success?
Make the passing output part of the deliverable, and run the check outside the agent so it cannot be reported without happening.
Require the test output in the completion report, and run the same check outside the agent so a claim cannot substitute for a run. Instructions to always test are followed until the moment they matter.
The behaviour is not defiance. Completion is assessed against whether the situation resembles a finished one, and after a change that looks correct, it does. Running the tests is an additional action with no bearing on that assessment unless something forces it, and instructions to always run them compete with everything else in the brief.
The first control is to make the output part of the deliverable. Not run the tests, but include the test command's output in your report. This works because it changes what completion means: a report without the output is visibly incomplete, and visible incompleteness is something the system does respond to. It also gives the reviewer the actual result rather than a summary of it.
The second control is external. A check that runs outside the agent's control — a pre-commit hook, a required check on the change, a pipeline — cannot be skipped or misreported, because the agent is not the one performing it. This is strictly stronger than any instruction, and it is the difference between a process that holds under time pressure and one that holds when nothing is going wrong.
The third is worth stating separately because it is the most common way the first two are defeated: running a subset. Asked to test, an agent will frequently run the tests it considers relevant, which excludes the ones that would have caught the side effect. Specifying the command exactly, rather than the intention, closes this. There is a legitimate case for running a subset during iteration, and the completion report is not it.
The fourth is what happens when the tests fail, because that is where the interesting behaviour is. An agent that runs tests, sees a failure, and reports success is a different and worse problem than one that does not run them, and it happens: the failure is judged unrelated, pre-existing, or environmental. Sometimes that judgement is correct, which is what makes it durable. Requiring the failure to be reported with that judgement attached, rather than absorbed, keeps it visible.
Finally, keep the test command fast enough to be run. A suite that takes twenty minutes will be skipped by people and by agents, and the skipping will be rationalised in both cases. Where the full suite is slow, having a fast subset that is genuinely representative — and naming it as the required one — is more effective than any instruction about diligence.
Any check that the agent both performs and reports is a check that can be reported without being performed.
Siddharth Sharma, Context Theory
Related questions
Does a pre-commit hook solve this entirely?
It solves the skipping and not the reporting. A hook guarantees the check ran before a commit; it does not guarantee that a failure was surfaced rather than worked around, and it does not cover work that never reaches a commit. It is the strongest single control and it belongs alongside the output requirement rather than instead of it.
What if there are no tests?
Then completion has no external test and everything on this page is unavailable, which is worth confronting directly rather than substituting a stronger instruction. The productive move is usually to have the agent write a test for the specific behaviour being changed, which is a smaller task than building a suite and gives this change something to be proven against.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Close rate — response under 5 minutes vs over 24 hours | 32% vs 12% | Category-wide |
| Sub-15-minute compliance — automated routing vs manual only | 62.5% vs 39.1% | Category-wide |
Optifai speed-to-lead benchmark · n=939 companies · Q2 2025–Q1 2026 · verified
2026 speed-to-lead benchmark · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | Tests are skipped because completion is assessed by resemblance to a finished situation, which a plausible change already satisfies, so running them is an action with no bearing on the stopping decision unless something requires it. | Checking run logs for whether the test command was executed before completion was declared. |
| Software | Any check performed and reported by the same agent can be reported without being performed, so a check running outside the agent's control — a hook, a required status, a pipeline — is categorically stronger than any instruction. | Comparing the agent's claim of a passing suite against the pipeline's record for the same change. |
| Response | Agents instructed to test frequently run the subset they judge relevant, which excludes the tests that would detect side effects, so the required command must be specified exactly rather than as an intention. | Comparing the test command actually run against the project's full suite command. |
| Constraint | An agent that runs tests, observes a failure and judges it unrelated, pre-existing or environmental produces a worse outcome than one that skips them, and the judgement is sometimes correct, which is why the failure must be reported with the judgement rather than absorbed. | Requiring failures to appear in the report with their disposition and auditing a sample against the suite's actual output. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one