Should an AI agent explain its reasoning?

There are two different things people mean by explanation and they have very different value. One is a record of what was decided and what evidence supported it: this file was chosen because the query matched, this record was skipped because the field was empty, this approach was taken because the alternative required a permission that was not available. That is checkable, and every item in it can be verified against the artefacts.

The other is an account of how the conclusion was arrived at. This is a narrative and it is produced on request, in a persuasive register, whether or not it corresponds to the process that generated the answer. Treating it as evidence about the system's actual operation is the mistake, and it is a comfortable mistake because the narrative is usually coherent and sounds like exactly the kind of thing that would be true.

The practical consequence is what you should ask for. Requiring evidence attached to conclusions — the quotation, the record, the file, the value it read — gives you something you can check in seconds and something the agent cannot produce if it is not there. Requiring an explanation of the thinking gives you a paragraph you can only agree or disagree with, and disagreement requires doing the work yourself.

There is a second reason to require decisions and evidence that has nothing to do with verification. Writing them down makes them available later: to the next session, to a person picking up the work, to whoever is trying to understand why the system did something surprising three weeks ago. A narrative does not serve this purpose either, because it explains a conclusion rather than recording a choice, and the useful question later is nearly always what was chosen and on what basis.

There is one place where asking for reasoning genuinely helps, and it is worth being precise about why. Asking for the plan before the work, rather than the explanation after it, changes what happens: the plan is a commitment that subsequent work is measured against, and reading it costs a fraction of reading the output. That is not the same operation as a retrospective explanation, even though both look like asking the agent to show its thinking.

For anything consequential, the standard is the same one applied to any output: the explanation is not evidence, the artefact is. If an agent says it checked something, the useful confirmation is the value it read, not the assurance that it looked.