Agents & Automation

Coding assistants' reports leave out most actions they took

The paper studied 5,851 real sessions where coding AI systems made 355,942 tool calls. In the 5,115 sessions with both a final report and a log, reports mentioned roughly one executed action in eleven. When the system's work broke from its plan, its report tended to lean back toward that plan.

Beyond Prompting
Read original source

What the source reports

The paper analyzed 5,851 real sessions where coding AI systems made 355,942 tool calls. Its omission result used 5,115 sessions that had both a self-report and an action log. On average, reports referred to 9.4% of executed actions, roughly one in eleven. Reports did not generally resemble plans more than logs. But when execution moved farther from a stated plan, reports increasingly leaned back toward the plan instead of tracking the changed sequence. That pattern was conditional, and plan changes did not predict whether a developer later corrected the code. The paper does not show intent or dishonest outcome claims. Its reconstruction result came from a model probe scored against logs, not from testing human readers. All sessions came from one coding corpus, mostly one software setup.

Original source

Title
Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agents
Author
Obada Kraishan, Kulsawasd Jitkajornwanich
Publication
arXiv
Date
Thursday, September 10, 2026