Closing a research issue is not checking every result
Dong, Zhang and Zheng report a conference review of 36 papers and 133 tables and figures. Authors filled out checklists; AI checks flagged issues; people made repair and review decisions. Twenty issues raised after review were closed with follow-up, but some work was not fully reproduced.
What the source reports
Dong, Zhang and Zheng report on a conference review of research results. The process joined author checklists, AI checks, author repair, and reviewer decisions. Across 36 papers and 133 tables and figures, reviewers raised 20 issues after review. Follow-up supported closing those issues, but some results were not fully reproduced. That distinction matters: an issue can be resolved without every finding being checked from start to finish. The Beyond Prompting digest adapts this into a low-stakes exercise for one public claim. The exercise has not been tested by this paper. Its curators reviewed the preprint abstract and metadata, not a full-text appraisal.
Original source
- Title
- Author-in-the-loop Agentic Reproducibility Auditing for a Machine Learning Conference
- Author
- Yukun Dong, Linjun Zhang, Tian Zheng
- Publication
- arXiv
- Date
- Thursday, October 15, 2026