When AI says it helped, ask what the person kept
The Kolab paper uses recordings of 50 professionals at work to examine AI help across longer tasks. Its tested models often overstated the quality of the teamwork. That is a warning about self-reports, not a finding that AI never helps. Look for what the person actually used, changed, or rejected.
What the source reports
Hayati and colleagues introduce Kolab, a test of how people and AI work together on longer jobs. The source digest describes over 115 hours of recordings from 50 professionals. In the paper, tested models often rated the quality of their teamwork too highly. A model saying it helped is not the same as a record of what the person kept. The practical distinction is to track contributions a person used, changed, or rejected. This test evaluates models. It does not test whether the daily review routine in the source digest makes a team more productive.
Original source
- Title
- Kolab: Benchmarking Human-Agent Collaboration in Long-Horizon Workflows
- Author
- Shirley Anugrah Hayati, Patricia Anugrah Setiani, Ruizi Wang, Zhouyi Li, Khanh Chi Le, Young-Jun Lee, Dongyeop Kang
- Publication
- arXiv
- Date
- Thursday, October 15, 2026