In math chats, the change in questions mattered
A preprint studied 880 meaningful chat turns from 112 secondary students doing math practice with Mistral. Planning and self-checking were rare. Students who moved toward asking for understanding scored better on a later test without AI, even after other patterns were considered. The study observed a link. It did not test a lesson that caused that gain.
What the source reports
Abdelghani and colleagues analyzed 880 meaningful messages from 112 secondary-school students using Mistral for math practice. They coded how students asked for help and checked their work. Planning, monitoring, and evaluation were uncommon. On average, requests became less focused on understanding. Yet students whose questions moved the other way did better on a later test taken without AI. The analysis accounted for an earlier test and broad chat patterns. It found a link within this group, not proof of cause. A student who asks deeper questions may differ in other ways. The abstract does not establish that a change to the chatbot or the teacher prompt would improve scores.
Original source
- Title
- Student-LLM Interactions During Math Learning: Limited Epistemic Proactivity Overall, but Within-chat Change Predicts Performance Without AI
- Author
- Rania Abdelghani, Peter Kaiser, Kou Murayama
- Publication
- arXiv cs.CY
- Date
- Friday, October 30, 2026