An AI companion said it remembered. What actually ran?
In a small prototype study of nine people, users rated an AI companion’s memory reasonably well. Yet two parts meant to build up that memory had not run. The preprint suggests checking a system’s claims against both harmless observed behavior and a record of the steps it took. It does not measure all AI companions.
What the source reports
Seiya Ikeda and Shin-nosuke Ishikawa tested a sample AI companion with nine people. Users thought it kept track of past talks fairly well. Yet two steps meant to save memories had not run. A friendly answer can make it seem as if a tool recalls earlier talks even when its memory step never worked. This was a small test of one sample system. It does not show how often other AI tools have the same flaw. The paper asks readers to look beyond what users felt. Check what a tool does with a harmless detail, then compare that with a record of the steps that ran. Both matter when an AI says it remembers.
Original source
- Title
- Where the Evidence Lives: Auditing AI Companions' Self-Descriptions
- Author
- Seiya Ikeda, Shin-nosuke Ishikawa
- Publication
- arXiv
- Date
- Wednesday, September 30, 2026