Fake audience panels lost to a simpler AI guess
Inventing ten reader profiles did not improve this headline test. A study of 399 Upworthy comparisons with clear winners found that a plain AI prediction chose the stronger headline 49.2% of the time; the ten-profile method reached 34.6%. That favors testing a simple starting point first. It does not make real audience research useless.
What the source reports
Alexandre Cristovão Maiorano compares AI predictions with real results from Upworthy headline tests. In 399 tests where one headline had a clear winner, a plain AI prediction ranked click outcomes better than a panel of ten invented reader profiles. The plain method picked the winning option 49.2% of the time, compared with 34.6% for the panel. On a measure of how well the full rankings agreed, the scores were 0.361 and 0.084. These are results for predicting combined clicks in this set of headline tests. The preprint favors checking a simple starting point against real outcomes before adding simulated people. It does not test whether interviews with actual readers help with a different decision.
Original source
- Title
- Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation
- Author
- Alexandre Cristovão Maiorano
- Publication
- arXiv
- Date
- Wednesday, September 23, 2026