AI can solve a task and still miss the clock
A system may solve a task when the clock pauses, yet fail while a user waits. Jack Clark reports on a 50-task test across web and game settings. The best real-time success rate was 36%. This is a newsletter account of a test, not a measure of school operations.
What the source reports
Jack Clark covers Realtime-Bench in the Import AI newsletter. It tests 50 tasks in interactive web and game settings. One setup uses real elapsed time. Others pause the clock or count the text a model produces while reasoning. Clark reports that the top system finished 36% of tasks under the real clock. Some models did much better when time was paused. That gap matters when a tool must act before a live chance closes. It does not show that any school system would succeed at the same rate. The newsletter is a secondhand account of the test, and the tasks are not school jobs. The supported point is about timing: a result can be correct too late to help.
Original source
- Title
- Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4
- Author
- Jack Clark
- Publication
- Import AI
- Date
- Monday, November 2, 2026