A screen recording can teach steps, not the reason
Work2Skill turns 350 human desktop demos into sets of steps an AI tool can reuse. It adds the order of tasks and checks for when a job is done. The authors report better results than tools without those saved steps on two tests. Those tests do not show that the tool knows why a person chose a step or when to make an exception. Do not record private screens to try it.
What the source reports
Work2Skill starts with 350 demos of people using desktop apps. The researchers use them to make step-by-step instructions for AI tools. Each set records what must happen first and how to check that a job is done. On two computer-use tests, the authors report better results than setups with no saved steps. A demo can show the click and the order. It may not show why a person made that choice or what would make them stop. Before reusing a step, ask the person about exceptions. The results come from computer-use tests, not a trial of school or workplace outcomes. Do not record private screens for this exercise.
Original source
- Title
- Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
- Author
- Yuqi Huang, Hui Li, Yifei Cao, Zikun Cai, Tianci Xie, Yangfan Gao, Pinxue Guo, Chaoyun Zhang, Bingxuan Li, Rui Zheng, Hang Yan, Ning Liao
- Publication
- arXiv
- Date
- Friday, October 9, 2026