Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

Computer-use agents, which leverage multimodal large lan-guage models (MLLMs) to operate computers and complete tasks, haveattracted significant attention for their utility and versatility. A majorchallenge in developing these agents is collecting large-scale, high-qualitytrajectories. The standard approach generates synthetic data through aself-improving loop: an agent is placed in a verifiable environment anditeratively fine-tuned on its successful trajectories. Despite its effective-ness, this paradigm exploits only successful trajectories and discards thefailed ones, even though failures carry rich information about a model’sweaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajecto-ries into agent improvements. Specifically, we employ an LLM to diag-nose failure modes, propose inference-time solutions, and generate codepatches—lightly verified by humans—that upgrade the agent. We vali-date this approach with the state-of-the-art OpenCUA-72B model on theOSWorld benchmark, improving the success rate from 42.3% to 48.9%,a gain of 6.6 percentage points, without any additional training costand with only modest inference overhead. Our results demonstrate thatfailure-driven self-improvement is a viable complement to success-basedpipelines, enabling more efficient agent improvement. Code is availableat https://github.com/snow10072740/LearningfromFailure.