RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

Summary

RSIAgent is a training-free multi-agent framework in which a curriculum agent proposes practice tasks, an actor agent executes code-as-policy actions, and a verifier agent grounds outcomes in environment feedback, iteratively building a persistent memory that is frozen and reused at test time. Exploration is broad-then-deep: Broad Recursive Self-exploration (BRS) runs parallel project waves for coverage, while Deep Recursive Self-exploration (DRS) sequentially targets hard cases, hidden constraints, and boundary conditions. On OSWorld 2.0 (82 offline tasks) and Agents' Last Exam Near-term (67 tasks), the authors report that RSIAgent lets GLM-5.3 and Kimi-K3 surpass frontier closed-source models including GPT-6 in partial score. On GameCraft-Bench (40 tasks), the full method improves all four base generators (e.g., GLM-5.3-Flash rises 30.55 to 48.72 overall) and outperforms Play2Code, which degrades strong baselines. The headline aggregates, however, are hybrids: the OSWorld RSI row replaces baseline scores on only 41 of 82 tasks, with exploration preferentially directed at tasks below full credit, and the ALE row combines 19 RSI scores with 48 baselines. On ALE binary accuracy RSIAgent (50.75) trails GPT-6 Astra (52.24). Comparisons are explicitly not matched-protocol; no significance tests or variance are reported; ablations and case studies use historical baselines. Limitations include heavy compute cost, verifier error propagation, and incomplete component isolation.