HuggingFace Papers
8/10 signal
DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
agenticresearchtool-use
Summary
This paper introduces DeepSearch-Evolve, a self-distillation framework for training web agents, and DeepSearch-World, a deterministic and verifiable environment for this training. DeepSearch-World contains 420K multi-hop QA tasks and supports agentic behaviors like reflection and failure recovery. Using an iterative process of trajectory generation, filtering, and fine-tuning, the resulting DeepSearch-World-9B model achieves competitive performance without distillation from more capable models, reaching 61.5% on GAIA and 31.2% on BrowseComp.
Problem
Training tool-use agents to improve from their own experience is challenging. Supervised fine-tuning is limited by its reliance on fixed, teacher-distilled trajectories, while reinforcement learning provides weak supervision for long-horizon interactions due to sparse rewards. These limitations hinder the ability of agents to learn and evolve effectively on complex web tasks.
Method
The authors created DeepSearch-World, a deterministic environment with 420K multi-hop QA tasks and reproducible search and page-reading tools. On this environment, they apply DeepSearch-Evolve, a self-distillation framework. This framework iteratively performs trajectory generation, filters for successful outcomes, mixes the successful data, and fine-tunes the agent, allowing it to progressively improve its own capabilities without a stronger teacher model.
Details
Benchmark Performance:
The DeepSearch-World-9B model was evaluated on several web agent benchmarks, achieving competitive results without distillation from more powerful models.
| Benchmark | Score |
|---|---|
| GAIA | 61.5% |
| BrowseComp | 31.2% |
| HotpotQA | 93.4% |
Framework and Environment Details:
- DeepSearch-World Environment: A deterministic and verifiable environment designed for agent training. It contains 420,000 multi-hop question-answering tasks constructed from entity-level random walks.
- DeepSearch-Evolve Framework: A self-distillation process that iteratively cycles through four stages: trajectory generation, filtering, data mixing, and fine-tuning.
- Agentic Behaviors: The environment is designed to support key cognitive behaviors for self-evolution, including progress verification, grounded reflection, and failure recovery.
What's new
The work introduces a self-distillation framework, DeepSearch-Evolve, paired with a new verifiable and deterministic environment, DeepSearch-World. This combination enables a web agent to iteratively improve its performance from its own successful experiences, removing the need for distillation from a more capable teacher model.
Conclusion
The authors conclude that verifiable environments like DeepSearch-World enable scalable self-evolution for long-horizon web agents. This approach demonstrates a path to creating more capable agents by allowing them to learn directly from their own generated experience, overcoming the typical constraints of supervised fine-tuning and reinforcement learning.
Don't read this site daily. Get it in your inbox.
The daily brief and Sunday deep dive — distilled, scored, and opinionated. For builders only.