HuggingFace Papers
★ 9/10 signal
SWE-Pruner Pro: The Coder LLM Already Knows What to Prune
contextagenticresearch
Summary
This paper proposes SWE-Pruner Pro, a method for pruning redundant context from coding agent tool outputs by reading line-level importance directly from the agent's own internal representations. A lightweight head, trained with a length-aware embedding and a per-sample balanced focal loss, is attached to the frozen agent backbone to make keep-or-prune decisions. Evaluated on two backbones and four benchmarks, SWE-Pruner Pro saves up to 39% of tokens while preserving task quality. On SWE-Bench Verified with the MiMo-V2-Flash backbone, it improves the resolve rate by 3.8%.
Problem
Coding agents accumulate long, redundant tool outputs in their context during multi-turn interactions, which increases cost and degrades performance. Prior pruning methods are either general-purpose compressors that are not task-aware, or task-specific pruners like SWE-Pruner that require a separate scoring model and an explicit goal-hint query from the agent at every turn. These approaches treat the agent's information need as something to be reconstructed externally, adding overhead.
Method
SWE-Pruner Pro attaches a lightweight, trainable head to the agent's frozen backbone LLM. This head reads the last-layer hidden states produced during the agent's normal prefill pass over a new tool output. It uses a length-aware embedding and a two-block feed-forward classifier trained with a per-sample balanced focal loss to predict a keep-or-prune label for each line. The method was evaluated on Qwen3-Coder-Next and MiMo-V2-Flash backbones across four benchmarks: SWE-Bench Verified, SWE-QA, SWE-QA-Pro, and Oolong.
Details
Motivation:
- A linear probe on the frozen last-layer hidden states of the Qwen3-Coder-Next backbone was able to distinguish between lines to keep and lines to prune.
- The probe achieved an AUC of 0.83 and a best-F1 of 0.63 on held-out trajectories, significantly above the majority-class F1 upper bound of 0.46, indicating the pruning signal is already present in the backbone's representations.
Read-Only Multi-Turn Benchmarks (SWE-QA, SWE-QA-Pro, Oolong):
| Backbone | Benchmark | Method | Score/Acc. | Total Tokens |
|---|---|---|---|---|
| Qwen3-Coder-Next | SWE-QA-Pro | No Pruning | 7.60 | 607K |
| Qwen3-Coder-Next | SWE-QA-Pro | SWE-Pruner Pro | 7.84 (↑0.24) | 368K (↓39.4%) |
| Qwen3-Coder-Next | Oolong | No Pruning | 81.7 | 3.6K |
| Qwen3-Coder-Next | Oolong | SWE-Pruner Pro | 80.3 (↓1.4) | 3.1K (↓13.9%) |
| MiMo-V2-Flash | SWE-QA-Pro | No Pruning | 7.97 | 438K |
| MiMo-V2-Flash | SWE-QA-Pro | SWE-Pruner Pro | 7.86 (↓0.11) | 339K (↓22.6%) |
| MiMo-V2-Flash | Oolong | No Pruning | 92.4 | 58.9K |
| MiMo-V2-Flash | Oolong | SWE-Pruner Pro | 94.6 (↑2.2) | 41.2K (↓30.1%) |
- SWE-Pruner Pro was the only method among seven evaluated that reduced end-to-end token use in every setting while maintaining task quality close to the unpruned agent.
- Other pruners like LLMLingua2, Selective Context, and RAG often inflated token counts or degraded performance significantly. For instance, LLMLingua2 increased tokens by 190% on Oolong with MiMo-V2-Flash.
Code-Modification Benchmark (SWE-Bench Verified):
| Backbone | Method | Resolved | Avg. Input Tokens | Avg. API Calls |
|---|---|---|---|---|
| MiMo-V2-Flash | No Pruning | 326/500 | 2,971K | 94.8 |
| MiMo-V2-Flash | SWE-Pruner | 347/500 (↑4.2%) | 3,414K (↑14.9%) | 103.8 |
| MiMo-V2-Flash | SWE-Pruner Pro | 345/500 (↑3.8%) | 3,190K (↑7.4%) | 111.8 |
| Qwen3-Coder-Next | No Pruning | 341/500 | 5,307K | 131.9 |
| Qwen3-Coder-Next | SWE-Pruner Pro | 335/500 (↓1.2%) | 4,590K (↓13.5%) | 139.8 |
- On MiMo-V2-Flash, all pruners improved the resolve rate. SWE-Pruner Pro achieved a +3.8% improvement with about half the token overhead of SWE-Pruner.
- On Qwen3-Coder-Next, all pruners lost resolves. SWE-Pruner Pro had the smallest degradation (-1.2 points) while achieving the largest input-token reduction (-13.5%).
Ablation Study:
| Design Axis | Variant | F1 | Judge Score |
|---|---|---|---|
| Loss function | BCE | 0.475 | 5.95 |
| Loss function | Focal | 0.593 | 6.37 |
| Loss function | Per-sample balanced focal (default) | 0.635 | 7.08 |
| Length-aware embedding | Without | 0.636 | 6.86 |
| Length-aware embedding | With (default) | 0.635 | 7.08 |
- The per-sample balanced focal loss outperformed standard BCE and corpus-level Focal loss, improving the judge score from 5.95 (BCE) to 7.08.
- Adding the length-aware embedding improved the judge score from 6.86 to 7.08 with no change in F1 score, by redistributing errors to longer responses where they are less harmful.
Latency and Training:
- The in-server pruning head adds 15.0% aggregate wall time on top of the agent's total generation time.
- The head was trained on a corpus of 22,609 samples from multi-turn agent trajectories, with per-line keep/prune labels generated by Claude Sonnet 4.6.
What's new
The core contribution is demonstrating that an agent's internal representations already encode the necessary signal for pruning tool outputs. This eliminates the need for a separate scoring model or explicit goal-hint queries, which were required by the closest prior work, SWE-Pruner. The paper introduces a method to extract this signal using a lightweight head with two novel components: a learned length-aware embedding and a per-sample balanced focal loss.
Limitations
The evaluation is limited to open-weight models because the method requires access to internal hidden states. While the benchmarks are Python-centric, the method was also tested on the natural-language Oolong benchmark, where it performed well. The authors state that broader coverage across programming languages is an area for future work.
Conclusion
The authors conclude that a coding agent's internal representations, formed while reading tool outputs, already contain sufficient information for line-level importance judgments. SWE-Pruner Pro successfully extracts this signal using a lightweight head, a length-aware embedding, and a per-sample balanced focal loss. This approach consistently reduces token consumption across multiple benchmarks and backbones while preserving task quality, suggesting a general principle: instead of building external mechanisms to reconstruct an agent's information needs, it is more efficient to read the relevance signals the agent has already formed.
Don't read this site daily. Get it in your inbox.
The daily brief and Sunday deep dive — distilled, scored, and opinionated. For builders only.