RECAP: Training Long-Horizon LLM Agents with Context Compaction

RECAP learns to compact context
while preserving how the agent acts.

Abstract

Context compaction enables large language model agents to tackle long-horizon tasks within limited context budgets by replacing interaction histories with compressed summaries. However, learning effective compaction jointly with the agent policy remains challenging, as summaries affect task success over many subsequent interactions.

We introduce RECAP, a reinforcement learning framework based on the principle that a summary should preserve the behavior induced by the original interaction history. We formalize this principle with a behavior distortion objective that measures the divergence between the agent’s subsequent behavior under compressed and original contexts. Its gradient decomposes into two complementary learning signals: a distortion-based reward for summary generation and self-distillation for subsequent actions.

On Qwen3.5-4B, RECAP improves over task-reward-only reinforcement learning by 4.6 percentage points on BrowseComp-Plus and 4.3 percentage points on SWE-bench Verified, with the same 32K working-context budget. Ablations highlight the value of jointly supervising summary generation and post-compaction behavior.

Method

Compact the context. Preserve the behavior.

A long-running agent repeatedly gathers observations, takes actions, and summarizes its history when the context fills up. A task-success reward arrives only at the end, making it difficult to learn which summaries helped. RECAP adds a direct signal for each compaction: how much did it change the agent’s subsequent behavior?

RECAP compresses the original history into a summary, retaining the instruction and recent interactions. A continuation sampled from the compacted context is scored under both contexts. Behavior distortion gives a reward to the summary and self-distillation feedback to subsequent actions.

We measure how compaction changes the agent’s behavior using token-level KL divergence, comparing the original and compacted contexts along the same continuation.

D_{KL} (π_{compacted} ∥ π_{original})

RECAP learns to minimize this behavior change. The gradient of its compaction objective decomposes into two learning signals:

\nabla_{θ} J_{comp} = g_{sum} + g_{post}

Reward the summary $g_{sum}$

Reward summaries that preserve the agent’s subsequent behavior.

Guide subsequent actions $g_{post}$

Use the original context to guide subsequent actions through self-distillation.

We train with both signals alongside the task-success reward, reusing the same sampled trajectories with no additional rollouts or environment interactions.

Experimental Results

Better results with the same context budget

RECAP improves performance on deep search and software engineering. The main comparison uses Qwen3.5-4B with a 32K working context. Both RECAP and the task-reward-only GRPO baseline use context compaction.

BrowseComp-Plus accuracy: task reward only 58.2%, RECAP 62.8%. Standard deviations are 1.5 and 3.9 percentage points.

SWE-bench Verified accuracy: task reward only 46.8%, RECAP 51.1%. Standard deviations are 1.4 and 1.0 percentage points.

Both learning signals matter

Combining the distortion-based summary reward with self-distillation achieves the best accuracy on both benchmarks. The two signals supervise different parts of the same process: what to preserve, and how to act after compaction.

Learning-signal ablation · Accuracy (%)
Training signals	BrowseComp-Plus	SWE-bench Verified
Task reward only	58.2	46.8
+ Summary reward	56.7	47.9
+ Self-distillation	60.3	47.1
+ Both (RECAP)	62.8	51.1

BibTeX

@misc{jiang2026recap,
  title  = {{RECAP}: Training Long-Horizon {LLM} Agents
            with Context Compaction},
  author = {Jiang, Yulun and Brbić, Maria and Moor, Michael},
  year   = {2026}
}

Abstract

Method

Compact the context. Preserve the behavior.

Reward the summary gsum

Guide subsequent actions gpost