Stop Context Bloat in Long-Running AI Agents

Long-running agent context bloat is the reason a deep tool-calling task that looked great in your demo becomes slow, expensive, and subtly wrong in production. Every tool call appends its request and result to the transcript, so by the twentieth call the model is re-reading a growing wall of its own past work on every turn — losing the thread, re-querying facts it already fetched, and paying for thousands of redundant tokens each time. This post is the Promptise Foundry decision guide for context_scope, the node-level lever that keeps a 30-tool task bounded instead of drowning in the middle. By the end you'll know which of the four modes to reach for, and you'll have a runnable example that stays flat while a naive loop balloons.

Why long-running agent context bloat happens

A naive tool-calling loop feeds the model the entire conversation on every turn. The message list grows like this:

turn 1:  [system, user]
turn 3:  [system, user, ai→tool, tool_result, ai→tool, tool_result]
turn 12: [system, user, + 22 more messages]   ← re-read in full, every turn

For a task that needs a dozen distinct facts, a naive loop can make dozens of tool calls — repeatedly looking up the same record because the relevant result is buried far back in the transcript. Three things degrade at once:

• Cost grows super-linearly. You pay to re-send the full history on every call.
• Latency climbs as the prompt gets longer each turn.
• Accuracy can drop. This is the well-known "lost in the middle" failure: the signal the model needs is there, but it's swamped by noise.

Context bloat in LLM agents is not a side effect to tolerate — on deep tasks it's the deciding factor for whether the run finishes correctly at all. So the fix isn't a bigger context window; it's controlling exactly how much history each reasoning step actually sees.

The lever: context_scope modes on PromptNode

In the Promptise reasoning engine, an agent is a graph of PromptNodes. Every node accepts a context_scope argument that controls what it sees on each LLM call. It is fully opt-in — a raw PromptNode defaults to "full", so existing behavior is unchanged until you decide to bound it.

Mode	What the node sees	Use it for
"full"	The whole accumulated transcript	Short tasks, or when every prior message genuinely matters
"scoped"	System prompt + the original task + only its own in-progress tool loop	Multi-stage graphs — drops the verbose output of other stages
"ledger"	System prompt + task + most-recent exchange + a compact deduplicated "facts gathered" ledger	Long single-node tool loops that gather many facts, then aggregate
"auto"	Behaves like "full" while the loop is short, then flips to "ledger" past a threshold	The safe default — simple tasks untouched, deep loops bounded automatically

The "ledger" mode is the one that tames a runaway tool loop. Instead of an ever-growing transcript, the node sees one line per tool(args) = result with last value wins per unique call, so duplicates collapse. The ledger is placed last — right before the model's turn, where it's most salient — and repeated calls are cache-served instead of re-executed. The model consults its own gathered facts rather than re-fetching them. The full mechanism is documented in the context lifecycle guide.

"auto" is what the built-in react pattern ships with, which is why most agents get bounded behavior without touching a single knob: a short Q&A sees the full transcript unchanged, and a deep chain switches to a ledger once it has accumulated enough tool results.

Bound agent tool loops with a facts ledger

Here's the shortest path to a bounded deep chain. The managed pattern is a single tool-using node run with context_scope="ledger" — ideal for traversing a database or graph where you gather many facts and then aggregate them. This is a complete, runnable script; set OPENAI_API_KEY and point it at any MCP server.

import asyncio
from promptise import build_agent
from promptise.config import StdioServerSpec


async def main():
    agent = await build_agent(
        model="openai:gpt-5-mini",
        servers={"company": StdioServerSpec(command="python", args=["company_tools.py"])},
        agent_pattern="managed",  # one tool node, run with context_scope="ledger"
        instructions=(
            "Answer by calling tools. A deduplicated ledger of the facts you have "
            "already gathered is provided every turn — consult it and never re-fetch "
            "a fact you already have."
        ),
        max_agent_iterations=30,  # deep chains make many calls
    )

    result = await agent.ainvoke({"messages": [
        {"role": "user", "content":
         "Which team has the highest average tenure? Traverse the org and report."}
    ]})
    print(result["messages"][-1].content)
    await agent.shutdown()


asyncio.run(main())

When you need mixed modes across a custom topology, drop context_scope onto individual nodes. Here gather runs a bounded tool loop, then write sees only the distilled output it inherits plus the task — neither stage drowns in the other's raw messages:

from promptise.engine import PromptGraph, PromptNode

graph = PromptGraph("research", nodes=[
    PromptNode("gather", inject_tools=True, context_scope="ledger", is_entry=True),
    PromptNode("write", context_scope="scoped",
               inherit_context_from="gather", is_terminal=True),
])

agent = await build_agent(model="openai:gpt-5-mini", servers=srv, agent_pattern=graph)

Both patterns keep the same tools and the same model — only the view of history changes.

Choosing full, scoped, ledger, or auto

Pick the mode that matches the shape of your task, not the size of your model:

• Short Q&A where every message matters → leave it on the default. The react pattern's "auto" already keeps this at zero overhead.
• One long tool chain over a dataset (gather → aggregate) → managed / context_scope="ledger". This is the mode that directly attacks context bloat in LLM agents by deduplicating the transcript.
• A multi-stage custom graph where stages pile up tokens → context_scope="scoped" on each stage so one stage's verbose output never leaks into the next.
• You're not sure and want a safe default → "auto". It's zero-regression: identical to "full" until the loop is deep enough to need bounding.

An honest note on what this does and doesn't buy you: context_scope="ledger" is an efficiency primitive. On long chains it cuts redundant tool calls and bounds token growth at equal accuracy — a real cost and latency win. It does not, by itself, make a weak model aggregate facts more correctly. If the model can gather the facts but still mis-reasons over them, that's a capability limit, not a context one. Don't expect a ledger to fix a reasoning problem.

Where context_scope fits — and when to reach for something else

context_scope manages the transcript. It composes with the two other big token levers in Promptise:

• Tool optimization trims the token cost of the tool definitions sent on every call — the framework reports 40–70% fewer tokens with semantic tool selection, which is the single largest cost after the conversation itself. Bloated tool schemas and a bloated transcript are separate problems; you usually want both levers on.
• The Context Engine governs the whole assembly — memory, prompt blocks, and history — counting tokens exactly and trimming by priority when the window is tight, so nothing silently truncates the user's message off the end.

There's also a case where a different approach fits better than any transcript trick. If your agent keeps looking up the same long-lived facts across many separate runs — user preferences, prior decisions, domain knowledge — the right tool is durable memory, not a per-run ledger. A ledger resets when the task ends; a memory provider persists. That distinction, and how to combine both, is the subject of AI Agent Memory: The Complete Guide for Python Devs. And if your chain is so deep that even a ledger is churning, consider the code-action pattern, which collapses a long tool loop into a single sandboxed program — a more radical context move than any scope setting.

Frequently asked questions

How do I reduce agent context tokens without losing accuracy?

Set context_scope="ledger" (or use the managed pattern) on the node that runs your deep tool loop. It replaces the growing transcript with a deduplicated ledger of the facts already gathered, so the model stops re-sending and re-fetching. Token growth is bounded at equal accuracy. Pair it with tool optimization to cut the tool-definition tokens too.

Does context_scope change behavior for my existing simple agents?

No. A raw PromptNode defaults to "full", and the built-in react pattern uses "auto", which stays identical to "full" while the tool loop is short. Bounding only kicks in once a chain grows past the threshold, so short tasks are untouched and there's no regression to reason about.

Is a bigger context window a substitute for managing context?

Not really. A larger window delays the cost and latency wall but doesn't remove the "lost in the middle" accuracy problem — a model re-reading 40 redundant messages degrades regardless of window size. Managing the agent context window with context_scope fixes the cause; a bigger window only raises the ceiling you eventually hit.

Next steps

Set context_scope on your reasoning nodes — or just adopt the managed pattern — and keep deep, multi-tool tasks bounded and accurate as they run. Start with the Quick Start to stand up an agent, then read the context lifecycle guide for the full decision table and the ledger internals.