Skip to content

Cost & Efficiency

Where Are Your Agent's Tokens Actually Going?

Agent token usage observability is the difference between guessing at your token bill and knowing, turn by turn, exactly where it goes. Most teams reach for a fix — a semantic cache, a facts ledger, tool selection — before they have a single number telling them which part of the agent is actually expensive. That's backwards. You can't cut what you can't see, and the tokens are almost never spread evenly: a couple of fat tool schemas and one or two long reasoning turns usually dominate, while your cache quietly misses more often than you think. This post shows how to attribute token usage across every LLM turn and every tool call with Promptise Foundry's built-in observability — no external tracing service, no code beyond one flag.

Cache MCP Tool Results to Cut Redundant API Calls

You can cache MCP tool results at the server so that the same expensive lookup — a database query, a paid pricing API, an Elasticsearch scan — runs once per TTL window instead of five times in one conversation. An agent-side semantic cache is a great lever, but it only saves LLM tokens: it stops the model re-reasoning over a paraphrased question. It does nothing for the other repeated cost, which is your tool handler re-hitting the backend every time the model decides it needs that fact again. Those are two different bills, and they need two different caches.

How to Cut Token Cost for a Multi-Tenant AI Agent

To cut token cost, multi-tenant AI agent deployments have to fight three inefficiencies at once — a leaky shared cache, unbounded tool-loop transcripts, and the full set of tool schemas resent on every call — because one agent serving many customers multiplies each of them per tenant. A waste that is annoying for a single-user demo becomes the dominant line on your bill when a thousand tenants hit the same code path. This post is the hub that connects the three cost levers Promptise Foundry ships as first-class, composable primitives on one build_agent() call — a per-tenant semantic cache, a context_scope="ledger" transcript, and optimize_tools — and then shows you how to prove the savings with built-in observability rather than trust a marketing number.

Encrypt Your LLM Response Cache at Rest in Redis

Learning to encrypt LLM cache at rest starts with reckoning with what a semantic cache actually is on disk: a running log of every prompt a user typed and every answer your model gave back, sitting in Redis as plaintext JSON. The feature that makes the cache cheap — remembering answers so you can serve them again — is exactly what turns it into a liability. An RDB snapshot copied to a backup bucket, a replica tapped by a read-only credential, a KEYS * from a misconfigured requirepass, or an instance accidentally bound to a public interface: any one of those hands over conversations verbatim. Promptise Foundry ships a one-flag answer to this. Set encrypt_values=True on SemanticCache and the prompt-and-answer payload is encrypted application-side with AES (via Fernet) before it ever reaches Redis, keyed by PROMPTISE_CACHE_KEY so it stays readable across restarts. This post walks the threat, the exact toggle, and how to verify ciphertext is really what landed at rest.

How to GDPR-Delete One User's Cached LLM Answers

To gdpr delete cached llm responses for a single data subject, you should be able to name the user, call one function, and be done — without touching any other customer's cached data. That is not how most LLM caches work. A semantic cache keys each entry by a prompt or content hash so it can serve a saved answer for a similar query, and that hash carries no notion of who asked. So when a user exercises their Article 17 right to erasure, you are left with two bad options: flush the entire cache and destroy every customer's hit rate to erase one person, or write and maintain a bespoke key-scanning script that walks the store looking for entries that "belong" to that subject — entries the cache never labelled as theirs in the first place. Promptise Foundry closes that gap structurally: each subject lives in its own cache partition, and SemanticCache.purge_user(user_id, tenant_id=...) drops exactly that partition and nothing else.

Cut Tool-Schema Tokens Without Per-Query Selection

The reliable way to reduce MCP tool schema tokens — the JSON definitions your agent ships to the model on every single call — is usually not the one teams reach for first. The headline move is semantic top-K selection: embed the query, embed each tool description, and send only the handful of tools that look relevant. It is the biggest single saver, and Promptise Foundry ships it. But it has two properties that rule it out for a lot of real deployments: it needs a per-query embedding pass, and it changes which tools the model sees from one request to the next. If your deployment is air-gapped, or your tool set has to be deterministic — every call presents the same tools, auditable and reproducible — per-query selection is off the table. This post is about the other half of optimize_tools: the graded static levels that shrink each tool's JSON schema deterministically, cutting tokens without ever changing tool availability.

Can a Paraphrase Leak Another Tenant's Cached Answer?

A semantic cache cross-tenant leak is the failure mode that only similarity caching can produce: tenant B types a paraphrase of a question tenant A already asked, the two queries embed to nearly the same vector, and — without a caller-scoped partition — the lookup hands tenant B the completion Promptise stored for tenant A. An exact-match cache can never do this. It keys on the literal prompt string, so a reworded question misses and goes to the model. The moment you cache by meaning to save money, you also open a channel where meaning, not identity, decides who sees what. This post isolates that leak vector precisely, and shows how Promptise Foundry confines every similarity search to the caller's own tenant-qualified partition — by default, and fail-closed.

Why trim_messages Drops Facts Your Agent Still Needs

Put trim_messages vs a facts ledger side by side on a deep tool loop and the difference stops being academic: trimming evicts messages by token or message count, so it can throw away the exact record the model still needs three turns later — while a ledger keeps that fact and drops the duplicates instead. When a tool-calling agent balloons in the middle of a long task, the reflex is to reach for one of two stock helpers: trim the message history, or summarize it. Both bound the transcript. Both also have a failure mode that only shows up on the tasks that matter most — the deep ones. This post compares those helpers head-to-head with Promptise Foundry's context_scope="ledger", a last-value-wins facts ledger that deduplicates repeated tool calls and cache-serves a repeat instead of re-running it, with no extra model call.