Skip to content

Cost & Efficiency

How to Cut Token Cost for a Multi-Tenant AI Agent

To cut token cost, multi-tenant AI agent deployments have to fight three inefficiencies at once — a leaky shared cache, unbounded tool-loop transcripts, and the full set of tool schemas resent on every call — because one agent serving many customers multiplies each of them per tenant. A waste that is annoying for a single-user demo becomes the dominant line on your bill when a thousand tenants hit the same code path. This post connects the three cost levers Promptise Foundry ships as first-class, composable primitives on one build_agent() call — a per-tenant semantic cache, a context_scope="ledger" transcript, and optimize_tools — and then shows you how to prove the savings with built-in observability rather than trust a marketing number.

Encrypt Your LLM Response Cache at Rest in Redis

Learning to encrypt LLM cache at rest starts with reckoning with what a semantic cache actually is on disk: a running log of every prompt a user typed and every answer your model gave back, sitting in Redis as plaintext JSON. The feature that makes the cache cheap — remembering answers so you can serve them again — is exactly what turns it into a liability. An RDB snapshot copied to a backup bucket, a replica tapped by a read-only credential, a KEYS * from a misconfigured requirepass, or an instance accidentally bound to a public interface: any one of those hands over conversations verbatim. Promptise Foundry ships a one-flag answer to this. Set encrypt_values=True on SemanticCache and the prompt-and-answer payload is encrypted application-side with AES (via Fernet) before it ever reaches Redis, keyed by PROMPTISE_CACHE_KEY so it stays readable across restarts. This post walks the threat, the exact toggle, and how to verify ciphertext is really what landed at rest.

Cut Tool-Schema Tokens Without Per-Query Selection

The reliable way to reduce MCP tool schema tokens — the JSON definitions your agent ships to the model on every single call — is usually not the one teams reach for first. The headline move is semantic top-K selection: embed the query, embed each tool description, and send only the handful of tools that look relevant. It is the biggest single saver, and Promptise Foundry ships it. But it has two properties that rule it out for a lot of real deployments: it needs a per-query embedding pass, and it changes which tools the model sees from one request to the next. If your deployment is air-gapped, or your tool set has to be deterministic — every call presents the same tools, auditable and reproducible — per-query selection is off the table. This post is about the other half of optimize_tools: the graded static levels that shrink each tool's JSON schema deterministically, cutting tokens without ever changing tool availability.

Can a Paraphrase Leak Another Tenant's Cached Answer?

A semantic cache cross-tenant leak is the failure mode that only similarity caching can produce: tenant B types a paraphrase of a question tenant A already asked, the two queries embed to nearly the same vector, and — without a caller-scoped partition — the lookup hands tenant B the completion Promptise stored for tenant A. An exact-match cache can never do this. It keys on the literal prompt string, so a reworded question misses and goes to the model. The moment you cache by meaning to save money, you also open a channel where meaning, not identity, decides who sees what. This post isolates that leak vector precisely, and shows how Promptise Foundry confines every similarity search to the caller's own tenant-qualified partition — by default, and fail-closed.

Why trim_messages Drops Facts Your Agent Still Needs

Put trim_messages vs a facts ledger side by side on a deep tool loop and the difference stops being academic: trimming evicts messages by token or message count, so it can throw away the exact record the model still needs three turns later — while a ledger keeps that fact and drops the duplicates instead. When a tool-calling agent balloons in the middle of a long task, the reflex is to reach for one of two stock helpers: trim the message history, or summarize it. Both bound the transcript. Both also have a failure mode that only shows up on the tasks that matter most — the deep ones. This post compares those helpers head-to-head with Promptise Foundry's context_scope="ledger", a last-value-wins facts ledger that deduplicates repeated tool calls and cache-serves a repeat instead of re-running it, with no extra model call.