Prompt Caching Explained: The AI Cost Cut Most Agents Miss

Think of it like calling a restaurant and re-explaining your dietary restrictions before every order. "I'm allergic to shellfish, I don't eat pork, I prefer grilled over fried, and I like extra vegetables." Same speech. Every call. The restaurant charges you by the minute.

Prompt caching is saving your preferences on file. The restaurant already knows. You just order the dish.

In technical terms: when you use an AI API, you send a system prompt (your instructions and context) plus the user message (the specific task). Most of the cost comes from the system prompt because it's long and it's the same every time. Prompt caching stores that system prompt so you only pay for it once. Every subsequent call only charges for the new user message.

Anthropic's prompt caching documentation puts the numbers directly: up to 90% cost reduction and up to 85% latency reduction. The break-even point is just two cache hits — meaning if you send the same base prompt more than twice, caching saves money.

For real estate teams running dozens or hundreds of AI queries per day with consistent instructions, this is not a minor optimization. It's the difference between a manageable AI budget and one that grows faster than the value it creates.

The Technical Version

When you send an API request, the system prompt gets tokenized and processed before the model generates a response. Prompt caching stores the processed system prompt in memory so subsequent requests skip the tokenization and processing step. The cached prefix is reused, and you only pay the reduced cost for a cache read instead of a full processing pass.

The economics are straightforward. Anthropic charges approximately 90% less for cached prompt tokens compared to new tokens. OpenAI's Assistants API achieves similar savings through persistent threads. Google Gemini offers context caching that stores conversation context server-side.

Break-even: two cache hits. If your system prompt is 2,000 tokens and you send 3 requests, you've already saved money.

The Plain English Version

Think of it like a textbook on the teacher's desk versus printing a new copy for every student. Same information. Fraction of the cost.

Your system prompt is the textbook — the instructions that tell the AI who it is, what it knows, and how to respond. Without caching, every single API call reprints the textbook from scratch. With caching, the textbook stays on the desk. Each student (each new query) just walks up and asks their question.

Which platforms support it: Anthropic Claude (native prompt caching), OpenAI (Assistants API with persistent threads), and Google Gemini (context caching). Each implements it differently, but the principle is the same.

Not every agent needs prompt caching. If you're using ChatGPT's web interface and typing prompts manually, caching doesn't apply to you — OpenAI handles that on their end.

Prompt caching matters when you're:

Running 50+ AI queries per day with the same base instructions. Teams that built custom tools for listing descriptions, lead responses, or market reports are sending the same system prompt hundreds of times. Each one costs money without caching.

Using AI for consistent output across a team. If your brokerage built an AI tool that ensures every agent's listing descriptions match brand standards, the brand guidelines live in the system prompt. Cache it once, use it thousands of times.

Building custom AI tools via API. This is the key distinction. If you're a solo agent using ChatGPT's web app, you don't need to think about this. If your team or brokerage built tools that call the API directly — through custom scripts, Zapier integrations, or dedicated software — prompt caching is where the savings are.

Andrew Ng's DeepLearning.AI course on prompt engineering for developers covers this principle: systematic prompt engineering treats prompts as software components, not throwaway text. Caching is part of that engineering mindset.

Sarah manages technology for an 80-agent brokerage in Denver. They built a custom AI listing description tool that every agent uses. The system prompt includes brand voice guidelines, formatting rules, compliance language, and style examples — about 2,200 tokens of instructions that go with every single request.

Before caching: $720 per month in API costs. Growing every month as more agents adopted the tool.

After implementing prompt caching on the system instructions: $72 per month. Same tool. Same output quality. Same number of agents using it.

The math: $648 per month saved × 12 months = $7,776 per year. Implementation time: about 2 hours of developer work to add cache headers to the API calls.

The ROI calculation barely needs a calculator. $7,776 saved for 2 hours of work is $3,888 per hour of implementation time. That's the kind of return that makes IT budgets look reasonable to brokerage owners who are skeptical about AI spending.

Mistake 1: Thinking you need prompt caching for ChatGPT web. If you're using ChatGPT, Claude, or Gemini through their web interfaces, you don't need to implement caching. The platforms handle optimization on their end. Caching is specifically for API users — teams and developers building custom tools.

Mistake 2: Caching dynamic content. Only cache the parts of your prompt that stay the same. Your system instructions, brand guidelines, and formatting rules are cacheable. The property details, client names, and market data that change with every request are not. Mixing dynamic content into cached prompts defeats the purpose.

Mistake 3: Not measuring your baseline costs first. You can't prove savings without a starting number. Before implementing caching, document your current monthly API spend, average queries per day, and average prompt size. Then implement caching and measure the delta. This is especially important for presenting the ROI to brokerage leadership.