← All Terms

Prompt Caching

A provider feature that stores a repeated prompt prefix, such as instructions or policy text, so later calls that reuse it are billed at a fraction of the normal input price.

Implementation

Prompt caching lets a model provider keep the processed form of the start of a prompt for a short time. When the next call begins with exactly the same text, the provider reads it from the cache instead of processing it again, which is faster and cheaper. Anthropic, for example, bills a cache read at a tenth of the base input price on most current models, with a higher one-off price to write the cache.

It pays off when a long, stable prefix is reused many times: system instructions, a policy document, a product catalogue or a long conversation history. It does little when every prompt is different, and a change of a single character early in the prefix means the cache is missed.

It differs from semantic caching, which stores whole answers and returns them for questions with the same meaning. Prompt caching still runs the model on every call and only discounts the repeated input, so it carries no risk of serving a stale answer.

Read more on this
AI Strategy

Inference Cost Accounting for Finance Teams

The price per token falls every quarter while the AI bill rises, because volume, tokens per task and model choice grow faster than prices fall. How to tag every model call to the use case that ran it, build a cost-per-outcome line that sits beside the benefit, and run a monthly AI cost review on three numbers. Part 4 of the AI Benefit Realisation series.