Models don’t read in whole words or sentences; they break text into smaller chunks called tokens, roughly four characters or three-quarters of a word on average in English. Every prompt you send, every document retrieved, and every word the model generates back gets counted in tokens, and most AI providers charge by the token, for both what goes in and what comes out.
This is why token counts, not word counts, are the real unit of cost and capacity in an AI system. A verbose prompt, an oversized retrieved document, or an unnecessarily long conversation history all burn tokens whether or not they add value to the answer.
Token optimisation, trimming unnecessary context, summarising long histories, retrieving only what’s relevant, is an actual engineering discipline in production systems, not a minor efficiency tweak. At scale, it’s frequently the difference between an AI feature that’s commercially viable and one that quietly loses money on every use.