Plain Prose

The cost of running large language models in production is dominated by input tokens. Most teams reach for a tokenizer published by their primary model vendor and use the count it returns as the answer to "how much will this prompt cost?" That answer is correct only when (1) the tokenizer is the one the vendor actually uses for billing, and (2) you intend to send the prompt verbatim. Both assumptions break in practice: tokenizers are sometimes proxies, prompts are usually templated and rendered at request time with dynamic data, and prompt caching can flip the per-request cost by an order of magnitude. The cheapest format is rarely the one you started with, and the cheapest model is rarely the one you reached for first. Treating cost as a property of the rendered prompt rather than of the source code is the first useful step toward keeping your inference bill predictable.