Training is the (very expensive, one-time or occasional) process of teaching a model from data. Inference is what happens every single time afterward, when a user sends a prompt and the already-trained model produces a response. Almost every cost, latency, and reliability question a business actually faces day to day is an inference-time question, not a training-time one.
This distinction matters commercially because the economics are completely different: training cost is largely fixed and paid once (by you, or usually by the model provider), while inference cost scales directly with usage, every query, every token, every user. A feature that looked cheap in a demo with ten test queries can look very different at ten thousand queries a day.
It’s also where most production reliability problems actually surface: latency under real load, cost per interaction, and consistency of output quality across thousands of different inputs, none of which a small-scale pilot reliably reveals.