Calling a model through an API is the default for most organisations: no hardware to manage, pay per use, always on the provider’s latest version. Running a model locally flips that trade: you take on the infrastructure and operational burden, but you also gain data residency, no per-token billing to a third party, and independence from a provider’s pricing or policy changes.
The economics only make sense past a certain volume and a certain data-sensitivity threshold. At low usage, the API is almost always cheaper once you account for hardware, maintenance, and the engineering time to run inference reliably. The case for local flips once query volume is high and consistent, or once regulatory or contractual requirements mean the data genuinely cannot leave your environment.
It’s not an all-or-nothing decision either. Many production systems run a mix: sensitive or high-volume workloads on local or self-hosted models, general-purpose tasks through an API, chosen deliberately per use case rather than as a blanket policy.