Solution · Inference cost

Self-hosted vs hosted LLM cost, compared on your workload

“Our inference bill is material and growing. Would self-hosting actually be cheaper?”

Assess this workload Buying questions
Short answer. Sometimes. Self-hosting is cheaper when volume is high and steady, a smaller open model passes your quality bar, and you have the people to operate it. At current hosted prices for small models, many moderate workloads never break even once hardware, power and admin time are counted. The only reliable answer compares both options on the same tasks, at the same quality, with operations costs included.
Illustrative cumulative cost: hosted versus self-hosted A schematic, not data. Hosted cost rises steadily from zero. Self-hosted cost starts with a hardware purchase and then rises more slowly, crossing the hosted line at a break-even point around month 17. If hosted prices halve, the hosted line rises more slowly and never crosses the self-hosted line within 36 months. Hosted API Self-hosted Hosted, prices halved Illustrative, not data 0122436 Months Cumulative cost hardware purchase Break-even Hosted Self-hosted Hosted, halved: no break-even Illustrative cumulative cost: hosted versus self-hosted A schematic, not data. Self-hosted starts with a hardware purchase and rises slowly, crossing the steadily rising hosted line at a break-even point. If hosted prices halve, there is no break-even within 36 months. Illustrative, not data Hosted Self-hosted Halved 0122436 Months Cumulative cost Break-even
Shape, not data: self-hosting front-loads cost and saves only if volume is high and steady. If hosted prices fall, the break-even can move past the hardware's life. Run the numbers for your workload.

When this works

  • You have at least a month of real usage data: requests, token counts, peaks
  • You can define what "good enough" means on a test set of your tasks
  • Someone can own the self-hosted service, or you are prepared to pay for that

The questions to settle before you buy

Why not just compare the price per token?

Because the models are not equivalent. A cheaper self-hosted model that fails 15% more of your tasks is not cheaper. We compare at a fixed quality bar on your own test set.

What costs do teams usually leave out?

Admin and on-call time, electricity at real duty cycles, redundancy, upgrades and regression testing, and the hosted calls you will still make for tasks the local model cannot handle.

Can we move only part of the workload?

Often that is the best answer: route high-volume, repetitive tasks to a self-hosted model and keep complex or rare tasks on a hosted model. The comparison models that split explicitly.

What if the answer is "stay hosted"?

Then that is the recommendation, with the evidence. You will also know the volume or price change that would flip it.

Where processing happens

On-device

Processing happens on the user's own computer or phone. Nothing is shared between users unless you build that in.

On-premises

Processing happens on infrastructure your organisation owns and operates, in your building or your colocation space.

Private hosted or hybrid

Explicitly approved external infrastructure, or a defined mix, with every external flow named in writing.

How we evaluate it

  1. Baseline current usage, cost and quality from real traffic
  2. Build a representative test set and agree a pass threshold
  3. Benchmark candidate self-hosted configurations on that set at realistic concurrency
  4. Model cash flow and annualised cost, including labour, power and residual hosted usage
  5. Report break-even, sensitivity to price changes, and a migration or stay recommendation

Every published result is labelled measured, reproduced, vendor-reported or estimated. Client data and results stay private unless the client approves publication.

When local is not the answer

Stay hosted, or go partly local, if:
  • Volume is low or spiky, so hardware sits idle
  • Only frontier-model quality passes your bar
  • Nobody can own operations and a managed option costs more than the saving

Ways to engage

Private AI Fit Assessment

Find out whether, and how, a specific AI workload can run within your constraints, before you buy hardware or sign an implementation contract.

From $1,500Typically 1–2 weeks

Performance Rescue

Diagnosis and agreed remediation of latency, memory, concurrency, reliability or cost problems in an existing deployment.

From $3,000Typically 1–3 weeks

Dipankar Sarkar · Engagement owner, Neul Labs Limited
One named person is accountable for scope, commitments and acceptance on every engagement, including work delivered with specialist partners.

Further reading

Sources

Frequently Asked Questions

Why not just compare the price per token?

Because the models are not equivalent. A cheaper self-hosted model that fails 15% more of your tasks is not cheaper. We compare at a fixed quality bar on your own test set.

What costs do teams usually leave out?

Admin and on-call time, electricity at real duty cycles, redundancy, upgrades and regression testing, and the hosted calls you will still make for tasks the local model cannot handle.

Can we move only part of the workload?

Often that is the best answer: route high-volume, repetitive tasks to a self-hosted model and keep complex or rare tasks on a hosted model. The comparison models that split explicitly.

What if the answer is "stay hosted"?

Then that is the recommendation, with the evidence. You will also know the volume or price change that would flip it.

How quickly does local AI pay for itself?

It varies from a few months to never. It depends on volume, the hosted price you would otherwise pay, hardware cost, power and admin time. Our calculator shows cash flow and annualised cost separately so you can see which applies.

Do you take a percentage of savings?

No. The comparison is a fixed fee, so we have no incentive to recommend migration.