What does it cost to run an LLM solution?
The operating cost of an LLM solution consists of model calls (price per token), hosting, and monitoring. A typical conversation costs from a fraction of a krona to several kronor, depending on the model tier. Total cost is driven by volume, model choice, and prompt length, and can often be halved with caching, batching, and the right model per task.
The development cost of an AI solution is a one-time item – the operating cost comes back every month, and it scales with usage. Yet operations is what’s most often missing from the math. This page is only about that: what it costs to run an LLM solution in production, and how you keep the cost down.
Price tiers by model level in 2026
Model vendors price per million tokens, with separate prices for input and output. Exact prices move fast, but the price tiers have been structurally stable – the difference between the cheapest and most expensive tier is roughly a factor of a hundred.
| Model tier | Order of magnitude | Good for |
|---|---|---|
| Small/fast model | ~SEK 1–5 per million tokens | Classification, sorting, simple rewrites, high volume |
| Mid-tier model | ~SEK 5–50 per million tokens | Customer service answers, summaries, RAG answers – the workhorse in most solutions |
| Top-tier model | ~SEK 50–500 per million tokens | Complex analysis, multi-step reasoning, agent tasks |
What does that mean per conversation? Assume a customer service conversation with five exchanges, where each call sends along history and knowledge material – say 20,000 tokens in and 2,000 out for the whole conversation. With a small model, the conversation costs a fraction of a krona; with a mid-tier model, on the order of SEK 0.10–1; and with a top-tier model, a few kronor. At 10,000 conversations a month, that’s the difference between a few hundred kronor and tens of thousands of kronor, for often equivalent perceived quality on simple cases.
Three optimizations that make the biggest difference
- Prompt caching. In most solutions, a large part of every call is identical: system instructions, examples, policy text. Vendors’ prompt caching gives a steep discount on reused input – cached input is often a fraction of the standard price. For solutions with long system prompts, this is the single biggest saving.
- The right model per step. Don’t let the top-tier model do everything. A common architecture has a cheap model classify and sort, while the expensive model only gets brought in for the cases that require it. Sending every call to the biggest model “just to be safe” is like driving a truck to the mailbox.
- Batching. Calls that don’t need a real-time answer – overnight summaries, document indexing, quality checks – can be run as a batch, which cuts the price in half with several vendors.
Together, these three can typically cut token costs by 50–80 percent compared with a naive implementation, with no noticeable loss of quality.
Beyond the three big ones, there’s a fourth habit that costs little and saves a lot: keep the context short. Many solutions routinely send the entire conversation history or an unnecessarily large number of retrieved documents with every call. Trimming away what doesn’t contribute to the answer – summarizing history, limiting the number of source passages – lowers both cost and response time, and often improves quality since the model doesn’t have to search through noise.
Build a forecast before you scale
An operating forecast doesn’t need to be advanced, but it needs to exist. Here’s how to build one:
- Measure cost per interaction in the pilot. Not in theory – log actual token usage per conversation or case over a few weeks.
- Multiply by volume scenarios. Calculate at three levels: expected volume, double volume, and ten times volume. Does the cost grow linearly, or are there items that explode, like context that grows with the history?
- Add the fixed items. Hosting, vector database, logging, monitoring, and evaluation runs.
- Set caps and alerts. Per-user cost caps and daily alerts on total cost should be in place from the first day of production. A misconfigured loop calling a top-tier model can otherwise turn into an expensive night.
Also ask every vendor to state their assumptions: which model, what prompt length, what volume. A quote without an operating budget is half a quote, and a forecast without stated assumptions can neither be reviewed nor tracked against when reality diverges.
Operations decides the architecture
The most important decision for operating cost isn’t made in operations but in the design: model choice per step, prompt length, caching strategy, and the ability to switch models when the pricing changes. At Weapp we build AI solutions with cost tracking as part of the delivery – get in touch if you’d like help forecasting or lowering the operating cost of your solution.
Frequently asked questions
What is a token?
The smallest unit of text a model works with, roughly a word fragment. A rule of thumb for Swedish is that one word becomes 1.5–2 tokens. Model vendors price per million tokens, with different prices for input (what you send in) and output (what the model writes). Long prompts and long answers therefore cost more.
Why did our LLM bill suddenly get much higher?
The most common causes are growing context (history sent along with every call), more calls per user interaction than planned, an upgrade to a more expensive model, or increased usage without a cost cap. Set alerts on daily costs and budget caps per user from day one, so anomalies are caught immediately.
Is it cheaper to self-host an open model?
At very high volumes or with strict data requirements, self-hosting can pay off, but GPU servers, operations, and expertise cost money too. For most Swedish companies, API models are cheaper overall until volumes are large and stable. Calculate the total cost including staff, not just the price per token.
Do token prices fall over time?
The trend has been clearly falling prices for equivalent capacity, while new top-tier models launch at higher price points. So build the solution so that switching models is easy, then you can take advantage of the price trend instead of being stuck with yesterday's price tag.
What else besides token cost is part of LLM operations?
Hosting for the application and any vector database, logging and monitoring, evaluation runs that safeguard quality over time, and maintenance when models are updated or replaced. For smaller solutions, the fixed operating costs often land at SEK 3,000–20,000 a month on top of the token cost.