AiGanakOrvella can cut its LLM bill by 42%–53% without losing quality.
What we found
Orvella sent 1,426,128 requests and 6.7 billion tokens to LLM APIs in September, for a bill of $14,001. Most of that money pays for text Orvella sends, not text it gets back: input is 95.7% of all tokens.
We tested every change on Orvella's own approved answers. No recommendation lowers any feature's quality score by more than 1 point on a 0–100 scale.
- The largest single saving is routing simple work to smaller models, worth $2,636 a month. Ticket triage can move to the smallest tier, and most help-centre questions to the mid tier.
- Help-centre answers send too many documents. The assistant retrieves 8 articles per question, but 95.8% of answers use only the top 6.
- Agent reply drafts should stay on the largest model. Smaller models lost too much quality. Savings there come from a shorter, cache-friendly system prompt and fewer retrieved documents.
- Self-hosting a model is not worth it yet. It would need 20.8× today's small-model traffic to break even.
- 4 reliability issues need fixing, including prompt-injection gaps in the customer-facing help-centre assistant.
Where the money goes
Help-centre answers and agent reply drafts make up 83.8% of the bill. Both run every request on the largest model and send long prompts.
Daily LLM spend, September
Monthly spend by feature
What each request is made of
9 changes, applied in order
We apply each change on top of the previous ones, so no saving is counted twice. The conservative case assumes only part of each saving holds up in production. The appendix shows the assumed share for every lever.
From today's bill to the optimised bill, per month
Help-centre answers retrieve more than they use
For each answer we found the lowest-ranked article it actually used. In 95.8% of answers, everything used came from the top 6 results.
Dropping the last 2 articles from every request saves $1,156 a month on this feature alone. The few answers that need deeper results can trigger a second retrieval.
Lowest-ranked article each answer used
Which model each task actually needs
Model Lab replayed Orvella's golden examples on each model tier. We pick the cheapest tier within 1 point of the largest model's score, for simple and complex requests separately. Features are never moved to a larger model than they use today.
Issues to fix alongside the savings
Prompt Lab ran injection and format tests on the busiest prompts. These issues cost little today, but they put customers and data at risk.
Should Orvella run its own model?
Not yet. After routing, the traffic that suits a small open model costs $99 a month on the API. A private GPU server with operations costs about $2,050 a month. Self-hosting pays off from about 18.3 billion tokens a month, 20.8× today's volume. Revisit if traffic grows or data rules require it.
The 30 / 60 / 90 day plan
- Trim system prompts
- Switch triage and refund agent to structured outputs
- Ask for shorter answers
- Move nightly summaries to batch
- Retrieve fewer help-centre articles
- Make prompt prefixes stable for caching
- Cache answers to repeat questions
- Route simple requests to smaller models, with monitoring
- Add agent memory summaries and a step cap with human hand-off
Data and method
- Data received: the September usage export, 6,000 logged requests with personal data removed, 200 golden examples, and Prompt Lab test runs.
- Reconciliation: our token model matches the September bill to within 1% for every feature.
- Prompt caching: counted only for prefixes of at least 1,024 tokens, a 5-minute cache life, and the measured share of requests whose prompt prefix is identical.
- Batch: priced at a 50% discount for work that does not need an answer in real time.
- Prices: illustrative list prices for three model tiers. A real audit uses the client's contract rates.
See what your LLM bill is really paying for.
Send a usage export and a sample of logs. You get a report like this one, reviewed by an AiGanak architect, in five to seven working days.