How to reduce LLM API costs: caching, routing, batching and more

The fastest ways to reduce LLM API costs are prompt caching for repeated context, routing easy requests to cheaper models, batch processing for anything that can wait, and sending fewer tokens in the first place. This guide shows each technique with current prices, code, worked math and the order to apply them in.

How do you reduce LLM API costs?

You reduce LLM API costs by paying less per token and sending fewer tokens. Paying less per token comes from prompt caching, batch discounts and cheaper models for easy work. Sending fewer tokens comes from trimming context, capping output and lowering reasoning effort where quality holds.

Apply the techniques in this order, because the first ones cost you nothing in quality:

Prices in this guide come from the Claude pricing page and OpenAI's API docs at the time of writing (October 2026). Check them before you plan a budget, because they change.

How do you measure what each LLM request costs?

Measure cost per completed task by logging the token counts every API response returns and multiplying by your model's prices. Per-request averages hide the expensive part: retries, long agent loops and the few prompts that carry huge context.

Every Claude response includes a usage object with input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens. Log these with a task ID and a route name, then compute:

cost = input_tokens        × input_price     + cache_write_tokens  × cache_write_price     + cache_read_tokens   × cache_read_price     + output_tokens       × output_price(prices per token = price per million / 1,000,000)

Group by route ("support answer", "invoice extraction", "code review") and by task outcome. A route where 30% of tasks need a second attempt costs 30% more than its per-request number suggests.

Before launch, estimate prompts with the token counting endpoint (POST /v1/messages/count_tokens), which accepts the same system prompt, tools, images and PDFs as a real request and returns the input token total. Different providers tokenize differently, so count with the tokenizer of the model you'll use.

For teams, connecting this to outcomes such as merged pull requests or resolved tickets is covered in how to measure AI ROI for engineering teams.

How does prompt caching cut LLM costs?

Prompt caching stores the processed form of a prompt prefix that repeats across requests, such as a long system prompt, tool definitions or a reference document, and bills later reads of that prefix at a fraction of the normal input price. It's the biggest free saving for most applications.

On Claude, according to the prompt caching docs:

A 5-minute cache pays for itself after a single read. Each read also refreshes the timer, so steady traffic keeps the cache warm. The minimum cacheable prefix is 512 tokens on the current Claude 5 models; shorter prefixes silently don't cache, with no error.

Mark the stable part of the prompt with cache_control:

import anthropicclient = anthropic.Anthropic()response = client.messages.create(    model="claude-sonnet-5-5",    max_tokens=2048,    system=[        {            "type": "text",            "text": SUPPORT_POLICY_AND_PRODUCT_DOCS,  # ~20,000 tokens, identical every call            "cache_control": {"type": "ephemeral"},        }    ],    messages=[{"role": "user", "content": customer_question}],)u = response.usageprint(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens)

Rules that decide whether caching works:

• Stable content first, variable content last. The cache matches an exact prefix in the order tools, system, messages. A timestamp or user name at the top of the system prompt breaks every read.
• Keep tool definitions and their order fixed. Changing tools or tool_choice invalidates the cache.
• Check the numbers. If cache_read_input_tokens stays at zero across repeated calls, something in your prefix changes between requests.

OpenAI applies prompt caching automatically for prefixes of at least 1,024 tokens on current models, with cached input discounted by up to 95% depending on the model. The same rule applies: put static content first.

Worked example: a support bot

Take a bot on Claude Sonnet 5.5 ($2 input, $10 output, $0.10 cache read per million tokens at the time of writing). Each request has a 20,000-token system prompt, a 500-token question and a 400-token answer, at 10,000 requests a day.

The cache write cost is a few cents a day when traffic is steady. This is illustrative arithmetic, not a benchmark, but the shape holds: when most of each request is repeated context, caching removes most of the bill.

When should you use batch processing?

Use a batch API for any work that doesn't need an answer within seconds: nightly classification, document extraction, evaluations, embeddings backfills, content generation queues. Anthropic and OpenAI both charge 50% less for batched requests.

Anthropic's Message Batches API accepts up to 100,000 requests or 256 MB per batch, whichever comes first. Most batches finish within an hour, and anything not finished within 24 hours expires. Results stay available for 29 days. Batch discounts stack with prompt caching.

batch = client.messages.batches.create(    requests=[        {            "custom_id": f"invoice-{invoice.id}",            "params": {                "model": "claude-haiku-5-5",                "max_tokens": 1024,                "messages": [{"role": "user", "content": build_prompt(invoice)}],            },        }        for invoice in invoices    ])print(batch.id)  # poll batches.retrieve(batch.id) until processing_status == "ended"

When processing_status is ended, read client.messages.batches.results(batch.id).

Results can come back in any order, so always key them by custom_id. OpenAI's Batch API works similarly with a JSONL file and a 24-hour completion window.

How do you route requests to cheaper models?

Send each request to the cheapest model that handles it well. Most applications have a mix of easy requests (classification, extraction, short rewrites) and hard ones (multi-step reasoning, code changes), and paying top-tier prices for the easy ones wastes most of the budget.

Current Claude prices per million tokens at the time of writing:

Haiku 5.5 is priced higher for prompts over 100,000 tokens ($0.50 input, $2.50 output), so long-document routes need their own math.

Three routing patterns, from simplest to most complex:

1. Static routing by task type. Your code already knows the task: invoice extraction always goes to Haiku, contract analysis to Opus. No extra calls.
2. Classifier routing. A cheap model labels each request as easy or hard, and your code picks the model.
3. Cascade. Try the cheap model, check the result with a validator or schema, and escalate on failure.

Before building a cascade, test the simpler option: the stronger model at a lower effort setting. Caches are per model, so routing one conversation across several models loses cache reuse. Compare cost per completed task, including escalations and retries, on a sample of real requests.

How do you send fewer tokens without losing quality?

Cut what the model doesn't need to see and limit what it writes back. Input bloat usually comes from stuffing whole documents, full chat histories and every possible tool into every request.

Trim the input

• Retrieve, don't paste. Send the three relevant sections of a manual, not the whole manual. Our guide to retrieval augmented generation covers how.
• Summarize old conversation turns instead of resending the full history every time.
• Load tools on demand. Large tool lists cost tokens on every call; only include what the route needs.
• Strip noise: HTML markup, repeated boilerplate, base64 blobs and log lines nobody asked about.
• Keep the cached part stable. Trimming the variable part is fine; editing the cached prefix costs a new cache write.

Context engineering goes deeper on choosing what belongs in the window.

Control the output

Output tokens cost five times as much as input on current Claude models, so long answers add up fast.

• Set max_tokens to a sensible ceiling for each route, but don't set it so low that answers get cut off and need a retry.
• Ask for the format you need: "Return JSON with fields x, y, z" or "Answer in at most three sentences."
• Use structured outputs for extraction, so the model doesn't wrap data in prose.

Lower the effort on routine routes

Claude models accept an effort setting that controls how many tokens the model spends thinking and responding. Levels run from low to max. Most models default to high; Claude Opus 5.5 and Haiku 5.5 default to medium. Anthropic recommends low for simple tasks such as subagents, where it gives significant token savings with some capability reduction.

response = client.messages.create(    model="claude-sonnet-5-5",    max_tokens=1024,    output_config={"effort": "low"},    messages=[{"role": "user", "content": "Classify this ticket: ..."}],)

Change effort per route, not globally, and re-run your quality checks after each change.

How do you keep coding agent costs under control?

Coding agents are the fastest-growing LLM cost for many teams, because a single task can run dozens of model calls while the agent reads files, runs tests and retries. The same levers apply, plus a few specific to agent loops.

• Write clear tasks. Vague tasks cause exploration and rework. A good spec with acceptance criteria is a cost control.
• Keep the memory file short. Project instructions such as CLAUDE.md are sent on every turn. Cut what doesn't change behavior.
• Use cheaper models for subagents that search or summarize, and keep the strong model for the main reasoning.
• Stop runaway loops. Set limits on turns or budget for unattended runs.
• Track cost per merged pull request, not per session, so a cheap agent that produces unmergeable work shows up as expensive.

If you use Claude Code on a subscription rather than the API, plan limits work differently; see Claude Code usage limits. For agents that run on Orkio, the Insights view shows real spend against API-equivalent usage per project, per model and per merged pull request, with project budgets and alerts, which makes these comparisons visible without building your own logging.

Your LLM cost reduction checklist

1. Log usage from every response with a route name and task ID.
2. Compute cost per completed task for your top three routes.
3. Move static content to the front of prompts and add cache_control; confirm cache reads are non-zero.
4. Send every non-urgent job through a batch API.
5. Trim context: retrieval instead of whole documents, summarized history, fewer tools.
6. Set max_tokens and output formats per route.
7. Test low or medium effort on routine routes against a fixed set of examples.
8. Route easy tasks to a smaller model, and measure cost per completed task, including escalations.
9. Review the numbers weekly, and set provider spending limits and alerts.

For a hands-on start with the API itself, our Claude API tutorial covers first calls, streaming, tools and structured output.

Frequently asked questions