Batch API guide: halve your LLM costs for work that can wait

Most LLM bills contain two kinds of work: requests where a person is waiting for the answer, and requests where nobody is. Product descriptions generated overnight, support tickets classified in bulk, documents extracted into fields, evaluation runs and backfills do not need an answer in two seconds. All three major providers sell that second kind of work at half price if you hand it over as a batch and collect the results later. This guide explains how each provider's batch service works, what it costs, how it combines with prompt caching, and how to decide what to move. All prices are in US dollars per million tokens, checked on each provider's own pages on 29 October 2026.

What a batch API is

You collect many requests into one job, submit it, and come back for the results. The provider runs the requests when it has spare capacity, within a stated window. Compared with calling the normal API one request at a time, three things change:

• Price: 50% off input and output tokens on all three providers.
• Timing: results arrive within hours, not seconds. Every provider states a 24-hour target or limit.
• Plumbing: you submit a file or a list, poll for completion and match results to requests with an ID you assign. There is no streaming.

The model, the prompts and the answers are the same as on the normal API. OpenAI also gives batch its own, separate rate-limit pool, so a large batch does not use up the capacity your live product depends on.

How each provider runs batches

Two practical points. First, because expiry is a possibility on every provider, split very large jobs into several batches and check for failed or expired requests when each finishes. Second, Gemini does not treat job creation as idempotent: sending the same creation request twice creates two jobs, and you pay for both.

OpenAI batches cover the Responses, Chat Completions, Embeddings, Completions, Moderation and image endpoints, and most models. Anthropic states that all active models support batches, and that batches can include vision and tool use, though streaming is not available. Gemini's Batch API works with the generateContent API and with the same input types as the interactive API.

Batch prices by model

The long-prompt surcharges carry over. OpenAI's batch price for GPT-6 Astra on prompts over 272K tokens is $10 input and $37.50 output, twice and one and a half times the short-prompt batch rates, and Claude Haiku 5.5 prompts over 100,000 tokens cost $0.25 input and $1.25 output in a batch. Google has announced that Gemini 4 Argon doubles on 1 December 2026, with batch prices doubling to $2 input and $10 output, and Gemini 3.8 Flash doubles on 1 January 2027; the Gemini price increase guide has the details. A batch price is always exactly half of that model's standard price, so the ranking of models does not change: what changes is how much each dollar buys. Every model's full price list is in the pricing calculator.

Worked example: what it saves

A company enriches product records overnight. Each month it sends 500,000 requests of 1,500 input tokens and gets 300 tokens back, 750 million input tokens and 150 million output tokens in all. The token counts are an illustration, not a measurement.

The percentage is the same everywhere, so the dollars follow the model. The same job costs 100 times more on a flagship than on a small model, and batch halves both. Choosing a cheaper model and using batch are separate savings that multiply: moving this job from GPT-6 Astra on the normal API ($15,000) to Claude Sonnet 5.5 in batch ($1,500) cuts the bill by 90%, if Sonnet 5.5 does the task well enough. That last condition decides it, and it is worth testing on your own examples before you move. Our small versus large model guide covers how.

Batch plus caching

Batch and prompt caching discounts combine. OpenAI's batch table lists cached input at half the standard cached rate, for example $0.05 per million tokens on GPT-6.1 Sol against $0.10 on the standard tier. Anthropic says the two discounts stack. Google says that a batch request which hits the cache is billed at the standard context caching rates, so check its pricing page for the cached rate that applies to your model. Our prompt caching guide has the full cache prices.

The catch is that cache hits in a batch are not guaranteed. Anthropic states that requests in a batch run concurrently and in any order, so cache hits are best-effort, and that customers typically see hit rates of 30% to 98%. It also recommends the 1-hour cache lifetime for batches with shared context, because a batch can take longer than the 5-minute default.

To see how much that range matters, take the support-assistant example from the caching guide on Claude Sonnet 5.5: 100,000 requests a month, each with a 20,000-token fixed prompt, a 500-token question and a 400-token answer. Cache write charges are left out of these figures.

Caching alone does most of the work on this job, so for a request that repeats a long prompt, batch adds a further halving only if the cache hit rate holds up. If the job can wait but your hit rate in a batch is low, compare the batch figure against the same job run on the standard API with a steady, cached stream of requests. Which is cheaper depends on your hit rate, and it is worth measuring on a small batch first.

Flex: batch prices without the file

OpenAI also offers Flex processing: you send ordinary Responses or Chat Completions requests with the service tier set to flex and get batch rates for them. OpenAI describes the trade as slower response times and occasional resource unavailability, and says the tier is in beta with limited model availability, so your code must handle a request being refused and retry later. It suits background jobs that fit a normal request-and-response loop, such as an agent running overnight, where building a batch file would be awkward. Google also lists a Flex inference option. Check each provider's pricing page for which models support it before relying on it.

What to move to batch

1. Move: bulk classification, tagging and routing of stored records; extracting fields from archives of documents; summaries and descriptions generated ahead of time; evaluation runs against a test set; embeddings for a content library; backfills after you change a prompt.
2. Keep live: chat, anything a user is watching, steps inside an agent where the next step depends on the answer, and anything with a deadline shorter than the batch window.
3. Redesign, then move: features that feel live but are not: a report generated when a user clicks, for example, can often be prepared overnight and shown when they arrive.

Before you switch, make sure you can handle three things: matching results to requests by your own ID, retrying the failed or expired ones, and a fallback to the normal API for the rare job that cannot wait. Stay under each provider's batch size limit, and keep one model per file on OpenAI.

How to find your batchable spend

Look at your usage export by feature or by job and ask of each line: would anyone notice if this answer arrived six hours later? The total of the lines where the answer is no is your batchable spend, and half of it is your saving. The free Bill Analyzer reads your usage export and lets you set the share of traffic that could move to batch, then shows the estimated saving alongside the effect of caching and model changes. To price your own volumes on every current model, standard and batch, use the API pricing calculator. If you want your real requests checked for what can be batched, cached or moved to a cheaper model, with a ranked plan, that is what the LLM Cost Report delivers.