NewLanguage token tax: what the same message costs in 23 languages on GPT, Claude and Gemini
Get alerts
AiGanak
Scope a project
Free index and checker · open dataNew

The language token tax: what the same message costs in 23 languages

LLM APIs bill by the token, and tokenizers favour English. The same customer message in Punjabi takes 6.7 times as many tokens as in English on Claude 5.x models, so it costs 6.7 times as much. We measured 23 languages on OpenAI, Claude and Gemini models. See the tax for your language and check your own text.

In short

AiGanak's language token tax index measures how many tokens the same 12 business messages take in English and 22 other languages, with each provider's official token counter (4 November 2026), as a multiple of English on the same model. OpenAI GPT-6 models: Odia 4.85, Punjabi 2.47, Hindi 1.43, German 1.25, Chinese 1.03. Claude 5.x models: Punjabi 6.67, Kannada 6.38, Odia 5.33, Hindi 2.28, German 1.99, Chinese 1.01. Gemini 3.x: Odia 3.37, Hindi 1.18, German 1.29, Chinese 0.96. Gemini 4 Argon: Odia 2.29, Hindi 1.28, Chinese 0.90. English itself takes 276 tokens on GPT-6, 292 on Gemini and 405 on Claude, so the same Kannada messages take 522 tokens on GPT-6, 532 on Gemini 3.x and 2,582 on Claude. Free checker estimates tokens and monthly cost for your own text; corpus, counts and script are open on GitHub under CC BY 4.0.

Check your text →India: 12 Indian languages
Token tax index · measured 4 November 2026

How many more tokens the same message costs, by language

1.00× = English. Tokens are what you are billed for, so 2.00× means the same conversation costs twice as much. Same 12 business messages in every language.

LanguageOpenAI GPT-6 familyClaude 5.x familyGemini 3.x familyGemini 4 ArgonIf English costs $1,000 (highest model)
Odia ଓଡ଼ିଆ 4.85×5.33×3.37×2.29× $5,331
Punjabi ਪੰਜਾਬੀ 2.47×6.67×2.46×2.23× $6,672
Telugu తెలుగు 1.99×3.05×1.76×1.90× $3,049
Tamil தமிழ் 1.91×3.21×1.36×1.65× $3,215
Kannada ಕನ್ನಡ 1.89×6.38×1.82×2.05× $6,375
Malayalam മലയാളം 1.85×4.50×1.73×2.00× $4,499
Marathi मराठी 1.78×2.60×1.26×1.47× $2,598
Gujarati ગુજરાતી 1.67×2.97×1.73×2.00× $2,968
Urdu اردو 1.63×2.63×1.39×1.59× $2,632
Bengali বাংলা 1.62×2.91×1.17×1.37× $2,914
Hindi हिन्दी 1.43×2.28×1.18×1.28× $2,281
Hinglish (Latin script) Hinglish 1.41×1.55×1.27×1.31× $1,553
Korean 한국어 1.37×1.64×1.27×1.15× $1,644
Japanese 日本語 1.37×1.26×1.02×0.96× $1,370
Russian Русский 1.30×1.50×1.24×1.19× $1,496
Indonesian Bahasa Indonesia 1.28×1.80×1.13×1.10× $1,802
Arabic العربية 1.25×1.89×1.31×1.27× $1,891
German Deutsch 1.25×1.99×1.29×1.27× $1,993
French Français 1.22×1.59×1.25×1.26× $1,590
Spanish Español 1.21×1.45×1.17×1.18× $1,454
Portuguese (Brazil) Português 1.15×1.40×1.19×1.16× $1,395
Chinese (Simplified) 中文 1.03×1.01×0.96×0.90× $1,029

Counted with each provider's official token counter. Models with identical counts are grouped: OpenAI GPT-6 family: gpt-6-astra, gpt-6.1-sol, gpt-6.1-terra, gpt-6-luna; Claude 5.x family: claude-opus-5-5, claude-sonnet-5-5, claude-haiku-5-5, claude-fable-5-1; Gemini 3.x family: gemini-3.1-pro-preview, gemini-3.8-flash, gemini-3.5-flash-lite; Gemini 4 Argon: gemini-4-argon. Each column compares a language with English on the same model. The English baseline itself differs: OpenAI GPT-6 family 276, Claude 5.x family 405, Gemini 3.x family 292, Gemini 4 Argon 292 tokens for the same 12 messages.

Same 12 messages, total tokens by model

LanguageOpenAI GPT-6 familyClaude 5.x familyGemini 3.x familyGemini 4 Argon
English276405292292
Odia1,3392,159983669
Punjabi6812,702719652
Telugu5491,235513555
Tamil5281,302397482
Kannada5222,582532599
Malayalam5111,822504584
Marathi4921,052369428
Gujarati4621,202504585
Urdu4491,066407464
Bengali4461,180341400
Hindi394924345373
Hinglish (Latin script)389629370383
Korean379666370335
Japanese378510299279
Russian358606361347
Indonesian354730331322
Arabic346766382372
German344807376371
French338644365369
Spanish334589343346
Portuguese (Brazil)317565348340
Chinese (Simplified)284410279263

Fewest tokens in each row highlighted. Multiply by each model's price per token to compare bills; the checker below does this for you.

Tokenizer generations matter: on OpenAI's previous tokenizer (cl100k), Odia cost 12.25× and Kannada 9.22× English. If you still run an older model in an Indian language, moving to a current one can cut the token count several times over.

Free checker · runs in your browser

Check your own text

Paste a typical message, prompt or reply. We estimate its tokens on each model from the measured index and price it at current list prices. Nothing you paste leaves this page.

Estimate
0tokens per message
–vs the same message in English
Monthly cost as input–
Monthly cost as output (replies)–
Extra vs English (avg. in/out)–

Estimate from measured tokens per character for this language and model family. Exact counts vary with wording.

Coming next · API

Token tax API waitlist

Get token and cost estimates per language and model inside your own product: price a message before you send it, forecast a multilingual bill, or pick the cheapest model per language. Join the waitlist for early access and a free tier.

By joining you agree to our Privacy Policy. One email when access opens; no newsletter unless you ask.

What to do about it

Five ways to pay less tax

Use a current model, but check it

OpenAI's current tokenizer uses a fraction of the tokens its previous one did for Indian languages. Newer is not always better, though: Gemini 4 Argon uses more tokens than Gemini 3.x for Kannada, Telugu, Malayalam and Gujarati, and far fewer for Odia.

Compare providers for your language

The tax differs a lot between model families. Marathi costs 1.26× English on Gemini 3.x, 1.78× on GPT-6 and 2.60× on Claude. The same Kannada messages take 522 tokens on GPT-6, 532 on Gemini 3.x and 2,582 on Claude. Combine token counts with each model's price before you choose.

Keep instructions in English

System prompts, tool definitions and examples can stay in English while the model answers in the user's language. Only the conversation pays the tax.

Cache the fixed part

Prompt caching cuts the cost of repeated instructions by 90% or more on all three major providers, whatever the language. See our caching guide.

Train a tokenizer for your language

At high volume, a small model with a tokenizer built for your language removes most of the tax. That's how we build custom small language models.

Measure, don't guess

We assumed Hinglish would be cheaper than Hindi script. On OpenAI it costs almost the same, on Gemini it costs more, and on Claude it uses 32% fewer tokens. Test your own traffic with the checker above.

Method and data

How we measure

The same 12 everyday business messages (order updates, refunds, appointments, invoices, password resets, shipping, opening hours, product copy, loan applications, fraud warnings and returns) were written in English and 22 other languages. We count each language's tokens with each provider's own counter and divide by the English count on the same model.

The corpus, raw counts and counting script are open on GitHub under CC BY 4.0. Native-speaker corrections are welcome. We re-measure monthly and whenever a provider releases a new model family.

FAQ

Questions people ask

Can't find it? Press ⌘K and ask.

What is the language token tax?

LLM APIs bill per token, and tokenizers split some languages into many more tokens than English. The token tax is how many times more tokens the same text takes in a language than in English on the same model, which is also how many times more it costs.

Which languages cost the most on LLM APIs?

In AiGanak's index (4 November 2026, official token counters), Indian languages cost the most. Odia takes 4.85 times English on OpenAI's GPT-6 models, 3.37 on Gemini 3.x and 5.33 on Claude; Punjabi takes 6.67 and Kannada 6.38 times English on Claude. European languages cost 1.15 to 1.30 times English on OpenAI and Gemini, and 1.40 to 1.99 on Claude.

Which LLM is cheapest for non-English languages?

It depends on the language. In AiGanak's measurements, Gemini 3.x used the fewest tokens for Hindi, Marathi, Tamil and Bengali, Gemini 4 Argon for Odia, and Claude's tokenizer used the most for every Indian language. Combine the token count with each model's price per token, which the free checker on this page does for you.

How can I reduce LLM costs for non-English users?

Compare model families for your language, keep system prompts and tool definitions in English, cache the fixed part of the prompt, and at high volume consider a small model with a tokenizer trained for your language.

Building an AI product in more than one language?

We find where multilingual LLM spend goes and how to cut it, from model choice to a custom small model for your language.

Talk to us about an audit →Free cost check