AI models do not charge by the word. They charge by the token, a piece of text that is often a whole word in English but only a fragment of a word in many other languages. The result is a hidden surcharge: ask the same thing in Hindi, Arabic, or Turkish and you pay for more tokens than in English, for exactly the same meaning.
How big is that surcharge, and can you avoid it? We measured it.
How we measured
We wrote the same four texts (a news item, a customer email, a technical explanation and travel tips) in 15 languages, translated to say exactly the same thing. Then we counted the tokens each version needs with the public tokenizers of 7 model families: GPT, Gemma, Llama, DeepSeek, Mistral, Qwen, and GLM. Anthropic (Claude) and xAI (Grok) do not publish their tokenizers, so they could not be measured. We repeat the measurement every month; the figures below date from September 25, 2026.
The results
Averaged over the 7 tokenizers, Hindi is the most expensive language we measured: the same text needs 177% more tokens than in English. Arabic needs 55% more, and the big European languages (French, Spanish, Portuguese, German and Italian) sit together at around 34% more.
Chinese is the exception: on average it needs about as many tokens as English, and fewer with DeepSeek (−16%). A single character can stand for a whole word, and tokenizers trained on a lot of Chinese keep the common ones whole.
| Language | Average | Most efficient family | Least efficient |
|---|---|---|---|
| Hindi | +177% | Gemma +30% | GLM +413% |
| Arabic | +55% | Mistral +26% | GLM +87% |
| Turkish | +54% | Gemma +30% | DeepSeek +103% |
| Vietnamese | +54% | Llama +34% | DeepSeek +124% |
| Korean | +48% | Llama +20% | GLM +89% |
| Italian | +48% | Llama +28% | Qwen +69% |
| Japanese | +47% | Gemma +17% | Mistral +70% |
| Indonesian | +42% | Gemma +18% | Qwen +66% |
| French | +35% | GPT +25% | DeepSeek +49% |
| Russian | +31% | Llama +6% | Qwen +60% |
| German | +30% | GPT +17% | DeepSeek +50% |
| Portuguese | +30% | GPT +15% | DeepSeek +47% |
| Spanish | +27% | GPT +17% | DeepSeek +45% |
| Chinese | +0% | DeepSeek −16% | Mistral +40% |
The model family matters as much as the language
The averages hide a bigger story. For Hindi, the most efficient tokenizer (Gemma) needs 30% more tokens than English, while the least efficient (GLM) needs 413% more: 3.9× as many for the same text. For Arabic, the surcharge goes from +26% with Mistral to +87% with GLM.
No family wins everywhere: GPT is the most efficient for 4 of the 14 languages, Gemma for 4, Llama for 4, DeepSeek for 1, and Mistral for 1. In practice, two models with the same price per token can send very different bills once your users write in their own language.
Why it happens
A tokenizer builds its vocabulary from a large sample of text. When most of that sample is English, common English words get a token of their own, while words in other languages are cut into several pieces. Scripts other than the Latin alphabet make it worse: a word written in Devanagari (Hindi) or in Arabic script can be split into many small fragments. Grammar plays a part too: Arabic and Turkish attach prefixes and suffixes to words, which creates many word forms that a tokenizer rarely sees whole. Chinese characters work differently: one character can stand for a whole word, and tokenizers trained on a lot of Chinese keep the common ones intact.
What it means for your bill
Take a service that would pay $1,000 a month in English. For the same volume in Hindi, it would pay about $2,774 with an average tokenizer, and anywhere from $1,303 to $5,128 depending on the model family. The surcharge applies both to what the model reads and to what it writes.
How to pay less
- Open the best AI for your language tool: it shows the measured cost of every family in your language, and the best models for it.
- Measure your own texts with the token counter: the gap depends on your vocabulary and subject.
- Consider writing the instructions (the system prompt) in English even when your users write in another language, since they are sent with every request. Check that the answers stay good in your language.
- Cache long instructions that repeat: cached input costs a fraction of the normal price.
- Compare total costs, not prices per token: the cost calculator prices your real volumes for every model.
Four texts are a small sample, and results vary with the kind of text. The full method is on our how we work page.
The figures in this article are recalculated from our data at every update (last: Sep 25, 2026). How we work