# Which Languages Do LLMs Support Well? > Frontier LLMs score 95 to 98% of their English level in about a dozen major languages. Below that, quality falls fast. A four-tier map with the evidence. Source: https://capitalandcompute.net/blog/which-languages-do-llms-support-well/ Published: 2026-10-06 Updated: 2026-10-13 Large language models handle about a dozen major languages at 95 to 98% of their English score: Spanish, Portuguese, French, German, Italian, Chinese, Japanese, Korean, Arabic, Hindi, Indonesian and Bengali. Swahili sits near 90%. Yoruba, Irish, Burmese, most Southern African languages and almost every indigenous, Creole and Pacific language fall well below that, and for many of them nobody has measured a current model at all. That is the short answer as of October 2026\. The longer one depends on three things a single leaderboard number hides: which model size you run, which dialect or script you need, and whether the text is legal or financial, where even English needs a human check. Key takeaways * Vendor tables put Claude Sonnet 4.5 at 98.2% of its English score in Spanish and 95.4% in Bengali, but 91.1% in Swahili and 79.7% in Yoruba. * Smaller models lose far more: Claude Haiku 4.5 keeps 96.4% in Spanish but only 52.7% in Yoruba. * On the September 2026 SEA-HELM leaderboard, the best Burmese scores are 71 to 76, while 46 of 91 tracked models score below 50. * The same sentence can cost 1.6x English tokens in Italian and 12 to 15x in Shan, Odia, Santali or Dzongkha on the GPT-4 tokenizer. * Per-language data for the newest flagships is scarce: most African, Indic and indigenous-language evidence still comes from 2024 and 2025 models. ## Which languages do LLMs handle at near-English quality? Two labs publish per-language numbers on the same test: MMLU, professionally translated into 14 languages. Anthropic’s [multilingual support documentation](https://platform.claude.com/docs/en/build-with-claude/multilingual-support) reports each language as a percentage of the model’s own English score. OpenAI’s [GPT-5 system card](https://cdn.openai.com/gpt-5-system-card.pdf) (August 2025) and its [GPT-5.2 update](https://cdn.openai.com/pdf/3a4153c8-c748-4b71-8e31-aecbde944f8d/oai%5F5%5F2%5Fsystem-card.pdf) (December 2025) report raw accuracy instead, so the two sets of numbers are on different scales and should not be compared directly. | Language | Claude Sonnet 4.5 (% of English) | Claude Haiku 4.5 (% of English) | GPT-5.2 Thinking (accuracy) | | ---------- | -------------------------------- | ------------------------------- | --------------------------- | | Spanish | 98.2 | 96.4 | 0.913 | | Italian | 97.9 | 96.0 | 0.910 | | Portuguese | 97.8 | 96.1 | 0.910 | | French | 97.5 | 95.7 | 0.899 | | Indonesian | 97.3 | 94.2 | 0.904 | | Arabic | 97.2 | 92.5 | 0.901 | | German | 97.0 | 94.3 | 0.903 | | Chinese | 96.9 | 94.2 | 0.901 | | Japanese | 96.8 | 93.5 | 0.897 | | Hindi | 96.7 | 92.4 | 0.900 | | Korean | 96.7 | 93.3 | 0.895 | | Bengali | 95.4 | 90.4 | 0.889 | | Swahili | 91.1 | 78.3 | 0.880 | | Yoruba | 79.7 | 52.7 | 0.808 | Every language in the top twelve rows clears 95% for the larger Claude model, and GPT-5.2 Thinking lands between 0.889 and 0.913 across the same twelve. For these languages, the honest summary is that the model is nearly as good as it is in English. Two caveats travel with the table. First, it is old: Anthropic’s page still shows only Sonnet 4.5 and Haiku 4.5, not the current Claude 5 family, and no newer OpenAI system card repeats the per-language table. Second, MMLU is a multiple-choice knowledge test. It measures comprehension and reasoning more than fluent writing, and the test itself is [saturated at the top](/blog/benchmark-saturation-mmlu/), which compresses differences between strong languages. Artificial Analysis runs the closest thing to a current cross-vendor comparison. Its [Global-MMLU-Lite leaderboard](https://artificialanalysis.ai/evaluations/global-mmlu-lite) covers 16 languages, including Burmese and Yoruba. When checked on 6 October 2026, Gemini 3.1 Pro Preview led at 93.2%. Claude Opus 4.6 (max), now deprecated, scored 92.2% overall: 95.2 in English, 94.0 in Spanish, 89.9 in Swahili, 88.8 in Burmese and 83.5 in Yoruba. ## Where does quality start to drop? The second tier is languages where the model is clearly competent but makes errors a native speaker will notice: Swahili, Afrikaans, Thai, Tamil, Filipino, and the smaller European languages. The best current evidence for Europe is a 2026 arXiv preprint, [_Language Equality has a Price_](https://arxiv.org/abs/2608.01395) (Hakimov et al., University of Potsdam). It ran two commercial models, GPT-5.4 and Claude Opus 4.8, and seven open-weight models through multi-turn dialogue games in all 24 official EU languages. In every language, including Irish, Maltese and Latvian, both commercial models beat every open model. The gap ran from 5.6 points in Greek to 35.8 points in Irish. Even in their weakest EU language, the commercial models kept 80.7% and 83.0% of their English score. The open models kept between 61.7% and 10.7%. For Southern Africa, a 2024 paper accepted at AAAI 2025, [_Bridging the Gap_](https://arxiv.org/abs/2412.12417) (Alhanai et al.), tested GPT-4o on the human-translated Belebele reading test. Afrikaans scored 94.4 against 95.9 in English, effectively top tier. isiXhosa scored 82.8, Sesotho 80.0, isiZulu 79.7, Sepedi 77.2, Setswana 77.0 and Xitsonga 76.8. For Southeast Asia, AI Singapore’s live [SEA-HELM leaderboard](https://leaderboard.sea-lion.ai) (dated 18 September 2026, 91 models) is one of the few sources that scores current flagships in a lower-resource language. On Thai, Gemini 3.8 Flash leads at 75.0, with Gemma 4 31B the best open model at 74.3\. Only 11 of the 91 models score below 50 on Thai. ## Which languages are LLMs still bad at? Below the second tier, the evidence turns from “noticeable errors” into “do not publish without a native speaker.” * **Yoruba:** 79.7% of English for Claude Sonnet 4.5, 52.7% for Haiku 4.5. * **Irish:** [IRLBench](https://arxiv.org/abs/2505.13498), a 2025 preprint built from Irish Leaving Certificate exams (Tran et al., University College Cork), found o4-mini, the top English scorer, fell from 76.2% in English to 55.8% in Irish. The best Irish score, GPT-4.1’s, was 58.4%. * **Burmese:** on SEA-HELM the leaders reach 71 to 76 (Gemini 3.8 Flash 75.7, GPT-6 Astra 73.5), but 46 of the 91 models score below 50\. Pick the model, not the language, carefully. * **isiZulu, isiXhosa, Sesotho, Luganda, Wolof:** [IrokoBench](https://arxiv.org/abs/2406.03368) (Adelani et al., NAACL 2025) scored GPT-4o on AfriMMLU in each language. English was 87.4; Swahili 77.4, isiXhosa 70.2, isiZulu 68.2, Sesotho 67.4, Luganda 52.8 and Wolof 37.8\. GPT-4o’s African-language average of 59.0 was the best of any model tested; the best open model, Gemma 2 27B, averaged 37.1. * **Dzongkha:** the 2025 DZEN preprint, [_Multilingual Question Answering in Low-Resource Settings_](https://arxiv.org/abs/2505.18638) (Hosain et al.), found GPT-4 averaged about 82.5 in English and 60.2 in Dzongkha across 15 subjects. The authors ran the open models in English only, because “the majority of open-source models don’t work well on Dzongkha.” * **Indigenous languages of the Americas:** a May 2025 report published by the Inter-American Development Bank, [_The Performance of Artificial Intelligence in the Use of Indigenous American Languages_](https://publications.iadb.org/en/performance-artificial-intelligence-use-indigenous-american-languages) (LLYC with IDB Lab and Microsoft), tested five models across seven languages. As reported in press coverage of the report, answers were apparently correct in 54% of cases and scored 2.4 out of 10 for correctness of expression. Quechua did best, at 3.72 out of 10. * **Maasai and other extremely low-resource targets:** the [WMT25 machine translation findings](https://aclanthology.org/2025.wmt-1.22.pdf) called translation into Maasai “largely unusable”, with systems producing “large portions of their output in Swahili.” Then there is the long tail with no data at all. No primary-source benchmark of a current frontier model could be found for Marshallese, Nauruan, Fijian, Bislama, Solomon Islands Pijin, Manx, Romansh, Garifuna, Inuktitut, most Mayan languages, or the regional languages of Namibia, Mozambique and Zambia. [CreoleVal](https://arxiv.org/abs/2310.19567) (Lent et al., TACL 2024) covers up to 28 Creoles, including Mauritian, Seychellois, Bislama and Tok Pisin, but its baselines are fine-tuned encoder and mBART models, not chat LLMs. An unmeasured language should be treated as unsupported until you test it. How the tiers were built Each language is placed by the published score for the strongest model that has one, read from the primary paper, vendor table or leaderboard and checked on 6 October 2026\. A language with no score for any model released in 2025 or 2026 is listed as unmeasured, not guessed into a tier. Benchmarks differ in scale, so compare languages within one source, not across sources. | Tier | Languages (examples) | What the evidence says | | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | | 1: near-English | Spanish, Portuguese, French, German, Italian, Chinese, Japanese, Korean, Arabic (MSA), Hindi, Indonesian, Bengali, Afrikaans | 95 to 98% of English on vendor tables; 94.4 vs 95.9 for Afrikaans | | 2: good, check the output | Swahili, Thai, smaller EU languages, isiXhosa, Sesotho | About 80 to 91% of English; commercial models above 80% in all EU-24 | | 3: unreliable without review | Yoruba, Irish, Burmese, isiZulu, Setswana, Luganda, Dzongkha | About 52 to 80% of English, or a wide spread between models | | 4: unusable or unmeasured | Wolof, Maasai, indigenous Americas, most Creoles, Pacific and Crown Dependency languages | Below 40 on knowledge tests, “largely unusable”, or no data | ## Does a smaller model lose more in other languages? Yes, and this is the most expensive mistake in the data. The gap between a lab’s large and small model is a point or two in Spanish and 27 points in Yoruba. __Large vs small model, % of English score by language__ | Language | Claude Sonnet 4.5 | Claude Haiku 4.5 | | ---------- | ----------------- | ---------------- | | Spanish | 98.2% | 96.4% | | Portuguese | 97.8% | 96.1% | | French | 97.5% | 95.7% | | Arabic | 97.2% | 92.5% | | German | 97.0% | 94.3% | | Chinese | 96.9% | 94.2% | | Hindi | 96.7% | 92.4% | | Bengali | 95.4% | 90.4% | | Swahili | 91.1% | 78.3% | | Yoruba | 79.7% | 52.7% | Claude Sonnet 4.5 and Claude Haiku 4.5 as a share of their own English score. The shaded gap is small in Spanish and French, then widens sharply from Bengali to Yoruba.Source: [Anthropic, Multilingual support documentation](https://platform.claude.com/docs/en/build-with-claude/multilingual-support) OpenAI’s numbers show the same pattern. In the GPT-5 system card, gpt-5-thinking scored 0.806 in Yoruba while the faster gpt-5-main scored 0.664\. If you route non-English traffic to a “mini”, “flash-lite” or “haiku” tier to save money, test each language on that tier first. The [AI model tracker](/ai-models/) lists the current price of each size. ## Which LLM is best for non-English languages? No single model wins every language, but the evidence points one way for each group: * **Major world languages:** the frontier models from Google, Anthropic and OpenAI sit within about a point of each other. Gemini leads the aggregate Global-MMLU-Lite board by a small margin. Choose on price, speed and instruction-following. * **Smaller European languages (Irish, Maltese, Baltic):** use a commercial frontier model. In the EU-24 study, open models trailed by up to 35.8 points in Irish, and every open model’s weakest language was Irish or Maltese. * **Southeast Asia:** Gemini 3.8 Flash and Gemini 3.1 Pro Preview top SEA-HELM in Burmese and Thai. For self-hosting, Gemma 4 31B and the Qwen 3.5 and 3.6 models lead the open tier. * **African languages:** closed models from OpenAI and Google led both IrokoBench and [AfroBench](https://arxiv.org/abs/2311.07978) (Findings of ACL 2025), where GPT-4o averaged 59.6 and Gemini 1.5 Pro 58.5\. Those are 2024-era models; newer per-language African scores for current flagships were not found. * **Arabic dialects:** see the next section; Arabic-specific models beat general ones on dialect work. * **Indigenous American, Pacific and Creole languages:** no model is adequate. Use human translators. For a wider view of how these benchmarks are built and gamed, the [AI benchmarks directory](/ai-benchmarks/) explains each test, and [Are AI benchmarks reliable?](/blog/are-ai-benchmarks-reliable/) covers the failure modes. ## Do LLMs cost more in other languages? Yes. APIs bill per token, and tokenizers are trained mostly on English, so the same sentence splits into more tokens in most other languages. You pay more, wait longer, and fit less into the context window. __Token premium vs English on the GPT-4 tokenizer__ | Item | Value | | --------- | ------ | | Shan | 15.05x | | Santali | 12.80x | | Odia | 12.48x | | Dzongkha | 12.33x | | Burmese | 11.70x | | Arabic | 3.04x | | Bulgarian | 2.64x | | Italian | 1.64x | | English | 1x | Tokens needed for the same text, relative to English, on the cl100k\_base tokenizer used by ChatGPT and GPT-4\. Italian costs 1.6 times as many tokens; Shan costs 15 times as many.Source: [Petrov et al., Language Model Tokenizers Introduce Unfairness Between Languages (NeurIPS 2023)](https://arxiv.org/abs/2305.15425) The figures above come from [_Language Model Tokenizers Introduce Unfairness Between Languages_](https://arxiv.org/abs/2305.15425) (Petrov et al., University of Oxford, NeurIPS 2023). A companion EMNLP 2023 paper, [_Do All Languages Cost the Same?_](https://aclanthology.org/2023.emnlp-main.614/) (Ahia et al.), found speakers of many supported languages are “overcharged while obtaining poorer results”, with Telugu and Amharic costing up to 4x more for prompting plus generation. Newer tokenizers narrow the gap but do not close it. The EU-24 preprint measured about 1.4 tokens per word for English against 3.0 for Maltese. For Irish, the median model cost 2.2x the English price while returning 58% of its English score. It also found Claude Opus 4.8 used about 50% more tokens per word than the median model on EU languages. Separately, Anthropic’s [token counting documentation](https://platform.claude.com/docs/en/build-with-claude/token-counting) says Claude 4.7 and later models produce “approximately 30 percent more tokens” for the same text than earlier models. For the money side of that, see how [cost per token has moved over time](/blog/cost-per-token-over-time/). ## Which dialects and scripts break first? A language score usually measures the standard written form. The variety your readers actually use can be much weaker. * **Arabic:** vendor scores measure Modern Standard Arabic. [AraDiCE](https://arxiv.org/abs/2409.11404) (Mousi et al., COLING 2025) found Arabic-specific models such as Jais and AceGPT beat multilingual models on Levantine and Egyptian dialect tasks, but “significant challenges persist in dialect identification, generation, and translation.” Use MSA for anything formal. * **Cantonese:** [HKCanto-Eval](https://arxiv.org/abs/2503.12440) (CoNLL 2025) found an average 7.46-point drop on MMLU questions asked in Cantonese, and GPT-4o scored 65.4% on Cantonese characters. Formal written Chinese in Traditional script is safe; colloquial written Cantonese is not. * **Chinese scripts and Portuguese variants:** Simplified and Traditional Chinese use different vocabulary, not just different characters, so prompt for the locale (zh-TW or zh-HK) rather than converting characters. Benchmarks measure Brazilian Portuguese; ask explicitly for European Portuguese spelling and terms. * **Cultural knowledge:** [Global-MMLU](https://aclanthology.org/2025.acl-long.919) (Singh et al., led by Cohere Labs, ACL 2025) found 28% of MMLU questions need culturally sensitive knowledge, and 84.9% of geography questions focus on North America or Europe. Model rankings shifted when scored only on the culturally sensitive questions. ## Can you trust an LLM for legal or financial translation? Not without review, in any language. A 2025 study in the _International Journal of Language & Law_, [_A Comparative Study of Accuracy in Human vs. AI Translation of Legal Documents into Arabic_](https://www.languageandlaw.eu/jll/article/download/190/100) (Altakhaineh et al., University of Jordan), found human translations “significantly outperformed” ChatGPT on legal terminology, clarity and fit with the Arabic legal framework. Using a model as a checker helps but does not replace a reviewer. Minder, Wisniewski and Kübler (MT Summit 2025) found GPT-4o caught about 65% of terminology errors (185 of 289) in specialist student translations, a figure the same team repeats in its 2026 EAMT paper, [_On the Use of LLMs for Specialised Terminology_](https://arxiv.org/abs/2607.24784). And a correct translation can still state a wrong rule: language quality and factual accuracy are separate checks. ## How do you check a language before you ship it? 1. **State the output language in the system prompt.** Anthropic’s documentation recommends it: Claude “infers the response language from the conversation, but for production applications you should state the target language explicitly.” Name the locale too (es-419, pt-PT, zh-Hant-TW, sr-Latn). 2. **Build a small eval from your own content.** Fifty to a hundred real questions per language, scored by a native speaker, tell you more than any public leaderboard. 3. **Test the tier you will actually run.** A large model’s Yoruba score says nothing about its small sibling’s. 4. **Measure tokens per language** on your provider before you budget, and keep the English fallback one click away for tier 3 and 4 languages. 5. **Give right-to-left and complex scripts their own QA.** Arabic and Hebrew model quality is strong; the bugs are in rendering mixed-direction text, numbers and codes. ## Frequently asked questions Which languages do LLMs support best? English first, then about a dozen major languages: Spanish, Portuguese, French, German, Italian, Chinese, Japanese, Korean, Arabic, Hindi, Indonesian and Bengali, where Claude Sonnet 4.5 scores 95 to 98% of its English result on Anthropic's published table. Which LLM is best for low-resource languages? Closed frontier models lead every benchmark that covers low-resource languages. Gemini models top the September 2026 SEA-HELM leaderboard in Burmese, and commercial models beat every open model in all 24 EU languages in a 2026 preprint. For indigenous, Pacific and Creole languages no model is adequate. Is ChatGPT good at translating African languages? It depends on the language. GPT-4o scored 94.4 in Afrikaans against 95.9 in English on Belebele, but 37.8 in Wolof against 87.4 in English on AfriMMLU in IrokoBench. Anything below Swahili and Afrikaans needs a native reviewer. Why do LLMs cost more in non-English languages? Tokenizers are trained mostly on English, so the same text splits into more tokens elsewhere. On GPT-4's tokenizer, Italian takes 1.6 times as many tokens as English, Arabic 3 times, and Shan 15 times, according to Petrov et al. (NeurIPS 2023). ## Sources * Anthropic (2026). _Multilingual support_. Claude documentation. * Anthropic (2026). _Token counting_. Claude documentation. * OpenAI (2025). _GPT-5 System Card_. * OpenAI (2025). _Update to GPT-5 System Card: GPT-5.2_. * Artificial Analysis (2026). _Global-MMLU-Lite leaderboard_. * AI Singapore (2026). _SEA-HELM leaderboard_ (18 September 2026). * Hakimov et al. (2026). _Language Equality has a Price: A Systematic Investigation of Multi-turn LLM Performance for EU-24+_. arXiv preprint. * Tran, O’Sullivan and Nguyen (2025). _IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark_. arXiv preprint. * Adelani et al. (2025). _IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models_. NAACL 2025\. * Ojo et al. (2025). _AfroBench: How Good are Large Language Models on African Languages?_ Findings of ACL 2025\. * Alhanai et al. (2025). _Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments_. AAAI 2025\. * Hosain, Das Gupta and Morol (2025). _Multilingual Question Answering in Low-Resource Settings: A Dzongkha-English Benchmark for Foundation Models_. arXiv preprint. * LLYC, IDB Lab and Microsoft (2025). _The Performance of Artificial Intelligence in the Use of Indigenous American Languages_. Inter-American Development Bank. * Kocmi et al. (2025). _Findings of the WMT25 General Machine Translation Shared Task_. WMT 2025\. * Lent et al. (2024). _CreoleVal: Multilingual Multitask Benchmarks for Creoles_. TACL. * Petrov, La Malfa, Torr and Bibi (2023). _Language Model Tokenizers Introduce Unfairness Between Languages_. NeurIPS 2023\. * Ahia et al. (2023). _Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models_. EMNLP 2023\. * Mousi et al. (2025). _AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs_. COLING 2025\. * Cheng et al. (2025). _HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMs_. CoNLL 2025\. * Singh et al. (2025). _Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation_. ACL 2025\. * Altakhaineh, Alghathian and Jarrah (2025). _A Comparative Study of Accuracy in Human vs. AI Translation of Legal Documents into Arabic_. International Journal of Language & Law. * Minder, Wisniewski and Kübler (2026). _On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora?_ EAMT 2026\.