Which Python language detection library gets the language right most often?
py3langid, Lingua, fast-langdetect, langdetect, langid: each one tested on the same 760 texts with a known language: 570 single sentences and 190 short paragraphs in 38 languages, from Tatoeba. New PyPI releases are tested again by themselves. Updated 06 November 2026.
760 files from Language test set (Tatoeba sentences, 38 languages);
760 of them give text to at least one library.
| Library | 🔴 Fails where another reads | 🟡 Empty where another reads |
⏱ Hangs or crashes | ✅ Correct | Median ms/file | Total seconds |
|---|---|---|---|---|---|---|
| 1 py3langid 0.4.0 | 0 | 0 | 0 | 97.0% | <1 | 0.5 |
| 2 Lingua 2.2.0 | 0 | 0 | 0 | 94.6% | <1 | 0.5 |
| 3 fast-langdetect 1.1.0 · model fastText lid.176 (lite) | 0 | 0 | 0 | 93.4% | <1 | 1.5 |
| 4 langdetect 1.0.9 | 0 | 0 | 0 | 93.0% | 1 | 1.3 |
| 5 langid 1.1.6 | 0 | 0 | 0 | 89.1% | 1 | 2.5 |
By kind of text: share correct
| Texts | py3langid | Lingua | fast-langdetect | langdetect | langid |
|---|---|---|---|---|---|
| paragraphs 190 texts | 99.0% | 97.4% | 95.8% | 99.5% | 99.5% |
| short-sentences 570 texts | 96.3% | 93.7% | 92.6% | 90.9% | 85.6% |
How this is measured
- Each library runs a short recipe from its own docs (board/targets.py):
open the file, get all text. Installed with the extras the docs recommend for reading
(
pypdf[full]: AES decryption, Brotli), nothing else. One worker process per CPU; per file 60 s / 120 s / 180 s / 300 s / 600 s at most (the any-document table allows longer: some tools load models), killed above the memory limit. - Fails where another reads: the library raised an error, hung or crashed on a file from which another library in the same table got text. Files that every library fails on (encrypted, truncated, fuzzed) are not counted against anyone.
- Empty where another reads: no error, but no text. This is often the recipe, not the library: for example
python-docx's paragraphs+tables recipe skips headers, text boxes and nested tables. Check the file list. - Any document → text: a file type the tool's docs don't list, or that the tool itself rejects as unsupported, counts as "not supported", not as a failure (lists in board/libs.json). Returning a file's raw RTF source counts as empty, for every tool. Each tool runs with default settings: no OCR switched on by hand.
- OCR: each image comes with the human-written ground-truth text published with its dataset. Words found is the share of its words that appear anywhere in the engine's output (reading order doesn't matter); an image that fails or comes back empty counts as 0. The ground truth of receipts and document pages covers only their annotated text, so a character error rate would punish engines for reading more than was annotated; it is in the raw results but not ranked. English models, CPU, default settings.
- Speech: each clip comes with its dataset's reference transcript. Both texts go through Whisper's English text normalizer (case, punctuation, "twenty" = "20"); word error rate is all substituted, missing and extra words over all reference words, the usual corpus-level WER. A clip that fails or comes back empty counts all its words as errors. The three Whisper implementations use the same base.en model; each other engine its documented default model. English, CPU, default decoding settings.
- Language detection: each text's language is known from Tatoeba. An answer counts as correct if the library's code matches after mapping variants to ISO 639-1 (zh-cn → zh, nb → no, iw → he). Close languages count as different: Croatian detected as Bosnian or Serbian, or Norwegian as Danish, is wrong. One call per text with default settings and all languages enabled.
- PDF → Markdown: pages from AI2's olmOCR-bench (ODC-BY), scored with its own checks (vendored, Apache-2.0): a sentence must be present, a page header or footer absent, two passages in the right order, a table cell next to the right neighbour. Checks passed is all passed checks over all checks; a page that fails or comes back empty passes none. Math pages are left out (their checks need a browser to render formulas). Default settings: whatever OCR a tool does on its own, it does.
- Text quality (order, tables, layout) is not ranked in the other tables; they are about not breaking. The raw results have every output's size and hash.
- Same files for every version: the test collections are pinned to one commit. GitHub-hosted runner, so times are comparable within one table, roughly.