multilingual-nlp 开源项目推荐（290 个）

embeddings-benchmark /mteb

MTEB：跨语言与跨模态嵌入的最先进评估

bigscience-workshop /xmtf

Crosslingual Generalization through Multitask Finetuning

salikhussain71-code /Urdu-English-Word-Frequency-Analyzer

Bilingual Python text analyzer for English and Urdu Roman text, with word-frequency analysis, UTF-8 support, and CSV results.

DmitryRyumin /EMNLP-2023-Papers

EMNLP 2023 论文：探索 EMNLP 2023 的前沿研究，这是推动自然语言处理实证方法发展的顶级会议。随时了解机器学习、深度学习和自然语言处理的最新动态，并包含代码。：star：支持NLP！

cisnlp /Glot500

[ACL 2023] Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages

FSoft-AI4Code /TheVault

[EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

shijie-wu /crosslingual-nlp

This repo supports various cross-lingual transfer learning & multilingual NLP models.

epfl-dlab /llm-latent-language

Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".

csebuetnlp /CrossSum

This repository contains the code, data, and models of the paper titled "CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs" published in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23), July 9-14, 2023.

techwolf-ai /workrb

WorkRB: Work Research Benchmark

posi-olomo /padie-extended

An open-source Python package that uses AI to predict Nigerian languages, including English, Pidgin, Yoruba, Hausa, and Igbo.

ceferisbarov /TUMLU

TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languages

swaggy66 /M-ABSA

M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis

kidist-amde /amharic-ir-benchmarks

Official codebase for the ACL 2025 Findings paper: Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval.

BatsResearch /LexC-Gen

Generate synthetic labeled data for extremely low-resource languages using bilingual lexicons.

swiss-ai /parity-aware-bpe

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization [ACL 2026]

BatsResearch /cross-lingual-detox

Code for "Preference Tuning For Toxicity Mitigation Generalizes Across Languages." Paper accepted at Findings of EMNLP 2024

deokhk /CBP

Official Repository for Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing (EMNLP 2024)

neulab /CulturalGround

This repository provides the official resources for EMNLP 2025 Paper Grounding Multilingual Multimodal LLMs With Cultural Knowledge

praneethreddyganta /Multilingual_News_Classifier

Multilingual news classification system using transformer-based NLP to classify news articles across multiple categories without requiring translation.