pretrained-language-model 开源项目推荐（158 个）

microsoft /torchscale

面向 (M)LLMs 的基础架构

wenge-research /YAYI2

YAYI 2 是中科闻歌研发的新一代开源大语言模型，采用了超过 2 万亿 Tokens 的高质量、多语言语料进行预训练。(Repo for YaYi 2 Chinese LLMs)

Separius /awesome-sentence-embedding

预先训练的句子和单词嵌入模型的精心策划列表

THUDM /P-tuning-v2

优化的深度即时调优策略，与跨规模和任务的微调相当

thunlp /OpenDelta

A plug-and-play library for parameter-efficient-tuning (Delta Tuning)

xcfcode /Summarization-Papers

Summarization Papers

AndrewZhe /lawyer-llama

中文法律LLaMA（LLaMA，中文法律域名）

gaoisbest /NLP-Projects

word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding

allenai /dont-stop-pretraining

Code associated with the Don't Stop Pretraining ACL 2020 paper

RenzeLou /awesome-instruction-learning

Papers and Datasets on Instruction Tuning and Following. ✨✨✨

OpenBMB /CPM-Live

Live Training for Open-source Big Models

studio-dots-ai /dots.llm1

The official repository of the dots.llm1 base and instruct models proposed by rednote-hilab.

Hzfinfdu /Diffusion-BERT

ACL'2023: DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models

zzz47zzz /awesome-lifelong-learning-methods-for-llm

[ACM Computing Surveys 2025] This repository collects awesome survey, resource, and paper for Lifelong Learning with Large Language Models. (Updated Regularly)

LYH-YF /MWPToolkit

MWPToolkit is an open-source framework for math word problem(MWP) solvers.

yueyu1030 /AttrPrompt

[NeurIPS 2023] This is the code for the paper `Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias`.

DC-research /TEMPO

The official code for "TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024)". TEMPO is one of the very first open source Time Series Foundation Models for forecasting task v1.0 version.

hyintell /awesome-refreshing-llms

EMNLP'23 survey: a curation of awesome papers and resources on refreshing large language models (LLMs) without expensive retraining.

SuperBruceJia /Awesome-LLM-Self-Consistency

Awesome LLM Self-Consistency: a curated list of Self-consistency in Large Language Models

ZhengZixiang /ATPapers

Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合