pre-trained-model 开源项目推荐（219 个）

microsoft /unilm

跨任务、语言和模态的大规模自监督预训练

thunlp /OpenPrompt

An Open-Source Framework for Prompt-Learning.

brightmart /albert_zh

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

ChineseGLUE /ChineseGLUE

Language Understanding Evaluation benchmark for Chinese: datasets, baselines, pre-trained models,corpus and leaderboard

ymcui /Chinese-ELECTRA

Pre-trained Chinese ELECTRA（中文ELECTRA预训练模型）

zjunlp /KnowLM

An Open-sourced Knowledgable Large Language Model Framework.

keyu-tian /SparK

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

d-li14 /involution

[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator

zjunlp /KnowledgeEditingPapers

Must-read Papers on Knowledge Editing for Large Language Models.

deepmodeling /Uni-Mol

Official Repository for the Uni-Mol Series Methods

NVlabs /FasterViT

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

MrGiovanni /ModelsGenesis

[MICCAI 2019 Young Scientist Award] [MedIA Best Paper Award] Models Genesis: self-supervised pre-training for 3D medical images. Learns transferable representations from unlabeled CT and MRI volumes, then fine-tunes for downstream segmentation and classification. Keras and PyTorch weights included.

salesforce /PCL

PyTorch code for "Prototypical Contrastive Learning of Unsupervised Representations"

mims-harvard /TFC-pretraining

Self-supervised contrastive learning for time series via time-frequency consistency

HKUDS /UrbanGPT

[KDD'2024] "UrbanGPT: Spatio-Temporal Large Language Models"

NVlabs /GCVit

[ICML 2023] Official PyTorch implementation of Global Context Vision Transformers

MrGiovanni /SuPreM

[ICLR 2024 oral; top 1.2%] Supervised Pre-Trained 3D Models for Medical Image Analysis (9,262 CT volumes + 25 annotated classes)

nogibjj /rust-mlops-template

A work in progress to build out solutions in Rust for MLOPs

ymcui /PERT

PERT: Pre-training BERT with Permuted Language Model

iflytek /MiniRBT

MiniRBT (中文小型预训练模型系列)