megatron 开源项目推荐（32 个）

modelscope /ms-swift

使用 PEFT 或全参数对 600+ LLMs（Qwen3.8、DeepSeek-V4、GLM-5.1、InternLM3、Llama4 等）和 300+ MLLMs（Qwen3-VL、Qwen3-Omni、InternVL3.5、Ovis2.5、GLM5.3、Gemma4、Llava、Phi4 等）进行 CPT/SFT/DPO/GRPO（AAAI 2025）。

Orchestra-Research /AI-Research-SKILLs

面向任何 AI 模型的综合性开源 AI 研究与工程技能库。将这些技能打包后，你的 Claude Code/Codex/Gemini 智能体将成为具备全速能力的 AI 研究智能体。由 Orchestra Research 维护。

baidu-baige /LoongForge

A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T-N1.7, FastWAM, Cosmos3, DreamZero, DeepSeek-V4, GLM-5.3, Kimi-K3 and more.

eyaadh /megadlbot_oss

Megatron was a telegram file management bot that helped a lot of users, specially movie channel managers to upload their files to telegram by just providing a link to it. The project initially started as roanuedhuru_bot which lately retired and came back as Megatron which was a side project of the famous Maldivian Telegram community - @baivaru u…

modelscope /mcore-bridge

MCore-Bridge：为前沿大模型提供 Megatron-Core 模型定义，让 Megatron 训练像 Transformers 一样简单——支持 300+ 大语言模型（Qwen3-Next、GLM-5.3、Deepseek-V4、MiniMax-2.7 等）和 200+ 多模态大模型（Qwen3.8、Qwen3-Omni、Gemma4 等）。

xrsrke /pipegoose

面向混合专家中 transformers 的大规模 4D 并行预训练（仍在进行中）

modal-projects /spindle

Own your Tinker-compatible training stack

plugyawn /Megaprop

Newton-Muon + Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.

liangyuwang /Tiny-Megatron

Tiny-Megatron, a minimalistic re-implementation of the Megatron library

chen-hao-chao /mdm-prime-v2

[EMNLP 2026] MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models

MoFHeka /LLaMA-Megatron

A LLaMA1/LLaMA12 Megatron implement.

zpqiu /rl-infra-notes

Source-code level analysis of LLM RL training infra: async RL, weight sync, FP8, MoE routing | LLM RL 训练基础设施源码级分析

janelu9 /EasyLLM

Running Large Language Model easily.

liangyuwang /Tinytron

A minimal, hackable pre-training stack for GPT-style language models

liangyuwang /train-large-model-from-scratch

A minimal, hackable pre-training stack for GPT-style language models

izyanrajwani /agent-skills-library

🛠️ Enhance programming workflows with a curated library of agent skills for tasks like code review, debugging, and planning for efficient project execution.

Sh-Jil /megadlbot

Megatron was a telegram file management bot that helped a lot of users, specially movie channel managers to upload their files to telegram by just providing a link to it. The project initially started as roanuedhuru_bot which lately retired and came back as Megatron which was a side project of the famous Maldivian Telegram community -

cfra5680 /MSRegFlow

Automate Microsoft Account OAuth registration and token imports with this Chrome extension for the Microsoft Account Manager API.

silenceminds /Frex

Frex: A high-performance, Unified Virtual Memory (UVM) Deep Learning framework. Engineered for efficient scaling with page-based memory spilling and fused-operator kernels.

ai-loren /vram-ledger

Estimate per-GPU memory for LLM training: parameters, gradients, optimizer states and activations under TP/PP/DP and ZeRO. 大模型训练显存账本