large-multimodal-models 开源项目推荐（87 个）

VITA-MLLM /VITA

✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

NVlabs /describe-anything

[ICCV 2025] Implementation for Describe Anything: Detailed Localized Image and Video Captioning

gokayfem /awesome-vlm-architectures

Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

ShareGPT4Omni /ShareGPT4Video

[NeurIPS 2024] An official implementation of "ShareGPT4Video: Improving Video Understanding and Generation with Better Captions"

TinyLLaVA /TinyLLaVA_Factory

小尺度大型多模态模型框架

richard-peng-xia /awesome-multimodal-in-medical-imaging

A collection of resources on applications of multi-modal learning in medical imaging.

LLaVA-VL /LLaVA-Plus-Codebase

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

MMMU-Benchmark /MMMU

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

ictnlp /LLaVA-Mini

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

xiaoachen98 /Open-LLaVA-NeXT

An open-source implementation for training LLaVA-NeXT.

shikiw /OPERA

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

ictnlp /Stream-Omni

Stream-Omni 是一款类似 GPT-4o 的语言-视觉-语音聊天机器人，能够同时支持多种模态组合的交互。

zjysteven /lmms-finetune

一个用于微调大型多模态模型的最小代码库，支持 llava-1.5/1.6、llava-interleave、llava-next-video、llava-onevision、llama-3.2-vision、qwen-vl、qwen2-vl、phi3-v 等。

friedrichor /Awesome-Multimodal-Papers

精选的多模态研究列表。

thunlp /LEGENT

Open Platform for Embodied Agents

EvolvingLMMs-Lab /LongVT

[CVPR 2026] LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

ShareGPT4Omni /ShareGPT4V

[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions

ritzz-ai /GUI-R1

GUI-R1的官方实现：一种通用的R1风格视觉语言动作模型，适用于图形界面代理

JinjieNi /MixEval

The official evaluation suite and dynamic data release for MixEval.

taco-group /GenAI4AD

a comprehensive and critical synthesis of the emerging role of GenAI across the full autonomous driving stack