visual-language-learning 开源项目推荐（14 个）

haotian-liu /LLaVA

[NeurIPS'23 口头报告] 视觉指令微调（LLaVA）面向达到并超越 GPT-4V 级别能力构建。

NExT-GPT /NExT-GPT

ICML 2024 论文 NExT-GPT：任意到任意多模态大语言模型的代码和模型

EvolvingLMMs-Lab /Otter

Otter，一个基于 OpenFlamingo（DeepMind Flamingo 的开源版本）的多模态模型，在 MIMIC-IT 上训练，并展现出改进的指令遵循和上下文学习能力。

InternLM /InternLM-XComposer

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

xiaoachen98 /Open-LLaVA-NeXT

An open-source implementation for training LLaVA-NeXT.

RLHF-V /RLHF-V

[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

mlpc-ucsd /BLIVA

(AAAI 2024) BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions

thomas-yanxin /KarmaVLM

🧘🏻‍♂️KarmaVLM (相生)：A family of high efficiency and powerful visual language model.

AdrianBZG /llama-multimodal-vqa

Multimodal Instruction Tuning for Llama 3

xinyanghuang7 /Basic-Visual-Language-Model

Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖

Skyline-9 /Shotluck-Holmes

[ACM MMGR '24] 🔍 Shotluck Holmes: A family of small-scale LLVMs for shot-level video understanding

ashleykleynhans /llava-docker

Docker image for LLaVA: Large Language and Vision Assistant

MuhammadAliS /CLIP

PyTorch implementation of OpenAI's CLIP model for image classification, visual search, and visual question answering (VQA).

ecoxial2007 /EffVideoQA

Efficient Video Question Answering