ttengwang/Awesome_Prompting_Papers_in_Computer_Vision：计算机视觉和视觉语言学习领域基于预算的论文精选列表

技术解读

以下内容由 GitRead 中文社区编辑团队基于项目 README 与官方资料整理，结合中文开发者的常见使用场景补充技术选型建议。具体以项目官方文档为准。

精准定位与系统梳理：GitHub 项目 “ttengwang/Awesome_Prompting_Papers_in_Computer_Vision” 深度分析

项目背景与痛点

ttengwang/Awesome_Prompting_Papers_in_Computer_Vision 是一个精心整理的视觉和视觉语言学习中基于提示的论文列表。计算机视觉（CV）和视觉语言学习是机器学习领域中非常活跃的两个分支，近年来，许多使用基于提示的方法来改进模型性能的论文不断涌现。然而，由于论文数量庞大且分散，对于从业者而言，查找并理解这些论文的工作是一个挑战。ttengwang/Awesome_Prompting_Papers_in_Computer_Vision 的出现解决了这一问题，提供了一个一站式的资源库，帮助研究者和开发者快速定位到关注的研究领域。

核心技术揭秘

通过分析项目的结构与实现，我们可以窥探到其背后的设计思想和技术难点。尽管作为一个论文列表项目，其核心更在于内容的组织而非技术的创新，但其系统化的方法对我们理解技术栈的使用和数据管理的策略提供了有价值的信息。

技术架构与实现原理

此项目的核心是精心组织的 Markdown 文档，包含对所有相关论文的分类索引与摘要。每位贡献者都遵循一套明确的规则来添加和维护文献，采取分布式版本控制来保证信息的实时更新与一致性。

为了更好地展示技术架构，可以通过 Mermaid 图来表示数据流中的关键步骤：

graph TD A[用户访问仓库] --> B[查看目录索引] B --> C[选择分类] C --> D[访问某篇论文] D --> E{"阅读摘要"} E -->|"无兴趣"| F[跳转其他论文] E -->|"感兴趣"| G[跳转论文原址] G --> H["下载与阅读"

数据流

用户访问仓库：用户通过浏览器访问项目的 GitHub 仓库。
查看目录索引：首页展示目录索引，让用户能够快速了解项目的大纲。
选择分类：用户根据个人兴趣选择特定的分类。
访问某篇论文：在分类下，用户可通过链接进入某个具体论文的详细页。
阅读摘要：摘要页提供了论文的关键信息，有助于用户快速决定是否需要深入了解。
跳转其他论文：如果用户对论文不感兴趣，可以继续点击浏览其他论文。
跳转论文原址：如果用户对某个论文有兴趣，可以点击链接跳转至其原始链接进行下载或阅读。

功能亮点与差异

该项目与其他项目最大的不同在于其系统化的论文整理和持续更新机制。反观同类竞品，多数只停留在简单的资源聚合上，缺乏系统化的整理与筛选。本项目以其细致的分类与标签化管理，提供了一个研究计算机视觉和视觉语言学习领域的高效工具。

应用场景与落地建议

对于在计算机视觉领域工作的开发者或研究者来说，该仓库可以作为日常工作中的重要参考资料库。在生产环境中使用时，需要注意定期对内容进行更新与核实，确保信息的时效性与准确性。适合的业务场景包括但不限于图像识别系统、视觉检索引擎以及视觉语言交互系统的设计与开发等。

综合评价

ttengwang/Awesome_Prompting_Papers_in_Computer_Vision 不仅为研究者提供了调研资源，也为从业人员提供了高效的学习工具。在现阶段，依旧是作为一个数据整理项目，其进一步发展的空间包括拓展分类边界和添加更多实用功能。同时，在协同维护和信息更新速度方面还有改进空间。总体而言，这是一个值得推荐的项目，希望它能在未来带来更多信息价值和创新思路。

关于

GitHub 原始描述（英文）

A curated list of prompt-based paper in computer vision and vision-language learning.

adapter few-shot-learning parameter-efficient-tuning prompt-learning prompt-tuning visual-prompt zero-shot-learning

931 Stars
66 Forks
32 Watching

描述取自 GitHub 官方 API，保持仓库作者书写的原文。

简要分析

热度分

1095

价值分

581

活跃状态

活跃

GitHub

贡献者

Awesome Prompting Papers in Computer Vision

A curated list of prompt-based papers in computer vision and vision-language learning.

Awesome Prompting Papers in Computer Vision

Keywords

Task tag, e.g.,
Abbreviation tag, e.g.,
Characteristic tag: Some characteristic makes this paper unique, e.g.,
Bold font: We highlight some pilot work that may contribute to the prevalence of visual prompting.

Vision Prompt

This section collects papers prompting pretrained vision foundation models (e.g., ViT) for parameter-efficient adaptation.

Learning to Prompt for Continual Learning [paper] [code]

CVPR 2022
Visual Prompt Tuning [paper] [code]

ECCV 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning [paper] [code]

ECCV 2022
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition [paper] [code]

NeurIPS 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [paper] [code]

NeurIPS 2022
P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting [paper] [code]

NeurIPS 2022
Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models [paper] [code]

NeurIPS 2022
Visual Prompting via Image Inpainting [paper] [code]

NeurIPS 2022
Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation [paper]

AAAI 2023
LPT: Long-tailed Prompt Tuning for Image Classification [paper]

ICLR 2023
Diversity-Aware Meta Visual Prompting [paper] [code]

CVPR 2023
Semantic Prompt for Few-Shot Image Recognition [paper]

CVPR 2023
Visual Prompt Tuning for Generative Transfer Learning [paper] [code]

CVPR 2023
CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching [paper] [code]

CVPR 2023
Images Speak in Images: A Generalist Painter for In-Context Visual Learning [paper] [code]

CVPR 2023
PIVOT: Prompting for Video Continual Learning [paper]

CVPR 2023
Learning Expressive Prompting With Residuals for Vision Transformers [paper]

CVPR 2023
BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning [paper] [code]

CVPR 2023
Visual Prompt Multi-Modal Tracking [paper] [code]

CVPR 2023
A-La-Carte Prompt Tuning (APT): Combining Distinct Data Via Composable Prompting [paper]

CVPR 2023
Understanding and Improving Visual Prompting: A Label-Mapping Perspective [paper] [code]

CVPR 2023
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning [paper] [code]

CVPR 2023
Explicit Visual Prompting for Low-Level Structure Segmentations low-level segmentation [paper] [code]

CVPR 2023
Understanding and Improving Visual Prompting: A Label-Mapping Perspective [paper] [code]

CVPR 2023

ArXiv Papers

Exploring Visual Prompts for Adapting Large-Scale Models [paper] [code]

arXiv 2022/03
Vision Transformer Adapter for Dense Predictions [paper] [code]

arXiv 2022/05
Neural Prompt Search [paper] [code]

arXiv 2022/06
Convolutional Bypasses Are Better Vision Transformer Adapters [paper] [code]

arXiv 2022/07
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets [paper]

arXiv 2022/08
Prompt Vision Transformer for Domain Generalization [paper]

arXiv 2022/08
Prompt-Matched Semantic Segmentation [paper]

arXiv 2022/08
Visual Prompt Tuning for Test-time Domain Adaptation [paper]

arXiv 2022/10
Visual Prompting for Adversarial Robustness [paper]

arXiv 2022/10
Prompt Generation Networks for Efficient Adaptation of Frozen Vision Transformers [paper] [code]

arXiv 2022/10
Towards a Unified View on Visual Parameter-Efficient Transfer Learning [paper] [code]

arXiv 2022/10
Multitask Vision-Language Prompt Tuning [paper] [code]

arXiv 2022/11

Vision-Language Prompt

This section collects papers prompting pretrained vision-language foundation models (e.g., CLIP) for parameter-efficient adaptation.

Learning Transferable Visual Models From Natural Language Supervision [paper] [code]

ICML 2021
Learning to Prompt for Vision-Language Models [paper] [code]

IJCV 2022
Prompt Distribution Learning [paper]

CVPR 2022
Conditional Prompt Learning for Vision-Language Models [paper] [code]

CVPR 2022
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting [paper] [code]

CVPR 2022
Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos [paper] [code]

CVPR 2022
PointCLIP: Point Cloud Understanding by CLIP [paper] [code]

CVPR 2022
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks [paper] [code]

CVPR 2022
A Good Prompt Is Worth Millions of Parameters? Low-resource Prompt-based Learning for Vision-Language Models [paper]

ACL 2022
Can Language Understand Depth? [paper] [code]

ACM MM 2022
Expanding Language-Image Pretrained Models for General Video Recognition [paper] [code]

ECCV 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification [paper] [code]

ECCV 2022
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression [paper]

NeurIPS 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models [paper] [code]

NeurIPS 2022
Learning to Decompose Visual Features with Latent Textual Prompts [paper]

ICLR 2023
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models [paper] [code]

ICLR 2023
Visual-Language Prompt Tuning with Knowledge-guided Context Optimization [paper] [code]

CVPR 2023
Open-Set Fine-Grained Retrieval Via Prompting Vision-Language Evaluator [paper]

CVPR 2023
Multimodal Prompting With Missing Modalities for Visual Recognition [paper] [code]

CVPR 2023
Efficient Multimodal Fusion Via Interactive Prompting [paper]

CVPR 2023
Hierarchical Prompt Learning for Multi-Task Learning [paper] [code]

CVPR 2023
Text-Visual Prompting for Efficient 2D Temporal Video Grounding [paper]

CVPR 2023
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval [paper] [code]

CVPR 2023
MaPLe: Multi-modal Prompt Learning [paper] [code]

CVPR 2023
Texts as Images in Prompt Tuning for Multi-Label Image Recognition [paper] [code]

CVPR 2023
Vita-CLIP: Video and Text Adaptive CLIP Via Multimodal Prompting [paper] [code]

CVPR 2023
LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models [paper] [code]

CVPR 2023
$\pi$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation [paper] [code]

ICML 2023
POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models [paper] [code]

ICML 2023
Rethinking the Openness of CLIP [paper] [code]

ACL 2023
PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization [paper] [code]

ICCV 2023

ArXiv Papers

Colorful Prompt Tuning for Pre-trained Vision-Language Models [paper]

arXiv 2021/08
ActionCLIP: A New Paradigm for Video Action Recognition [paper] [code]

arXiv 2021/09
CLIP-Adapter: Better Vision-Language Models with Feature Adapters [paper] [code]

arXiv 2021/10
Amortized Prompt: Lightweight Fine-Tuning for CLIP in Domain Generalization [paper]

arXiv 2021/11
Prompting Visual-Language Models for Efficient Video Understanding [paper] [code]

arXiv 2021/12
Unsupervised Prompt Learning for Vision-Language Models [paper] [code]

arXiv 2022/04
Prompt-aligned Gradient for Prompt Tuning [paper] [code]

arXiv 2022/05
Parameter-Efficient Image-to-Video Transfer Learning [paper]

arXiv 2022/06
DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations [paper]

arXiv 2022/06
Prompt Tuning for Generative Multimodal Pretrained Models [paper] [code]

arXiv 2022/06
Prompt Tuning with Soft Context Sharing for Vision-Language Models [paper]

arXiv 2022/08
CPL: Counterfactual Prompt Learning for Vision and Language Models [paper] [code]

arXiv 2022/10
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models [paper] [code]

arXiv 2022/10
Unified Vision and Language Prompt Learning [paper]

arXiv 2022/10
Multi-Prompt Alignment for Multi-source Unsupervised Domain Adaptation [paper]

arXiv 2022/10
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition [paper] [code]

arXiv 2023/04

Language-Interactable Prompt

Language-interactable prompter develops zero/few-shot capabilities by prompting several independent foundational models (VLMs, LLMs, VMs, etc.) with the language interface. One of the most attractive applications is multimodal chatbot.

Multimodal Few-Shot Learning with Frozen Language Models [paper]

NeurIPS 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA [paper] [code]

AAAI 2022
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning [paper] [code]

CVPR 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language [paper] [code]

ICLR 2023

Arxiv Papers

Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models [paper] [code] [demo] arXiv 2023/03
Chameleon: Plug-and-play compositional reasoning with large language models [paper] [code] arXiv 2023/04
ClipCap: CLIP Prefix for Image Captioning [paper] [code]

arXiv 2021/11
Flamingo: a Visual Language Model for Few-Shot Learning [paper]

arXiv 2022/04
Language Models Can See: Plugging Visual Controls in Text Generation [paper] [code]

arXiv 2022/05
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models [paper]

arXiv 2022/06
Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning [paper]

arXiv 2022/06

Vision-Language Instruction Tuning

The goal of vision-language instruction tuning is to train a model that can effectively understand instructions for general-purpose multimodal tasks.

Visual Instruction Tuning [paper] [code] [demo]

arXiv 2023/04
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models [paper] [code] [demo]

arXiv 2023/04
Otter: A Multi-Modal Model with In-Context Instruction Tuning [paper] [code] [demo]

arXiv 2023/05
MultiModal-GPT: A Vision and Language Model for Dialogue with Humans [paper] [code]

arXiv 2023/05
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning [paper] [code]

arXiv 2023/05
InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists [paper] [code] [demo]

arXiv 2023/09

More Resources

PromptPapers: A comprehensive curated list for prompting papers (mainly in natural language processing)
Awesome Multimodal Assistant: a curated list for vision-language instruction tuning and LLM-based chatbot.

暂无评论

Awesome_Prompting_Papers_in_Computer_Vision

技术解读

精准定位与系统梳理：GitHub 项目 “ttengwang/Awesome_Prompting_Papers_in_Computer_Vision” 深度分析

项目背景与痛点

核心技术揭秘

技术架构与实现原理

数据流

功能亮点与差异

应用场景与落地建议

综合评价

相关仓库

关于

简要分析

贡献者

Awesome Prompting Papers in Computer Vision

Keywords

Vision Prompt

Vision-Language Prompt

Language-Interactable Prompt

Vision-Language Instruction Tuning

More Resources

评论