cross-modal 开源项目推荐（77 个）

jina-ai /discoart

用一行代码创作 Disco Diffusion 艺术作品

docarray /docarray

表示、发送、存储和搜索多模式数据

shaoxiongji /knowledge-graphs

A collection of research on knowledge graphs

krantiparida /awesome-audio-visual

A curated list of different papers and datasets in various areas of audio-visual processing

kuanghuei /SCAN

PyTorch source code for "Stacked Cross Attention for Image-Text Matching" (ECCV 2018)

towhee-io /examples

Analyze the unstructured data with Towhee, such as reverse image search, reverse video search, audio classification, question and answer systems, molecular search, etc.

QiangChunyu /SecoustiCodec

Ultra-low bitrate speech codec (0.27-1 kbps) with cross-modal alignment and real-time capabilities

yisun98 /SOLC

遥感 Sar-Optical 土地利用分类 Pytorch 高分辨率遥感语义分割/地物分割/地物分类

JizhiziLi /RIM

[CVPR 2023] Referring Image Matting

QizhiPei /BioT5

BioT5 (EMNLP 2023) and BioT5+ (ACL 2024 Findings)

DRSY /MoTIS

[NAACL 2022]Mobile Text-to-Image search powered by multimodal semantic representation models(e.g., OpenAI's CLIP)

qcraftai /distill-bev

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation (ICCV 2023)

Zengyi-Qin /Weakly-Supervised-3D-Object-Detection

Weakly Supervised 3D Object Detection from Point Clouds (VS3D), ACM MM 2020

yangli18 /VLTVG

Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoning, CVPR 2022

marslanm /Multimodality-Representation-Learning

This repository provides a comprehensive collection of research papers focused on multimodal representation learning, all of which have been cited and discussed in the survey just accepted https://dl.acm.org/doi/abs/10.1145/3617833 .

rohitrango /objects-that-sound

Unofficial Implementation of Google Deepmind's paper `Objects that Sound`

Paranioar /UniPT

[CVPR2024] The code of "UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory"

kywen1119 /DSRAN

Code for journal paper "Learning Dual Semantic Relations with Graph Attention for Image-Text Matching", TCSVT, 2020.

sarahESL /AlignCLIP

AlignCLIP: Improving Cross-Modal Alignment in CLIP (ICLR 2025)

Eaphan /UPIDet

Unleash the Potential of Image Branch for Cross-modal 3D Object Detection [NeurIPS2023]