audio-visual-learning 开源项目推荐（62 个）

ali-vilab /dreamtalk

Official implementations for paper: DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models

tanshuai0219 /EDTalk

[ECCV 2024 Oral/TPAMI] EDTalk/EDTalk++ - Official PyTorch Implementation

OpenNLPLab /AVSBench

[ECCV 2022] & [IJCV 2024] Official implementation of the paper: Audio-Visual Segmentation (with Semantics)

xid32 /NAACL_2025_TWM

我们介绍了时间工作记忆（TWM），旨在增强多模态基础模型（MFM）的时间建模能力。该即插即用模块可以轻松集成到现有MFM。通过我们的TWM，九个最先进的模型在质量保证、字幕和检索任务中展现出显著的性能提升。

YapengTian /AVE-ECCV18

Audio-Visual Event Localization in Unconstrained Videos, ECCV 2018

alvinliu0 /HA2G

[CVPR 2022] Code for "Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation"

rhgao /co-separation

Co-Separating Sounds of Visual Objects (ICCV 2019)

yanbeic /CCL

PyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning

ttgeng233 /UnAV

Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline (CVPR 2023)

roger-tseng /av-superb

A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models (ICASSP 2024)

praveena2j /JointCrossAttentional-AV-Fusion

ABAW3 (CVPRW): A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition

praveena2j /Joint-Cross-Attention-for-Audio-Visual-Fusion

IEEE T-BIOM : "Audio-Visual Fusion for Emotion Recognition in the Valence-Arousal Space Using Joint Cross-Attention"

stoneMo /DeepAVFusion

Official codebase for "Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling".

stoneMo /EZ-VSL

Official Codebase of "Localizing Visual Sounds the Easy Way" (ECCV 2022)

jasongief /PSP_CVPR_2021

[2021 CVPR] Positive Sample Propagation along the Audio-Visual Event Line

stoneMo /AVGN

Official implementation for AVGN

MengyuanChen21 /CVPR2023-CMPAE

[CVPR 2023] Collecting Cross-Modal Presence-Absence Evidence for Weakly-Supervised Audio-Visual Event Perception

DragonLiu1995 /xRIR_code

[CVPR 2025] Pytorch implementation of the paper "Hearing Anywhere in Any Environment"

praveena2j /Cross-Attentional-AV-Fusion

FG2021: Cross Attentional AV Fusion for Dimensional Emotion Recognition

ttgeng233 /UniAV

Unified Audio-Visual Perception for Multi-Task Video Localization