Awesome-Talking-Head-Synthesis

Awesome-Talking-Head-Synthesis

🌐 Website: https://kedreamix.github.io/Awesome-Talking-Head-Synthesis/

The README below starts with runnable open-source systems that do not have a paper, then the complete paper list. For search, filters, bilingual browsing, and category overview, please use the website.

• Website
• Open-source Projects
• Datasets
• Survey
• Funny Work
• Audio-driven
• Portrait Animation
• Text-driven
• NeRF & 3D Head Avatar
• 3D Gaussian Splatting
• Conversational & Dialogue
• Talking Body & Avatar
• Robot-Driven Face & Body
• Metrics
• Tools & Software
• Slides & Presentations
• References
• Star History

This repository organizes papers, codes, datasets and project pages for Talking Head Synthesis, covering audio-driven avatars, portrait animation, NeRF / 3D heads, 3D Gaussian Splatting, conversational agents, talking body, and more. 👤

Papers for Talking Head Synthesis, released codes collections. ✍️

Most papers are linked to PDFs on “arXiv” or journal/conference websites 📚. However, some papers require an academic license to view 🔐.

🔆 This project Awesome-Talking-Head-Synthesis is ongoing - pull requests are welcome! If you have any suggestions (missing papers, new papers, key researchers or typos), please feel free to edit and submit a PR. You can also open an issue or contact me directly via email. 📩

⭐ If you find this repo useful, please give it a star! 🤩

2026.09 Update 📆

The paper list has grown large enough that browsing only in the README is no longer convenient, so I launched an interactive website:

👉 https://kedreamix.github.io/Awesome-Talking-Head-Synthesis/

You can search papers, browse runnable open-source projects, filter by category and year, switch between English / 中文, and use dark mode. If you find a missing paper or project, newly released code, an accepted venue, or want to suggest a new category, please submit it through the website or GitHub Issues. 🙌

I also added Open-source Projects for runnable talking-head systems that are useful to try, but do not have a paper (for example Linly-Talker and NanoAvatar). SadTalker, MuseTalk, Hallo, and similar works stay in the paper tables.

⭐ If the site helps you, please star the repo — it really motivates continued updates!

2023.12 Update 📆

Thank you to https://github.com/Curated-Awesome-Lists/awesome-ai-talking-heads, I have added some of its contents, such as Tools & Software and Slides & Presentations. 🙏 I hope this will be helpful.😊

If you have any feedback or ideas on extending this aggregated resource, please open an issue or PR - community contributions are vital to advancing this shared knowledge. 🤝

Let’s keep pushing forward to recreate ever more realistic digital human faces! 💪 We’ve come so far but still have a long way to go. With continued research 🔬 and collaboration, I’m sure we’ll get there! 🤗

Please feel free to star ⭐ and share this repo if you find it a valuable resource. Your support helps motivate me to keep maintaining and improving it. 🥰 Let me know if you have any other questions!

---

Open-source Projects

Runnable talking-head systems, apps, and integration frameworks that do not have a paper listing. Research papers with official code stay in the sections below (Audio-driven, Conversational, and so on).

Year	Project	Code	Resources	Description
2026	NanoAvatar	Code	Weights · APKs	On-device audio-driven talking avatars for Android and local NVIDIA GPUs, with offline APKs.
2026	Linly-Talker-Stream	Code		Full-duplex, low-latency conversational digital human built on a real-time WebRTC streaming pipeline.
2026	CyberVerse	Code	Site	Self-hosted real-time digital-human agent platform with WebRTC, memory, tools, RAG, and optional avatar video.
2025	OpenAvatarChat	Code	Docs · Demo	Modular interactive avatar chat with replaceable ASR, LLM, TTS, and avatar backends.
2025	LiteAvatar	Code	Gallery	Real-time CPU audio-to-face 2D chat avatar and an avatar backend for OpenAvatarChat.
2024	Duix-Avatar	Code	Site	Offline avatar toolkit for appearance and voice cloning plus text- or audio-driven video generation.
2024	Ultralight-Digital-Human	Code	FeatherTalk	Mobile-friendly real-time 2D digital human, with FeatherTalk as its lighter successor.
2024	DH_live	Code	MatesX	Lightweight real-time 2D digital human for web and mobile, followed by the multi-platform MatesX engine.
2023	Linly-Talker	Code	Weights · Page	Conversational digital human WebUI combining LLM, ASR, TTS, voice cloning, and multiple talking-head backends.
2023	LiveTalking	Code	Site	Real-time interactive streaming digital human engine supporting Wav2Lip, MuseTalk, ER-NeRF, and other backends.
2023	TalkingHead	Code		Browser JavaScript class for real-time lip-sync using full-body 3D avatars.
2022	VU-VRM	Code	Demo	Browser-based real-time lip-sync VRM avatar driven by a microphone without a webcam.

---

Datasets

Year	Dataset	Conference/Journal	Download Link	Description
2026	The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset	ArXiv 2026	Download	Large-scale disentangled human-evaluation challenge for speech-driven co-speech gesture generation (GENEA 2026).
2026	REACT 2026: The Fourth Multiple Appropriate Facial Reaction Generation Challenge: Personalised MAFRG and Appropriate EEG Reaction Prediction	ArXiv 2026	Download	Fourth Multiple Appropriate Facial Reaction Generation Challenge for personalized listener reactions in dyadic interaction.
2026	MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation	ArXiv 2026	Download	Benchmark diagnosing cinematic-expressiveness failure modes in multi-talker audio-visual generation.
2026	AVAPrintDB: Leveraging Avatar Fingerprinting: A Multi-Generator Photorealistic Talking-Head Public Database and Benchmark	ArXiv 2026	N/A	Multi-generator benchmark for talking-head avatar fingerprinting and photorealistic avatar generation evaluation.
2026	A Near-Raw Talking-Head Video Dataset for Various Computer Vision Tasks	ArXiv 2026	N/A	Near-raw talking-head video dataset for computer vision tasks including detection, recognition, and generation.
2026	Face-to-Face: A Video Dataset for Multi-Person Interaction Modeling	ArXiv 2026	Download	70-hour, 14k-clip dataset of two-person talk-show exchanges for multi-person interaction modeling.
2026	SFQA	ArXiv	N/A	A dataset for singing face generation quality assessment with 5,184 videos generated from 100 photographs and 36 music clips using 12 generation methods.
2025	TalkCuts	ArXiv 2025	Download	A large-scale dataset with 164k clips totaling over 500 hours of human speech videos featuring diverse camera shots and detailed annotations including textual descriptions, 2D keypoints, and 3D SMPL-X motions for multi-shot speech video generation.
2025	EmojiBench++	IJCV 2025	Download	A comprehensive benchmark for portrait animation comprising diverse portraits, driving videos, and landmark sequences.
2025	Multi-human Interactive	ArXiv 2025	Download	12 hours of high-res footage with 2-4 speakers, fine-grained body pose and speech interaction annotations.
2025	THQA-10K	ArXiv 2025	Download	Largest AGTH quality assessment dataset with 10,457 samples from 12 T2I models and 14 talkers.
2025	SpeakerVid-5M	ArXiv	Download	Large-scale dataset with 5.2M video clips (8,743 hours) for audio-visual dyadic interactive virtual human generation, covering monadic talking, listening, and dyadic conversations, with pre-training and SFT subsets.
2025	TalkingHeadBench	WACV 2026	Download	Comprehensive benchmark for talking-head deepfake detection with multi-model generators.
2025	Motion-X++	ArXiv 2025	N/A	19.5M 3D whole-body pose annotations covering 120.5K motion sequences with 80.8K RGB videos.
2024	GLCF (MSTF)	ArXiv 2024	N/A	First large-scale multi-scenario talking face dataset with 22 audio/video forgery techniques.
2024	SAVEE	ArXiv 2024	Download	480 British English utterances from 4 male actors expressing 7 emotions.
2024	Allo-AVA	ArXiv 2024	N/A	~1,250 hours of conversational content for allocentric avatar gesture animation.
2024	MMHead	ACMMM 2024	Download	Large-scale multi-modal 3D facial animation dataset with 49 hours of 3D facial motion sequences, speech audios, and hierarchical text annotations for text-induced 3D talking head animation and text-to-3D facial motion generation.
2024	DH-FaceVid-1K	ICCV 2025	Download	1,200 hours, 270K+ clips from 20K+ individuals with speech audio, keypoints, and text annotations.
2024	MultiTalk	CVPR 2024	Download	420+ hours across 20 languages, 293K clips (512x512, 25fps, avg 5.19s duration).
2024	THQA	ArXiv 2024	Download	800 talking head videos from 8 speech-driven methods with subjective quality assessments.
2023	GRID	ArXiv 2023	Download	34 volunteers each speaking 1000 phrases (34K utterances) with 6-word sentence structures.
2023	ViCo	ArXiv 2023	Download	ViCo and ViCo-X are datasets for conversational head generation, with ViCo for sentence-level independent talking and listening tasks, and ViCo-X for multi-turn conversational scenarios.
2023	TalkingHead-1KH	ArXiv 2023	Download	500K video clips with ~80K greater than 512x512 resolution. Only permissive license videos included.
2023	CelebV	CVPR 2023	Download	Includes CelebV-Text with 70,000 in-the-wild face video clips for text-to-video generation.
2023	MMFace4D	ArXiv 2023	Download	Large-scale multi-modal 4D dataset with 35,000+ sequences from 431 subjects (age 15-68).
2022	CelebV-HQ	ECCV 2022	Download	35,666 clips with 15,653 identities, each labeled with 83 facial attributes.
2022	Multiface	NeurIPS 2022	Download	High-quality multi-view recordings of 13 people with 12K-23K frames per subject at 30fps. 65TB dataset.
2022	VFHQ	CVPRW 2022	Download	16,000+ high-fidelity clips for video face super-resolution research.
2021	HDTF	CVPR 2021	Download	High-definition Talking-Face Dataset with ~362 videos (15.8 hours) in 720P/1080P resolution.
2020	MEAD	ECCV 2020	Download	Large-scale audio-visual dataset with 60 actors expressing 8 emotions at 3 intensity levels.
2019	BIWI	ArXiv 2019	Download	3D Audiovisual Corpus of Affective Communication with 40 sentences spoken by 14 subjects.
2019	VOCA	SIGGRAPH 2019	Download	4D-face dataset with ~29 minutes of 4D face scans and synchronized audio from 12 speakers.
2019	CN-CVS	ArXiv 2019	Download	Large-scale continuous visual-speech dataset in Mandarin Chinese from TV news and speech shows.
2019	FaceForensics++	ICCV 2019	Download	Large-scale dataset for detecting manipulated facial images with over 1.8M images.
2018	VoxCeleb2	Interspeech 2018	Download	Largest public audio-visual dataset with video URLs and timestamps. Requires 300GB+ storage.
2018	LRS2	ArXiv 2018	Download	Lip reading dataset with videos recorded in diverse settings from BBC television.
2018	LRW	ACCV 2018	Download	Diverse English-speaking dataset from BBC with 1000+ speakers. Each video is 1.16s (29 frames).
2017	VoxCeleb1	Interspeech 2017	Download	Contains over 100,000 utterances for 1,251 celebrities, extracted from YouTube videos.
2017	ObamaSet	SIGGRAPH 2017	Download	Specialized audio-visual dataset focused on analyzing visual speech of Barack Obama from weekly address footage.
2014	CREMA-D	ACM TOCC 2014	Download	Diverse dataset with 7,442 clips featuring 91 actors (48 male, 43 female) aged 20-74, expressing six emotions at four intensity levels.

---

Survey

Year	Title	Conference/Journal
2026	How to Build Digital Humans? From Priors to Photorealistic Avatars	Eurographics 2026
2025	A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques	ArXiv 2025
2025	Human Motion Video Generation: A Survey	TPAMI
2025	A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation	ArXiv 2025
2025	Controllable Video Generation: A Survey	ArXiv 2025
2025	Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions	ArXiv 2025
2025	Survey of Video Diffusion Models: Foundations, Implementations, and Applications	TMLR
2025	A Survey on Human Interaction Motion Generation	ArXiv 2025
2024	Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey	EMNLP 2025
2024	Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey	ArXiv 2024
2024	3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities	ArXiv 2024
2024	A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights	ArXiv 2024
2024	A Survey on 3D Human Avatar Modeling — From Reconstruction to Generation	ArXiv 2024
2024	Video Diffusion Models: A Survey	ArXiv 2024
2024	Deepfake Generation and Detection: A Benchmark and Survey	ACM Computing Surveys
2024	ADTH-QA: A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos	ArXiv 2024
2024	How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey	ArXiv 2024
2024	3D Gaussian as a New Vision Era: A Survey	IEEE TVCG
2024	Advances in 3D Generation: A Survey	ArXiv 2024
2024	A Survey on 3D Gaussian Splatting	ArXiv 2024
2024	Neural Radiance Fields: Past, Present, and Future	ArXiv 2024
2023	From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications	ArXiv 2023
2023	Human Motion Generation: A Survey	TPAMI
2023	Human 3D Avatar Modeling with Implicit Neural Representation: A Brief Survey	ArXiv 2023
2023	Human-Computer Interaction System: A Survey of Talking-Head Generation	IEEE
2023	Talking human face generation: A survey	ACM
2022	Face Generation and Editing with StyleGAN: A Survey	ArXiv 2022
2022	Deep Learning for Visual Speech Analysis: A Survey	ArXiv 2022
2022	A Survey on Applications of Digital Human Avatars toward Virtual Co-presence	ArXiv 2022
2021	A Review of 3D Face Reconstruction From a Single Image	ArXiv 2021
2021	Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis	ArXiv 2021
2021	AudioVisual Speech Synthesis: A brief literature review	ArXiv 2021
2020	What comprises a good talking-head video generation?: A Survey and Benchmark	ArXiv 2020

---

Funny Work

Year	Title	Code	Project	Keywords
2024	LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control	Code	Project	🔥快手
2024	From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations	Code	Project	Photoreal
2024	Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation	Code	Project	🔥Animate (阿里科目三驱动)
2024	What You See Is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs		Project	🔥Nvidia

---

Audio-driven

Year	Title	Conference/Journal	Code	Project	Keywords
2026	EfficientSync: Real-Time Lip Synchronization via Deformation-Based Reference Texture Mixing	ArXiv 2026		Project	lip synchronization, audio-driven, texture mixing, real-time
2026	DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation	ACM MM 2026			audio-driven, streaming avatar, lip-sync, self-forcing
2026	Xemo-Talker: Unlock Emotions Explicitly for Audio-Driven Talking Portrait Synthesis	ArXiv 2026	Code		audio-driven, emotion control, talking portrait, lip-sync
2026	Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Video Avatar Generation	ArXiv 2026	Code	Project	joint audio-video, streaming avatar, real-time, prompt planning
2026	Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars	ArXiv 2026	Code	Project	audio-driven, streaming avatar, real-time, long-horizon, distillation
2026	Proxy Avatar Meets Low-Rank Caching: Real-Time One-Shot Emotion-Controllable Portrait Animation	ArXiv 2026			audio-driven, portrait animation, emotion control, one-shot, real-time
2026	GemTalk: Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation	ACM MM 2026			audio-driven, emotional talking face, blendshape prior, diffusion
2026	LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation	ArXiv 2026	Code	Project	audio-driven, talking head, real-time, diffusion distillation
2026	TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation	ArXiv 2026	Code	Project	digital human, audio-video, real-time, talking head, TaoMate
2026	AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars	ArXiv 2026	Code	Project	audio-driven, avatar generation, distillation, long-form, real-time
2026	SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation	ECCV 2026			audio-driven, portrait animation, caching, lip-sync, DiT
2026	ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance	ArXiv 2026			speech-driven, portrait animation, lip-sync, co-speech
2026	Lip Forcing: Few-Step Autoregressive Diffusion for Real-time Lip Synchronization	ArXiv 2026	Code	Project	lip synchronization, audio-driven, autoregressive diffusion, real-time
2026	Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation	ICME 2026			audio-driven, portrait animation, implicit motion, diffusion
2026	Real-Time Generation of Streamable Talking Portrait Video with Reference-Guided Deep Compression VAEs	CVPR 2026			audio-driven, talking portrait, streaming, causal VAE, real-time
2026	FreeTalkDiff: IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation	ArXiv 2026	Code		talking face, diffusion, IP-Adapter, lip-sync
2026	CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planning	ArXiv 2026			audio-driven, portrait animation, eye control, lip-sync, DiT
2026	Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation	ArXiv 2026			audio-driven, talking head, test-time adaptation, identity stability
2026	HighSync: High-Quality Lip Synchronization via Latent Diffusion Models	ArXiv 2026	Code		lip synchronization, diffusion model, talking face, high-resolution, audio-driven
2026	MoCoTalk: Multi-Conditional Diffusion with Adaptive Router for Controllable Talking Head Generation	ArXiv 2026			talking head, multi-conditional, diffusion, 3DMM, controllable
2026	AudioFace: Language-Assisted Speech-Driven Facial Animation with Multimodal Language Models	ArXiv 2026			speech-driven, facial animation, blendshape, multimodal LLM, language-assisted
2026	TAVR: Generate Your Talking Avatar from Video Reference	ArXiv 2026		Project	talking avatar, video reference, cross-scene, reinforcement learning, identity preservation
2026	Talking Slide Avatars: Open-Source Multimodal Communication Approach for Teaching	ArXiv 2026			talking slide avatars, text-to-speech, audio-driven synthesis, educational
2026	Personalizing Causal Audio-Driven Facial Motion via Dynamic Multi-modal Retrieval	ArXiv 2026			causal audio-driven, facial motion, multi-modal retrieval, personalization
2026	Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation	ArXiv 2026	Code		real-time avatar, audio-video generation, diffusion, streaming
2026	Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling	ArXiv 2026	Code	Project	joint audio-video generation, autoregressive diffusion, talking head synthesis
2026	EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence	ICMR 2026			emotion-aware, talking head generation, lip-sync, temporal coherence
2026	Learning Spatial-Temporal Coherent Correlations for Speech-Preserving Facial Expression Manipulation	ArXiv 2026			speech-preserving, facial expression manipulation, spatial-temporal correlation, emotion editing
2026	Polyglot: Multilingual Style Preserving Speech-Driven Facial Animation	ArXiv 2026		Project	multilingual, speaker style, speech-driven, facial animation
2026	TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation	ArXiv 2026			distillation, audio-driven, avatar, head avatar
2026	SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization	ArXiv 2026	Code	Project	audio-driven, 3D, emotion, facial animation
2026	C-MET: Cross-Modal Emotion Transfer for Emotion Editing in Talking Face Video	CVPR 2026	Code	Project	emotion transfer, emotion editing, talking face, cross-modal
2026	MMTalker: Multiresolution 3D Talking Head Synthesis with Multimodal Feature Fusion	ArXiv 2026			3D, Audio-Driven, Multimodal Fusion, Mesh Parameterization
2026	EmoTaG: Emotion-Aware Talking Head Synthesis on Gaussian Splatting with Few-Shot Personalization	CVPR 2026	Code	Project	Gaussian Splatting, 3DGS, Audio-Driven, Emotion, Few-Shot
2026	EARTalking: End-to-end GPT-style Autoregressive Talking Head Synthesis with Frame-wise Control	ArXiv 2026			Autoregressive, GPT-style, Talking Head
2026	FreeTalk: Emotional Topology-Free 3D Talking Heads	ArXiv 2026			3D Talking Heads, Emotional, Topology-Free
2026	AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising	ArXiv 2026	Code	Project	Streaming Avatar, Real-Time, Diffusion
2026	DiFlowDubber: Discrete Flow Matching for Automated Video Dubbing via Cross-Modal Alignment and Synchronization	CVPR 2026 Findings	Code	Project	Video Dubbing, Flow Matching, Cross-Modal
2026	OmniEdit: A Training-free Framework for Lip Synchronization and Audio-Visual Editing	ArXiv 2026	Code		Lip Sync, Audio-Visual Editing, Training-Free
2026	EmbedTalk: Triplane-Free Talking Head Synthesis using Embedding-Driven Gaussian Deformation	Preprint			Gaussian Splatting, 3DGS, Audio-Driven, Talking Head
2026	TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation	ArXiv 2026	Code	Project	Diffusion, Audio-Driven, Talking Head, VAE, Latent
2026	UniSync: Towards Generalizable and High-Fidelity Lip Synchronization for Challenging Scenarios	ArXiv 2026			Lip Sync, Pose-Anchored, Generalizable
2026	UniTalking: A Unified Audio-Video Framework for Talking Portrait Generation	CVPR 2026			Audio-Driven, Portrait Animation, Talking Head, CVPR, Transformer, Attention
2026	FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation	ArXiv 2026			Audio-Driven, Portrait Animation, Reinforcement Learning, GRPO
2026	Narrating For You: Prompt-guided Audio-visual Narrating Face Generation Employing Multi-entangled Latent Space	WACV 2026	Code		Audio-Driven, WACV, Latent
2026	VineetVC: Adaptive Video Conferencing Under Severe Bandwidth Constraints Using Audio-Driven Talking-Head Reconstruction	ArXiv 2026			audio-driven, video conferencing, talking head
2026	DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation	ArXiv 2026	Code	Project	Audio-Driven, Transformer, Attention
2026	3DXTalker: Unifying Identity, Lip Sync, Emotion, and Spatial Dynamics in Expressive 3D Talking Avatars	ArXiv 2026	Code	Project	3D, Emotional, Lip Sync, Avatar, Talking Head, Transformer
2026	AUHead: Realistic Emotional Talking Head Generation via Action Units Control	ArXiv 2026	Code		Action Units, Audio-Driven Generation, Emotion Control, Diffusion Model
2026	MOVA: Towards Scalable and Synchronized Video-Audio Generation	ArXiv 2026	Code	Project	Audio-Driven
2026	VedicTHG: Symbolic Vedic Computation for Low-Resource Talking-Head Generation in Educational Avatars	ArXiv 2026	Code	Project	Avatar, Talking Head
2026	SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Heads	ArXiv 2026	Code	Project	Real-time, Streaming, Talking Head
2026	Asymmetric Hierarchical Anchoring for Audio-Visual Joint Representation	ArXiv 2026			Audio-Driven
2026	JoyAvatar: Unlocking Highly Expressive Avatars via Harmonized Text-Audio Conditioning	ArXiv 2026		Project	Audio-Driven, Avatar
2026	LPIPS-AttnWav2Lip: Generic Audio-Driven lip synchronization for Talking Head Generation in the Wild	ArXiv 2026	Code	Project	Lip Sync, Audio-Driven, Talking Head, Latent
2026	MIRRORTALK: Forging Personalized Avatars Via Disentangled Style and Hierarchical Motion Control	ICASSP 2026			Personalized Avatars, Lip Sync, Style Disentanglement, Diffusion Model
2026	JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusion	ArXiv 2026	Code	Project	Audio-Visual Diffusion, LoRA, Lip Sync
2026	EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers	ArXiv 2026		Project	Diffusion, Audio-Driven, Talking Head
2026	UA-3DTalk: Uncertainty-Aware 3D Emotional Talking Face Synthesis with Emotion Prior Distillation	ICASSP 2026	Code	Project	3D, Emotional, Talking Head, ICASSP, Attention
2026	Audio-Driven Talking Face Generation with Blink Embedding and Hash Grid Landmarks Encoding	ArXiv 2026			Audio-Driven, Talking Head, Transformer
2026	SkyReels-V3 Technique Report	ArXiv 2026	Code		Video Generation, Audio-Guided, Talking Avatar, Diffusion Transformers
2026	THFEM: Exploring Talking Head Models With Adjacent Frame Prior for Speech-Preserving Facial Expression Manipulation	ACM Trans. Multimedia	Code	Project	Speech-Driven, Talking Head
2026	Now You See Me, Now You Don’t: A Unified Framework for Expression Consistent Anonymization in Talking Head Videos	ArXiv 2026			Talking Head
2026	EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing	ArXiv 2026			3D, Speech-Driven
2026	ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting	ArXiv 2026			3D, Gaussian Splatting, 3DGS, Emotional, Audio-Driven
2026	DyStream: Streaming Dyadic Talking Heads Generation via Flow Matching-based Autoregressive Model	ArXiv 2026	Code	Project	Streaming, Talking Head, Flow Matching
2026	SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation	ArXiv 2026	Code	Project	Real-time, Streaming, Audio-Driven, Avatar, Attention, VAE
2026	SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wild	ArXiv 2026		Project	Transformer
2026	JoyAvatar-Flash: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion	ArXiv 2026		Project	Diffusion, Real-time, Audio-Driven, Avatar
2026	REST: Diffusion-based Real-time End-to-end Streaming Talking Head Generation via ID-Context Caching and Asynchronous Streaming Distillation	ArXiv 2026			Diffusion, Real-time, Streaming, Talking Head, Latent
2026	Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generation	ArXiv 2026			Talking Head, Attention, Latent
2025	X-Dub: From Inpainting to Editing: A Self-Bootstrapping Framework for Context-Rich Visual Dubbing	ArXiv 2025	Code	Project	Visual dubbing, Diffusion Transformer, Self-bootstrapping, Lip sync
2025	PTalker: Personalized Speech-Driven 3D Talking Head Animation via Style Disentanglement and Modality Alignment	ArXiv 2025			3D, Speech-Driven, Talking Head, Attention
2025	FlashLips: 100-FPS Mask-Free Latent Lip-Sync using Reconstruction Instead of Diffusion or GANs	ArXiv 2025			Diffusion, Transformer, GAN, Latent
2025	In-Context Audio Control of Video Diffusion Transformers	ArXiv 2025			Diffusion, Audio-Driven, Transformer, Attention
2025	Asynchronous Pipeline Parallelism for Real-Time Multilingual Lip Synchronization in Video Communication Systems	IEEE Big Data 2025			lip sync, real-time, multilingual
2025	TalkVerse: Democratizing Minute-Long Audio-Driven Video Generation	ArXiv 2025	Code	Project	Audio-Driven, VAE, Latent
2025	VASA-3D: Lifelike Audio-Driven Gaussian Head Avatars from a Single Image	NeurIPS 2025			3D, Gaussian Splatting, Audio-Driven, Avatar
2025	FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling	ArXiv 2025		Project	Talking Head, Transformer, Attention, Flow Matching
2025	JoVA: Unified Multimodal Learning for Joint Video-Audio Generation	ArXiv 2025	Code	Project	Audio-Driven, Transformer, Attention, GAN
2025	Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model	Tech Report			Audio-Driven, Transformer, Reinforcement Learning
2025	STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraits	ArXiv 2025	Code	Project	Diffusion, Portrait Animation, Talking Head
2025	GaussianHeadTalk: Wobble-Free 3D Talking Heads with Audio Driven Gaussian Splatting	WACV 2026			3D, Gaussian Splatting, 3DGS, Audio-Driven, Talking Head
2025	UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speaking	CVPR 2026	Code	Project	Audio-Driven, Avatar
2025	Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length	ArXiv 2025	Code	Project	Real-time, Streaming, Audio-Driven, Avatar
2025	EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans	ArXiv 2025			Portrait Animation, Talking Head
2025	AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinement	ArXiv 2025	Code	Project	Talking Head, Transformer, Attention
2025	AI killed the video star. Audio-driven diffusion model for expressive talking head generation	ArXiv 2025			audio-driven, diffusion, talking head
2025	IMTalker: Efficient Audio-driven Talking Face Generation with Implicit Motion Transfer	ArXiv 2025			Audio-Driven, Talking Head, Attention, Latent
2025	Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy	ArXiv 2025			Audio-Driven, Attention, Latent
2025	StreamingTalker: Audio-driven 3D Facial Animation with Autoregressive Diffusion Model	ArXiv 2025		Project	3D, Diffusion, Streaming, Audio-Driven
2025	ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search	AAAI 2026			Diffusion, Talking Head, AAAI, Knowledge Distillation
2025	Shared Latent Representation for Joint Text-to-Audio-Visual Synthesis	ArXiv 2025			Audio-Driven, Latent
2025	UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactions	ArXiv 2025			Audio-Driven, Transformer, Attention, Latent
2025	See the Speaker: Crafting High-Resolution Talking Faces from Speech with Prior Guidance and Region Refinement	TASLP 2025			High-Resolution, Talking Faces, Speech-to-Face, Diffusion
2025	Learning Disentangled Speech- and Expression-Driven Blendshapes for 3D Talking Face Animation	ICXR 2025			Blendshapes, FLAME, Disentanglement, 3D Animation
2025	MAGIC-Talk: Motion-aware Audio-Driven Talking Face Generation with Customizable Identity Control	ArXiv 2025
2025	LSF-Animation: Label-Free Speech-Driven Facial Animation via Implicit Feature Representation	SIGGRAPH Asia 2025	Code		Label-Free, Speech-Driven, Facial Animation, FLAME
2025	Playmate2: Training-Free Multi-Character Audio-Driven Animation via Diffusion Transformer with Reward Feedback	ArXiv 2025			Diffusion, Audio-Driven, AAAI, Transformer
2025	DEMO: Disentangled Motion Latent Flow Matching for Fine-Grained Controllable Talking Portrait Synthesis	ArXiv 2025			Disentangled Motion, Flow Matching, Talking Portrait, Controllable
2025	SyncLipMAE: Contrastive Masked Pretraining for Audio-Visual Talking-Face Representation	ArXiv 2025			Contrastive Masked Pretraining, Audio-Visual, Talking-Face
2025	EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformation	IEEE SMC 2025			Real-Time, Audio-Driven, Gaussian Deformation, Talking Head
2025	A Bridge from Audio to Video: Phoneme-Viseme Alignment Allows Every Face to Speak Multiple Languages	ArXiv 2025			Phoneme-Viseme Alignment, Multilingual TFS, Mixture-of-Experts
2025	IASA: Input-Aware Sparse Attention for Real-Time Co-Speech Video Generation	ArXiv 2025	Code	Project	Diffusion models, co-speech video, real-time, sparse attention
2025	Audio Driven Real-Time Facial Animation for Social Telepresence	SIGGRAPH Asia 2025		Project	Real-time, Audio-Driven, SIGGRAPH, Transformer, Latent
2025	StableDub: Taming Diffusion Prior for Generalized and Efficient Visual Dubbing	ArXiv 2025		Project	Visual Dubbing, Diffusion, Mamba-Transformer
2025	KSDiff: Keyframe-Augmented Speech-Aware Dual-Path Diffusion for Facial Animation	ArXiv 2025			Keyframe, Diffusion, Dual-Path, Facial Animation
2025	SynchroRaMa: Lip-Synchronized and Emotion-Aware Talking Face Generation via Multi-Modal Emotion Embedding	WACV 2026	Code	Project	Multi-Modal, Emotion-Aware, LLM
2025	Talking Head Generation via AU-Guided Landmark Prediction	ArXiv 2025			Action Units, Landmark Prediction, Diffusion
2025	PGSTalker: Real-Time Audio-Driven Talking Head Generation via 3D Gaussian Splatting with Pixel-Aware Density Control	ICONIP 2025			3DGS, Real-Time, Pixel-Aware, Audio-Driven
2025	A Lightweight Pipeline for Noisy Speech Voice Cloning and Accurate Lip Sync Synthesis	ArXiv 2025			lip sync, voice cloning
2025	Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis	ArXiv 2025		Project	Multimodal Instructions, Avatar Synthesis, Lip Synchronization
2025	Think2Sing: Orchestrating Structured Motion Subtitles for Singing-Driven 3D Head Animation	ArXiv 2025			Singing-Driven, 3D Head, Diffusion
2025	EmoCAST: Emotional Talking Portrait via Emotive Text Description	ArXiv 2025	Code	Project	Emotional, Portrait Animation, Talking Head, Attention
2025	Wan-S2V: Audio-Driven Cinematic Video Generation	ArXiv 2025			Cinematic, Audio-Driven, Video Generation
2025	Warm Chat: Diffuse Emotion-aware Interactive Talking Head Avatar with Tree-Structured Guidance	ArXiv 2025 (Withdrawn)			Emotional, Avatar, Talking Head, Transformer, Latent
2025	Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars	ArXiv 2025			Audio-driven Realistic Facial Animation, Digital Avatars
2025	D^3-Talker: Dual-Branch Decoupled Deformation Fields for Few-Shot 3D Talking Head Synthesis	ECAI 2025			Few-Shot, 3DGS, Deformation Fields
2025	InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing	ArXiv 2025			Sparse-Frame Dubbing, Full-Body
2025	CEM-Net: Cross-Emotion Memory Network for Emotional Talking Face Generation	ArXiv 2025			Cross-emotion memory, audio emotion enhancement, expression displacement, lip sync
2025	RealTalk: Realistic Emotion-Aware Lifelike Talking-Head Synthesis	ICCV 2025 Workshop			Emotion, NeRF, VAE
2025	FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation	ArXiv 2025	Code	Project	Audio-Driven, Portrait Animation, Preference Optimization
2025	HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis	ArXiv 2025			Hybrid Motion, High-Fidelity, Talking Head
2025	StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation	ArXiv 2025	Code	Project	Stable Diffusion
2025	DiTalker: A Unified DiT-based Framework for High-Quality and Speaking Styles Controllable Portrait Animation	ArXiv 2025		Project	DiT, Portrait Animation, Speaking Styles
2025	READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation	ArXiv 2025		Project	Diffusion, Real-time, Audio-Driven, Talking Head, Transformer, VAE
2025	X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio	ArXiv 2025		Project	Emotional Portrait, Long-range, Audio-driven
2025	SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware Video Generation	ACM MM 2025			Spatial Audio, Video Generation, MLLM
2025	Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation	ArXiv 2025			Mask-Free, Identity Preservation, Audio-driven
2025	Learning Phonetic Context-Dependent Viseme for Enhancing Speech-Driven 3D Facial Animation	Interspeech 2025	Code	Project	Phonetic Context, Viseme, 3D Facial Animation
2025	MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylization	ICCV 2025	Code	Project	Personalized, 3D Facial Animation, Memory
2025	JOLT3D: Joint Learning of Talking Heads and 3DMM Parameters with Application to Lip-Sync	ArXiv 2025			3DMM, Joint Learning, Talking Head
2025	Livatar-1: Real-Time Talking Heads Generation with Tailored Flow Matching	Technical Report		Project	real-time, flow matching, lip-sync
2025	ATL-Diff: Audio-Driven Talking Head Generation with Early Landmarks-Guide Noise Diffusion	ArXiv 2025	Code		Diffusion, Landmarks-Guide, Real-time, Identity Preservation
2025	MOSPA: Human Motion Generation Driven by Spatial Audio	NeurIPS 2025	Code		Spatial Audio, Human Motion Generation, Virtual Human
2025	M2DAO-Talker: Harmonizing Multi-granular Motion Decoupling and Alternating Optimization for Talking-head Generation	ArXiv 2025		Project	Multi-granular Motion, Decoupling, Optimization
2025	MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding	ArXiv 2025	Code		Multimodal, 3D Facial Animation, Dynamic Emotions
2025	MoDiT: Learning Highly Consistent 3D Motion Coefficients with Diffusion Transformer for Talking Head Generation	ArXiv 2025			3DMM, Diffusion Transformer, Temporal Consistency, Blinking Dynamics
2025	MoDA: Multi-modal Diffusion Architecture for Talking Head Generation	ArXiv 2025	Code	Project	Multi-modal, Diffusion, Talking Head Generation
2025	FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases	ArXiv 2025			Identity Leakage, Extreme Cases
2025	JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching	ArXiv 2025			Flow Matching, Audio-Motion
2025	FIAG: Few-Shot Identity Adaptation for 3D Talking Heads via Global Gaussian Field	ArXiv 2025	Code		Few-Shot, Global Gaussian Field, 3DGS
2025	GGTalker: Talking Head Synthesis with Generalizable Gaussian Priors and Identity-Specific Adaptation	ICCV 2025	Code	Project	3D Talking Head, Gaussian Priors, Identity Adaptation
2025	SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting	ArXiv 2025	Code	Project	3DGS, Synchronization
2025	LLIA – Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models	ArXiv 2025			Low-Latency, Real-Time, Interactive
2025	TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models	ArXiv 2025	Code	Project	Real-Time, Autoregressive Diffusion
2025	SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers	ArXiv 2025			video diffusion transformer, multimodal, talking portrait, audio-conditioned
2025	Video Editing for Audio-Visual Dubbing	ArXiv 2025			Video Editing, Dubbing
2025	Wav2Sem: Plug-and-Play Audio Semantic Decoupling for 3D Speech-Driven Facial Animation	CVPR 2025	Code		3D, Semantic Decoupling
2025	MMGT: Motion Mask Guided Two-Stage Network for Co-Speech Gesture Video Generation	ArXiv 2025	Code		Co-Speech Gesture, Two-Stage
2025	FaceEditTalker: Interactive Talking Head Generation with Facial Attribute Editing	ArXiv 2025		Project	Attribute Editing, Interactive
2025	OmniSync: Towards Universal Lip Synchronization via Diffusion Transformers	ArXiv 2025			Lip Sync, Universal, Visual Prosody
2025	OT-Talk: Animating 3D Talking Head with Optimal Transportation	ArXiv 2025			FLAME, 3D
2025	GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting	CVPRW 2025			3DGS
2025	Model See Model Do: Speech-Driven Facial Animation with Style Control	SIGGRAPH 2025
2025	KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution	ArXiv 2025	Code	Project
2025	IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos	CVPR 2025		Project	3D-aware, Video Diffusion
2025	Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning	ArXiv 2025			Joint uncertainty learning, Audio-driven talking face, Lip sync, Visual uncertainty
2025	DICE-Talk: Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation	ACM MM 2025			Emotional Portrait, Identity Preservation, Emotion Cooperation
2025	PASE: Phoneme-Aware Speech Encoder to Improve Lip Sync Accuracy for Talking Head Synthesis	ArXiv 2025			Lip Sync, Phoneme-Aware, Speech Encoder
2025	Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation	TMM 2025			Talking Head Animation, Temporal Correlation, One-Shot
2025	FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesis	ArXiv 2025	Code	Project	talking portrait, motion synthesis, video diffusion, audio-visual alignment
2025	ACTalker: Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation	ArXiv 2025	Code	Project
2025	MGGTalk：Monocular and Generalizable Gaussian Talking Head Animation	CVPR 2025		Project	One Shot, 3DGS
2025	STSA: Spatial-Temporal Semantic Alignment for Visual Dubbing	ICME 2025	Code		Spatial-Temporal Alignment, Semantic Features, Visual Dubbing, Stability
2025	Dual Audio-Centric Modality Coupling for Talking Head Generation	ArXiv 2025			NeRF
2025	Audio-Plane: Audio Factorization Plane Gaussian Splatting for Real-Time Talking Head Synthesis	ArXiv 2025		Project	3DGS
2025	Audio-driven Gesture Generation via Deviation Feature in the Latent Space	ArXiv 2025			Gesture
2025	Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics	CVPR 2025	Code	Project
2025	AudCast: Audio-Driven Human Video Generation by Cascaded Diffusion Transformers	CVPR 2025		Project	DiT
2025	EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters	ArXiv 2025			Neural Radiance Fields, Expression Parameters, Emotion Control, Audio-Driven
2025	DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model	ICME 2025		Project
2025	Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation	CVPR 2025		Project	Autoregressive
2025	DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation	ICME 2025	Code		Diffusion, 3D
2025	[KDTalker: Unlock Pose Diversity: Accurate and Efficient Implicit Keypoint-based Spatiotemporal Diffusion for Audio-driven Talking Portrait]

README 内容过长，已截取前 64 KB。完整内容请前往 GitHub 仓库查看。