Autoregressive-Models-in-Vision-Survey

[TMLR 2025] Awesome Autoregressive Models in Vision

If you like our project, please give us a star ⭐ on GitHub for the latest update.

Autoregressive models have shown significant progress in generating high-quality content by modeling the dependencies sequentially. This repo is a curated list of papers about the latest advancements in autoregressive models in vision.

Paper: [TMLR 2025🔥] Autoregressive Models in Vision: A Survey | [中文解读]

Authors: Jing Xiong^1,†, Gongye Liu^2,†, Lun Huang³, Chengyue Wu¹, Taiqiang Wu¹, Yao Mu¹, Yuan Yao⁴, Hui Shen⁵, Zhongwei Wan⁵, Jinfa Huang⁴, Chaofan Tao^1,‡, Shen Yan⁶, Huaxiu Yao⁷, Lingpeng Kong¹, Hongxia Yang⁹, Mi Zhang⁵, Guillermo Sapiro^8,10, Jiebo Luo⁴, Ping Luo¹, Ngai Wong¹

¹The University of Hong Kong, ²Tsinghua University, ³Duke University, ⁴University of Rochester, ⁵The Ohio State University, ⁶Bytedance, ⁷The University of North Carolina at Chapel Hill, ⁸Apple, ⁹The Hong Kong Polytechnic University, ¹⁰Princeton University

^† Core Contributors, ^‡ Corresponding Authors

💡 We also have other generative projects that may interest you ✨.

Personalized Video Generation: Progress, Applications, and Challenges
Jinfa Huang, Shenghai Yuan, Kunyang Li, and Meng Cao etc.

📑 Citation

Please consider citing 📑 our papers if our repository is helpful to your work. Thanks sincerely!

@misc{xiong2024autoregressive,
    title={Autoregressive Models in Vision: A Survey},
    author={Jing Xiong and Gongye Liu and Lun Huang and Chengyue Wu and Taiqiang Wu and Yao Mu and Yuan Yao and Hui Shen and Zhongwei Wan and Jinfa Huang and Chaofan Tao and Shen Yan and Huaxiu Yao and Lingpeng Kong and Hongxia Yang and Mi Zhang and Guillermo Sapiro and Jiebo Luo and Ping Luo and Ngai Wong},
    year={2024},
    eprint={2411.05902},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

📣 Update News

[2025-11-01] ⏸️ After a year of rapid progress in autoregressive visual generation, two clear trends now define the field: unified multimodal models and autoregressive diffusion-forcing video generation. Our current repository categories no longer capture this evolving landscape, so we’re moving to maintenance mode and pausing proactive updates as of today. The repo remains available as a reference, and targeted PRs are welcome (additions, corrections, or reorganizations with new trends). Thanks for your support! 🙏

[2025-05-31] 🔥 Our survey has been revised in arXiv! The revised paper streamlines content and enhances discussions on:

• Continuous autoregressive methods
• Computational costs
• More details about metrics
• Expands future application roadmaps

[2025-03-11] 🔥 Our survey has been accepted by TMLR 2025!

[2024-11-11] We have released the survey: Autoregressive Models in Vision: A Survey.

[2024-10-13] We have initialed the repository.

⚡ Contributing

We welcome feedback, suggestions, and contributions that can help improve this survey and repository and make them valuable resources for the entire community. We will actively maintain this repository by incorporating new research as it emerges. If you have any suggestions about our taxonomy, please take a look at any missed papers, or update any preprint arXiv paper that has been accepted to some venue.

If you want to add your work or model to this list, please do not hesitate to email [email protected] or pull requests. Markdown format:

* [**Name of Conference or Journal + Year**] Paper Name. [Paper](link) [Code](link)

📖 Table of Contents

• Image Generation
  • Unconditional/Class-Conditioned Image Generation
  • Text-to-Image Generation
  • Image-to-Image Translation
  • Image Editing
• Video Generation
  • Unconditional Video Generation
  • Conditional Video Generation
  • Embodied AI
• 3D Generation
  • Motion Generation
  • Point Cloud Generation
  • 3D Medical Generation
• Multimodal Generation
  • Unified Understanding and Generation Multi-Modal LLMs
• Other Generation
• Benchmark / Analysis
• Reasoning Alignment
• Safety
• Accelerating
• Stability & Scaling
• Tutorial
• Evaluation Metrics

---

Image Generation

Unconditional/Class-Conditioned Image Generation

• Pixel-wise Generation

  • [ICML, 2021 oral] Improved Autoregressive Modeling with Distribution Smoothing Paper Code
  • [ICML, 2020] ImageGPT: Generative Pretraining from Pixels Paper
  • [ICML, 2018] Image Transformer Paper Code
  • [ICML, 2018] PixelSNAIL: An Improved Autoregressive Generative Model Paper Code
  • [ICML, 2017] Parallel Multiscale Autoregressive Density Estimation Paper
  • [ICLR workshop, 2017] Gated PixelCNN: Generating Interpretable Images with Controllable Structure Paper
  • [ICLR, 2017] PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications Paper Code
  • [NeurIPS, 2016] PixelCNN Conditional Image Generation with PixelCNN Decoders Paper Code
  • [ICML, 2016] PixelRNN Pixel Recurrent Neural Networks Paper Code

• Token-wise Generation

  Tokenizer

  • [Arxiv, 2025.07] Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Generation Paper Code
  • [Arxiv, 2025.07] Holistic Tokenizer for Autoregressive Image Generation Paper Code
  • [Arxiv, 2025.06] Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generation Paper
  • [Arxiv, 2025.05] D-AR: Diffusion via Autoregressive Models Paper Code
  • [Arxiv, 2025.05] Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space Paper Code
  • [Arxiv, 2025.04] Distilling semantically aware orders for autoregressive image generation Paper
  • [Arxiv, 2025.04] Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models Paper
  • [CVPR, 2025] Improving Autoregressive Visual Generation with Cluster-Oriented Token Prediction Code Paper
  • [Arxiv, 2025.03] Equivariant Image Modeling Paper Code
  • [Arxiv, 2025.03] V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation Paper Code
  • [Arxiv, 2025.02] FlexTok: Resampling Images into 1D Token Sequences of Flexible Length Paper
  • [Arxiv, 2025.01] ARFlow: Autogressive Flow with Hybrid Linear Attention Paper Code
  • [Arxiv, 2024.12] TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation Paper Code
  • [Arxiv, 2024.12] Next Patch Prediction for Autoregressive Visual Generation Paper Code
  • [Arxiv, 2024.12] XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation Paper Code
  • [Arxiv, 2024.12] RandAR: Decoder-only Autoregressive Visual Generation in Random Orders. Paper Code Project
  • [Arxiv, 2024.11] Randomized Autoregressive Visual Generation. Paper Code Project
  • [Arxiv, 2024.09] Open-MAGVIT2: Democratizing Autoregressive Visual Generation Paper Code
  • [Arxiv, 2024.06] OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation Paper Code
  • [Arxiv, 2024.06] Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99% Paper Code
  • [Arxiv, 2024.06] Titok An Image is Worth 32 Tokens for Reconstruction and Generation Paper Code
  • [Arxiv, 2024.06] Wavelets Are All You Need for Autoregressive Image Generation Paper
  • [Arxiv, 2024.06] LlamaGen Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Paper Code
  • [ICLR, 2024] MAGVIT-v2 Language Model Beats Diffusion – Tokenizer is Key to Visual Generation Paper
  • [ICLR, 2024] FSQ Finite scalar quantization: Vq-vae made simple Paper Code
  • [ICCV, 2023] Efficient-VQGAN: Towards High-Resolution Image Generation with Efficient Vision Transformers Paper
  • [CVPR, 2023] Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantization Paper Code
  • [CVPR, 2023, Highlight] MAGVIT: Masked Generative Video Transformer Paper
  • [NeurIPS, 2023] MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation Paper
  • [BMVC, 2022] Unconditional image-text pair generation with multimodal cross quantizer Paper Code
  • [CVPR, 2022] RQ-VAE Autoregressive Image Generation Using Residual Quantization Paper Code
  • [ICLR, 2022] ViT-VQGAN Vector-quantized Image Modeling with Improved VQGAN Paper
  • [PMLR, 2021] Generating images with sparse representations Paper
  • [CVPR, 2021] VQGAN Taming Transformers for High-Resolution Image Synthesis Paper Code
  • [NeurIPS, 2019] Generating Diverse High-Fidelity Images with VQ-VAE-2 Paper Code
  • [NeurIPS, 2017] VQ-VAE Neural Discrete Representation Learning Paper

  Autoregressive Modeling

  • [Arxiv, 2025.11] InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generation Paper Code
  • [Arxiv, 2025.10] FARMER: Flow AutoRegressive Transformer over Pixels Paper
  • [Arxiv, 2025.10] SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generation Paper
  • [NeurIPS, 2025] Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling Paper
  • [NeurIPS, 2025] Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy Paper Code
  • [Arxiv, 2025.09] Hyperspherical Latents Improve Continuous-Token Autoregressive Generation Paper Code
  • [Arxiv, 2025.09] Go with Your Gut: Scaling Confidence for Autoregressive Image Generation Paper Code
  • [NeurIPS, 2025] Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation Paper Code
  • [Arxiv, 2025.08] Exploiting Discriminative Codebook Prior for Autoregressive Image Generation Paper
  • [Arxiv, 2025.08] NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale Paper Code
  • [Arxiv, 2025.07] Frequency-Aware Autoregressive Modeling for Efficient High-Resolution Image Synthesis Paper Code
  • [Arxiv, 2025.07] TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation Paper Code
  • [Arxiv, 2025.07] Transition Matching: Scalable and Flexible Generative Modeling Paper
  • [Arxiv, 2025.07] Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis Paper
  • [CVPR, 2025] OmniGen: Unified Image Generation Paper Code
  • [Arxiv, 2025.06] AR-RAG: Autoregressive Retrieval Augmentation for Image Generation Paper Code
  • [Arxiv, 2025.06] Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregression Paper Code
  • [Arxiv, 2025.06] MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation Paper
  • [Arxiv, 2025.06] SpectralAR: Spectral Autoregressive Visual Generation Paper Code
  • [Arxiv, 2025.06] AliTok: Towards Sequence Modeling Alignment between Tokenizer and Autoregressive Model Paper Code
  • [Arxiv, 2025.05] DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via Next-Detail Prediction Paper Code
  • [Arxiv, 2025.05] TensorAR: Refinement is All You Need in Autoregressive Image Generation Paper
  • [Arxiv, 2025.05] MVAR: Visual Autoregressive Modeling with Scale and Spatial Markovian Conditioning Paper Code
  • [ICML, 2025] Continuous Visual Autoregressive Generation via Score Maximization Paper Code
  • [Arxiv, 2025.04] GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation Paper Code
  • [Arxiv, 2025.03] D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens Paper
  • [Arxiv, 2025.03] Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation Paper Code
  • [Arxiv, 2025.03] Autoregressive Image Generation with Randomized Parallel Decoding Paper Code
  • [Arxiv, 2025.03] Direction-Aware Diagonal Autoregressive Image Generation Paper
  • [Arxiv, 2025.03] Neighboring Autoregressive Modeling for Efficient Visual Generation Paper Code
  • [Arxiv, 2025.03] NFIG: Autoregressive Image Generation with Next-Frequency Prediction Paper
  • [Arxiv, 2025.03] Frequency Autoregressive Image Generation with Continuous Tokens Paper Code
  • [Arxiv, 2025.03] ARINAR: Bi-Level Autoregressive Feature-by-Feature Generative Models Paper Code
  • [Arxiv, 2025.02] Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation Paper Code Project
  • [Arxiv, 2025.02] Fractal Generative Models Paper Code
  • [Arxiv, 2025.01] An Empirical Study of Autoregressive Pre-training from Videos Paper
  • [Arxiv, 2024.12] E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Paper
  • [Arxiv, 2024.12] Taming Scalable Visual Tokenizer for Autoregressive Image Generation Paper Code
  • [Arxiv, 2024.11] Sample- and Parameter-Efficient Auto-Regressive Image Models Paper Code
  • [Arxiv, 2024.01] Scalable Pre-training of Large Autoregressive Image Models Paper Code
  • [Arxiv, 2024.10] ImageFolder: Autoregressive Image Generation with Folded Tokens Paper Code
  • [Arxiv, 2024.10] SAR Customize Your Visual Autoregressive Recipe with Set Autoregressive Modeling Paper Code
  • [Arxiv, 2024.08] AiM Scalable Autoregressive Image Generation with Mamba Paper Code
  • [Arxiv, 2024.06] ARM Autoregressive Pretraining with Mamba in Vision Paper Code
  • [Arxiv, 2024.06] MAR Autoregressive Image Generation without Vector Quantization Paper Code
  • [Arxiv, 2024.06] LlamaGen Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Paper Code
  • [ICML, 2024] DARL: Denoising Autoregressive Representation Learning Paper
  • [ICML, 2024] DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents Paper Code
  • [ICML, 2024] DeLVM: Data-efficient Large Vision Models through Sequential Autoregression Paper Code
  • [AAAI, 2023] SAIM Exploring Stochastic Autoregressive Image Modeling for Visual Representation Paper Code
  • [NeurIPS, 2021] ImageBART: Context with Multinomial Diffusion for Autoregressive Image Synthesis Paper Code
  • [CVPR, 2021] VQGAN Taming Transformers for High-Resolution Image Synthesis Paper Code
  • [ECCV, 2020] RAL: Incorporating Reinforced Adversarial Learning in Autoregressive Image Generation Paper
  • [NeurIPS, 2019] Generating Diverse High-Fidelity Images with VQ-VAE-2 Paper Code
  • [NeurIPS, 2017] VQ-VAE Neural Discrete Representation LearningPaper

• Scale-wise AutoRegressive Generation

  • [Arxiv, 2025.10] Dynamic Mixture-of-Experts for Visual Autoregressive Model Paper
  • [Arxiv, 2025.09] SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Model Paper
  • [Arxiv, 2025.09] Not All Tokens are Guided Equal: Improving Guidance in Visual Autoregressive Models Paper
  • [Arxiv, 2025.09] Scale-Wise VAR is Secretely Discrete Diffusion Paper Code Project
  • [Arxiv, 2025.05] Generative Autoregressive Transformers for Model-Agnostic Federated MRI Reconstruction Paper Code
  • [ICML, 2025] Continuous Visual Autoregressive Generation via Score Maximization Paper Code
  • [Arxiv, 2025.02] FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction Paper Code
  • [Arxiv, 2024.12] FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching Paper Code
  • [Arxiv, 2024.11] M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation Paper Code
  • [NeurIPS 2024 Best Paper] Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction Paper Code

Text-to-Image Generation

• Token-wise Generation

  • [ICML, 2025] Discrete JEPA: Learning Discrete Token Representations without Reconstruction Paper
  • [Arxiv, 2025.04] Lumina-mGPT 2.0: Stand-alone Autoregressive Image Modeling Paper Code
  • [Arxiv, 2025.03] Lumina-Image 2.0: A Unified and Efficient Image Generative Framework Paper Code
  • [Arxiv, 2024.12] Liquid: Language Models are Scalable Multi-modal Generators Paper Code
  • [Arxiv, 2024.12] Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis Paper Code
  • [Arxiv, 2024.12] TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation Paper Code
  • [Arxiv, 2024.11] High-Resolution Image Synthesis via Next-Token Prediction Paper Code
  • [Arxiv, 2024.10] Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens Paper
  • [Arxiv, 2024.10] DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation Paper Code
  • [Arxiv, 2024.10] DnD-Transformer: A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Fine-grained Image Generation Paper Code
  • [Arxiv, 2024.08] Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Paper Code
  • [Arxiv, 2024.07] MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis Paper Code
  • [Arxiv, 2024.06] LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generation Paper Code
  • [Arxiv, 2024.06] STAR: Scale-wise Text-to-image generation via Auto-Regressive representations Paper Code
  • [Arxiv, 2024.05] Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling Paper
  • [CVPR, 2024] Beyond Text: Frozen Large Language Models in Visual Signal Comprehension Paper Code
  • [TOG, 2023] IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers (svg image) Paper Code
  • [NeurIPS, 2023] LQAE Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment Paper Code
  • [TMLR, 2022.06] Parti: Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Paper Code
  • [NeurIPS, 2022] CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers Paper Code
  • [ECCV, 2022] Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Paper
  • [CVPR, 2022] VQ-Diffusion: Vector Quantized Diffusion Model for Text-to-Image Synthesis Paper Code
  • [CVPR, 2022] Make-A-Story: Visual Memory Conditioned Consistent Story Generation (storytelling) Paper
  • [NeurIPS, 2021] CogView: Mastering Text-to-Image Generation via Transformers Paper Code
  • [Arxiv, 2021.02] DALL-E 1: Zero-Shot Text-to-Image Generation Paper

• Scale-wise Generation

  • [Arxiv, 2024.12] Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis Paper Code
  • [Arxiv, 2024.12] SWITTI: Designing Scale-Wise Transformers for Text-to-Image Synthesis Paper Code Page
  • [Arxiv, 2024.10] HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Paper Code
  • [Arxiv, 2024.08] VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modeling Paper Code
  • [Arxiv, 2024.06] STAR: Scale-wise Text-to-image generation via Auto-Regressive representations Paper Code

Image-to-Image Translation

• [ICCV, 2025] CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation Paper
• [ICML Workshop, 2024] MIS Many-to-many Image Generation with Auto-regressive Diffusion Models Paper
• [Arxiv, 2024.03] SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model Paper Project
• [CVPR, 2024] Sequential modeling enables scalable learning for large vision models Paper
• [ECCV, 2022] QueryOTR: Outpainting by Queries Paper Code
• [NeurIPS, 2022] Visual prompting via image inpainting Paper
• [MM, 2021] Diverse image inpainting with bidirectional and autoregressive transformers Paper

Image Editing

• [Arxiv, 2025.09] Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing Paper
• [Arxiv, 2025.08] Visual Autoregressive Modeling for Instruction-Guided Image Editing Paper Code
• [Arxiv, 2025.08] NEP: Autoregressive Image Editing via Next Editing Token Prediction Paper Code
• [Arxiv, 2025.07] SCALAR: Scale-wise Controllable Visual Autoregressive Learning Paper
• [Arxiv, 2025.04] Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing Paper Code
• [ICCV, 2025] Training-Free Text-Guided Image Editing with Visual Autoregressive Model Paper Code
• [Arxiv, 2025.01] EditAR: Unified Conditional Generation with Autoregressive Models Paper Code
• [Arxiv, 2024,06] CAR: Controllable Autoregressive Modeling for Visual Generation Paper Code
• [ICLR, 2025] ControlAR: Controllable Image Generation with Autoregressive Models Paper Code
• [Arxiv, 2024,06] ControlVAR: Exploring Controllable Visual Autoregressive Modeling Paper Code
• [Arxiv, 2024,06] Medical Vision Generalist: Unifying Medical Imaging Tasks in Context Paper
• [Arxiv, 2024,04] M2M Many-to-many Image Generation with Auto-regressive Diffusion Models Paper
• [ECCV, 2022] VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance Paper
• [ECCV, 2022] Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Paper
• [ICIP, 2021] MSGNet: Generating annotated high-fidelity images containing multiple coherent objects Paper

Video Generation

Unconditional Video Generation

• [Arxiv, 2025.03] FAR: Frame Autoregressive Model for Both Short- and Long-Context Video Modeling Paper Code
• [Arxiv, 2025.03] HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models Paper
• [Arxiv, 2025.03] AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion Paper Code
• [Arxiv, 2025.02] Next Block Prediction: Video Generation via Semi-Autoregressive Modeling Paper Code
• [Arxiv, 2025.01] Taming Teacher Forcing for Masked Autoregressive Video Generation Paper Code
• [Arxiv, 2024.10] LARP: Tokenizing Videos with a Learned Autoregressive Generative Prior Paper
• [ECCV 2024] ST-LLM: Large Language Models Are Effective Temporal Learners Paper Code
• [ICLR, 2024] MAGVIT-v2 Language Model Beats Diffusion – Tokenizer is Key to Visual Generation Paper
• [CVPR, 2023] PVDM Video Probabilistic Diffusion Models in Projected Latent Space Paper
• [ECCV, 2022] Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer Paper Code
• [Arxiv, 2021.04] VideoGPT: Video generation using VQ-VAE and transformers Paper
• [Arxiv, 2020.06] Latent Video Transformer Paper Code
• [ICLR, 2020] Scaling Autoregressive Video Models Paper
• [CVPR, 2018] MoCoGAN: Decomposing Motion and Content for Video Generation Paper Code
• [ICML, 2017] Video Pixel Networks Paper

Conditional Video Generation

• Text-to-Video Generation

  • [Arxiv, 2025.10] Uniform Discrete Diffusion with Metric Path for Video Generation Paper Code Page
  • [Arxiv, 2025.10] Autoregressive Video Generation beyond Next Frames Prediction Paper
  • [Arxiv, 2025.07] Lumos-1: On Autoregressive Video Generation from a Unified Model Perspective Paper Code
  • [Arxiv, 2025.05] Generative Pre-trained Autoregressive Diffusion Transformer Paper
  • [Arxiv, 2024.12] DiCoDe: Diffusion-Compressed Deep Tokens for Autoregressive Video Generation with Language Models Paper Page
  • [Arxiv, 2024.11] Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing Paper Code
  • [Arxiv, 2024.10] ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation Paper Code
  • [Arxiv, 2024.10] Progressive Autoregressive Video Diffusion Models Paper Code
  • [Arxiv, 2024.10] Pyramid Flow: Pyramidal Flow Matching for Efficient Video Generative Modeling Paper Code
  • [Arxiv, 2024.10] Loong: Generating Minute-level Long Videos with Autoregressive Language Models Paper
  • [Arxiv, 2024.06] Pandora: Towards General World Model with Natural Language Actions and Video States Paper Code
  • [Arxiv, 2024.06] iVideoGPT: Interactive VideoGPTs are Scalable World Models Paper Code
  • [Arxiv, 2024.06] ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models Paper Code
  • [Arxiv, 2024.02] LWM World Model on Million-Length Video And Language With Blockwise RingAttention Paper Code
  • [CVPR, 2024] ART-V: Auto-Regressive Text-to-Video Generation with Diffusion Models Paper
  • [NeurIPS, 2022] NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis Paper Code
  • [ECCV, 2022] NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion Paper Code
  • [Arxiv, 2022.05] CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Paper Code
  • [Arxiv, 2022.05] GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions Paper
  • [IJCAI, 2021] IRC-GAN: Introspective Recurrent Convolutional GAN for Text-to-video Generation. Paper

• Visual Conditional Video Generation

  • [Arxiv, 2025.06] VideoMAR: Autoregressive Video Generation with Continuous Tokens Paper Code
  • [Arxiv, 2025.06] DeepVerse: 4D Autoregressive Video Generation as a World Model Paper Code
  • [Arxiv, 2025.05] Video-GPT via Next Clip Diffusion Paper Code
  • [Arxiv, 2025.04] GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography Paper Code
  • [Arxiv, 2024.10] MarDini: Masked Autoregressive Diffusion for Video Generation at Scale Paper
  • [CVPR, 2024] LVM Sequential Modeling Enables Scalable Learning for Large Vision Models Paper Code
  • [ICIP, 2022] HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator Paper
  • [Arxiv, 2021.03] Predicting Video with VQVAE Paper
  • [CVPR, 2021] Stochastic Image-to-Video Synthesis using cINNs Paper Code
  • [ICLR, 2019] Eidetic 3d lstm: A model for video prediction and beyond Paper
  • [ICLR, 2018] Stochastic variational video prediction Paper
  • [NeurIPS, 2017] Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms Paper
  • [NeurIPS, 2015] Convolutional LSTM network: A machine learning approach for precipitation nowcasting Paper

• Multimodal Conditional Video Generation

  • [Arxiv, 2025.01] VideoAuteur: Towards Long Narrative Video Generation Paper Code
  • [Arxiv, 2024.12] Autoregressive Video Generation without Vector Quantization Paper Code
  • [ICML, 2024] Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Paper Code
  • [ICML, 2024] VideoPoet: A Large Language Model for Zero-Shot Video Generation Paper
  • [CVPR, 2023] MAGVIT: Masked Generative Video Transformer Paper
  • [CVPR, 2022] Make it move: controllable image-to-video generation with text descriptions Paper Code

AutoRegressive Diffusion-Forcing Video Generation

• [Arxiv, 2025.10] Real-Time Motion-Controllable Autoregressive Video Diffusion Paper Code
• [Arxiv, 2025.10] Self-Forcing++: Towards Minute-Scale High-Quality Video Generation Paper Code
• [Arxiv, 2025.10] Pack and Force Your Memory: Long-form and Consistent Video Generation Paper Code
• [Arxiv, 2025.09] LongLive: Real-time Interactive Long Video Generation Paper Code
• [Arxiv, 2025.09] Rolling Forcing: Autoregressive Long Video Diffusion in Real Time Paper Page Code
• [Arxiv, 2025.08] MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generation Paper Project
• [Arxiv, 2025.08] Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation Paper Project Code
• [Arxiv, 2025.05] Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion Paper Code
• [Arxiv, 2025.04] MAGI-1: Autoregressive Video Generation at Scale Paper Code
• [Arxiv, 2025.04] SkyReels-V2: Infinite-length Film Generative Model Paper Code
• [Arxiv, 2025.04] Packing Input Frame Context in Next-Frame Prediction Models for Video Generation Paper Code
• [CVPR, 2025] AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion Paper Code
• [CVPR, 2025] From Slow Bidirectional to Fast Autoregressive Video Diffusion Models Paper Code
• [NeurIPS, 2024] FIFO-Diffusion: Generating Infinite Videos from Text without Training Paper Code

Embodied AI

• [Arxiv, 2025.03] HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Paper Code
• [Arxiv, 2024.12] Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression Paper Page
• [Arxiv, 2024.10] Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation Paper
• [Arxiv, 2024.05] iVideoGPT: Interactive VideoGPTs are Scalable World Models Paper
• [ICML, 2024] Genie: Generative interactive environments Paper
• [ICLR, 2024] GR-1 Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation Paper
• [ICLR, 2023] IRIS Transformers are sample-efficient world models Paper

3D Generation

Motion Generation

• [Arxiv, 2025.06] Auto-Regressive Surface Cutting Paper Code
• [CVPR, 2025] Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation Paper Page
• [CVPR, 2025] ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model Paper Code
• [AAAI, 2024] AMD: Autoregressive Motion Diffusion Paper Code
• [ECCV, 2024] BAMM: Bidirectional Autoregressive Motion Model Paper Code
• [CVPR, 2023] T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations Paper
• [Arxiv, 2022] HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE Paper
• [ICCV, 2021 oral] HuMoR: 3D Human Motion Model for Robust Pose Estimation Paper Code

Point Cloud Generation

• [Arxiv, 2025.06] ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding Paper Code
• [Siggraph, 2025] OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation Paper Code
• [Arxiv, 2025.04] Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenization Paper Page
• [CVPR, 2025] TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencing Paper Code
• [Arxiv, 2025] 3D Point Cloud Generation via Autoregressive Up-sampling Paper
• [Arxiv, 2024.02] Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability Paper
• [ECCV, 2022] Autoregressive 3D Shape Generation via Canonical Mapping Paper
• [CVPR workshop, 2023] Octree transformer: Autoregressive 3d shape generation on hierarchically structured sequences Paper

3D Medical Generation

• [Arxiv, 2024] Autoregressive Sequence Modeling for 3D Medical Image Representation Paper
• [Arxiv, 2024] Medical Vision Generalist: Unifying Medical Imaging Tasks in Context Paper Code
• [MIDL, 2024] Conditional Generation of 3D Brain Tumor ROIs via VQGAN and Temporal-Agnostic Masked Transformer Paper
• [NMI, 2024] Realistic morphology-preserving generative modelling of the brain Paper Code
• [Arxiv, 2023] Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks Paper
• [ICCV, 2023] Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers Paper Code
• [MICCAI, 2022] Morphology-preserving Autoregressive 3D Generative Modelling of the Brain Paper Code

Multimodal Generation

Unified Understanding and Generation Multi-Modal LLMs

• [NeurIPS, 2025] JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generation Paper
• [Arxiv, 2025.10] Wave-Particle (Continuous-Discrete) Dualistic Visual Tokenization for Unified Understanding and Generation Paper Code
• [Arxiv, 2025.10] NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching Paper
• [Arxiv, 2025.10] Emu3.5: Native Multimodal Models are World Learners Paper Code
• [Arxiv, 2025.10] PairUni: Pairwise Training for Unified Multimodal Language Models Paper Code
• [Arxiv. 2025.10] LightBagel: A Light-weighted, Double Fusion Framework for Unified Multimodal Understanding and Generation Paper Page
• [Arxiv. 2025.10] Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation Paper Code
• [Arxiv, 2025.10] SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models Paper Code
• [Arxiv, 2025.10] UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generation Paper Code
• [Arxiv, 2025.10] UniVideo Unified Understanding, Generation, and Editing for Videos Paper Page
• [Arxiv, 2025.10] Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer Paper Code
• [Arxiv, 2025.10] Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding Paper Code Page
• [Arxiv, 2025.09] Query-Kontext: An Unified Multimodal Model for Image Generation and Editing Paper
• [Arxiv, 2025.09] Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation Paper
• [Arxiv, 2025.09] MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer Paper
• [Arxiv, 2025.09] RecA: Reconstruction Alignment Improves Unified Multimodal Models Paper Code
• [Arxiv, 2025.09] Interleaving Reasoning for Better Text-to-Image Generation Paper Code
• [Arxiv, 2025.09] OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation Paper Code Page
• [Arxiv, 2025.08] TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuning Paper Code
• [Arxiv, 2025.08] Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents Paper Code
• [Arxiv, 2025.08] Uni-COT: Towards Unified Chain-of-Thought Reasoning Across Text and Vision Paper Code
• [Arxiv, 2025.08] UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying Paper
• [Arxiv, 2025.08] Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation Paper Code
• [Arxiv, 2025.07] Omni-Video: Democratizing Unified Video Understanding and Generation Paper Code
• [Arxiv, 2025.07] Ovis-U1 Technical Report Paper Code
• [Qwen, 2025.07] Qwen VLo: From “Understanding” the World to “Depicting” It Blog
• [ICCV, 2025] USP: Unified Self-Supervised Pretraining for Image Generation and Understanding Paper Code
• [Arxiv, 2025.06] UniCode²: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation Paper
• [Arxiv, 2025.06] OmniGen2: Exploration to Advanced Multimodal Generation Paper Code
• [Arxiv, 2025.06] Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations Paper Code
• [Arxiv, 2025.06] UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation Paper Code
• [Arxiv, 2025.06] Show-o2: Improved Native Unified Multimodal Models Paper Code
• [Arxiv, 2025.06] Ming-Omni: A Unified Multimodal Model for Perception and Generation Paper Code
• [Arxiv, 2025.06] Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation Paper
• [Arxiv, 2025.06] UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation Paper Code
• [Arxiv, 2025.06] ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding Paper Code
• [Arxiv, 2025.05] Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model Paper Code
• [Arxiv, 2025.05] OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generation Paper Code
• [Arxiv, 2025.05] FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities Paper
• [Arxiv, 2025.05] MMaDA: Multimodal Large Diffusion Language Models Paper Code
• [Arxiv, 2025.05] Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction Paper Code
• [Arxiv, 2025.05] Emerging Properties in Unified Multimodal Pretraining Paper Code
• [Arxiv, 2025.05] BLIP3-o: A Family of Fully Open Unified Multimodal Models—Architecture, Training and Dataset Paper Code
• [Arxiv, 2025.05] Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning Paper Project
• [Arxiv, 2025.05] Nexus-Gen: A Unified Model for Image Understanding, Generation, and Editing Paper Code
• [Arxiv, 2025.05] TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation Paper Code
• [Arxiv, 2025.05] Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation Paper
• [Arxiv, 2025.04] VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning Paper Code
• [Arxiv, 2025.04] ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement Paper Code
• [OpenAI, 2025.03] Addendum to GPT-4o System Card: Native image generation Paper
• [Arxiv, 2025.03] Harmonizing Visual Representations for Unified Multimodal Understanding and Generation Paper Code
• [Arxiv, 2025.03] Unified Autoregressive Visual Generation and Understanding with Continuous Tokens Paper
• [Arxiv, 2025.03] DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies Paper
• [Arxiv, 2025.03] OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models Paper Code
• [Arxiv, 2025.02] UniTok: A Unified Tokenizer for Visual Generation and Understanding Paper Code
• [Arxiv, 2025.02] HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation Paper Code
• [Arxiv, 2025.02] QLIP: Text-Aligned Visual Tokenization Unifies Auto-Regressive Multimodal Understanding and Generation Paper Code
• [Arxiv, 2025.01] Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling Paper Code
• [Arxiv, 2025.01] VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model Paper Code
• [Arxiv, 2024.12] LlamaFusion: Adapting Pretrained Lang

README 内容过长，已截取前 64 KB。完整内容请前往 GitHub 仓库查看。