MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP graduate students, faculty members, and corporate researchers. The community’s vision is to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners. Reprinted from | RUC AI Box Author | Zhang Gaowei Institution | Renmin University of China, Gaoling School of Artificial Intelligence This article selects over 100 papers related to diffusion models presented at ICLR 2023 and categorizes them according to different research themes for reference. The article is also published in the AI Box Zhihu column (search for AI Box column on Zhihu), and everyone is welcome to comment and discuss below the articles in the Zhihu column! Introduction: ICLR is one of the top conferences in the field of artificial intelligence, covering topics including deep learning, statistics, and data science, as well as important applications such as computer vision, computational biology, speech recognition, text understanding, gaming, and robotics. ICLR 2023 will be held from May 1 to May 5, 2023, in Kigali, Rwanda. The official list of accepted papers has not yet been released, but from the submitted papers, diffusion models remain a hot topic with a high frequency and average rating. This article selects over 100 papers related to diffusion models and categorizes them according to different research themes for reference. The open review link for ICLR 2023 submission papers is as follows: https://openreview.net/group?id=ICLR.cc/2023/Conference

1

『Efficient Sampling』

• Dynamic Scheduled Sampling with Imitation Loss for Neural Text Generation
• Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders
• Denoising Diffusion Samplers
• Denoising MCMC for Accelerating Diffusion-Based Generative Models
• DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
• Quasi-Taylor Samplers for Diffusion Generative Models based on Ideal Derivatives
• Fast Sampling of Diffusion Models with Exponential Integrator
• Accelerating Guided Diffusion Sampling with Splitting Numerical Methods
• Boomerang: Local sampling on image manifolds using diffusion models
• Markup-to-Image Diffusion Models with Scheduled Sampling

2

『Combination with Other Generative Models』

• Diffusion-GAN: Training GANs with Diffusion
• in Conversation based on offline reinforcement learning
• FastDiff 2: Dually Incorporating GANs into Diffusion Models for High-Quality Speech Synthesis
• Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
• Geometric Networks Induced by Energy Constrained Diffusion
• Progressive Image Synthesis from Semantics to Details with Denoising Diffusion GAN
• Flow Matching for Generative Modeling
• SPI-GAN: Denoising Diffusion GANs with Straight-Path Interpolations
• Building Normalizing Flows with Stochastic Interpolants
• Guiding Energy-based Models via Contrastive Latent Variables
• Your Denoising Implicit Model is a Sub-optimal Ensemble of Denoising Predictions
• Thinking fourth dimensionally: Treating Time as a Random Variable in EBMs

3

『Applications in CV and NLP』

• Novel View Synthesis with Diffusion Models
• Pyramidal Denoising Diffusion Probabilistic Models
• Compositional Image Generation and Manipulation with Latent Diffusion Models
• Towards the Detection of Diffusion Model Deepfakes
• DifFace: Blind Face Restoration with Diffused Error Contraction
• Restoration based Generative Models
• Generative Modelling with Inverse Heat Dissipation
• Deep Watermarks for Attributing Generative Models
• Learning multi-scale local conditional probability models of images
• Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules
• Self-conditioned Embedding Diffusion for Text Generation
• Sequence to sequence text generation with diffusion models
• DiffusER: Diffusion via Edit-based Reconstruction
• SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation
• Universal Speech Enhancement with Score-based Diffusion
• Score-based Generative 3D Mesh Modeling
• CAN: A simple, efficient and scalable contrastive masked autoencoder framework for learning visual representations
• Neural Volumetric Mesh Generator
• SketchKnitter: Vectorized Sketch Generation with Diffusion Models
• $DDM^2$: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Models
• Neural Image Compression with a Diffusion-based Decoder
• Lossy Compression with Gaussian Diffusion
• Distilling Model Failures as Directions in Latent Space
• Lossy Image Compression with Conditional Diffusion Models
• Quantized Compressed Sensing with Score-Based Generative Models
• Out-of-distribution Detection with Diffusion-based Neighborhood

4

『Applications in Multimodal Domains』

• DreamFusion: Text-to-3D using 2D Diffusion
• Diffusion-based Image Translation using disentangled style and content representation
• CUSTOMIZING PRE-TRAINED DIFFUSION MODELS FOR YOUR OWN DATA
• Human Motion Diffusion Model
• Prosody-TTS: Self-Supervised Prosody Pretraining with Latent Diffusion For Text-to-Speech
• Text-Guided Diffusion Image Style Transfer with Contrastive Loss Fine-tuning
• Meta-Learning via Classifier(-free) Guidance
• KNN-Diffusion: Image Generation via Large-Scale Retrieval
• DiffEdit: Diffusion-based semantic image editing with mask guidance
• Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis
• Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
• Unified Discrete Diffusion for Simultaneous Vision-Language Generation
• ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech
• Re-Imagen: Retrieval-Augmented Text-to-Image Generator
• Prompt-to-Prompt Image Editing with Cross-Attention Control

5

『Combination with Reinforcement Learning』

• Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
• Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
• Provably Efficient Reinforcement Learning for Online Adaptive Influence Maximization
• Variational Reparametrized Policy Learning with Differentiable Physics
• Is Conditional Generative Modeling all you need for Decision Making?

6

『Molecular Graph Modeling』

• Diffusion Probabilistic Modeling of Protein Backbones in 3D for the motif-scaffolding problem
• Protein structure generation via folding diffusion
• Pre-training Protein Structure Encoder via Siamese Diffusion Trajectory Prediction
• DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking
• Pocket-specific 3D Molecule Generation by Fragment-based Autoregressive Diffusion Models
• Equivariant 3D-Conditional Diffusion Models for Molecular Linker Design
• Structure-based Drug Design with Equivariant Diffusion Models
• Equivariant Energy-Guided SDE for Inverse Molecular Design
• 3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction
• Exploring Chemical Space with Score-based Out-of-distribution Generation
• Protein Sequence and Structure Co-Design with Equivariant Translation

7

『Understanding and Theorizing Diffusion Models』

• Information-Theoretic Diffusion
• Analyzing diffusion as serial reproduction
• Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
• Diffusion Models Already Have A Semantic Latent Space
• Unifying Diffusion Models’ Latent Space, with Applications to CycleDiffusion and Guidance
• Understanding DDPM Latent Codes Through Optimal Transport
• Interpreting Neural Networks Through the Lens of Heat Flow
• gDDIM: Generalized denoising diffusion implicit models

8

『Generalization and Extension of Diffusion Models』

• Soft Diffusion: Score Matching For General Corruptions
• Where to Diffuse, How to Diffuse and How to get back: Learning in Multivariate Diffusions
• Blurring Diffusion Models
• Diffusion Probabilistic Fields
• Neural Diffusion Processes
• Pseudoinverse-Guided Diffusion Models for Inverse Problems
• Removing Structured Noise with Diffusion Models
• f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation
• Iterative α-(de)Blending: Learning a Deterministic Mapping Between Arbitrary Densities
• Score-Based Graph Generative Modeling with Self-Guided Latent Diffusion
• Self-Guided Diffusion Models
• From Points to Functions: Infinite-dimensional Representations in Diffusion Models
• Score Matching via Differentiable Physics
• Approximated Anomalous Diffusion: Gaussian Mixture Score-based Generative Models
• Action Matching: A Variational Method for Learning Stochastic Dynamics from Samples
• Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation
• DIFFUSION GENERATIVE MODELS ON SO(3)
• Diffusion Posterior Sampling for General Noisy Inverse Problems

9

『Transfer of Diffusion Models』

• Transferring Pretrained Diffusion Probabilistic Models
• Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
• Dual-Domain Diffusion Based Progressive Style Rendering towards Semantic Structure Preservation
• Dual Diffusion Implicit Bridges for Image-to-Image Translation
• Learning to Learn with Generative Models of Neural Network Checkpoints
• Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model

10

『Modeling Special Structured Data』

• Autoregressive Diffusion Model for Graph Generation
• Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
• TabDDPM: Modelling Tabular Data with Diffusion Models
• ChiroDiff: Modelling chirographic data with Diffusion Models
• Modeling Temporal Data as Continuous Functions with Process Diffusion
• Domain Specific Denoising Diffusion Probabilistic Models for Brain Dynamics
• Discrete Predictor-Corrector Diffusion Models for Image Synthesis
• Diffusion-based point cloud generation with smoothness constraints
• Computational Doob h-transforms for Online Filtering of Discretely Observed Diffusions
• Imitating Human Behaviour with Diffusion Models
• Learning Diffusion Bridges on Constrained Domains
• DiGress: Discrete Denoising diffusion for graph generation
• Score-based Continuous-time Discrete Diffusion Models
• Brain Signal Generation and Data Augmentation with a Single-Step Diffusion Probabilistic Model

11

『Robustness and Stability』

• DensePure: Understanding Diffusion Models towards Adversarial Robustness
• Defending against Adversarial Audio via Diffusion Model
• PointDP: Diffusion-driven Purification against 3D Adversarial Point Clouds
• Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation
• Improving Adversarial Robustness by Contrastive Guided Diffusion Process
• Robustness for Free: Adversarially Robust Anomaly Detection Through Diffusion Model
• (Certified!!) Adversarial Robustness for Free!
• The Biased Artist: Exploiting Cultural Biases via Homoglyphs in Text-Guided Image Generation Models
• Expected Perturbation Scores for Adversarial Detection
• Input Perturbation Reduces Exposure Bias in Diffusion Models
• Stable Target Field for Reduced Variance Score Estimation

12

『Privacy Protection of Diffusion Models』

• Membership Inference Attacks Against Text-to-image Generation Models
• Differentially Private Diffusion Models

13

『Other Directions』

• OCD: Learning to Overfit with Conditional Diffusion Models
• Denoising Diffusion Error Correction Codes
• Neural Lagrangian Schrodinger Bridge: Diffusion Modeling for Population Dynamics
• Diffusion Models for Causal Discovery via Topological Ordering
• Transport with Support: Data-Conditional Diffusion Bridges
• A Score-Based Model for Learning Neural Wavefunctions

Technical Group Invitation

△ Long press to add the assistant

Scan the QR code to add the assistant’s WeChat

Please note: Name – School/Company – Research Direction(e.g., Xiaozhang – Harbin Institute of Technology – Dialogue Systems) to apply to join the Natural Language Processing/PyTorch technical group

About Us

MLNLP community is a grassroots academic community built by scholars in machine learning and natural language processing from home and abroad. It has developed into a well-known machine learning and natural language processing community, aiming to promote progress among the academic and industrial sectors of machine learning and natural language processing, as well as enthusiasts. The community can provide an open communication platform for practitioners’ further education, employment, and research. Everyone is welcome to follow and join us.