Conditional Diffusion Model Acceleration with First-Demonstrated RRAM-Based In-Memory Neural Differential Equation Solver

Diffusion models generate high-quality images and videos, closely mirroring the imagination of human brain. Specifically, score-based diffusion models generate by solving neural differential equations. However, their digital computer implementations are discrete in time and inherently digital, with energy efficiency constrained by the von Neumann architecture. Herein, we firstly demonstrate a chip-level solution that embodies the implementation of time-continuous and analog conditional score-based diffusion using a Resistive Random Access Memory (RRAM) in-memory neural differential equation solver. Notably, the score-based diffusion process is intrinsically robust to analog computing noise. Under equivalent distribution generation quality, our system achieves a sampling time of 0.02ms (software baseline 3.0 ms) and energy consumption of 7.2μJ (software baseline 22.5μJ) per sample, denoting ×150 enhancement on generation speed and 68% reduction on energy consumption. We validate our solution on a two-dimensional conditional diffusion task. Our in-memory neural differential equation solver opens up a brand-new hardware solution for edge generative AI.