On Guided and Reversible Solvers for Neural Differential Equations
Zander W. Blasingame, Ph.D. Candidate
Abstract: Neural differential equations have quickly become a powerful framework for modelling state-of-the-art generative models, in particular flow/diffusion models. These models work by modeling an ordinary differential equation (ODE) or stochastic differential equation (SDE) which transports samples from a source distribution (e.g., the unit Gaussian) to a desired target distribution (the data distribution). We explore two primary directions within this framework, namely guided generation and reversible solvers.
Guided generation. We study how to guide the generative process of neural differential equations via gradient-based guidance. First, we explore optimize-then-discretize methods which create a separate system of equations known as the continuous adjoint equations which model the continuous-time dynamics of the gradients. We develop bespoke ODE/SDE solvers for such continuous adjoint equations for affine conditional flow models and diffusion SDEs. Secondly, we explore a greedy strategy on guidance enabling greater computational efficiency. We show that this method can be viewed as a unifying view between end-to-end optimization techniques for guidance and posterior sampling guidance techniques.
Reversible solvers. We study how to construct algebraically reversible numerical solvers for neural ODEs/SDEs which guarantee the exact inversion of samples back to the source distribution. We then develop a bespoke reversible solver for diffusion SDEs which does not require storing the entire realization of Brownian motion in memory by taking a rough paths view of SDEs using Brownian Intervals. This enables us to invert diffusion SDEs for image editing (and other) applications.
We provide numerous theoretical results to explore the topics above and perform some numerical experiments within the topic of face morphing.
Date: April 21, 2025
Time: 09:00 — 11:00
Location: SC 166
Advisor: Dr. Chen Liu
Committee members: Dr. Joseph Skufca, Dr. Christino Tamon, Dr. Faraz Hussain, and Dr. Mahesh Banavar