Sampling via Annealed controlled dynamics
This builds on Ben's paper [1]
Suppose we have a reference and target density satisfying for some for all .
For Brownian motion , consider the Ornstein-Uhlenbeck (OU) process, defined by the SDE
This defines a Markov generator for the OU process , which has value
As Ben says, "this is a linear operator that acts on the space of twice-continuously differentiable functions and describes the evolution of expectations of the SDE through the Kolmogorov backward equation (KBE)". We define the conditional expectation operator
, which allows us to write the KBE as
where the latter term is the boundary condition in time. Expanding , we get
As opposed to diffusion samplers, we may want to use control to sample according to the following SDE at finite terminal time ,
The control that achieves this is the Doob -transform (why ??). Further, if everywhere and solves the time- KBE for ^ab1843, we can use the following control
which will give a sample distributed according to for initial condition . Therefore, if we set and sample , we will get that satisfies .
- Bayesian perspective: Suppose we have prior and likelihood ; we would like to sample from posterior . Therefore, one can set and get samples from the posterior .
- If is large, then is unfortunately going to be ill-conditioned, as there must be large regions where dominates , i.e., they place their concentrations in vastly different locations. Ideally, one would choose a reference to be as close to as possible, and then everything falls into place from there.
- The full SDE would be which looks like the reverse diffusion in score matching, except that we've chosen a control that gets us to the reference in finite time. If we choose for some particular covariance then we can reparameterize the SDE with the eigendecomposition and to get
- As alluded to by Panos, Aimee, and Max, you probably don't get "something" from "nothing" here. The time schedule is probably going to hamper you. However, you are instead working with Gaussian convolutions, which might help you!
- Obviously, the problem here is solving the KBE! If we knew the solution, then everything would be great.
OU Process
As seen in Ben's other paper on OU processes, the KBE on the OU process induces an analytical set of eigenfunctions. Suppose we have the left eigenvectors/values of as , i.e., with . Then, the OU process will have eigenvalues and eigenfunctions defined for each multi-index as
which is easy to verify by setting each and checking the solution.