Suppose we have a scalar-valued stochastic process for and , where describes our physical domain (spatial + temporal) and describes the sample space. In other words, suppose for some symmetric positive definite operator and for some . Making some relatively mild assumptions about , i.e., that is compact and bounded, we can see the KL expansion as

where we can define

Then, by construction,

which simply comes from the orthonormality of . Suppose further that is a Gaussian process (GP) (i.e., any collection of gives s.t. are jointly Gaussian). Then, one can prove that , meaning we can easily sample from given the eigenvalues and eigenfunctions (see the Nystrom method).

It suffices to show that are jointly Gaussian (identity covariance and zero mean give the exact shape). This loosely comes from the fact that if you use a convergent scheme to converge the integral in ^b94337 , each step will be a sum of Gaussian variables, which itself is Gaussian.

The problem arises when is not a GP; we of course maintain zero mean and identity covariance, but the higher moments will become nontrivial (and thus the joint distribution of nongaussian). We can express the covariance function as

and consider the term truncation of such a series. We can express as the leading eigenfunction of . I believe, though obviously this isn't rigorous, that this implies a certain "triangular" nature of , in that they are well-ordered in the Knothe-Rosenblatt sense.

Suppose we, for the time being, can perform the integrals in ^992e28,^b94337 analytically (or, at the very worst, "close enough"), and thus can find the eigenfunctions and -values. Therefore, for a given set of samples of the stochastic process, we can obtain samples with where is our truncation level of the KL expansion and for some unknown distribution . We can calculate an acceptable before any discussion of how to describe using samples; the acceptable truncation error exactly comes from the decay of the covariance kernel spectrum.

If we could sample exactly, we could create samples of "easily" and "cheaply". The problem comes down to approximating the distribution of . In practice, we generally have a stochastic process where is a stochastic input to the process (consider, e.g., random diffusivity or initial condition, where is the solution to a PDE). More on this in On stochastic inputs.

Suppose, though, we only have access to samples of the solution (and no knowledge/desire for any stochastic inputs). Then, we create an invertible map such that , i.e., we find a transport map where . If we can find such a map from the samples , then we can sample from via the relationship for !

From there, we can sample new approximate solutions

If we truly have , where is a deterministic function of and random variable following distribution , then there must exist a deterministic mapping , since must then be deterministic for any given realization of . In a distributional sense, we could create a map such that the operator satisfies . This is what is traditionally done in our previous triangular transport work. However, this is not well-posed. We know for certain that is a degenerate distribution! There's only one possible mapping from a choice of input to a given KL sample .

What to do? A few options, I suppose.

1. Find a map such that . Note that this is ill-posed; the measures can be equivalent even when the samples get transported poorly. For example, suppose the input is a coin-flip on 0 1 and the output is a coin flip on 1 2. There's two ways of transporting these distributions, but one of them will guarantee you're always wrong.
2. Find the map as above such that . This totally disregards interpolation and just matches points (though I suppose, with enough parameters, you could pass overfitting?)
3. Use something like entropically-regularized OT? Idk enough to say much. Would I need the input/output to match?
4. Suppose that is in fact still stochastic for fixed , i.e., doesn't capture all of the randomness. Then, the approach described by is perfectly well-posed, I think?