Global inducing points implementation #50
Conversation
sebastianober
changed the title
There was a problem hiding this comment.
This is a good draft PR. Left some comments and questions. Would be good to have a notebook to see how to use the code. A few tests (as discussed in person) will also be required. I also think the formatting is off in some places; make sure to run make format before pushing.
| class GIGPLayer(tf.keras.layers.Layer): | ||
| """ | ||
| A sparse variational multioutput GP layer. This layer holds the kernel, | ||
| inducing variables and variational distribution, and mean function. |
There was a problem hiding this comment.
| inducing variables and variational distribution, and mean function. | |
| inducing variables, variational distribution, and mean function. |
| Calculates the log probability of a zero-mean multivariate Gaussian with covariance sigma | ||
| and evaluation points X, with batching of both the covariance and X. | ||
|
|
||
| TODO: look into whether this can be replaced with a tfp.distributions.Distribution |
There was a problem hiding this comment.
yes, this should be possible and I would advice doing that.
| from gpflux.layers import LayerWithObservations, SampleBasedGaussianLikelihoodLayer | ||
|
|
||
|
|
||
| class GIDeepGP(Module): |
There was a problem hiding this comment.
There appears to be a lot of code duplication from DeepGP. Can you not inherit from DeepGP and overwrite the appropriate methods?
| """ | ||
| mean_function = self.mean_function(inputs) | ||
|
|
||
| Kuu = self.kernel(inputs[..., :self.num_inducing, :]) |
There was a problem hiding this comment.
Can you give some information in the form of comments how inputs is structured. What exactly are the inputs and outputs of call?
| """ | ||
| Samples function values f based off samples of u. | ||
|
|
||
| :param u: Samples of the inducing points, shape [S, Lout, M, 1] |
Quick implementation of "global inducing" approximate posteriors for basic DGPs (see Ober and Aitchison (2021): https://arxiv.org/abs/2005.08140). TODOs: