Repository navigation
Approximation of the marginal GP #39
Description
A very cool addition would be the approximation of the marginal GP presented in https://arxiv.org/abs/1310.6740. This approach requires both the gradient as well as the Hessian of the lilkelihood w.r.t its hyperparameters which should be doable with tensorflow. The hessian however is currently a problem with r1.2 of tensorflow, as the cholesky_grad has no gradient op. I saw that in the rc for r1.3, the cholesky_grad is no longer the default gradient op for tf.cholesky, rather it is computed using tensorflow ops. I quickly tested this, the following code now runs:
import tensorflow as tf
import numpy as np
A = tf.Variable(np.zeros((10,)), dtype=tf.float32)
with tf.Session() as sess:
Ae = tf.expand_dims(A, -1)
Xs = tf.matmul(Ae, Ae, transpose_b=True) + 1e-2 * tf.eye(10)
X = tf.cholesky(Xs)
Xg = tf.hessians(X, A)
sess.run(tf.global_variables_initializer())
print (sess.run(Xg, feed_dict={A: np.random.rand(10)}))So we can try this again, @nknudde could you look at this?
Activity
gpfins commentedon Jul 22, 2017
I will definitely look at this!
gpfins commentedon Jul 22, 2017
I'm putting it together in this branch https://github.com/nknudde/GPflowOpt/tree/mgp. I made a rough version, but still needs some cleaning up and in particular the prediction code needs some looking into. Right now I evaluate for every point individually to circumvent the collocation of gradients in tf.gradients.
javdrher commentedon Jul 24, 2017
Lots of progress there, looking very good! Thanks a lot for the effort.
Some thoughts:
gpfins commentedon Aug 6, 2017
Here I am again.
I wrote some tests and now it also support multi-output GPs. I think the notebook would best be made after the datascaler reform
javdrher commentedon Sep 11, 2017
Postponed until we manage to find a way to accomplish this in GPflow, some of the changes planned might facilitate this.