[theano-users] Theano custom Op: difference between perform() and grad()

[Jørgen Eriksson Midtbø]

Hi,

I'm not sure if this forum is meant for user questions, but I'll try my
luck as I'm getting a little desperate for help. My question is probably
quite basic, but I'm having such trouble understanding Theano.

I am trying to use PyMC3 to optimize a chisquare function, using HMC/NUTS.
My chisquare calculation involves calls to an external program. Therefore,
during each chisquare evaluation, I need to write the current parameters to
file and call the external program. Following the PyMC3 docs, I have tried
to implement this using a custom Theano Op:
http://docs.pymc.io/advanced_theano.html

It works fine in the chisquare evaluation itself (in the perform() method
of the Theano Op class). However, in the grad() method I run into trouble.
When I try to write the contents of the input argument to file, it gives an
error because the input is not a numpy array, but a Theano tensor. I see in
the Theano docs
(http://deeplearning.net/software/theano/extending/extending_theano.html#op-s-basic-methods)
that this is how it's meant to be:

For perform(), "inputs is a list of references to data which can be
operated on using non-symbolic statements, (i.e., statements in Python,
Numpy)."

While for grad(), "It takes two arguments inputs and output_gradients which
are both lists of symbolic Theano Variables and those must be operated on
using Theano’s symbolic language."

My question is how to write inputs to file from inside the grad() method. I
have been trying with a shared variable which I update from the perform()
method, but that is probably too much of a hack because I suppose I can't
assume that perform() is called every time grad() is evaluated.

Any help is greatly appreciated!