[theano-users] custom Theano operation that accepts either a vector or matrix and produces either a scalar or vector

[Brad Buran]

Given the following function where:

- y is a ND array
- g has one less dimension than y
- x is a 1D array
- the result is the same number of dimensions as g

How do I write a Theano operation that wraps this function? I can get it
to work for the case where y is a 1D array and g is a scalar (see below).
However, when y is a 2D array, I get the following error. I understand why
I'm getting the error, but I'm not clear how to write a more suitably
polymorphic function that can handle broadcasting. Any pointers would be
appreciated.

TypeError: We expected inputs of types '[TensorType(float64, vector), TensorType(float64, scalar)]' but got types '[TensorType(float64, matrix), TensorType(float64, vector)]'


class Threshold(theano.Op):

__props__ = ('threshold', 'lower', 'upper', 'step')

itypes = [theano.tensor.dvector, theano.tensor.dscalar]
otypes = [theano.tensor.dscalar]

def __init__(self, threshold, lower, upper, step):
self.threshold = threshold
self.lower = lower
self.upper = upper
self.step = step
self.x = np.arange(lower, upper, step)
super().__init__()

def perform(self, node, inputs, output_storage):
y = inputs[0]
g = inputs[1]
z = output_storage[0]

y = norm.ppf(y)-norm.ppf(g)
fx = lambda y: np.interp([1], y, x)[0]
z[0] = np.apply_along_axis(fx, 0, y)

[Frédéric Bastien]

Theano broadcasting is hardcoded in the Theano graph. You will need to
build 2 nodes and probably compile 2 Theano function for that.

If you would like to have a partially working version that don't hardcode
in the graph the number of dimensions, you could have the inputs/outputs of
the nodes that changes be a Generic() variable instead of a TensorVariable.
But you won't have any operation working on it.

Fred

Post by Brad Buran
- y is a ND array
- g has one less dimension than y
- x is a 1D array
- the result is the same number of dimensions as g
How do I write a Theano operation that wraps this function? I can get it
to work for the case where y is a 1D array and g is a scalar (see below).
However, when y is a 2D array, I get the following error. I understand why
I'm getting the error, but I'm not clear how to write a more suitably
polymorphic function that can handle broadcasting. Any pointers would be
appreciated.
TypeError: We expected inputs of types '[TensorType(float64, vector), TensorType(float64, scalar)]' but got types '[TensorType(float64, matrix), TensorType(float64, vector)]'
__props__ = ('threshold', 'lower', 'upper', 'step')
itypes = [theano.tensor.dvector, theano.tensor.dscalar]
otypes = [theano.tensor.dscalar]
self.threshold = threshold
self.lower = lower
self.upper = upper
self.step = step
self.x = np.arange(lower, upper, step)
super().__init__()
y = inputs[0]
g = inputs[1]
z = output_storage[0]
y = norm.ppf(y)-norm.ppf(g)
fx = lambda y: np.interp([1], y, x)[0]
z[0] = np.apply_along_axis(fx, 0, y)
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Pascal Lamblin]

Just to clarify: in the actual graph, the types of tensors are static,
and so you will need different Apply nodes for different input types.

However, you can get away with only one Op.
The way of doing that is not to use the static `itypes` and `otypes`,
but to implement the `make_node` function, where you can check the type
of inputs (for instance, their number of dimensions) and create outputs
of a type that depends on them.

For instance, for your class, it could look like:

def make_node(self, y, g, x):
y = as_tensor_variable(y)
g = as_tensor_variable(g)
x = as_tensor_variable(x)

assert g.ndim == y.ndim - 1
assert x.ndim == 1

out_var = g.type()
return Apply(self, [y, g, x], [out_var]

You can also check the .dtype and .broadcastable attribute of each variable,
and build your own TensorType(...) output type depending on them (for instance,
to get the right broadcastable pattern for the output).

See http://deeplearning.net/software/theano/sandbox/how_to_make_ops.html#make-node

Post by Frédéric Bastien
Theano broadcasting is hardcoded in the Theano graph. You will need to
build 2 nodes and probably compile 2 Theano function for that.
If you would like to have a partially working version that don't hardcode
in the graph the number of dimensions, you could have the inputs/outputs of
the nodes that changes be a Generic() variable instead of a TensorVariable.
But you won't have any operation working on it.
Fred

Post by Brad Buran
- y is a ND array
- g has one less dimension than y
- x is a 1D array
- the result is the same number of dimensions as g
How do I write a Theano operation that wraps this function? I can get it
to work for the case where y is a 1D array and g is a scalar (see below).
However, when y is a 2D array, I get the following error. I understand why
I'm getting the error, but I'm not clear how to write a more suitably
polymorphic function that can handle broadcasting. Any pointers would be
appreciated.
TypeError: We expected inputs of types '[TensorType(float64, vector), TensorType(float64, scalar)]' but got types '[TensorType(float64, matrix), TensorType(float64, vector)]'
__props__ = ('threshold', 'lower', 'upper', 'step')
itypes = [theano.tensor.dvector, theano.tensor.dscalar]
otypes = [theano.tensor.dscalar]
self.threshold = threshold
self.lower = lower
self.upper = upper
self.step = step
self.x = np.arange(lower, upper, step)
super().__init__()
y = inputs[0]
g = inputs[1]
z = output_storage[0]
y = norm.ppf(y)-norm.ppf(g)
fx = lambda y: np.interp([1], y, x)[0]
z[0] = np.apply_along_axis(fx, 0, y)
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
For more options, visit https://groups.google.com/d/optout.

[Aima Aima]

*hello @nouiz*

*..I read many posts of yours.. but still i could not map my problem
accordingly. please help me. *

TypeError: ('An update must have the same type as the original shared
variable (shared_var=1lstm1_U_rgrad2, shared_var.type=TensorType(float32,
matrix), update_val=Elemwise{add,no_inplace}.0,
update_val.type=TensorType(float64, matrix)).', 'If the difference is
related to the broadcast pattern, you can call the tensor.unbroadcast(var,
axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

the error appears on this line of code:

f_grad_shared = theano.function([emb11, mask11, emb21, mask21, y], cost,
updates=zgup + rg2up,

name='adadelta_f_grad_shared',allow_input_downcast=True)


whereas the created parameters that are passe dto this function are :

y = tensor.vector('y', dtype='float32')
mask11 = tensor.matrix('mask11', dtype='float32')
mask21 = tensor.matrix('mask21', dtype='float32')
emb11 = theano.tensor.ftensor3('emb11')
emb21 = theano.tensor.ftensor3('emb21')
trng = RandomStreams(1234)
self.tnewp = init_tparams(newp)

rate = 0.5
rrng = trng.binomial(emb11.shape, p=1 - rate, n=1,
dtype=emb11.dtype)

proj11 = getpl2(emb11, '1lstm1', mask11, False, rrng, 50,
self.tnewp)[-1]
proj21 = getpl2(emb21, '2lstm1', mask21, False, rrng, 50,
self.tnewp)[-1]
dif = (proj21 - proj11).norm(L=1, axis=1)
s2 = T.exp(-dif)
sim = T.clip(s2, 1e-7, 1.0 - 1e-7)
lr = tensor.scalar(name='lr',dtype='float32')
ys = T.clip((y - 1.0) / 4.0, 1e-7, 1.0 - 1e-7)
cost = T.mean((sim - ys) ** 2)

i checked for data types:

cost.type()
<TensorType(float64, scalar)>
where as all others are in float32.
now please tell me how can i deal with the issue. Its urgent .

Regards,