[theano-users] Theano variable sent to for slicing instead of constant while using theano.grad

[Siddhartha Saxena]

Hi

I am training a custom LSTM model on theano with LSTM layers as in
(https://github.com/asheshjain399/NeuralModels/tree/master/neuralmodels/layers/LSTM.py)
and
(https://github.com/asheshjain399/NeuralModels/blob/master/neuralmodels/layers/multilayerLSTM.py).
Now the model that I have created makes a valid theano graph but at the
step the find the updates (RMSprop) of the model with a cost (mean square)
function via the theano.grad function, I keep getting the following error.
The source of the error is that a theano op
(Elemwise{minimum,no_inplace}.0) is being sent for slicing to the
subtensor.py file instead of a theano constant. The issue is that I am not
sure how it ends up there. The model works if I remove the LSTM layer, but
this LSTM layer works with some other model perfectly.

I'll be extremely thankful if someone can help me through this error which
is bugging me for days now.

Best
Siddhartha

-------------------------------------------------------------
File "trainGCNN_NoGraph.py", line 251, in trainGCNN
gcnn =
GCNNmodelRegression(preGraphNets,nodeList,nodeFeatureLength,temporalNodeFeatureLength,new_idx,featureRange)
File "trainGCNN_NoGraph.py", line 202, in GCNNmodelRegression
gcnn =
GCNN(graphLayers,finalLayer,preGraphNets,nodeNames,temporalNodeRNN,nodeRNNs,topLayer,euclidean_loss,nodeLabels,learning_rate,adjacency,new_idx,featureRange,clipnorm=args.clipnorm,update_type=gradient_method,weight_decay=args.weight_decay)
File "/home/siddhartha/Graph_CNNs/neuralmodels/models/GCNN.py", line 169,
in __init__
[self.updates,self.grads] =
self.update_type.get_updates(self.params_all,self.cost)
File "/home/siddhartha/Graph_CNNs/neuralmodels/updates.py", line 108, in
get_updates
grads_unclipped = T.grad(cost, params)
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
555, in grad
grad_dict, wrt, cost_name)
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1317, in _populate_grad_dict
rval = [access_grad_cache(elem) for elem in wrt]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
967, in access_term_cache
output_grads = [access_grad_cache(var) for var in node.outputs]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1272, in access_grad_cache
term = access_term_cache(node)[idx]
File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py", line
1108, in access_term_cache
new_output_grads)
File
"/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py",
line 2250, in L_op
outer_inp_seqs = [s_[:grad_steps] for s_ in outer_inp_seqs]
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/var.py", line
519, in __getitem__
theano.tensor.subtensor.Subtensor.convert(arg)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py",
line 378, in convert
slice_b = Subtensor.convert(b, False)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py",
line 348, in convert
raise TypeError("Expected an integer")
TypeError: Expected an integer

[Pascal Lamblin]

Can you open that trace in pdb, and let us know what the dtype of
`grad_steps` here is?
What about the dtype of `self.truncate_gradient`?
And the type of `s_`?

Hi
I am training a custom LSTM model on theano with LSTM layers as in
(https://github.com/asheshjain399/NeuralModels/tree/master/neuralmodels/layers/LSTM.py)
and
(https://github.com/asheshjain399/NeuralModels/blob/master/neuralmodels/layers/multilayerLSTM.py).
Now the model that I have created makes a valid theano graph but at the
step the find the updates (RMSprop) of the model with a cost (mean
square) function via the theano.grad function, I keep getting the
following error. The source of the error is that a theano op
(Elemwise{minimum,no_inplace}.0) is being sent for slicing to the
subtensor.py file instead of a theano constant. The issue is that I am
not sure how it ends up there. The model works if I remove the LSTM
layer, but this LSTM layer works with some other model perfectly.
I'll be extremely thankful if someone can help me through this error
which is bugging me for days now.
Best
Siddhartha
-------------------------------------------------------------
  File "trainGCNN_NoGraph.py", line 251, in trainGCNN
    gcnn =
GCNNmodelRegression(preGraphNets,nodeList,nodeFeatureLength,temporalNodeFeatureLength,new_idx,featureRange)
  File "trainGCNN_NoGraph.py", line 202, in GCNNmodelRegression
    gcnn =
GCNN(graphLayers,finalLayer,preGraphNets,nodeNames,temporalNodeRNN,nodeRNNs,topLayer,euclidean_loss,nodeLabels,learning_rate,adjacency,new_idx,featureRange,clipnorm=args.clipnorm,update_type=gradient_method,weight_decay=args.weight_decay)
  File "/home/siddhartha/Graph_CNNs/neuralmodels/models/GCNN.py", line
169, in __init__
    [self.updates,self.grads] =
self.update_type.get_updates(self.params_all,self.cost)
  File "/home/siddhartha/Graph_CNNs/neuralmodels/updates.py", line 108,
in get_updates
    grads_unclipped = T.grad(cost, params)
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 555, in grad
    grad_dict, wrt, cost_name)
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1317, in _populate_grad_dict
    rval = [access_grad_cache(elem) for elem in wrt]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 967, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1272, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/usr/local/lib/python2.7/dist-packages/theano/gradient.py",
line 1108, in access_term_cache
    new_output_grads)
  File
"/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py",
line 2250, in L_op
    outer_inp_seqs = [s_[:grad_steps] for s_ in outer_inp_seqs]
  File "/usr/local/lib/python2.7/dist-packages/theano/tensor/var.py",
line 519, in __getitem__
    theano.tensor.subtensor.Subtensor.convert(arg)
  File
"/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py",
line 378, in convert
    slice_b = Subtensor.convert(b, False)
  File
"/usr/local/lib/python2.7/dist-packages/theano/tensor/subtensor.py",
line 348, in convert
    raise TypeError("Expected an integer")
TypeError: Expected an integer
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[Siddhartha Saxena]

grad_steps itself is of <class 'theano.tensor.var.TensorVariable'> with
value "Elemwise{minimum,no_inplace}.0". So here a tensor that is s_ ( of
type Subtensor{::int64}.0}) is being sliced by a variable. Again how it is
reaching there is what i am unable to understand.

Thanks

[Pascal Lamblin]

OK, but what is the `dtype` (data type) of those variables?

Post by Siddhartha Saxena
grad_steps itself is of <class 'theano.tensor.var.TensorVariable'> with
value "Elemwise{minimum,no_inplace}.0". So here a tensor that is s_ ( of
type Subtensor{::int64}.0}) is being sliced by a variable. Again how it
is reaching there is what i am unable to understand.
Thanks
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[Siddhartha Saxena]

The dtype of grad_steps and s_ is float64 while self.truncate_gradient is
a python float.
Sorry I didn't answer it properly previously.

Thanks

[Siddhartha Saxena]

Thanks a lot Pascal, I have solved the problem now, the issue was that
self.truncate_gradient was a float instead of being an int.

[Pascal Lamblin]

OK, thanks.

self.truncate_gradient should not be a Python float, it should be an
integer.
This is probably why the dtype of grad_steps is float64, instead of
int64 (or another integer dtype).

Do you have any idea why self.truncate_gradient would not be "-1" (the
default value)? Did you set it?

The dtype of  grad_steps and s_ is float64 while self.truncate_gradient
is a python float.
Sorry I didn't answer it properly previously.
Thanks
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.