[theano-users] Error computing theano grad

[Giuseppe Angora]

Hi,
I'm trying to develop an unsupervised clustering (Neural Gas) using theano.
I want to compute the gradient of the quantizzation error, to minimize it.
When i eval the gradient, i have an error, and i can not understand why,
this is the code:

import numpy as npimport theanoimport theano.tensor as T

theano.config.exception_verbosity='high'

W=T.matrix('W')
x=T.matrix('x')

n_neurons = 3
batch_size = 4

xx=np.zeros((4,2))
xx[1]+=[1,1]
xx[2]+=[0,1]
xx[3]+=[1,0]


WW=np.zeros((3,2))
WW[0]+=[0,3]
WW[1]+=[2,0]
WW[2]+=[4,4]

wL2S = T.sum(W*W, axis=-1) # [n_neurons]
xL2S = T.sum(x*x, axis=-1) # [batch_size]
wL2SM = T.zeros((n_neurons, batch_size)) + wL2S # broadcasting, [n_neurons, batch_size]
xL2SM = T.zeros((batch_size, n_neurons)) + xL2S # # broadcasting, [batch_size, n_neurons]
squaredPairwiseDistances = wL2SM + xL2SM.T - 2.0*T.dot(x, W.T) # [batch_size, n_neurons]
bestIndices = T.argsort(squaredPairwiseDistances, axis=1)

eq_bmu = T.eq(bestIndices, [0])
bmu=eq_bmu.nonzero()[1]

cost = T.sum(squaredPairwiseDistances[range(batch_size), bmu])
grads = T.grad(cost, W)
grads.eval({x:np.asarray(xx, dtype=theano.config.floatX), W:np.asarray(WW, dtype=theano.config.floatX)})

The error:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 884, in __call__
self.fn() if output_subset is None else\
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\op.py", line 891, in rval
r = p(n, [x[0] for x in i], o)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\tensor\subtensor.py", line 2218, in perform
np.add.at(out[0], tuple(inputs[2:]), inputs[1])IndexError: index 3 is out of bounds for axis 0 with size 3
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\graph.py", line 520, in eval
rval = self._fn_cache[inputs](*args)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 898, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "C:\ProgramData\Anaconda3\lib\site-packages\six.py", line 685, in reraise
raise value.with_traceback(tb)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 884, in __call__
self.fn() if output_subset is None else\
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\op.py", line 891, in rval
r = p(n, [x[0] for x in i], o)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\tensor\subtensor.py", line 2218, in perform
np.add.at(out[0], tuple(inputs[2:]), inputs[1])IndexError: index 3 is out of bounds for axis 0 with size 3Apply node that caused the error: AdvancedIncSubtensor{inplace=False, set_instead_of_inc=False}(Alloc.0, TensorConstant{(4,) of 1.0}, TensorConstant{[0 1 2 3]}, Subtensor{int64}.0)Toposort index: 14Inputs types: [TensorType(float32, matrix), TensorType(float32, vector), TensorType(int32, vector), TensorType(int64, vector)]Inputs shapes: [(3, 4), (4,), (4,), (4,)]Inputs strides: [(16, 4), (4,), (4,), (8,)]Inputs values: ['not shown', array([ 1., 1., 1., 1.], dtype=float32), array([0, 1, 2, 3]), array([1, 1, 0, 1], dtype=int64)]Inputs type_num: [11, 11, 7, 9]

[Pascal Lamblin]

The error is:

IndexError: index 3 is out of bounds for axis 0 with size 3


You are trying to update elements outside of a tensor.
More precisely, an update is trying to add 1. to elements (0, 1), (1, 1),
(2, 0), and (3, 1) of a (3, 4) matrix. There seems to be a dimension
mismatch or indexing issue.

Post by Giuseppe Angora
Hi,
I'm trying to develop an unsupervised clustering (Neural Gas) using
theano. I want to compute the gradient of the quantizzation error, to
minimize it. When i eval the gradient, i have an error, and i can not
import numpy as npimport theanoimport theano.tensor as T
theano.config.exception_verbosity='high'
W=T.matrix('W')
x=T.matrix('x')
n_neurons = 3
batch_size = 4
xx=np.zeros((4,2))
xx[1]+=[1,1]
xx[2]+=[0,1]
xx[3]+=[1,0]
WW=np.zeros((3,2))
WW[0]+=[0,3]
WW[1]+=[2,0]
WW[2]+=[4,4]
wL2S = T.sum(W*W, axis=-1) # [n_neurons]
xL2S = T.sum(x*x, axis=-1) # [batch_size]
wL2SM = T.zeros((n_neurons, batch_size)) + wL2S # broadcasting, [n_neurons, batch_size]
xL2SM = T.zeros((batch_size, n_neurons)) + xL2S # # broadcasting, [batch_size, n_neurons]
squaredPairwiseDistances = wL2SM + xL2SM.T - 2.0*T.dot(x, W.T) # [batch_size, n_neurons]
bestIndices = T.argsort(squaredPairwiseDistances, axis=1)
eq_bmu = T.eq(bestIndices, [0])
bmu=eq_bmu.nonzero()[1]
cost = T.sum(squaredPairwiseDistances[range(batch_size), bmu])
grads = T.grad(cost, W)
grads.eval({x:np.asarray(xx, dtype=theano.config.floatX), W:np.asarray(WW, dtype=theano.config.floatX)})
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 884, in __call__
self.fn() if output_subset is None else\
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\op.py", line 891, in rval
r = p(n, [x[0] for x in i], o)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\tensor\subtensor.py", line 2218, in perform
np.add.at(out[0], tuple(inputs[2:]), inputs[1])IndexError: index 3 is out of bounds for axis 0 with size 3
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\graph.py", line 520, in eval
rval = self._fn_cache[inputs](*args)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 898, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "C:\ProgramData\Anaconda3\lib\site-packages\six.py", line 685, in reraise
raise value.with_traceback(tb)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 884, in __call__
self.fn() if output_subset is None else\
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\op.py", line 891, in rval
r = p(n, [x[0] for x in i], o)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\tensor\subtensor.py", line 2218, in perform
np.add.at(out[0], tuple(inputs[2:]), inputs[1])IndexError: index 3 is out of bounds for axis 0 with size 3Apply node that caused the error: AdvancedIncSubtensor{inplace=False, set_instead_of_inc=False}(Alloc.0, TensorConstant{(4,) of 1.0}, TensorConstant{[0 1 2 3]}, Subtensor{int64}.0)Toposort index: 14Inputs types: [TensorType(float32, matrix), TensorType(float32, vector), TensorType(int32, vector), TensorType(int64, vector)]Inputs shapes: [(3, 4), (4,), (4,), (4,)]Inputs strides: [(16, 4), (4,), (4,), (8,)]Inputs values: ['not shown', array([ 1., 1., 1., 1.], dtype=float32), array([0, 1, 2, 3]), array([1, 1, 0, 1], dtype=int64)]Inputs type_num: [11, 11, 7, 9]