[theano-users] AssertionError when adding Sparse to float

[Amir Alavi]

I'm new to theano, and my research group is using it as the backend for
Keras. We are using some Sparse matrices for our weights, and I wanted to
use RMSprop as our optimizer, so I had to write my own to work with these
Sparse matrices. However, I am running into errors that I don't understand.
For example, here is the end of the Traceback:

File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 83, in
get_updates
new_p = p - lr * g / (sparse.sqrt(new_a) + self.epsilon)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 225, in __add__
return add(left, right)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2174, in add
return add_s_d(x, y)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2039, in make_node
assert y.type.ndim == 2
AssertionError


To put into context, here is the part of the built-in RMSprop optimizer
from Keras, which I am trying to get to work with Sparse:

for p, g, a in zip(params, grads, accumulators):
# update accumulator
new_a = self.rho * a + (1. - self.rho) * K.square(g)
self.updates.append(K.update(a, new_a))
new_p = p - lr * g / (K.sqrt(new_a) + self.epsilon)


# apply constraints
if p in constraints:
c = constraints[p]
new_p = c(new_p)
self.updates.append(K.update(p, new_p))
return self.updates

I originally had an error with the line:
new_a = self.rho * a + (1. - self.rho) * K.square(g)

and the error was:
File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 73, in
get_updates
new_a = self.rho * a + (1. - self.rho) * K.square(g)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/keras/backend/theano_backend.py"
, line 472, in square
return T.sqr(x)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/elemwise.py"
, line 576, in make_node
inputs = list(map(as_tensor_variable, inputs))
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/basic.py"
, line 171, in as_tensor_variable
"Variable type field must be a TensorType.", x, x.type)
theano.tensor.var.AsTensorError: ('Variable type field must be a
TensorType.', SparseVariable{csr,float32}, Sparse[float32, csr])
I fixed this by using theano.sparse.sqr(g) in the calculation for new_a,
but now I can't get paste the error in calculating new_p, even after trying
theano.sparse.sqrt(new_a) as above.

I'd appreciate any help on this

Is this similar to below?
Discussion about comparing sparse to
scalar: https://groups.google.com/d/topic/theano-users/sbKdzoWOCDI/discussion

[Frédéric Bastien]

The problem is probably the + in
sparse.sqrt(new_a) + self.epsilon

Here you add a scalar to a sparse variable. WHich will make it non sparse!
We force the user to make explicit conversion from sparse to dense to
prevent unexpected memory grow. You can manually force the conversion of
the sparse variable like this:

sparse.dense_from_sparse(sparse.sqrt(new_a)) + self.epsilon

Post by Amir Alavi
I'm new to theano, and my research group is using it as the backend for
Keras. We are using some Sparse matrices for our weights, and I wanted to
use RMSprop as our optimizer, so I had to write my own to work with these
Sparse matrices. However, I am running into errors that I don't understand.
File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 83,
in get_updates
new_p = p - lr * g / (sparse.sqrt(new_a) + self.epsilon)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 225, in __add__
return add(left, right)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2174, in add
return add_s_d(x, y)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/sparse/basic.py"
, line 2039, in make_node
assert y.type.ndim == 2
AssertionError
To put into context, here is the part of the built-in RMSprop optimizer
# update accumulator
new_a = self.rho * a + (1. - self.rho) * K.square(g)
self.updates.append(K.update(a, new_a))
new_p = p - lr * g / (K.sqrt(new_a) + self.epsilon)
# apply constraints
c = constraints[p]
new_p = c(new_p)
self.updates.append(K.update(p, new_p))
return self.updates
new_a = self.rho * a + (1. - self.rho) * K.square(g)
File "/home/aalavi/single_cell_reducer/sparse_optimizers.py", line 73,
in get_updates
new_a = self.rho * a + (1. - self.rho) * K.square(g)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/keras/backend/theano_backend.py"
, line 472, in square
return T.sqr(x)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/gof/op.py"
, line 615, in __call__
node = self.make_node(*inputs, **kwargs)
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/elemwise.py"
, line 576, in make_node
inputs = list(map(as_tensor_variable, inputs))
File
"/home/aalavi/anaconda2/envs/scrna_new/lib/python3.6/site-packages/theano/tensor/basic.py"
, line 171, in as_tensor_variable
"Variable type field must be a TensorType.", x, x.type)
theano.tensor.var.AsTensorError: ('Variable type field must be a
TensorType.', SparseVariable{csr,float32}, Sparse[float32, csr])
I fixed this by using theano.sparse.sqr(g) in the calculation for new_a,
but now I can't get paste the error in calculating new_p, even after trying
theano.sparse.sqrt(new_a) as above.
I'd appreciate any help on this
Is this similar to below?
https://groups.google.com/d/topic/theano-users/sbKdzoWOCDI/discussion
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.