[theano-users] TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, matrix)>, shared_var.type=TensorType(float32, matrix)

[Haree Varshan Jeyaram]

I am trying to train an image-sentence ranking model, where I get the below
error while I am calling the optimizer function. The error is as follows.

train_model = train.trainer()
{'grad_clip': 2.0, 'dim': 1024, 'optimizer': 'adam', 'dim_word': 300,
'data': 'coco', 'lrate': 0.0002, 'batch_size': 128, 'encoder': 'gru',
'maxlen_w': 100, 'saveto': 'D:/practicum/kiros data/train/coco.npz',
'max_epochs': 15, 'dim_image': 4096, 'dispFreq': 10, 'decay_c': 0.0,
'margin': 0.2, 'reload_': False, 'validFreq': 100}
Loading dataset
Creating dictionary
Dictionary size: 32198
Building model
Building f_log_probs... Done
Building f_cost... Done
Building sentence encoder
Building image encoder
Building f_grad... Building optimizers...Traceback (most recent call last):

File "<ipython-input-29-635970d00143>", line 1, in <module>
train_model = train.trainer()

File "train.py", line 155, in trainer
f_grad_shared, f_update = eval(optimizer)(lr, tparams, grads, inps,
cost)

File "optim.py", line 39, in adam
f_update = theano.function([lr], [], updates=updates,
on_unused_input='ignore', profile=False)

File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\function.py",
line 317, in function
output_keys=output_keys)

File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py",
line 449, in pfunc
no_default_updates=no_default_updates)

File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py",
line 208, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)

TypeError: ('An update must have the same type as the original shared
variable (shared_var=<TensorType(float32, matrix)>,
shared_var.type=TensorType(float32, matrix),
update_val=Elemwise{add,no_inplace}.0, update_val.type=TensorType(float64,
matrix)).', 'If the difference is related to the broadcast pattern, you can
call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to
remove broadcastable dimensions.')


Kindly help me in resolving this issue.


Thanks.

[Haree Varshan Jeyaram]

Code is as follows:

def adam(lr, tparams, grads, inp, cost):
gshared = [theano.shared(p.get_value() * 0., name='%s_grad'%k) for k, p
in tparams.iteritems()]
gsup = [(gs, g) for gs, g in zip(gshared, grads)]

f_grad_shared = theano.function(inp, cost, updates=gsup, profile=False)

b1 = 0.1
b2 = 0.001
e = 1e-8

updates = []

i = theano.shared(numpy.float32(0.))
i_t = i + 1.
fix1 = 1. - b1**(i_t)
fix2 = 1. - b2**(i_t)
lr_t = lr * (tensor.sqrt(fix2) / fix1)

for p, g in zip(tparams.values(), gshared):
m = theano.shared(p.get_value() * 0.)
v = theano.shared(p.get_value() * 0.)
m_t = (b1 * g) + ((1. - b1) * m)
v_t = (b2 * tensor.sqr(g)) + ((1. - b2) * v)
g_t = m_t / (tensor.sqrt(v_t) + e)
p_t = p - (lr_t * g_t)
updates.append((m, m_t))
updates.append((v, v_t))
updates.append((p, p_t))
updates.append((i, i_t))

f_update = theano.function([lr], [], updates=updates,
on_unused_input='ignore', profile=False)

return f_grad_shared, f_update


I'm not able to crack the issue. Please help me in resolving this.

Post by Haree Varshan Jeyaram
I am trying to train an image-sentence ranking model, where I get the
below error while I am calling the optimizer function. The error is as
follows.
train_model = train.trainer()
{'grad_clip': 2.0, 'dim': 1024, 'optimizer': 'adam', 'dim_word': 300,
'data': 'coco', 'lrate': 0.0002, 'batch_size': 128, 'encoder': 'gru',
'maxlen_w': 100, 'saveto': 'D:/practicum/kiros data/train/coco.npz',
'max_epochs': 15, 'dim_image': 4096, 'dispFreq': 10, 'decay_c': 0.0,
'margin': 0.2, 'reload_': False, 'validFreq': 100}
Loading dataset
Creating dictionary
Dictionary size: 32198
Building model
Building f_log_probs... Done
Building f_cost... Done
Building sentence encoder
Building image encoder
File "<ipython-input-29-635970d00143>", line 1, in <module>
train_model = train.trainer()
File "train.py", line 155, in trainer
f_grad_shared, f_update = eval(optimizer)(lr, tparams, grads, inps,
cost)
File "optim.py", line 39, in adam
f_update = theano.function([lr], [], updates=updates,
on_unused_input='ignore', profile=False)
File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\function.py",
line 317, in function
output_keys=output_keys)
File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py",
line 449, in pfunc
no_default_updates=no_default_updates)
File
"C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py",
line 208, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared
variable (shared_var=<TensorType(float32, matrix)>,
shared_var.type=TensorType(float32, matrix),
update_val=Elemwise{add,no_inplace}.0, update_val.type=TensorType(float64,
matrix)).', 'If the difference is related to the broadcast pattern, you can
call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to
remove broadcastable dimensions.')
Kindly help me in resolving this issue.
Thanks.

[Arnaud Bergeron]

You can probably make use of http://deeplearning.net/software/theano/library/config.html#config.warn_float64 <http://deeplearning.net/software/theano/library/config.html#config.warn_float64> to pinpoint the expression that causes an upcast.

gshared = [theano.shared(p.get_value() * 0., name='%s_grad'%k) for k, p in tparams.iteritems()]
gsup = [(gs, g) for gs, g in zip(gshared, grads)]
f_grad_shared = theano.function(inp, cost, updates=gsup, profile=False)
b1 = 0.1
b2 = 0.001
e = 1e-8
updates = []
i = theano.shared(numpy.float32(0.))
i_t = i + 1.
fix1 = 1. - b1**(i_t)
fix2 = 1. - b2**(i_t)
lr_t = lr * (tensor.sqrt(fix2) / fix1)
m = theano.shared(p.get_value() * 0.)
v = theano.shared(p.get_value() * 0.)
m_t = (b1 * g) + ((1. - b1) * m)
v_t = (b2 * tensor.sqr(g)) + ((1. - b2) * v)
g_t = m_t / (tensor.sqrt(v_t) + e)
p_t = p - (lr_t * g_t)
updates.append((m, m_t))
updates.append((v, v_t))
updates.append((p, p_t))
updates.append((i, i_t))
f_update = theano.function([lr], [], updates=updates, on_unused_input='ignore', profile=False)
return f_grad_shared, f_update
I'm not able to crack the issue. Please help me in resolving this.
I am trying to train an image-sentence ranking model, where I get the below error while I am calling the optimizer function. The error is as follows.
train_model = train.trainer()
{'grad_clip': 2.0, 'dim': 1024, 'optimizer': 'adam', 'dim_word': 300, 'data': 'coco', 'lrate': 0.0002, 'batch_size': 128, 'encoder': 'gru', 'maxlen_w': 100, 'saveto': 'D:/practicum/kiros data/train/coco.npz', 'max_epochs': 15, 'dim_image': 4096, 'dispFreq': 10, 'decay_c': 0.0, 'margin': 0.2, 'reload_': False, 'validFreq': 100}
Loading dataset
Creating dictionary
Dictionary size: 32198
Building model
Building f_log_probs... Done
Building f_cost... Done
Building sentence encoder
Building image encoder
File "<ipython-input-29-635970d00143>", line 1, in <module>
train_model = train.trainer()
File "train.py", line 155, in trainer
f_grad_shared, f_update = eval(optimizer)(lr, tparams, grads, inps, cost)
File "optim.py", line 39, in adam
f_update = theano.function([lr], [], updates=updates, on_unused_input='ignore', profile=False)
File "C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\function.py", line 317, in function
output_keys=output_keys)
File "C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py", line 449, in pfunc
no_default_updates=no_default_updates)
File "C:\Users\hareevarshan\Anaconda2\lib\site-packages\theano\compile\pfunc.py", line 208, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, matrix)>, shared_var.type=TensorType(float32, matrix), update_val=Elemwise{add,no_inplace}.0, update_val.type=TensorType(float64, matrix)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')
Kindly help me in resolving this issue.
Thanks.
--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.