[theano-users] How to use pretrained word2vec vectors in Theano

[Laura CP]

I have implemented a seq2seq model in Theano for a summarization task. It
has an initial Embedding layer which I initialized using GlorotUniform()
and it works fine:


self.initialization_weight = GlorotUniform()
self.W = theano.shared(
self.initialization_weight.sample((self.number_of_input_symbols, self.embedding_dimension)), name='W_emb')
self.trainable_variables = [self.W]

However, I want to improve the training phase setting pretrained word2vec or GloVe vectors as my Embedding layer's weights. They're stored in an external file all.glove.vectors.txt.
After loading it in the variable init_weights (a Numpy matrix), I do as follows:

self.initialization_weight = init_weights
self.W = theano.shared(self.initialization_weight, name='W_emb')
self.trainable_variables = [self.W]

It doesn't through any error message and the network starts training. The
thing is I think something is wrong since I get similar results than before
(without word2vec initialization) and the elapsed time in training takes 4
times longer (all experiments run in GPU).


*Note:* word2vec vectors were trained over Wikipedia corpus. I tried with
both dimension 100 and 300, but with 300-dimensional vectors training was
extremely slow.


I'd appreciate if anybody could give me a hint of what's going on. Thanks!