[theano-users] Scan error message that neved made sense to me

[Dzmitry Bahdanau]

Hi all,

I have significant experience with theano.scan, but its error messages were
always incomprehensible for me. Perhaps this is the day to figure out how
to read them..

Here is a typical error message that I got this morning:


* When compiling the inner function of scan (the function called by scan in
each of its iterations) the following error has been encountered: The
initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s) (it should be one less than the initial state).
For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). The first dimension of this matrix
corresponds to the number of previous time-steps that scan uses in each of
its iterations. In order to solve this issue if the two varialbe currently
have the same dimensionality, you can increase the dimensionality of the
variable in the initial state of scan by using dimshuffle or shape_padleft.
*
Now, I know that it means that my "outputs_info" don't match the variable
returned by inner scan function. But let me try to read this error message
literally:



*The initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s)*All right, let's look what we have in outputs_info





*ipdb> outputs_info[None, Rebroadcast{0}.0]ipdb> outputs_info[1].ndim2*
Oops! First of all, there is no *IncSubtensor{Set;:int64:}.0, *in this
list. In addition, there is no variable with 3 dimensions. But let's look
further:



*For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). *While I believe that providing an
initial state of shape (1, d) would probably work, nobody does this,
including the official scan tutorial! The standard practice in such case is
to provide a d-vector as an initial state. In the common case when only
previous state results are used, it is perfectly fine to have the initial
state and the value returned by scan function with the same shape!

This error message gives me a double blow: first it is incomprehensible,
second, it postulates that I have always been using Scan in wrong way, and
makes me wonder, how it ever worked. The only useful information in this
message is that the result of the inner function has 3 dimensions. This was
actually helpful in fixing the bug.

I would be curious to hear the opinion of Theano developers, can we somehow
make this super common error message less confusing?

[Pascal Lamblin]

Believe it or not, this is already an improved version over what was
there earlier, but given what you say, it is indeed confusing.

Someone would have to investigate to be sure, but I believe that the
confusion comes from a layer or syntactic sugar that enables you _not_
to pass an initial state of shape (1, d) when not explicitly providing
taps (previous timesteps), and takes care of adding that dimension (the
IncSubtensor from the error message).

When taps are provided explicitly, then scan considers that there are
possibly multiple initial states passed, so the additional dimension has
to be provided, and the message displayed is accurate.

We should sort this out, but it will not be as simple as just rewriting
that error message.

Post by Dzmitry Bahdanau
Hi all,
I have significant experience with theano.scan, but its error messages were
always incomprehensible for me. Perhaps this is the day to figure out how
to read them..
* When compiling the inner function of scan (the function called by scan in
each of its iterations) the following error has been encountered: The
initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s) (it should be one less than the initial state).
For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). The first dimension of this matrix
corresponds to the number of previous time-steps that scan uses in each of
its iterations. In order to solve this issue if the two varialbe currently
have the same dimensionality, you can increase the dimensionality of the
variable in the initial state of scan by using dimshuffle or shape_padleft.
*
Now, I know that it means that my "outputs_info" don't match the variable
returned by inner scan function. But let me try to read this error message
*The initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s)*All right, let's look what we have in outputs_info
*ipdb> outputs_info[None, Rebroadcast{0}.0]ipdb> outputs_info[1].ndim2*
Oops! First of all, there is no *IncSubtensor{Set;:int64:}.0, *in this
list. In addition, there is no variable with 3 dimensions. But let's look
*For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). *While I believe that providing an
initial state of shape (1, d) would probably work, nobody does this,
including the official scan tutorial! The standard practice in such case is
to provide a d-vector as an initial state. In the common case when only
previous state results are used, it is perfectly fine to have the initial
state and the value returned by scan function with the same shape!
This error message gives me a double blow: first it is incomprehensible,
second, it postulates that I have always been using Scan in wrong way, and
makes me wonder, how it ever worked. The only useful information in this
message is that the result of the inner function has 3 dimensions. This was
actually helpful in fixing the bug.
I would be curious to hear the opinion of Theano developers, can we somehow
make this super common error message less confusing?
--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
For more options, visit https://groups.google.com/d/optout.

[Dzmitry Bahdanau]

I know that this error message used to be even worse, and I think that the
current version is a way better. But it still does more harm, than good, I
guess, given the this syntactic sugar is a very popular way of using scan.

Post by Pascal Lamblin
Believe it or not, this is already an improved version over what was
there earlier, but given what you say, it is indeed confusing.
Someone would have to investigate to be sure, but I believe that the
confusion comes from a layer or syntactic sugar that enables you _not_
to pass an initial state of shape (1, d) when not explicitly providing
taps (previous timesteps), and takes care of adding that dimension (the
IncSubtensor from the error message).
When taps are provided explicitly, then scan considers that there are
possibly multiple initial states passed, so the additional dimension has
to be provided, and the message displayed is accurate.
We should sort this out, but it will not be as simple as just rewriting
that error message.

Post by Dzmitry Bahdanau
Hi all,
I have significant experience with theano.scan, but its error messages

were

Post by Dzmitry Bahdanau
always incomprehensible for me. Perhaps this is the day to figure out how
to read them..
* When compiling the inner function of scan (the function called by scan

in

Post by Dzmitry Bahdanau
each of its iterations) the following error has been encountered: The
initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s) (it should be one less than the initial state).
For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). The first dimension of this

matrix

Post by Dzmitry Bahdanau
corresponds to the number of previous time-steps that scan uses in each

of

Post by Dzmitry Bahdanau
its iterations. In order to solve this issue if the two varialbe

currently

Post by Dzmitry Bahdanau
have the same dimensionality, you can increase the dimensionality of the
variable in the initial state of scan by using dimshuffle or

shape_padleft.

Post by Dzmitry Bahdanau
*
Now, I know that it means that my "outputs_info" don't match the variable
returned by inner scan function. But let me try to read this error

message

Post by Dzmitry Bahdanau
*The initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s)*All right, let's look what we have in

outputs_info

Post by Dzmitry Bahdanau
*ipdb> outputs_info[None, Rebroadcast{0}.0]ipdb> outputs_info[1].ndim2*
Oops! First of all, there is no *IncSubtensor{Set;:int64:}.0, *in this
list. In addition, there is no variable with 3 dimensions. But let's look
*For example, if the inner function of scan returns a vector of size d

and

Post by Dzmitry Bahdanau
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). *While I believe that providing

an

Post by Dzmitry Bahdanau
initial state of shape (1, d) would probably work, nobody does this,
including the official scan tutorial! The standard practice in such case

is

Post by Dzmitry Bahdanau
to provide a d-vector as an initial state. In the common case when only
previous state results are used, it is perfectly fine to have the initial
state and the value returned by scan function with the same shape!
This error message gives me a double blow: first it is incomprehensible,
second, it postulates that I have always been using Scan in wrong way,

and

Post by Dzmitry Bahdanau
makes me wonder, how it ever worked. The only useful information in this
message is that the result of the inner function has 3 dimensions. This

was

Post by Dzmitry Bahdanau
actually helpful in fixing the bug.
I would be curious to hear the opinion of Theano developers, can we

somehow

Post by Dzmitry Bahdanau
make this super common error message less confusing?
--
---
You received this message because you are subscribed to the Google

Groups "theano-users" group.

Post by Dzmitry Bahdanau
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
Pascal
--
---
You received this message because you are subscribed to a topic in the
Google Groups "theano-users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/theano-users/_Lts71NpnLA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
For more options, visit https://groups.google.com/d/optout.

[Cinar]

Here is more details on the code and error:
https://groups.google.com/d/msg/theano-users/bNnNHGFRXjs/tk_pB1YdBgAJ

Hello,
I'm facing a similar error and I don't have significant experience with
scan. When I attempted to solve it with dimshuffle, I get another error
that I don't expect.
Any suggestions?
Many thanks.

Post by Dzmitry Bahdanau
Hi all,
I have significant experience with theano.scan, but its error messages
were always incomprehensible for me. Perhaps this is the day to figure out
how to read them..
* When compiling the inner function of scan (the function called by scan
in each of its iterations) the following error has been encountered: The
initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s) (it should be one less than the initial state).
For example, if the inner function of scan returns a vector of size d and
scan uses the values of the previous time-step, then the initial state in
scan should be a matrix of shape (1, d). The first dimension of this matrix
corresponds to the number of previous time-steps that scan uses in each of
its iterations. In order to solve this issue if the two varialbe currently
have the same dimensionality, you can increase the dimensionality of the
variable in the initial state of scan by using dimshuffle or shape_padleft.
*
Now, I know that it means that my "outputs_info" don't match the variable
returned by inner scan function. But let me try to read this error message
*The initial state (`outputs_info` in scan nomenclature) of variable
IncSubtensor{Set;:int64:}.0 (argument number 0) has 3 dimension(s), while
the corresponding variable in the result of the inner function of scan
(`fn`) has 3 dimension(s)*All right, let's look what we have in outputs_info
*ipdb> outputs_info[None, Rebroadcast{0}.0]ipdb> outputs_info[1].ndim2*
Oops! First of all, there is no *IncSubtensor{Set;:int64:}.0, *in this
list. In addition, there is no variable with 3 dimensions. But let's look
*For example, if the inner function of scan returns a vector of size d
and scan uses the values of the previous time-step, then the initial state
in scan should be a matrix of shape (1, d). *While I believe that
providing an initial state of shape (1, d) would probably work, nobody does
this, including the official scan tutorial! The standard practice in such
case is to provide a d-vector as an initial state. In the common case when
only previous state results are used, it is perfectly fine to have the
initial state and the value returned by scan function with the same shape!
This error message gives me a double blow: first it is incomprehensible,
second, it postulates that I have always been using Scan in wrong way, and
makes me wonder, how it ever worked. The only useful information in this
message is that the result of the inner function has 3 dimensions. This was
actually helpful in fixing the bug.
I would be curious to hear the opinion of Theano developers, can we
somehow make this super common error message less confusing?