[theano-users] Can we place .theanorc file in a location other than the default $HOME directory?

Discussion:

Abhishek Shivkumar

2015-07-22 10:34:57 UTC

We have Theano installed on a cluster of nodes and we want to have just
one copy of the .theanorc file that everyone can access. We don't want to
put it in everyone's home directory and maintain multiple copies.

Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?

Thanks
Abhishek S

--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Daniel Renshaw

2015-07-22 10:43:17 UTC

Permalink

Yes, change the THEANORC environment variable to point at your shared
.theanorc file.

This is described in the documentation [1]
<http://deeplearning.net/software/theano/library/config.html#envvar-THEANORC>
.

Daniel

[1]
http://deeplearning.net/software/theano/library/config.html#envvar-THEANORC

Post by Abhishek Shivkumar
We have Theano installed on a cluster of nodes and we want to have just
one copy of the .theanorc file that everyone can access. We don't want to
put it in everyone's home directory and maintain multiple copies.
Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?
Thanks
Abhishek S
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

Abhishek Shivkumar

2015-07-22 10:47:16 UTC

Permalink

Thanks Daniel. Not sure how I overlooked that part of the documentation.
Thanks again !

Post by Daniel Renshaw
Yes, change the THEANORC environment variable to point at your shared
.theanorc file.
This is described in the documentation [1]
<http://deeplearning.net/software/theano/library/config.html#envvar-THEANORC>
.
Daniel
[1]
http://deeplearning.net/software/theano/library/config.html#envvar-THEANORC

t***@gmail.com

2015-09-17 19:48:42 UTC

Permalink

Hi Abhishek,

I am also running theano on a cluster and I don't have permissions to
create compile directories in $HOME/. So I want to change THEANORC
environment variable to point to my .theanorc file where I specify
base_compiledir=/tmp/theano/. The problem is when I run this on the
cluster, theano still wants to create compile directories in $HOME/.
Can you explain how you were able to change THEANORC globally?

Thanks.

Post by Abhishek Shivkumar
Thanks Daniel. Not sure how I overlooked that part of the documentation.
Thanks again !

Post by Abhishek Shivkumar
We have Theano installed on a cluster of nodes and we want to have just
one copy of the .theanorc file that everyone can access. We don't want to
put it in everyone's home directory and maintain multiple copies.
Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?
Thanks
Abhishek S
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

Frédéric Bastien

2015-09-17 20:09:32 UTC

Permalink

How did you create it? You need to export it in your .bashrc file

export THEANORC=...

Fred

Post by t***@gmail.com
Hi Abhishek,
I am also running theano on a cluster and I don't have permissions to
create compile directories in $HOME/. So I want to change THEANORC
environment variable to point to my .theanorc file where I specify
base_compiledir=/tmp/theano/. The problem is when I run this on the
cluster, theano still wants to create compile directories in $HOME/.
Can you explain how you were able to change THEANORC globally?
Thanks.

Post by Abhishek Shivkumar
Thanks Daniel. Not sure how I overlooked that part of the documentation.
Thanks again !

Post by Abhishek Shivkumar
We have Theano installed on a cluster of nodes and we want to have
just one copy of the .theanorc file that everyone can access. We don't
want to put it in everyone's home directory and maintain multiple copies.
Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?
Thanks
Abhishek S
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

t***@gmail.com

2015-09-17 21:08:59 UTC

Permalink

Hi Fred,

Yes this is exactly how I did it. A compile directory was created to where
I specified it, but it still wants to create another one in $HOME/ where I
don't have access to.

Here is the error:

ValueError: Unable to create the compiledir directory
'/home/.theano/compiledir_Linux-2.6.32-504.el6.x86_64-x86_64-with-redhat-6.5-Santiago-x86_64-2.7.8-64'.
Check the permissions.

I am using version 0.6, as this is the approved version by my company.

I see this problem only when I run theano in spark in yarn-cluster mode,
but when I run it in standalone mode everything works perfectly fine and
the performance is great.

The only solution I can think of right now (not ideal) is to modify the
compiledir.py file in site-packages/theano/gof, specifically line 141
instead of: home = os.getenv('HOME')
use: home = '/tmp/theano/'

Any thoughts?

Post by FrÃ©dÃ©ric Bastien
How did you create it? You need to export it in your .bashrc file
export THEANORC=...
Fred

Post by Abhishek Shivkumar
Thanks Daniel. Not sure how I overlooked that part of the documentation.
Thanks again !

Post by Abhishek Shivkumar
We have Theano installed on a cluster of nodes and we want to have
just one copy of the .theanorc file that everyone can access. We don't
want to put it in everyone's home directory and maintain multiple copies.
Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?
Thanks
Abhishek S
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

Pascal Lamblin

2015-09-17 22:05:58 UTC

Permalink

Post by t***@gmail.com
I see this problem only when I run theano in spark in yarn-cluster mode,
but when I run it in standalone mode everything works perfectly fine and
the performance is great.

Is it possible that spark does not use all the environment variables
that you defined in your .bashrc, or does not re-read your .bashrc until
it is restarted or reloaded?

Do you have a way to access os.env['THEANORC'] from inside the
spark environment, and check if it is defined? For instance, you
could add additional prints in theano/configparser.py around
"os.getenv('THEANORC', '~/.theanorc').split(os.pathsep)".

Post by t***@gmail.com
The only solution I can think of right now (not ideal) is to modify the
compiledir.py file in site-packages/theano/gof, specifically line 141
instead of: home = os.getenv('HOME')
use: home = '/tmp/theano/'
Any thoughts?

Post by FrÃ©dÃ©ric Bastien
How did you create it? You need to export it in your .bashrc file
export THEANORC=...
Fred

Post by Abhishek Shivkumar
Thanks Daniel. Not sure how I overlooked that part of the documentation.
Thanks again !

Post by Abhishek Shivkumar
We have Theano installed on a cluster of nodes and we want to have
just one copy of the .theanorc file that everyone can access. We don't
want to put it in everyone's home directory and maintain multiple copies.
Is there a way to point Theano to the .theanorc file located at any
specific location by mentioning the path?
Thanks
Abhishek S
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
For more options, visit https://groups.google.com/d/optout.

--
Pascal
--
---
You received this message because you are subscribed to the Google Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

t***@gmail.com

2015-09-17 22:44:09 UTC

Permalink

Hi Pascal,

I can actually access THEANORC within the spark environment:
os.getenv('THEANORC') -> points to the location of my .theanorc.

So if I were to make changes to theano files in site-packages, making
changes to compiledir.py would guarantee that the changes are in effect
right?

Post by Pascal Lamblin

Post by t***@gmail.com
I see this problem only when I run theano in spark in yarn-cluster mode,
but when I run it in standalone mode everything works perfectly fine and
the performance is great.

Is it possible that spark does not use all the environment variables
that you defined in your .bashrc, or does not re-read your .bashrc until
it is restarted or reloaded?
Do you have a way to access os.env['THEANORC'] from inside the
spark environment, and check if it is defined? For instance, you
could add additional prints in theano/configparser.py around
"os.getenv('THEANORC', '~/.theanorc').split(os.pathsep)".

Post by FrÃ©dÃ©ric Bastien
How did you create it? You need to export it in your .bashrc file
export THEANORC=...
Fred

Post by Abhishek Shivkumar
Thanks Daniel. Not sure how I overlooked that part of the

documentation.