[theano-users] Cannot do a simple theano install (Python 2.7, Ubuntu 16.04, Theano 0.9, Cuda 8.0, TitanX GPU) due to pygpu errors

[Daniel Seita]

I'm trying to follow the *simplest theano installation instructions I can
find*. I'm using Ubuntu 16.04 (along with other settings described in the
title) and the instructions seem to be here:
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.

I'm using Anaconda for Python 2.7.

Running

conda install numpy scipy mkl nose sphinx pydot-ng

works without issues.

Then the instructions say to get the GPU and CUDA set up. I've installed my
CUDA drivers and they should be working, since I'm able to run TensorFlow.
I set up these in the bashrc:

export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64

Finally, I ran

conda install theano pygpu

I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
of periods):

In [1]: import pygpu

In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:
09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s

OK

I have this in my theanorc:

[global]
floatX = float64
device = cuda0

[cuda]
root = /usr/local/cuda-8.0

Now let's see what happens:

$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])

I cannot even import theano successfully. There are two problems. The first
is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?

Another thread
(https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has a
similar question but `sudo ldconfig` did not work for me and results in the
same errors.

[Daniel Seita]

I can run Theano using `device = gpu` in the configuration. It complains
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions I can
find*. I'm using Ubuntu 16.04 (along with other settings described in the
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've installed
my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23
:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has a
similar question but `sudo ldconfig` did not work for me and results in the
same errors.

[Frédéric Bastien]

Ça an you update to Theano dev version? It contains fixes. It don't have
the old backend anymore. Cudnn should be optional in the dev version and
Should give better error msg.

Fred

Post by Daniel Seita
I can run Theano using `device = gpu` in the configuration. It complains
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions I
can find*. I'm using Ubuntu 16.04 (along with other settings described
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've installed
my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-packages/
numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016,
23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])'
)
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has a
similar question but `sudo ldconfig` did not work for me and results in the
same errors.

--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

nouiz, thanks for the response, but I ran into basically the same exact
errors with the bleeding edge version (that's the dev, right?). At this
point it seems to be harder to switch to the new backend.

Post by Frédéric Bastien
Ça an you update to Theano dev version? It contains fixes. It don't have
the old backend anymore. Cudnn should be optional in the dev version and
Should give better error msg.
Fred

Post by Daniel Seita
I can run Theano using `device = gpu` in the configuration. It complains
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions I
can find*. I'm using Ubuntu 16.04 (along with other settings described
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've installed
my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-packages
/pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-packages
/numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016,
23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15
)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has a
similar question but `sudo ldconfig` did not work for me and results in the
same errors.

--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Frédéric Bastien]

Check the Theano flag dnn.include_path and dnn.librairy_path. you can set
them to the corresponding path. This will probably make Theano find them.

Post by Daniel Seita
nouiz, thanks for the response, but I ran into basically the same exact
errors with the bleeding edge version (that's the dev, right?). At this
point it seems to be harder to switch to the new backend.

Post by Frédéric Bastien
Ça an you update to Theano dev version? It contains fixes. It don't have
the old backend anymore. Cudnn should be optional in the dev version and
Should give better error msg.
Fred

I can run Theano using `device = gpu` in the configuration. It complains

Post by Frédéric Bastien

Post by Daniel Seita
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions I
can find*. I'm using Ubuntu 16.04 (along with other settings described
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've
installed my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016,
23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has
a similar question but `sudo ldconfig` did not work for me and results in
the same errors.

--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send an

Post by Daniel Seita
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

Thanks. I checked them and also that I was on the dev version:

In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])

In [2]: theano.config.dnn
Out[2]: <theano.configparser.SubObj at 0x7f6dc56239d0>

In [3]: theano.config.dnn.include_path
Out[3]: '/usr/local/cuda-8.0/include'

In [4]: theano.config.dnn.library_path
Out[4]: '/usr/local/cuda-8.0/lib64

In [5]: theano.__version__
Out[5]: '0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291'


Unfortunately, they make sense. This is exactly where my CUDA libraries are
stored. For the above, I had this as my configuration:

[global]
floatX = float64
device = cuda

[cuda]
root = /usr/local/cuda-8.0

What do you recommend now?

Post by Frédéric Bastien
Check the Theano flag dnn.include_path and dnn.librairy_path. you can set
them to the corresponding path. This will probably make Theano find them.

Post by Daniel Seita
nouiz, thanks for the response, but I ran into basically the same exact
errors with the bleeding edge version (that's the dev, right?). At this
point it seems to be harder to switch to the new backend.

Post by Frédéric Bastien
Ça an you update to Theano dev version? It contains fixes. It don't have
the old backend anymore. Cudnn should be optional in the dev version and
Should give better error msg.
Fred

I can run Theano using `device = gpu` in the configuration. It complains

Post by Frédéric Bastien

Post by Daniel Seita
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions I
can find*. I'm using Ubuntu 16.04 (along with other settings
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've
installed my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016
, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k) has
a similar question but `sudo ldconfig` did not work for me and results in
the same errors.

--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send

Post by Daniel Seita
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

I was running into some more difficulties, so I gave up on getting this to
work and tried to uninstall and then reinstall Theano. Just to be extra
clear, here is my setup:

- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal

cuDNN is here:

$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h

To verify that I can use my GPU I started this quick TensorFlow computation:

In [1]: import tensorflow as tf

In [2]: tf.__version__
Out[2]: '1.1.0'

In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions

In [4]: with tf.device('/gpu:0'):
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3],
name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2],
name='b')
...: c = tf.matmul(a,b)
...:

In [5]: with tf.Session() as sess:
...: print(sess.run(c))
...:
2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard.cc:
45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but
these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available on your
machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard.cc:
45] The TensorFlow library wasn't compiled to use AVX instructions, but
these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on your
machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard.cc:
45] The TensorFlow library wasn't compiled to use FMA instructions, but
these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
properties:
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id:
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]


This looks like it indicates a successful GPU and/or cuDNN installation.

Great, now let's install the *development version* of Theano. The
instructions I'm following step-by-step:
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html

The first step seems to be to install miniconda. I downloaded the bash
script for Python 2.7 and ran it:

~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh

Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)

In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue

and it seemed to work without issues.

The next step is to install requirements through conda. Here I did:

$ conda install numpy scipy mkl nose sphinx pydot-ng
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /home/daniel/miniconda2:

The following NEW packages will be INSTALLED:

alabaster: 0.7.10-py27_0
babel: 2.4.0-py27_0
docutils: 0.13.1-py27_0
imagesize: 0.7.1-py27_0
jinja2: 2.9.6-py27_0
libgfortran: 3.0.0-1
markupsafe: 0.23-py27_2
mkl: 2017.0.1-0
nose: 1.3.7-py27_1
numpy: 1.13.0-py27_0
pydot-ng: 1.0.0.15-py27_0
pygments: 2.2.0-py27_0
pytz: 2017.2-py27_0
scipy: 0.19.0-np113py27_0
snowballstemmer: 1.2.1-py27_0
sphinx: 1.6.2-py27_0
sphinxcontrib: 1.0-py27_0
sphinxcontrib-websupport: 1.0.1-py27_0
typing: 3.6.1-py27_0

The following packages will be UPDATED:

conda: 4.3.21-py27_0 --> 4.3.22-py27_0

Proceed ([y]/n)? y

libgfortran-3. 100% |#############################################################################################################################################################| Time: 0:00:00 12.51 MB/s
mkl-2017.0.1-0 100% |#############################################################################################################################################################| Time: 0:00:05 24.09 MB/s
alabaster-0.7. 100% |#############################################################################################################################################################| Time: 0:00:00 17.42 MB/s
docutils-0.13. 100% |#############################################################################################################################################################| Time: 0:00:00 20.24 MB/s
imagesize-0.7. 100% |#############################################################################################################################################################| Time: 0:00:00 14.09 MB/s
markupsafe-0.2 100% |#############################################################################################################################################################| Time: 0:00:00 45.16 MB/s
nose-1.3.7-py2 100% |#############################################################################################################################################################| Time: 0:00:00 16.95 MB/s
numpy-1.13.0-p 100% |#############################################################################################################################################################| Time: 0:00:00 20.15 MB/s
pygments-2.2.0 100% |#############################################################################################################################################################| Time: 0:00:00 28.41 MB/s
pytz-2017.2-py 100% |#############################################################################################################################################################| Time: 0:00:00 33.31 MB/s
snowballstemme 100% |#############################################################################################################################################################| Time: 0:00:00 36.62 MB/s
sphinxcontrib- 100% |#############################################################################################################################################################| Time: 0:00:00 2.55 MB/s
typing-3.6.1-p 100% |#############################################################################################################################################################| Time: 0:00:00 15.68 MB/s
babel-2.4.0-py 100% |#############################################################################################################################################################| Time: 0:00:00 32.66 MB/s
jinja2-2.9.6-p 100% |#############################################################################################################################################################| Time: 0:00:00 39.55 MB/s
pydot-ng-1.0.0 100% |#############################################################################################################################################################| Time: 0:00:00 78.96 MB/s
scipy-0.19.0-n 100% |#############################################################################################################################################################| Time: 0:00:01 25.00 MB/s
sphinxcontrib- 100% |#############################################################################################################################################################| Time: 0:00:00 44.77 MB/s
sphinx-1.6.2-p 100% |#############################################################################################################################################################| Time: 0:00:00 39.27 MB/s
conda-4.3.22-p 100% |#############################################################################################################################################################| Time: 0:00:00 42.98 MB/s


Everything still looks good from here.

Next, it tells me to install and configure the CUDA drivers. I'm pretty
sure this is working. The instructions here tell me to test using
nvidia-smi. Here you go:

$ nvidia-smi
Fri Jun 16 16:21:41 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 370.28 Driver Version: 370.28
|
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr.
ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M
. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 0000:01:00.0 On | N/A
|
| 23% 42C P0 61W / 250W | 385MiB / 12186MiB | 0%
Default |
+-------------------------------+----------------------+----------------------+


+-----------------------------------------------------------------------------+
| Processes: GPU
Memory |
| GPU PID Type Process name Usage
|
|=============================================================================|
| 0 981 G /usr/lib/xorg/Xorg
201MiB |
| 0 1394 G compiz
179MiB |
| 0 2006 G /usr/lib/firefox/firefox
2MiB |
+-----------------------------------------------------------------------------+

It looks good to me. Also:

$ which nvcc
/usr/local/cuda-8.0/bin/nvcc

And regarding path variables:

$ echo $LD_LIBRARY_PATH
/usr/local/lib:/usr/local/cuda-8.0/lib64:/home/daniel/mjpro131/bin:/home/
daniel/mjpro131

Again, this looks good. It has the `lib64` directory there.

Finally, the moment of truth, the next step is to install theano and pygpu.
Here's the output:

$ conda install theano pygpu
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /home/daniel/miniconda2:

The following NEW packages will be INSTALLED:

libgpuarray: 0.6.4-0
mako: 1.0.6-py27_0
mkl-service: 1.1.2-py27_3
pygpu: 0.6.4-py27_1
theano: 0.9.0-py27_0

Proceed ([y]/n)? y

libgpuarray-0. 100% |#############################################################################################################################################################|
Time: 0:00:00 10.58 MB/s
mkl-service-1. 100% |#############################################################################################################################################################|
Time: 0:00:00 23.83 MB/s
mako-1.0.6-py2 100% |#############################################################################################################################################################|
Time: 0:00:00 12.78 MB/s
pygpu-0.6.4-py 100% |#############################################################################################################################################################|
Time: 0:00:00 13.81 MB/s
theano-0.9.0-p 100% |#############################################################################################################################################################|
Time: 0:00:00 19.51 MB/s

Once again, it looks good. With a non-existent `.theanorc` file, here's
what I get:
$ ./miniconda2/bin/ipython
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import theano

In [2]: theano.__version__
Out[2]: '0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291'

This is a successful import using miniconda's ipython.

Now I'm following how to use the GPU (again, for the dev version):
http://deeplearning.net/software/theano_versions/dev/tutorial/using_gpu.html

I modified my theanorc file and copied the output to the command line:

$ cat ~/.theanorc
[global]
device = cuda
floatX = float32



And then ... boom.

$ ./miniconda2/bin/ipython
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File
"/home/daniel/miniconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/miniconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/miniconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/miniconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/miniconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])

In [2]: theano.config.dnn.include_path
Out[2]: '/usr/local/cuda-8.0/include'

In [3]: theano.config.dnn.library_path
Out[3]: '/usr/local/cuda-8.0/lib64'

Did I miss a step somewhere in the installation?

Post by Daniel Seita
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
In [2]: theano.config.dnn
Out[2]: <theano.configparser.SubObj at 0x7f6dc56239d0>
In [3]: theano.config.dnn.include_path
Out[3]: '/usr/local/cuda-8.0/include'
In [4]: theano.config.dnn.library_path
Out[4]: '/usr/local/cuda-8.0/lib64
In [5]: theano.__version__
Out[5]: '0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291'
Unfortunately, they make sense. This is exactly where my CUDA libraries
[global]
floatX = float64
device = cuda
[cuda]
root = /usr/local/cuda-8.0
What do you recommend now?

Post by Frédéric Bastien
Check the Theano flag dnn.include_path and dnn.librairy_path. you can set
them to the corresponding path. This will probably make Theano find them.

Post by Daniel Seita
nouiz, thanks for the response, but I ran into basically the same exact
errors with the bleeding edge version (that's the dev, right?). At this
point it seems to be harder to switch to the new backend.

Post by Frédéric Bastien
Ça an you update to Theano dev version? It contains fixes. It don't
have the old backend anymore. Cudnn should be optional in the dev version
and Should give better error msg.
Fred

I can run Theano using `device = gpu` in the configuration. It complains

Post by Frédéric Bastien

Post by Daniel Seita
about not being able to find the new backend, but at this point I see no
reason why to switch to the new backend.

Post by Daniel Seita
I'm trying to follow the *simplest theano installation instructions
I can find*. I'm using Ubuntu 16.04 (along with other settings
http://deeplearning.net/software/theano/install_ubuntu.html# for Theano 0.9.
I'm using Anaconda for Python 2.7.
Running
conda install numpy scipy mkl nose sphinx pydot-ng
works without issues.
Then the instructions say to get the GPU and CUDA set up. I've
installed my CUDA drivers and they should be working, since I'm able to run
export PATH="/usr/local/cuda-8.0/bin:$PATH"
export CUDA_HOME=/usr/local/cuda-8.0
export LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
Finally, I ran
conda install theano pygpu
I was hoping this is it, because that automatically installs the GPU
backend. In addition, pygpu seems to be working (I deleted the long line
In [1]: import pygpu
In [2]: pygpu.test()
pygpu is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/pygpu
NumPy version 1.12.1
NumPy relaxed strides checking option: True
NumPy is installed in /home/daniel/anaconda2/lib/python2.7/site-
packages/numpy
Python version 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20
2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
nose version 1.3.7
*** Testing for TITAN X (Pascal)
----------------------------------------------------------------------
Ran 7051 tests in 138.494s
OK
[global]
floatX = float64
device = cuda0
[cuda]
root = /usr/local/cuda-8.0
$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09
:15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details
.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
I cannot even import theano successfully. There are two problems. The
first is that pygpu isn't initialized (?) and the second is that the cudnn
library couldn't be found. It was working for TensorFlow and the cudnn
header is found in `/usr/local/cuda-8.0/lib64/cudnn.h` so I'm guessing the
issue lies with the pygpu library. I have no idea how to fix this but there
must be a common fix since this seems to be the default way to install
theano?
Another thread (
https://groups.google.com/forum/#!topic/theano-users/zzaJVrYBD1k)
has a similar question but `sudo ldconfig` did not work for me and results
in the same errors.

--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.

To unsubscribe from this group and stop receiving emails from it, send

Post by Daniel Seita
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

Ack, sorry, half of my post got deleted! Hopefully you can still see it (i
can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting this to
work and tried to uninstall and then reinstall Theano. Just to be extra
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3],
name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2],
name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use AVX instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on your
machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use FMA instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems to be to install miniconda. I downloaded the bash
~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh
Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
and it seemed to work without issues.
$ conda install numpy scipy mkl nose sphinx pydot-ng
Fetching package metadata .........
Solving package specifications: .
alabaster: 0.7.10-py27_0
babel: 2.4.0-py27_0
docutils: 0.13.1-py27_0
imagesize: 0.7.1-py27_0
jinja2: 2.9.6-py27_0
libgfortran: 3.0.0-1
markupsafe: 0.23-py27_2
mkl: 2017.0.1-0
nose: 1.3.7-py27_1
numpy: 1.13.0-py27_0
pydot-ng: 1.0.0.15-py27_0
pygments: 2.2.0-py27_0
pytz: 2017.2-py27_0
scipy: 0.19.0-np113py27_0
snowballstemmer: 1.2.1-py27_0
sphinx: 1.6.2-py27_0
sphinxcontrib: 1.0-py27_0
sphinxcontrib-websupport: 1.0.1-py27_0
typing: 3.6.1-py27_0
conda: 4.3.21-py27_0 --> 4.3.22-py27_0
Proceed <span style="color: #660;" class="styled-by-prettify

[Frédéric Bastien]

Your cudnn.h file should not be in the lib64 directory, but in an include
directory. Tensorflow does none standard stuff related to import and cause
problem in other setup, but it seem to tolerate your non standard setup.
Theano does the standard setup.

You can use the Theano flag dnn.include_path and dnn.library_path to tell
Theano where your cudnn.h and cudnn.so* files are.

I did not see your last error in full.

Post by Daniel Seita
Ack, sorry, half of my post got deleted! Hopefully you can still see it (i
can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting this
to work and tried to uninstall and then reinstall Theano. Just to be extra
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3],
name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2],
name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use SSE4.1
instructions, but these are available on your machine and could speed up
CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use AVX instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard.
cc:45] The TensorFlow library wasn't compiled to use FMA instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems to be to install miniconda. I downloaded the bash
~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh
Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
and it seemed to work without issues.
$ conda install numpy scipy mkl nose sphinx pydot-ng
Fetching package metadata .........
Solving package specifications: .
alabaster: 0.7.10-py27_0
babel: 2.4.0-py27_0
docutils: 0.13.1-py27_0
imagesize: 0.7.1-py27_0
jinja2: 2.9.6-py27_0
libgfortran: 3.0.0-1
markupsafe: 0.23-py27_2
mkl: 2017.0.1-0
nose: 1.3.7-py27_1
numpy: 1.13.0-py27_0
pydot-ng: 1.0.0.15-py27_0
pygments: 2.2.0-py27_0
pytz: 2017.2-py27_0
scipy: 0.19.0-np113py27_0
snowballstemmer: 1.2.1-py27_0
sphinx: 1.6.2-py27_0
sphinxcontrib: 1.0-py27_0
sphinxcontrib-websupport: 1.0.1-py27_0
typing: 3.6.1-py27_0
conda: 4.3.21-py27_0 --> 4.3.22-py27_0
Proceed <span style="color: #660;" class="styled-by-prettify
--

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Pascal Lamblin]

Something weird seems to be happening:
- theano detects that cuDNN is available, in fact it called
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage to
find the cudnn library.

Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to
LD_LIBRARY_PATH)?

Post by Frédéric Bastien
Your cudnn.h file should not be in the lib64 directory, but in an include
directory. Tensorflow does none standard stuff related to import and cause
problem in other setup, but it seem to tolerate your non standard setup.
Theano does the standard setup.
You can use the Theano flag dnn.include_path and dnn.library_path to tell
Theano where your cudnn.h and cudnn.so* files are.
I did not see your last error in full.

Post by Daniel Seita
Ack, sorry, half of my post got deleted! Hopefully you can still see it
(i can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting this
to work and tried to uninstall and then reinstall Theano. Just to be extra
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3
], name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2
], name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/cpu_feature_guard
.cc:45] The TensorFlow library wasn't compiled to use SSE4.1
instructions, but these are available on your machine and could speed up
CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/cpu_feature_guard
.cc:45] The TensorFlow library wasn't compiled to use AVX instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/cpu_feature_guard
.cc:45] The TensorFlow library wasn't compiled to use FMA instructions,
but these are available on your machine and could speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems to be to install miniconda. I downloaded the bash
~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh
Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
and it seemed to work without issues.
$ conda install numpy scipy mkl nose sphinx pydot-ng
Fetching package metadata .........
Solving package specifications: .
alabaster: 0.7.10-py27_0
babel: 2.4.0-py27_0
docutils: 0.13.1-py27_0
imagesize: 0.7.1-py27_0
jinja2: 2.9.6-py27_0
libgfortran: 3.0.0-1
markupsafe: 0.23-py27_2
mkl: 2017.0.1-0
nose: 1.3.7-py27_1
numpy: 1.13.0-py27_0
pydot-ng: 1.0.0.15-py27_0
pygments: 2.2.0-py27_0
pytz: 2017.2-py27_0
scipy: 0.19.0-np113py27_0
snowballstemmer: 1.2.1-py27_0
sphinx: 1.6.2-py27_0
sphinxcontrib: 1.0-py27_0
sphinxcontrib-websupport: 1.0.1-py27_0
typing: 3.6.1-py27_0
conda: 4.3.21-py27_0 --> 4.3.22-py27_0
Proceed <span style="color: #660;" class="styled-by-prettify
--

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

Hi nouiz and Pascal, thanks for the responses. I've been busy using the CPU
version of Theano in the meantime, so sorry for the delay in responding.

nouiz:

I actually had my `cudnn.h` file in both a `lib64` directory and an
`include` directory:

~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/include/cudnn.h
~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/lib64/cudnn.h

I must have copied them to both when I was installing it. Also, here is my
error message in full, assuming that my `~/.theanorc` file is

~$ cat ~/.theanorc
[global]
device = cuda
floatX = float64

[cuda]
root = /usr/local/cuda-8.0
~$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
Traceback (most recent call last):
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py",
line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])

This happens if I also set the device to be `cuda0` instead of `cuda`, and
trying with `float32` instead of `float64`.

Post by Pascal Lamblin
- theano detects that cuDNN is available, in fact it called
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage to
find the cudnn library.
Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to
LD_LIBRARY_PATH)?

Post by Frédéric Bastien
Your cudnn.h file should not be in the lib64 directory, but in an include
directory. Tensorflow does none standard stuff related to import and cause
problem in other setup, but it seem to tolerate your non standard setup.
Theano does the standard setup.
You can use the Theano flag dnn.include_path and dnn.library_path to tell
Theano where your cudnn.h and cudnn.so* files are.
I did not see your last error in full.

Post by Daniel Seita
Ack, sorry, half of my post got deleted! Hopefully you can still see it
(i can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting this
to work and tried to uninstall and then reinstall Theano. Just to be extra
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3
], name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2
], name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use
SSE4.1 instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use
AVX instructions, but these are available on your machine and could speed
up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use
FMA instructions, but these are available on your machine and could speed
up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems to be to install miniconda. I downloaded the bash
~/Downloads$ bash Miniconda2-latest-Linux-x86_64.sh
Welcome to Miniconda2 4.3.21 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
and it seemed to work without issues.
$ conda install numpy scipy mkl nose sphinx pydot-ng
Fetching package metadata .........
Solving package specifications: .
alabaster: 0.7.10-py27_0
babel: 2.4.0-py27_0
docutils: 0.13.1-py27_0
imagesize: 0.7.1-py27_0
jinja2: 2.9.6-py27_0
libgfortran: 3.0.0-1
markupsafe: 0.23-py27_2
mkl: 2017.0.1-0
nose: 1.3.7-py27_1
numpy: 1.13.0-py27_0
pydot-ng: 1.0.0.15-py27_0
pygments: 2.2.0-py27_0
pytz: 2017.2-py27_0
scipy: 0.19.0-np113py27_0
snowballstemmer: 1.2.1-py27_0
sphinx: 1.6.2-py27_0
sphinxcontrib: 1.0-py27_0
sphinxcontrib-websupport: 1.0.1-py27_0
typing: 3.6.1-py27_0
conda: 4.3.21-py27_0 --> 4.3.22-py27_0
Proceed <span style="color: #660;" class="styled-by-prettify
--

---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

[Daniel Seita]

Pascal:

your plan worked! I previously had:

$ echo $LD_LIBRARY_PATH
/usr/local/lib:/usr/local/cuda-8.0:/usr/local/cuda-8.0/lib64:/usr/local/cuda
-8.0/include:/home/daniel/mjpro131/bin:/home/daniel/mjpro131
~$ echo $LIBRARY_PATH
/usr/local/cuda/lib64


AH! My library path actually had `cuda` and not `cuda-8.0`. Gaaaah. I reset
my bashrc file with the correct cuda folder and:


$ cat ~/.theanorc
[global]
device = cuda
floatX = float64

[cuda]
root = /usr/local/cuda-8.0
~$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
Using cuDNN version 5105 on context None
Mapped name None to device cuda: TITAN X (Pascal) (0000:01:00.0)

In [2]:



Awesome! No wonder this was a problem, it was a bit hard for me to see.
Anyway I think things look OK. Thank you very much.

Pascal and nouiz, just one last thing, this GPU backend is the first time
that Theano can run float64 with the GPU, right? I'm hoping to take
existing code which uses float64 only and running it on the GPU. (That's
why I've been using the CPU recently, to use float64.)

Post by Daniel Seita
Hi nouiz and Pascal, thanks for the responses. I've been busy using the
CPU version of Theano in the meantime, so sorry for the delay in responding.
I actually had my `cudnn.h` file in both a `lib64` directory and an
~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/include/cudnn.
h
~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/lib64/cudnn.h
I must have copied them to both when I was installing it. Also, here is my
error message in full, assuming that my `~/.theanorc` file is
~$ cat ~/.theanorc
[global]
device = cuda
floatX = float64
[cuda]
root = /usr/local/cuda-8.0
~$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])')
RuntimeError: Could not find cudnn library (looked for v5[.1])
This happens if I also set the device to be `cuda0` instead of `cuda`, and
trying with `float32` instead of `float64`.

Post by Pascal Lamblin
- theano detects that cuDNN is available, in fact it called
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage
to find the cudnn library.
Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to
LD_LIBRARY_PATH)?

Post by Frédéric Bastien
Your cudnn.h file should not be in the lib64 directory, but in an
include directory. Tensorflow does none standard stuff related to import
and cause problem in other setup, but it seem to tolerate your non standard
setup. Theano does the standard setup.
You can use the Theano flag dnn.include_path and dnn.library_path to
tell Theano where your cudnn.h and cudnn.so* files are.
I did not see your last error in full.

Post by Daniel Seita
Ack, sorry, half of my post got deleted! Hopefully you can still see it
(i can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting
this to work and tried to uninstall and then reinstall Theano. Just to be
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2,
3], name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3,
2], name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use SSE4.1 instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available
on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use AVX instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use FMA instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN
installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems t

[Pascal Lamblin]

Post by Daniel Seita
your plan worked!

Nice to hear that :)

Post by Daniel Seita
Pascal and nouiz, just one last thing, this GPU backend is the first time
that Theano can run float64 with the GPU, right? I'm hoping to take
existing code which uses float64 only and running it on the GPU. (That's
why I've been using the CPU recently, to use float64.)

Yes, indeed.
Note that float64 performance can be much slower than float32, depending on
the GPU. For instance, on a Titan X (Pascal), it would be 32x slower than
float32, K40 and K80 should be only 3x slower, and P100 / GP100 2x slower.

Post by Daniel Seita

Post by Daniel Seita
Hi nouiz and Pascal, thanks for the responses. I've been busy using the
CPU version of Theano in the meantime, so sorry for the delay in responding.
I actually had my `cudnn.h` file in both a `lib64` directory and an
~$ ls -lh /usr/local/cuda-8.0/include/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/include/cudnn
.h
~$ ls -lh /usr/local/cuda-8.0/lib64/cudnn.h
-r--r--r-- 1 root root 98K Oct 17 2016 /usr/local/cuda-8.0/lib64/cudnn.h
I must have copied them to both when I was installing it. Also, here is
my error message in full, assuming that my `~/.theanorc` file is
~$ cat ~/.theanorc
[global]
device = cuda
floatX = float64
[cuda]
root = /usr/local/cuda-8.0
~$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import theano
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 164, in <module>
use(config.device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 151, in use
init_dev(device)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/__init__.py"
, line 68, in init_dev
context.cudnn_handle = dnn._make_handle(context)
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 80, in _make_handle
cudnn = _dnn_lib()
File
"/home/daniel/anaconda2/lib/python2.7/site-packages/theano/gpuarray/dnn.py"
, line 67, in _dnn_lib
raise RuntimeError('Could not find cudnn library (looked for v5[.1])'
)
RuntimeError: Could not find cudnn library (looked for v5[.1])
This happens if I also set the device to be `cuda0` instead of `cuda`,
and trying with `float32` instead of `float64`.

Post by Pascal Lamblin
- theano detects that cuDNN is available, in fact it called
_dnn_check_compile() and _dnn_check_version() successfully
- however, calling _dnn_lib() failed, which means ctypes did not manage
to find the cudnn library.
Is /usr/local/cuda-8.0/lib64 in your LIBRARY_PATH (in addition to
LD_LIBRARY_PATH)?

Post by Frédéric Bastien
Your cudnn.h file should not be in the lib64 directory, but in an
include directory. Tensorflow does none standard stuff related to import
and cause problem in other setup, but it seem to tolerate your non standard
setup. Theano does the standard setup.
You can use the Theano flag dnn.include_path and dnn.library_path to
tell Theano where your cudnn.h and cudnn.so* files are.
I did not see your last error in full.

Post by Daniel Seita
Ack, sorry, half of my post got deleted! Hopefully you can still see
it (i can find it by looking at the original post but it's in a really ugly
format, sorry).

Post by Daniel Seita
I was running into some more difficulties, so I gave up on getting
this to work and tried to uninstall and then reinstall Theano. Just to be
- Ubuntu 16.04
- Cuda 8.0, stored in `usr/local/cuda-8.0`
- Titan X GPU with Pascal
$ ls /usr/local/cuda-8.0/lib64/cudnn.h
/usr/local/cuda-8.0/lib64/cudnn.h
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '1.1.0'
In [3]: tf.GPUOptions
Out[3]: tensorflow.core.protobuf.config_pb2.GPUOptions
...: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2,
3], name='a')
...: b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3,
2], name='b')
...: c = tf.matmul(a,b)
...: print(sess.run(c))
2017-06-16 16:10:54.402311: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use SSE4.1 instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.402328: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use SSE4.2 instructions, but these are available
on your machine and could speed up CPU computations.
2017-06-16 16:10:54.402346: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use AVX instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.402350: W
tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library
wasn't compiled to use AVX2 instructions, but these are available on
your machine and could speed up CPU computations.
2017-06-16 16:10:54.402356: W tensorflow/core/platform/
cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to
use FMA instructions, but these are available on your machine and could
speed up CPU computations.
2017-06-16 16:10:54.527167: I
tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:901] successful NUMA
node read from SysFS had negative value (-1), but there must be at least
one NUMA node, so returning NUMA node zero
2017-06-16 16:10:54.527553: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:887] Found device 0 with
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:01:00.0
Total memory: 11.90GiB
Free memory: 11.38GiB
2017-06-16 16:10:54.527565: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0
2017-06-16 16:10:54.527568: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0: Y
2017-06-16 16:10:54.527590: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow
0000:01:00.0)
[[ 22. 28.]
[ 49. 64.]]
This looks like it indicates a successful GPU and/or cuDNN installation.
Great, now let's install the *development version* of Theano. The
http://deeplearning.net/software/theano_versions/dev/install_ubuntu.html
The first step seems t