[theano-users] Could not initialize pygpu error with CUDA 9.0 and latest Theano

[Frédéric Bastien]

creating 2 env seem the right thing to do. I do not think dnn.libary_path
override LD_LIBRARY_PATH.

What is the best way to do that? Should I use separate conda environments
for theano and tensorflow, and create LD_LIBRARY_PATH in each? Does
dnn.libary_path in theanorc override LD_LIBRARY_PATH?

You could have multiple cuda version installed to have TF working.

After struggling with this error for a day, I decided to upgrade CUDA to
9.1 and CuDNN to 7.1. After that I got "your driver might be too old"
error, which was resolved by updating the driver to 396.24. Also, in the
process I found out I had older CuDNN files in /usr/lib/x86_64-linux-gnu/
directory. Not sure how they got there, perhaps because sometimes I
installed CuDNN using .deb package, and sometimes by manually copying the
files. So it's probably not a good idea to mix .deb and .run cuda
installation methods.
Anyway, now theano works fine now, but unfortunately my Tensorflow is
broken because it does not support cuda 9.1 yet... Will probably have to
compile it from source.

This is a new one. It is also very weird since gemm doesn't involve
cuLinkAddData. This may be an error message from something else.
First things first, since you are on cuda 9.0, I would recommend that
you update your driver to 384.111 or 390.*. If that doesn't help, then
I'll need some help reproducing the problem since I don't get that in any
of my environments.
I have CUDA 9.0 and CuDNN 7.0.5 on my Ubuntu 16.04, and Tensorflow
works fine. In order to install theano, I first installed miniconda, then
ran "conda install theano pygpu" and it seemed to have installed fine.
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import theano

Using cuDNN version 7005 on context None
ERROR (theano.gpuarray): Could not initialize pygpu, support disabled
File "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 227, in <module>
use(config.device)
File "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 214, in use
init_dev(device, preallocate=preallocate)
File "/home/michael/miniconda2/envs/las/lib/python3.6/site-packages/theano/gpuarray/__init__.py", line 159, in init_dev
pygpu.blas.gemm(0, tmp, tmp, 0, tmp, overwrite_c=True)
File "pygpu/blas.pyx", line 149, in pygpu.blas.gemm
File "pygpu/blas.pyx", line 47, in pygpu.blas.pygpu_blas_rgemm
pygpu.gpuarray.GpuArrayException: (b'cuLinkAddData: CUDA_ERROR_UNKNOWN: unknown error', 3)
$ conda list
#
# Name Version Build Channel
binutils_impl_linux-64 2.28.1 had2808c_3
binutils_linux-64 7.2.0 26
ca-certificates 2018.03.07 0
certifi 2018.4.16 py36_0
gcc_impl_linux-64 7.2.0 habb00fd_3
gcc_linux-64 7.2.0 26
gxx_impl_linux-64 7.2.0 hdf63c60_3
gxx_linux-64 7.2.0 26
intel-openmp 2018.0.0 8
libedit 3.1 heed3624_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 7.2.0 hdf63c60_3
libgfortran-ng 7.2.0 hdf63c60_3
libgpuarray 0.7.5 h14c3975_0
libstdcxx-ng 7.2.0 hdf63c60_3
mako 1.0.7 py36h0727276_0
markupsafe 1.0 py36hd9260cd_1
mkl 2018.0.2 1
mkl-service 1.1.2 py36h17a0993_4
mkl_fft 1.0.1 py36h3010b51_0
mkl_random 1.0.1 py36h629b387_0
ncurses 6.0 h9df7e31_2
nose 1.3.7 py36hcdf7029_2
numpy 1.14.2 py36hdbf6ddf_1
openssl 1.0.2o h20670df_0
pip 10.0.1 py36_0
pygpu 0.7.5 py36h14c3975_0
python 3.6.5 hc3d631a_2
readline 7.0 ha6073c6_4
scipy 1.0.1 py36hfc37229_0
setuptools 39.1.0 py36_0
six 1.11.0 py36h372c433_1
sqlite 3.23.1 he433501_0
theano 1.0.1 py36h6bb024c_0
tk 8.6.7 hc745277_3
wheel 0.31.0 py36_0
xz 5.2.3 h5e939de_4
zlib 1.2.11 ha838bed_2
[global]
device = cuda0
optimizer_including = cudnn
floatX = float32
[dnn]
include_path = /usr/local/cuda/include
library_path = /usr/local/cuda/lib64
[lib]
cnmem = 0.7
[nvcc]
fastmath = True
[blas]
# Only used for device = cpu
ldflags = -lopenblas
[cuda]
root = /usr/local/cuda/bin
Nvidia driver: Driver Version: 384.81
--
---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--

---
You received this message because you are subscribed to the Google
Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.