Showing posts with label CUDA. Show all posts

CUDA GPU Concurrent (parallel) Programming

C, C++, and Python 3 code running asynchronously using

threads
queues
other concurrent programming techniques

Relevance

CUDA
OpenCL
Metal
OpenAcc
PyCUDA
jCuda

Hardware:

AMD
Apple
FPGA
multi-core CPUs

Pitfalls of Concurrent Programming

race conditions

the expected order of thread operations is not followed

resource contention

two or more threads attempt to modify the same memory

deadlock

one or more processes are blocked by waiting for a resource

live locks

two or more processes are stuck in a loop, but cannot finish while waiting for resources

resource over-utilization

too few or too many threads, context switching
memory required is too large
memory changes too often

resource under-utilization

sitting idle

Semaphore, for all intents and purposes, is an atomic variable that has more than one thread requiring it, which means that a predefined number of threads can use the semaphore to enter a critical section of code.

A lock is the more restrictive parent asynchronous mechanism for a single thread to enter a critical section of code. Thus a semaphore is a more relaxed form of lock.

Concurrent Programming Algorithms

Dining Philosophers

multiple processes require the same resources to complete their jobs

Producer-Consumer

consumers need to read the data

in order
no duplication

Producers add data in order it needs to be processed

Sleeping Barber

customers are waiting
single barber
if the barber is sleeping customer should wake him
If there is no space in the queue, customers are not added

Data and Code Synchronization

References

As an Amazon Associate I earn from qualifying purchases.

Julia CUDA

import Pkg
Pkg.add("CUDAapi")

Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions... Installed CUDAapi ─ v4.0.0 Updating `~/.julia/environments/v1.4/Project.toml` [3895d2a7] + CUDAapi v4.0.0 Updating `~/.julia/environments/v1.4/Manifest.toml` [3895d2a7] + CUDAapi v4.0.0

using CUDAapi
has_CUDA = "false"
if has_cuda()
@info "CUDA is on"
has_CUDA = "true"
end
println("it is ", has_CUDA, " that I have CUDA.")

As an Amazon Associate I earn from qualifying purchases.

Julia CUDA

import Pkg
Pkg.add("CUDAapi")

using CUDAapi
has_CUDA = "false"
if has_cuda()
@info "CUDA is on"
has_CUDA = "true"
end
println("it is ", has_CUDA, " that I have CUDA.")

As an Amazon Associate I earn from qualifying purchases.

Ubuntu: installing TensorFlow for NVidia (CUDA) GPU

I am setting TensorFlow on:

Ubuntu 16.04 LTS 64-bit
16 GiB RAM
AMD Athlon(tm) II X4 640 Processor × 4
GeForce GTX 1050 Ti/PCIe/SSE2

Check if you have NVidia CUDA GPU

uki@uki-p6710f:~$  lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1c82 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fb9 (rev a1)

Check the name of your OS

uki@uki-p6710f:~$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Check C compiler

uki@uki-p6710f:~$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ubuntu Headers

uki@uki-p6710f:~$ sudo apt-get install linux-headers-$(uname -r)
[sudo] password for uki:
Reading package lists... Done
Building dependency tree
Reading state information... Done
linux-headers-4.4.0-62-generic is already the newest version (4.4.0-62.83).
linux-headers-4.4.0-62-generic set to manually installed.
The following packages were automatically installed and are no longer required:
linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 89 not upgraded.

Download newest CUDA installer (1.4GB)

https://developer.nvidia.com/cuda-downloads

https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run

Execute CUDA installer

cd ~/Downloads/

uki@uki-p6710f:~/Downloads$ ls -alt

Mar 5 14:54 cuda_8.0.61_375.26_linux.run

Feb 2 09:53 NVIDIA-Linux-x86_64-375.10.run

$ sudo sh cuda_8.0.61_375.26_linux.run

Set environment variables

uki@uki-p6710f:~$ nano ~/.bashrc

##### CUDA
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64

uki@uki-p6710f:~$ echo $LD_LIBRARY_PATH

/usr/local/cuda-8.0/lib64

Install TensorFlow via pip3 (Python 3.5)

uki@uki-p6710f:~$ python --version

Python 3.5.2 :: Anaconda 4.3.0 (64-bit)

$ sudo apt install python3-pip

$ conda info --envs
# conda environments:
#
tensorflow /home/uki/anaconda3/envs/tensorflow
root * /home/uki/anaconda3

$ conda env create -f /Users/ukilucas/dev/uki.guru/conda_enviroment_GPU.yml

$source activate tensorflow

Setting Jupyter kernel to match Python conda environment

http://ukitech.blogspot.com/2017/02/kernel.html

As an Amazon Associate I earn from qualifying purchases.

Ubuntu: installing TensorFlow for NVidia (CUDA) GPU

I am setting TensorFlow on:

Ubuntu 16.04 LTS 64-bit
16 GiB RAM
AMD Athlon(tm) II X4 640 Processor × 4
GeForce GTX 1050 Ti/PCIe/SSE2

Check if you have NVidia CUDA GPU

uki@uki-p6710f:~$  lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1c82 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fb9 (rev a1)

Check the name of your OS

uki@uki-p6710f:~$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Check C compiler

uki@uki-p6710f:~$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ubuntu Headers

Download newest CUDA installer (1.4GB)

https://developer.nvidia.com/cuda-downloads

https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run

Execute CUDA installer

cd ~/Downloads/

uki@uki-p6710f:~/Downloads$ ls -alt

Mar 5 14:54 cuda_8.0.61_375.26_linux.run

Feb 2 09:53 NVIDIA-Linux-x86_64-375.10.run

$ sudo sh cuda_8.0.61_375.26_linux.run

Set environment variables

uki@uki-p6710f:~$ nano ~/.bashrc

##### CUDA
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64

uki@uki-p6710f:~$ echo $LD_LIBRARY_PATH

/usr/local/cuda-8.0/lib64

Install TensorFlow via pip3 (Python 3.5)

uki@uki-p6710f:~$ python --version

Python 3.5.2 :: Anaconda 4.3.0 (64-bit)

$ sudo apt install python3-pip

$ conda info --envs
# conda environments:
#
tensorflow /home/uki/anaconda3/envs/tensorflow
root * /home/uki/anaconda3

$ conda env create -f /Users/ukilucas/dev/uki.guru/conda_enviroment_GPU.yml

$source activate tensorflow

Setting Jupyter kernel to match Python conda environment

http://ukitech.blogspot.com/2017/02/kernel.html

As an Amazon Associate I earn from qualifying purchases.

Installing TensorFlow 1.0 as conda environment with yml definition file on MacBook Pro with NVidia GPU support

My conda environment yml file on GitHub

https://github.com/UkiDLucas/uki.guru/blob/master/conda_enviroment_GPU.yml

Execute yml file, set environment

$ conda env create --file .../conda_environment.yml
$ conda info --envs

$ source activate py352_tf_gpu

Update jupyter notebook (iPython) kernel with new environment we created

$ python -m ipykernel install --user --name py352_tf_gpu --display-name "conda env py352_tf_gpu"$ jupyter notebook

Test run in jupyter notebook

You can see that jupyter notebook recognize GPU

You can see execution on MacBook Pro (late 2013 with NVidia). Both CPU and GPU are measured.

PLEASE READ THIS if you get DEAD KERNELS in GPU mode:
https://github.com/aymericdamien/TensorFlow-Examples/issues/38#issuecomment-280468579

As an Amazon Associate I earn from qualifying purchases.

Installing TensorFlow 1.0 as conda environment with yml definition file on MacBook Pro with NVidia GPU support

My conda environment yml file on GitHub

https://github.com/UkiDLucas/uki.guru/blob/master/conda_enviroment_GPU.yml

Execute yml file, set environment

$ conda env create --file .../conda_environment.yml
$ conda info --envs

$ source activate py352_tf_gpu

Update jupyter notebook (iPython) kernel with new environment we created

$ python -m ipykernel install --user --name py352_tf_gpu --display-name "conda env py352_tf_gpu"$ jupyter notebook

Test run in jupyter notebook

You can see that jupyter notebook recognize GPU

You can see execution on MacBook Pro (late 2013 with NVidia). Both CPU and GPU are measured.

PLEASE READ THIS if you get DEAD KERNELS in GPU mode:
https://github.com/aymericdamien/TensorFlow-Examples/issues/38#issuecomment-280468579

As an Amazon Associate I earn from qualifying purchases.

Google TensorFlow 1.0 - Mac with Nvidia GPU installation and testing in jupyter notebook

TensorFlow 1.0 is promising more stable GPU build for Mac, let's test it.

Make sure you have CUDA installed

$ echo $DYLD_LIBRARY_PATH/usr/local/cuda/lib:

http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/#axzz4Ynrb42hi

$ echo $CUDA_HOME/Developer/NVIDIA/CUDA-8.0/

https://developer.nvidia.com/cudnn

$ sudo easy_install pip

Set variable with TensorFlow 1.0 binary

export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl
$ echo $TF_BINARY_URL https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl

Use pip to install TensorFlow

$ sudo pip3 install --upgrade $TF_BINARY_URL Password:The directory '/Users/ukilucas/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/ukilucas/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting tensorflow-gpu==1.0.0 from https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl Downloading https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl (89.0MB) 100% |████████████████████████████████| 89.0MB 8.8kB/s Requirement already up-to-date: numpy>=1.11.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: wheel>=0.26 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: six>=1.10.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: protobuf>=3.1.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Collecting setuptools (from protobuf>=3.1.0->tensorflow-gpu==1.0.0) Downloading setuptools-34.2.0-py2.py3-none-any.whl (389kB) 100% |████████████████████████████████| 399kB 2.0MB/s Requirement already up-to-date: appdirs>=1.4.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: packaging>=16.8 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: pyparsing in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Installing collected packages: tensorflow-gpu, setuptools Found existing installation: tensorflow-gpu 0.12.1 Uninstalling tensorflow-gpu-0.12.1: Successfully uninstalled tensorflow-gpu-0.12.1 Found existing installation: setuptools 34.1.1 Uninstalling setuptools-34.1.1: Successfully uninstalled setuptools-34.1.1Successfully installed setuptools-34.2.0 tensorflow-gpu-1.0.0(tensorflow_gpu) uki@UkiPEsMcBookPro 192.168.1.24 19:44 dev $

Start jupyter notebook in the right (same) environment

(tensorflow_gpu) $ jupyter notebook

Try in jupyter notebook

import tensorflow as tf
import time
from tensorflow.python.client import device_lib

def get_available_CPU_GPU():
devices = device_lib.list_local_devices()
#return [x.name for x in devices if x.device_type == 'CPU']
return [x.name for x in devices ]

print(get_available_CPU_GPU())
['/cpu:0', '/gpu:0']

start = timeit.timeit()
print ("starting")with tf.device('/gpu:0'):
# [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)starting
[[ 22. 28.]
[ 49. 64.]]
elapsed -0.003607149003073573

It is a win so far, time will show if it is usable.

Reference:

https://www.tensorflow.org/install/install_mac

As an Amazon Associate I earn from qualifying purchases.

Google TensorFlow 1.0 - Mac with Nvidia GPU installation and testing in jupyter notebook

TensorFlow 1.0 is promising more stable GPU build for Mac, let's test it.

Make sure you have CUDA installed

$ echo $DYLD_LIBRARY_PATH/usr/local/cuda/lib:

http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/#axzz4Ynrb42hi

$ echo $CUDA_HOME/Developer/NVIDIA/CUDA-8.0/

https://developer.nvidia.com/cudnn

$ sudo easy_install pip

Set variable with TensorFlow 1.0 binary

export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl
$ echo $TF_BINARY_URL https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl

Use pip to install TensorFlow

$ sudo pip3 install --upgrade $TF_BINARY_URL Password:The directory '/Users/ukilucas/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/ukilucas/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting tensorflow-gpu==1.0.0 from https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl Downloading https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl (89.0MB) 100% |████████████████████████████████| 89.0MB 8.8kB/s Requirement already up-to-date: numpy>=1.11.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: wheel>=0.26 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: six>=1.10.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: protobuf>=3.1.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Collecting setuptools (from protobuf>=3.1.0->tensorflow-gpu==1.0.0) Downloading setuptools-34.2.0-py2.py3-none-any.whl (389kB) 100% |████████████████████████████████| 399kB 2.0MB/s Requirement already up-to-date: appdirs>=1.4.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: packaging>=16.8 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: pyparsing in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Installing collected packages: tensorflow-gpu, setuptools Found existing installation: tensorflow-gpu 0.12.1 Uninstalling tensorflow-gpu-0.12.1: Successfully uninstalled tensorflow-gpu-0.12.1 Found existing installation: setuptools 34.1.1 Uninstalling setuptools-34.1.1: Successfully uninstalled setuptools-34.1.1Successfully installed setuptools-34.2.0 tensorflow-gpu-1.0.0(tensorflow_gpu) uki@UkiPEsMcBookPro 192.168.1.24 19:44 dev $

Start jupyter notebook in the right (same) environment

(tensorflow_gpu) $ jupyter notebook

Try in jupyter notebook

import tensorflow as tf
import time
from tensorflow.python.client import device_lib

def get_available_CPU_GPU():
devices = device_lib.list_local_devices()
#return [x.name for x in devices if x.device_type == 'CPU']
return [x.name for x in devices ]

print(get_available_CPU_GPU())
['/cpu:0', '/gpu:0']

start = timeit.timeit()
print ("starting")with tf.device('/gpu:0'):
# [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)starting
[[ 22. 28.]
[ 49. 64.]]
elapsed -0.003607149003073573

It is a win so far, time will show if it is usable.

Reference:

https://www.tensorflow.org/install/install_mac

As an Amazon Associate I earn from qualifying purchases.

Nvidia just released an update to their CUDA drivers for Mac

Nvidia CUDA just released an update to their Mac OS X driver.

As an Amazon Associate I earn from qualifying purchases.

Nvidia just released an update to their CUDA drivers for Mac

Nvidia CUDA just released an update to their Mac OS X driver.

As an Amazon Associate I earn from qualifying purchases.

TensorFlow: MacBook Pro: detect which CPU and GPU devices are available

from tensorflow.python.client import device_lib

def get_available_gpus():
devices = device_lib.list_local_devices()
#return [x.name for x in devices if x.device_type == 'CPU']
return [x.name for x in devices ]

print(get_available_gpus())

['/cpu:0']

Currently, I can see and execute only on CPU.


MacBook Pro i7 Late 2013

Device 0: "GeForce GT 750M" CUDA Driver Version / 

Runtime Version 8.0 / 8.0 CUDA Capability Major/Minor version number: 3.0 

Total amount of global memory: 2048 MBytes (2147024896 bytes) 

( 2) Multiprocessors, 

(192) CUDA Cores/MP: 384 CUDA Cores GPU 

Max Clock rate: 926 MHz (0.93 GHz) 

Memory Clock rate: 2508 Mhz Memory Bus Width: 128-bit L2 

Cache Size: 262144 bytes

http://osxdaily.com/2017/01/08/disable-gpu-switching-macbook-pro/

Still not working with TensorFlow

start = timeit.timeit()
print ("starting")

with tf.device('/cpu:0'):
    # [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)

starting
[[ 22.  28.]
 [ 49.  64.]]
elapsed -0.0033593010011827573

As an Amazon Associate I earn from qualifying purchases.

TensorFlow: MacBook Pro: detect which CPU and GPU devices are available

['/cpu:0']

Currently, I can see and execute only on CPU.


MacBook Pro i7 Late 2013


Device 0: "GeForce GT 750M" CUDA Driver Version / 


Runtime Version 8.0 / 8.0 CUDA Capability Major/Minor version number: 3.0 


Total amount of global memory: 2048 MBytes (2147024896 bytes) 


( 2) Multiprocessors, 


(192) CUDA Cores/MP: 384 CUDA Cores GPU 


Max Clock rate: 926 MHz (0.93 GHz) 


Memory Clock rate: 2508 Mhz Memory Bus Width: 128-bit L2 


Cache Size: 262144 bytes

http://osxdaily.com/2017/01/08/disable-gpu-switching-macbook-pro/

Still not working with TensorFlow

start = timeit.timeit()
print ("starting")

with tf.device('/cpu:0'):
    # [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)

starting
[[ 22.  28.]
 [ 49.  64.]]
elapsed -0.0033593010011827573

As an Amazon Associate I earn from qualifying purchases.

CUDA: late 2013 MacBook Pro GPU: GeForce GT 750M 384 Cores

Installing CUDA on MacBook Pro

$ brew update
$ brew upgrade

$ id -g
20

$ sudo chown -R uki:20 *
Password:

$ brew link pcre

$ brew install coreutils swig
Warning: coreutils-8.26 already installed
==> Downloading https://homebrew.bintray.com/bottles/swig-3.0.11.sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring swig-3.0.11.sierra.bottle.tar.gz
🍺 /usr/local/Cellar/swig/3.0.11: 754 files, 5.5M

$ brew cask install cuda
🍺 cuda was successfully installed!

$ brew cask info cuda

cuda: 8.0.55

https://developer.nvidia.com/cuda-zone

$ kextstat | grep -i cuda

... com.nvidia.CUDA (1.1.0) ..

$ cd /usr/local/cuda/samples/
$ sudo make -C 1_Utilities/deviceQuery

Makefile NsightEclipse.xml deviceQuery deviceQuery.cpp deviceQuery.o readme.txt

$ /usr/local/cuda/samples/1_Utilities/deviceQuery/deviceQuery

Device 0: "GeForce GT 750M"
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 3.0
Total amount of global memory: 2048 MBytes (2147024896 bytes)
( 2) Multiprocessors, (192) CUDA Cores/MP: 384 CUDA Cores
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GeForce GT 750M
Result = PASS

cuDNN Download

https://developer.nvidia.com/rdp/cudnn-download

Download file for OSX: cudnn-8.0-osx-x64-v5.1.tgz

and copy the file to your favorite place..

cd ~/Dropbox/dev/NVidia_CUDA/
NVidia_CUDA $ tar zxvf cudnn-8.0-osx-x64-v5.1.tgz
cd $ /cuda/include
$ sudo cp cudnn.h /usr/local/cuda/include/
cd ../lib/
$ sudo cp libcudnn* /usr/local/cuda/lib/

Add to your bash_profile

########## CUDA cuDNN ########## created: February 6, 2017

export DYLD_LIBRARY_PATH="/usr/local/cuda/lib":$DYLD_LIBRARY_PATH

$ brew cask install java
$ brew install bazel

(carnd-term1) uki@Uki-PEs-MacBookPro 16:46 tensorflow $ TF_UNOFFICIAL_SETTING=1 ./configure

Please specify the location of python. [Default is /Users/ukilucas/anaconda3/envs/carnd-term1/bin/python]: /Users/ukilucas/anaconda3/envs/carnd-term1/bin/python

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] N

No Google Cloud Platform support will be enabled for TensorFlow

Do you wish to build TensorFlow with Hadoop File System support? [y/N] N

No Hadoop File System support will be enabled for TensorFlow

Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] y

XLA JIT support will be enabled for TensorFlow

Found possible Python library paths:

/Users/ukilucas/anaconda3/envs/carnd-term1/lib/python3.5/site-packages

Please input the desired Python library path to use. Default is [/Users/ukilucas/anaconda3/envs/carnd-term1/lib/python3.5/site-packages]

Using python library path: /Users/ukilucas/anaconda3/envs/carnd-term1/lib/python3.5/site-packages

Do you wish to build TensorFlow with OpenCL support? [y/N] N

No OpenCL support will be enabled for TensorFlow

Do you wish to build TensorFlow with CUDA support? [y/N] Y

CUDA support will be enabled for TensorFlow

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to use system default]:

Please specify the location where CUDA toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify the Cudnn version you want to use. [Leave empty to use system default]:

Please specify the location where cuDNN library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify a list of comma-separated Cuda compute capabilities you want to build with.

You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.

Please note that each additional compute capability significantly increases your build time and binary size.

[Default is: "3.5,5.2"]: 3.0

INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.

............

INFO: All external dependencies fetched successfully.

Configuration finished

bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-0.6.0-py2-none-any.whl

As an Amazon Associate I earn from qualifying purchases.

CUDA: late 2013 MacBook Pro GPU: GeForce GT 750M 384 Cores

cuDNN Download

https://developer.nvidia.com/rdp/cudnn-download

Download file for OSX: cudnn-8.0-osx-x64-v5.1.tgz

and copy the file to your favorite place..

########## CUDA cuDNN ########## created: February 6, 2017

export DYLD_LIBRARY_PATH="/usr/local/cuda/lib":$DYLD_LIBRARY_PATH

$ brew cask install java
$ brew install bazel

bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-0.6.0-py2-none-any.whl

As an Amazon Associate I earn from qualifying purchases.

My interests

Pitfalls of Concurrent Programming

Concurrent Programming Algorithms

References

Check if you have NVidia CUDA GPU

Check the name of your OS

Check C compiler

Ubuntu Headers

Download newest CUDA installer (1.4GB)

Execute CUDA installer

Set environment variables

Install TensorFlow via pip3 (Python 3.5)

Setting Jupyter kernel to match Python conda environment

Check if you have NVidia CUDA GPU

Check the name of your OS

Check C compiler

Ubuntu Headers

Download newest CUDA installer (1.4GB)

Execute CUDA installer

Set environment variables

Install TensorFlow via pip3 (Python 3.5)

Setting Jupyter kernel to match Python conda environment

My conda environment yml file on GitHub

Execute yml file, set environment

Update jupyter notebook (iPython) kernel with new environment we created

Test run in jupyter notebook

My conda environment yml file on GitHub

Execute yml file, set environment

Update jupyter notebook (iPython) kernel with new environment we created

Test run in jupyter notebook

Make sure you have CUDA installed

Set variable with TensorFlow 1.0 binary

Use pip to install TensorFlow

Start jupyter notebook in the right (same) environment

Try in jupyter notebook

Make sure you have CUDA installed

Set variable with TensorFlow 1.0 binary

Use pip to install TensorFlow

Start jupyter notebook in the right (same) environment

Try in jupyter notebook

cuDNN Download

cuDNN Download

apt quotation..