How To Share Your Host GPU With Docker Container

Is our host can share GPU with a docker container? Yes, and this is an important feature to apply to our programming environment.

For example, we build many environments in our previous labs that utilize our GPU CUDA features, such as the TensorFlow and PyTorch labs. As we can optimize and deploy our AI machine learning applications on GPU-accelerated.

And it’s time to apply the same labs, but using the docker containers, as it’s the easiest and the fastest way to deploy our environment, with GPU and CUDA support, as a piece of cake.

In This lab, we will do:

• Update Our Ubuntu 20.04 LTS System
• Install Docker Engine
• Install Nvidia GPU Driver
• Install Nvidia Docker Support
• Test Our System And Run Our Hello World GPU Docker Container
• Install TensorFlow GPU Docker Container

Step1: Update Ubuntu 20.04

# apt update
# apt upgrade

Step2: Install Docker Engine

Installing Docker Repository and GPG

# apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
# echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker Engine

# apt-get update
# apt-get install docker-ce docker-ce-cli containerd.io

Step3: Install Nvidia GPU Driver

We will only need to install Nvidia Driver and Nvidia Docker Support, and no need to install the CUDA GPU ToolKit, so we will start by:

Pre-requests

Make sure that all Linux required headers are installed.

# apt install linux-headers-$(uname -r) -y

Install development and other required building tools

# apt install build-essential

To ensure that we have an Nvidia GPU Graphic Card with the CUDA Enabled feature, the compute capability is listed here for Nvidia CUDA GPUs Graphic Cards.

# lspci | grep -i nvidia

Adding the Nvidia Official Repositories

Downloading and adding the official NVIDIA repository Pin file from the NVIDIA website for Ubuntu 20.04 and adding the GPG key of the official NVIDIA package repository.

# wget -O /etc/apt/preferences.d/cuda-repository-pin-600 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
# apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
# add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
# apt update
# apt upgrade

Install Nvidia GPU Driver

# apt install nvidia-driver-510

And you can get more information about your GPU by:

# apt install hwinfo
# hwinfo --gfxcard --short

# lshw -C display

# nvidia-smi

Step4: Install Nvidia Docker Support

For newly docker versions, including and after the 19.03, we will need to use the nvidia-container-toolkit package, and the --gpus all flag.

Enable Nvidia Docker Repository

# distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
# curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
# curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
# apt update

Do not worry if it automatically applies Ubuntu 18.04 repo. It’s compatible and works fine.

Install Nvidia Docker Container Toolkit package

# apt install nvidia-container-toolkit

Restart The Docker Service

# systemctl restart docker

Step5: Verify and Test The Docker Engine And Official Nvidia GPU Docker

Verify that Docker Engine is installed correctly by running:

# docker run hello-world

The output is a message that shows that your installation is working correctly.

Test Nvidia Container Single GPU With the Latest CUDA Docker Image

# docker run --gpus all nvidia/cuda:11.0-base nvidia-smi

Testing with Multiple GPUS

# docker run --gpus 2 nvidia/cuda:11.0-base nvidia-smi

The output is nvidia-smi information.

Step6: Install And Run TensorFlow GPU Docker Container

Run And Test TensorFlow GPU Container Image 

# docker run --gpus all -it --rm tensorflow/tensorflow:latest-gpu \
   python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

Start Use TensorFlow GPU Container With Bash Shell Session

# docker run --gpus all -it tensorflow/tensorflow:latest-gpu bash

You can find more examples and configurations about TensorFlow docker here