In actual applications, you need install the right versions of libraries or software such as drivers, CUDA toolkit, cuDNN, and PyTorch on a GPU cloud server.
How to Select a CUDA Version
The Compute Unified Device Architecture (CUDA) is a computing platform developed by NVIDIA, a graphics card manufacturer. CUDA™ is a general-purpose parallel computing architecture introduced by NVIDIA that enables GPUs to address complex computing problems. It includes the CUDA instruction set architecture (ISA) as well as the parallel computing engine inside the GPU. Developers can use the C language to write programs for the CUDA™ architecture that can run at ultra-high performance on CUDA-enabled processors.
Before selecting a CUDA version, learn about the computing power of the graphics card mounted to the GPU cloud server and select a CUDA version based on the computing power of the graphics card.
The steps are as follows:
Step 1: Check the computing power of the graphics card in NVIDIA official website. NVIDIA T4, for example, has the computing power of the graphics card of 7.5.
Step 2: Check the supported CUDA versions according to the computing power of the graphics card. For more information, see NVIDIA Data Center. As an example, for NVIDIA T4, CUDA 10 and later versions are supported. We recommend that you select the latest CUDA version.
How to Select a Graphics Card Driver Version
Select the driver version for the graphics card according to the identified CUDA version, as shown in the figure below. For example, if the CUDA version 11.4.3 is selected, the driver version must be later than 450.80.02 in a Linux OS. For more information, see https://docs.nvidia.com/datacenter/tesla/drivers/index.html#cuda-drivers.
How to Select a cuDNN Version
The NVIDIA CUDA Deep Neural Network Library (cuDNN) is a GPU-accelerated deep neural network primitives library capable of implementing standard routines (such as forward and reverse convolutions, pooling layers, normalization and activation layers) in a highly optimized manner. With cuDNN, researchers and developers can focus on training neural networks and developing software applications, instead of spending time making low-level GPU performance adjustments. cuDNN accelerates a wide range of deep learning frameworks including Caffe2, Chainer, Keras, MATLAB, MxNet, PaddlePaddle, PyTorch, and TensorFlow. Select the cuDNN version according to the selected CUDA version. For version correspondence and cuDNN download address, see cuDNN Archive | NVIDIA Developer.
How to Select a PyTorch Version
Select the PyTorch version according to the selected CUDA version. For version correspondence, see Previous PyTorch Versions | PyTorch.