Click on the above“Beginner Learning Vision” to add it to your favorites or “pin”
Important content delivered to you first
Introduction
This article introduces the detailed process and code for setting up a deep learning environment using RTX 3090 with Pytorch, TensorFlow, Keras, etc.
The author is a graduate student at Sun Yat-sen University, a combination of a medical student and a computer science student, and a machine learning enthusiast.
Recently, I just got a 3090 and found that the various environment setups written online are quite chaotic and slow. So I tested the fastest 3090 configuration environment, and welcome any additions!
Basic environment (the whole process takes about 5 minutes or even less)
py37 or py38 cuda11.0 cudnn8.0.4 tf2.5 (tf-nightly) or tf1.15.4 pytorch1.7 keras2.3(1) Download from the official website and install the graphics card driver:
bash NVIDIA-Linux-x86_64-455.23.04.run(2) Install Anaconda and change the source
bash Anaconda3-5.2.0-Linux-x86_64.sh vim ~/.bashrc export PATH=/home/XXX/anaconda3/bin:$PATH (XXX is your username) (add this line at the end of the file) source ~/.bashrc conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/ conda config --set show_channel_urls yes after vim ~/.condarc, delete defaults(3) Create a virtual environment, usually use py37 or py38 (all operations below are in the virtual environment)
conda create -n exp38 python==3.8 conda activate exp38(4) Install cuda11.0 and pytorch1.7 (no need to conda install cudatoolkit==11.0)
conda install pytorch torchvision cudatoolkit=11(5) Install cudnn8 (because conda does not yet support downloading cudnn with cudatoolkit=11)
Download cudnn from https://developer.nvidia.com/rdp/cudnn-download, unzip it, go to the cuda/lib64 path, and copy all the files inside to the corresponding lib of the virtual environment (exp38)
(6) Install tf2.5 (do not install tensorflow-gpu==2.4.0rc0, it will report an error ‘NoneType’ object has no attribute ‘TFE_MonitoringDeleteBuckets’)
pip install tf-nightly-gpu -i http://pypi.douban.com/simple --trusted-host pypi.douban.com pip install tf-nightly -i http://pypi.douban.com/simple --trusted-host pypi.douban.com(7) Install tf1.15.4
Refer to this expert’s tf1.15.4 installation steps: https://blog.csdn.net/wu496963386/article/details/109583045?utm_medium=distribute.wap_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-2.wap_blog_relevant_pic
pip install google_pasta-0.2.0-py3-none-any.whl nvidia_cublas-11.2.1.74-cp36-cp36m-linux_x86_64.whl nvidia_cuda_cupti-11.1.69-cp36-cp36m-linux_x86_64.whl nvidia_cuda_nvcc-11.1.74-cp36-cp36m-linux_x86_64.whl nvidia_cuda_nvrtc-11.1.74-cp36-cp36m-linux_x86_64.whl nvidia_cuda_runtime-11.1.74-cp36-cp36m-linux_x86_64.whl nvidia_cudnn-8.0.4.30-cp36-cp36m-linux_x86_64.whl nvidia_cufft-10.3.0.74-cp36-cp36m-linux_x86_64.whl nvidia_curand-10.2.2.74-cp36-cp36m-linux_x86_64.whl nvidia_cusolver-11.0.0.74-cp36-cp36m-linux_x86_64.whl nvidia_cusparse-11.2.0.275-cp36-cp36m-linux_x86_64.whl nvidia_dali_cuda110-0.26.0-1608709-py3-none-manylinux2014_x86_64.whl nvidia_dali_nvtf_plugin-0.26.0+nv20.10-cp36-cp36m-linux_x86_64.whl nvidia_nccl-2.7.8-cp36-cp36m-linux_x86_64.whl nvidia_tensorrt-7.2.1.4-cp36-none-linux_x86_64.whl tensorflow_estimator-1.15.1-py2.py3-none-any.whl nvidia_tensorboard-1.15.0+nv20.10-py3-none-any.whl nvidia_tensorflow-1.15.4+nv20.10-cp36-cp36m-linux_x86_64.whl -i http://pypi.douban.com/simple --trusted-host pypi.douban.com(8) Install keras2.3
pip install keras==2.3 -i http://pypi.douban.com/simple --trusted-host pypi.douban.com(9) Testing (you can also test using cuda10.2, but it seems you cannot write data to the GPU)
Pytorch
TensorFlow-2.5 or 1.15.4
Keras (testing requires modifying part of the source code _get_available_gpus())
import tensorflow as tf import keras.backend.tensorflow_backend as tfback print("tf.__version__ is", tf.__version__) print("tf.keras.__version__ is:", tf.keras.__version__) def _get_available_gpus(): if tfback._LOCAL_DEVICES is None: devices = tf.config.list_logical_devices() tfback._LOCAL_DEVICES = [x.name for x in devices] return [x for x in tfback._LOCAL_DEVICES if 'device:gpu' in x.lower()] tfback._get_available_gpus = _get_available_gpus from keras import backend as KK tensorflow_backend._get_available_gpus()
Postscript: In fact, the 3090 requires cuda11.1, but pytorch and tf currently only support 11.0. Moreover, to be honest, there is no need to configure cuda and cudnn separately, just do it in the virtual environment.
Update 20210102: Tested tf1.15.4, confirmed usable, data can be written to GPU. The article has been updated accordingly.
Good News!
Beginner Learning Vision Knowledge Group
Is now open to the public👇👇👇
Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial
Reply "Extension Module Chinese Tutorial" in the "Beginner Learning Vision" public account to download the first Chinese version of the OpenCV extension module tutorial, covering more than twenty chapters including extension module installation, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, etc.
Download 2: Python Vision Practical Project 52 Lectures
Reply "Python Vision Practical Project" in the "Beginner Learning Vision" public account to download 31 vision practical projects including image segmentation, mask detection, lane line detection, vehicle counting, adding eyeliner, license plate recognition, character recognition, emotion detection, text content extraction, facial recognition, etc., to help quickly learn computer vision.
Download 3: OpenCV Practical Project 20 Lectures
Reply "OpenCV Practical Project 20 Lectures" in the "Beginner Learning Vision" public account to download 20 practical projects based on OpenCV, achieving advanced learning of OpenCV.
Group Chat
Welcome to join the reader group of the public account to exchange ideas with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat number below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format, otherwise, it will not be approved. After successfully adding, you will be invited to enter the relevant WeChat group according to your research direction. Please do not send advertisements in the group, otherwise you will be removed. Thank you for your understanding~