Recommended! Learn PyTorch From Scratch

Click on "Beginner Learning Vision" above, select "Star" or "Pin"

Heavy content delivered to you first

Source: GitHub

Author: zergtant

[Guide] Today we strongly recommend a Chinese PyTorch book — PyTorch Chinese Manual, along with a preview. This book provides a quick start guide for PyTorch and aligns with the latest version, including PyTorch tutorials that have been tested to ensure they run successfully.

PyTorch is a deep learning framework designed for simple and flexible experimentation.

Since its initial release in early 2017, PyTorch has quickly become a popular choice among AI researchers and is highly regarded. PyTorch has many advantages, such as using the Python language, dynamic graph mechanism, flexible network building, and a strong community. Due to its flexible, dynamic programming environment and user-friendly interface, PyTorch is an ideal choice for rapid experimentation.

PyTorch is now the second fastest-growing open-source project on GitHub, with contributors increasing by 2.8 times in the past 12 months. Moreover, the stable version of PyTorch 1.0 was finally released at the NeurIPS conference last December. PyTorch 1.0 has introduced a series of powerful new features, rivaling its older brother TensorFlow in the deep learning framework space.

Therefore, learning PyTorch is highly beneficial!

Today we strongly recommend a Chinese PyTorch book — PyTorch Chinese Manual (pytorch handbook), along with a preview. This is an open-source book aimed at helping those who wish to use PyTorch for deep learning development and research to quickly get started, with all PyTorch tutorials guaranteed to run successfully.

Open-source address:

https://github.com/zergtant/pytorch-handbook

This is an open-source book aimed at helping those who wish to use PyTorch for deep learning development and research to quickly get started.

Due to my limited level, I referenced some online materials while writing this tutorial, and I would like to express my respect to them. I will attach the original address in each citation for everyone’s reference.

The technology of deep learning is developing rapidly, and PyTorch is constantly being updated. I will gradually improve the relevant content.

Due to the version changes of PyTorch, the tutorial version will align with the PyTorch version.

On December 8, PyTorch 1.0 was released as a stable version. The API changes are not significant, and this tutorial has been tested to ensure it runs normally in 1.0. However, the impact seems minimal, as the content is still limited. Version 0.4.1 has been archived in a new branch and will no longer be updated.

Chapter 1: Introduction to PyTorch

1. Introduction to PyTorch

2. Setting Up the PyTorch Environment

3. Deep Learning with PyTorch: A 60 Minute Blitz (Official)

• Tensors

• Autograd: Automatic Differentiation

• Neural Networks

• Training a Classifier

• Recommended Reading: Data Parallelism (Multi GPU)

4. Introduction to Related Resources

Chapter 2: Basics

Section 1: Basics of PyTorch

1. Tensors

2. Automatic Differentiation

3. Neural Network Package nn and Optimizer optm

4. Data Loading and Preprocessing

Section 2: Basics of Deep Learning and Mathematical Principles

• Basics of Deep Learning and Mathematical Principles

Section 3: Introduction to Neural Networks

• Introduction to Neural Networks

Section 4: Convolutional Neural Networks

• Convolutional Neural Networks

Section 5: Recurrent Neural Networks

• Recurrent Neural Networks

Chapter 3: Practice

Section 1: Logistic Regression Binary Classification

• Logistic Regression Binary Classification

Section 2: CNN: Handwritten Digit Recognition with MNIST Dataset

• CNN: Handwritten Digit Recognition with MNIST Dataset

Section 3: RNN Example: Predicting Cosine from Sine

• RNN Example: Predicting Cosine from Sine

Chapter 4: Improvement

Section 1: Fine-tuning

Fine-tuning

Section 2: Visualization

visdom

tensorboardx

Section 3: fastai

Section 4: Data Processing Techniques

Section 5: Parallel Computing

Chapter 5: Applications

Section 1: Introduction to Kaggle

Section 2: Structured Data

Section 3: Computer Vision

Section 4: Natural Language Processing

Section 5: Collaborative Filtering

Chapter 6: Resources

1.1 Introduction to PyTorch

1.1.1 Origin of PyTorch

Many people compare PyTorch with Google’s TensorFlow, which is certainly valid as they are the two most popular deep learning frameworks. However, when it comes to PyTorch, we should first mention Torch.

1.1.2 What is Torch?

Torch translated into Chinese: 火炬

A Tensor library like Numpy, unlike Numpy it has strong GPU support. Lua is a wrapper for Torch (Yes! you need to have a good understanding of Lua), and for that you will need LuaRocks package manager.

Torch is a tensor operation library similar to Numpy, but unlike Numpy, Torch has strong GPU support. Lua is the upper layer wrapper for Torch.

Torch is not going anywhere. PyTorch and Torch use the same C libraries that contain all the performance: TH, THC, THNN, THCUNN and they will continue to be shared.

We still and will have continued engineering on Torch itself, and we have no immediate plan to remove that.

PyTorch and Torch use the same C libraries that contain all the performance: TH, THC, THNN, THCUNN, and they will continue to share these libraries.

This answer is very clear; in fact, both PyTorch and Torch use the same underlying layers, just with different upper layer wrapper languages.

Note: Although LUA is fast, it is too niche, which is why PyTorch emerged.

1.1.3 Reintroducing PyTorch

PyTorch is an open source machine learning library for Python, based on Torch, used for applications such as natural language processing. It is primarily developed by Facebook's artificial-intelligence research group, and Uber's "Pyro" software for probabilistic programming is built on it.

PyTorch is an open-source machine learning library for Python, based on Torch, used for applications such as natural language processing. It is primarily developed by Facebook’s artificial intelligence research group, and Uber’s “Pyro” is also built on this library.

PyTorch is a Python package that provides two high-level features:


    Tensor computation (like NumPy) with strong GPU acceleration

    Deep neural networks built on a tape-based autograd system

You can reuse your favorite Python packages such as NumPy, SciPy and Cython to extend PyTorch when needed.

PyTorch is a Python package that provides two high-level features:

• Tensor computation (like NumPy) with strong GPU acceleration

• Deep neural networks built on a tape-based autograd system

1.1.4 Comparing PyTorch and TensorFlow

There are no superior frameworks, only suitable frameworks. Here is a simple comparison, so I won’t go into detail: https://zhuanlan.zhihu.com/p/28636490. Moreover, technology is evolving, and this comparison is not absolute. For example, TensorFlow introduced Eager Execution mechanism in version 1.5 to achieve dynamic graphs, while issues like visualization, Windows support, and tensor flipping along dimensions in PyTorch have already been resolved.

1.1.5 Summary Again

• PyTorch is a relatively concise, elegant, and efficient framework.

• The design pursues minimal encapsulation, avoiding reinventing the wheel as much as possible.

• It is one of the most elegantly designed frameworks in object-oriented design, aligning closely with people’s thinking, allowing users to focus as much as possible on implementing their ideas.

• Supported by big names, similar to Google’s TensorFlow, FAIR’s support ensures that PyTorch receives continuous development and updates.

• Good documentation (compared to other FB projects, PyTorch’s documentation is relatively complete, referencing Thrift), and a forum maintained by the authors of PyTorch for user communication and problem-solving.

• Easy to get started.

So if the above information attracts you, please make sure to read this book 🙂

1.2 Setting Up the PyTorch Environment

Installing PyTorch is very simple. According to the PyTorch official website, you can flexibly choose the system and installation method. Here, I will take Anaconda as an example to briefly describe the steps and key points. For domestic installation of Anaconda, it is recommended to use Tsinghua or USTC mirrors [http://mirrors.ustc.edu.cn/help/anaconda.html] as they are significantly faster.

1.2.1 Installing PyTorch

After Anaconda is installed, you can start creating an environment. Here, I will take Windows 10 as an example. Open Anaconda Prompt.

#pytorch is the environment name, creating python 3.6 version here.

conda create -n pytorch python=3.6

# Switch to the pytorch environment

activate pytorch

# Install the GPU version, choose cuda 80, cuda 92 based on the cuda version. If cuda is version 9.0, no need to install.

# Simply run conda install pytorch -c pytorch

# To check the cuda version on Windows, use the command nvcc -V

conda install pytorch cuda92 -c pytorch

# For CPU version use

# conda install pytorch-cpu -c pytorch


# torchvision is the computer vision toolkit provided by torch, to be introduced later.

pip install torchvision

It should be noted that if you are using the Tsinghua source, you can directly add the pytorch source mirror and remove the -c pytorch to use the mirror source.

To verify, enter python.

import torch

torch.__version__

# Result should be '0.4.1'

1.2.2 Configuring Jupyter Notebook

The newly created environment does not have ipykernel installed, so it cannot be registered in Jupyter Notebook. Therefore, the environment needs to be prepared first.

# Install ipykernel

conda install ipykernel

# Register the environment

python -m ipykernel install  --name pytorch --display-name "Pytorch for Deeplearning"

The next step is to customize Jupyter Notebook.

# Switch back to the base environment

activate base

# Create Jupyter Notebook configuration file

jupyter notebook --generate-config

## This will display the specific location of the created jupyter_notebook_config.py

Open the file and modify it.

c.NotebookApp.notebook_dir = '' # Default directory location

c.NotebookApp.iopub_data_rate_limit = 100000000 # Increase this limit to avoid possible errors

1.2.3 Testing

At this point, the PyTorch development environment installation is complete. You can open Jupyter Notebook from the start menu and select “Pytorch for Deeplearning” from the New menu to create the relevant PyTorch development environment.

1.2.4 Problem Solving

Problem 1: Encoding error when starting Python

Delete .python_history source.

Problem 2: Default directory setting not effective

Open the shortcut and check if the shortcut is the same as this screenshot. If so, delete %USERPROFILE% to change the parameter, which may override the notebook_dir setting and cause it to be ineffective.

If you find any other problems, please leave a message directly.

1.3 Deep Learning with PyTorch: A 60 Minute Blitz (Official)

This chapter is a Chinese translation of the official website’s [Deep Learning with PyTorch: A 60 Minute Blitz] (https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html), as all the Chinese translations available online are outdated, so I translated it again to ensure synchronization with the official version.

Table of Contents

• Tensors

• Autograd: Automatic Differentiation – This chapter is a conflict hotspot, recommended to read.

• Neural Networks

• Training a Classifier

• Recommended Reading: Data Parallelism (Multi GPU)

Note

All images in this chapter are from the PyTorch official website, copyright belongs to PyTorch.

Good news!
The Beginner Learning Vision Knowledge Planet
is now open to the public 👇👇👇



Download 1: OpenCV-Contrib Extension Module Chinese Tutorial
Reply "Extension Module Chinese Tutorial" in the background of "Beginner Learning Vision" official account to download the first OpenCV extension module tutorial in Chinese, covering installation of extension modules, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, and more than twenty chapters of content.

Download 2: Python Vision Practical Project 52 Lectures
Reply "Python Vision Practical Project" in the background of "Beginner Learning Vision" official account to download 31 practical projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, face recognition, etc., to help quickly learn computer vision.

Download 3: OpenCV Practical Projects 20 Lectures
Reply "OpenCV Practical Projects 20 Lectures" in the background of "Beginner Learning Vision" official account to download 20 practical projects based on OpenCV, achieving advanced learning of OpenCV.

Discussion Group

Welcome to join the official account reader group to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will be gradually subdivided later). Please scan the WeChat number below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiaotong University + Vision SLAM". Please follow the format; otherwise, it will not be approved. After successful addition, you will be invited to the relevant WeChat group based on your research direction. Please do not send ads in the group, or you will be removed. Thank you for your understanding ~