5 Useful Tensor Operations in PyTorch

Click on the above“Beginner’s Guide to Vision” to select and add a Bookmark or “Pin”

Important content delivered promptly

PyTorch is a scientific package based on Python that performs advanced operations using a special data type called a tensor. A tensor is a numerical, vector, matrix, or multi-dimensional array with a regular shape and the same data type. PyTorch is an alternative to the NumPy package that can be used on a GPU. It is also used as a framework for deep learning research.

The 5 operations are:

• expand()
• permute()
• tolist()
• narrow()
• where()

Expands an existing tensor along dimensions with a value of 1 to new dimensions. A tensor can be expanded along any one or multiple dimensions simultaneously. If you do not want to expand the tensor along a specific dimension, you can set its parameter value to -1.

Note: Only a single dimension can be expanded

# Example 1 - working 
a=torch.tensor([[[1,2,3],[4,5,6]]])
a.size()
>>torch.Size([1, 2, 3])

a.expand(2,2,3)
>>tensor([[[1, 2, 3],
         [4, 5, 6]],

        [[1, 2, 3],
         [4, 5, 6]]])

In this example, the original dimensions of the tensor are [1,2,3]. It is expanded to [2,2,3].

This function returns a view of the tensor, with the original tensor’s dimensions changed according to our selection. For example, if the original dimensions are [1,2,3], we can change it to [3,2,1]. The function takes the desired dimension order as parameters.

# Example 1 - working
a=torch.tensor([[[1,2,3],[4,5,6]]])
a.size()
>>torch.Size([1, 2, 3])

a.permute(2,1,0).size()
>>torch.Size([3, 2, 1])

a.permute(2,1,0)
>>tensor([[[1],
         [4]],

        [[2],
         [5]],

        [[3],
         [6]]])

In this example, the original tensor’s dimensions are [1,2,3]. Using permuting, I set the order to (2,1,0), which means the new dimensions should be [3,2,1]. As shown, the new view of the tensor rearranges the numbers, resulting in dimensions of [3,2,1].

This function can be used when we want to reorder tensors of different dimensions or perform matrix multiplication with matrices of different orders.

This function returns the tensor in the form of Python numbers, lists, or nested lists. After this, we can perform any Python logic and operations on it.

# Example 1 - working
a=torch.tensor([[1,2,3],[4,5,6]])
a.tolist()
>> [[1, 2, 3], [4, 5, 6]]

In this example, the tensor is returned in the form of a nested list.

This function returns a new tensor that is a reduced version of the original tensor. The parameters of this function are the input tensor, the dimension to narrow, the starting index, and the length of the new tensor along that dimension. It returns the elements from index start to index (start+length-1).

# Example 1 - working
a=torch.tensor([[1,2,3,4],[5,6,7,8],[9,10,11,12],[14,15,16,17]])
torch.narrow(a,1,2,2)
>> tensor([[ 3,  4],
        [ 7,  8],
        [11, 12],
        [16, 17]])

In this example, the tensor is narrowed along the second dimension, which is the innermost dimension. It takes elements from the list starting from index 2 to index 3 (=2+2 -1, i.e., start+length-1).

Narrow() works similarly to advanced indexing. For example, in a 2D tensor, using [:,0:5] selects all rows in columns 0 to 5. Similarly, torch.narrow(1,0,5) can be used. However, using range operations for each dimension in high-dimensional tensors can be cumbersome. Using narrow() can achieve this faster and more conveniently.

This function returns a new tensor whose values change at each index based on the given condition. The parameters for this function are: condition, first tensor, and second tensor. It checks the condition on the values of each tensor (using the condition), and if true, replaces it with the value from the first tensor at the same position; if false, it replaces it with the value from the second tensor at the same position.

# Example 1 - working
a=torch.tensor([[[1,2,3],[4,5,6]]]).to(torch.float32)
b=torch.zeros(1,2,3)
torch.where(a%2==0,b,a)
>>tensor([[[1., 0., 3.],
         [0., 5., 0.]]])

Here, it checks whether the values in tensor a are even. If so, it replaces them with the values from tensor b, which are all 0; otherwise, it retains the original values.

This function can be used to set thresholds. If the values in the tensor are greater than or less than a certain number, they can be easily replaced.

Good news! 
The Beginner's Guide to Vision Knowledge Planet is now open to the public 👇👇👇



Download 1: Chinese Version of OpenCV-Contrib Extension Module Tutorial
Reply "Chinese Extension Module Tutorial" in the background of the "Beginner's Guide to Vision" official account to download the first Chinese version of the OpenCV extension module tutorial, covering installation, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, and more than twenty chapters of content.

Download 2: 52 Python Vision Practical Projects
Reply "Python Vision Practical Projects" in the background of the "Beginner's Guide to Vision" official account to download 31 practical projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, and face recognition to help quickly learn computer vision.

Download 3: 20 OpenCV Practical Projects
Reply "20 OpenCV Practical Projects" in the background of the "Beginner's Guide to Vision" official account to download 20 practical projects based on OpenCV to advance OpenCV learning.

Group Chat

Welcome to join the reader group of the official account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat number below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiaotong University + Vision SLAM". Please follow the format in the remarks; otherwise, you will not be approved. After successfully adding, you will be invited to relevant WeChat groups based on your research direction. Please do not send advertisements in the group; otherwise, you will be removed from the group. Thank you for your understanding~