Why study pytorch

• Replace numpy to make better use of GPUs
• Provide maximum flexibility and speed for deep learning

Learning premise

Make sure that the torch package is installed, and the following sentence can be executed smoothly

import torch

Variable operation

Source: https://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#sphx-glr-beginner-blitz-tensor-tutorial-py

statement

• Not initialized (garbage value)

x = torch.empty(5, 3)
print(x)

out:
tensor([[8.4490e-39, 9.2755e-39, 1.0653e-38],
        [8.4489e-39, 9.6429e-39, 8.4490e-39],
        [9.6429e-39, 9.2755e-39, 1.0286e-38],
        [9.0919e-39, 8.9082e-39, 9.2755e-39],
        [8.4490e-39, 9.2755e-39, 9.6429e-39]])

• Initialize with random values

x = torch.rand(5, 3)
print(x)

out:
tensor([[0.8901, 0.3888, 0.8734],
        [0.8738, 0.0639, 0.2489],
        [0.8157, 0.1134, 0.3066],
        [0.7617, 0.9803, 0.4271],
        [0.1756, 0.1482, 0.2855]])

• Assign 0 (or 1) to all

x = torch.zeros(5, 3, dtype=torch.long)
print(x)
# The same: ones

out:
tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]])

• Initialize with value

x = torch.tensor([1, 2, 5.5])
print(x)

out:
tensor([1.0000, 2.0000, 5.5000])

Variable attribute

• Get the format size of the variable: x.size()

x = torch.zeros(5, 3, dtype=torch.long)
print(x.size())

out:
torch.Size([5, 3]) # Return a tuple

Variable operation

• Addition and subtraction

x = torch.ones(5, 3)
y = torch.ones(5, 3)
res = x + y

• Direct operation of original data

y.add_(x)
# Add x to y
# Any addition of '' will change the original value

• Take part: x[:,1]

print(y)
print(y[:,1])

out:
tensor([[2., 2., 2.],
        [2., 2., 2.],
        [2., 2., 2.],
        [2., 2., 2.],
        [2., 2., 2.]])
tensor([2., 2., 2., 2., 2.])

• reshape: use torch.view

x = torch.randn(4, 4)
y = x.view(16)
print(y.size())
print(y)
z = x.view(-1, 8) # -1 will automatically calculate dimensions
print(z.size())
print(z)

out:
torch.Size([16])
tensor([-0.9323,  0.0361,  0.6241, -0.4999, -1.1119, -2.3558,  1.6778,  0.5107,
         0.0895,  1.2031,  0.2117,  1.2669, -0.5290,  0.5751, -0.2783,  0.3618])
torch.Size([2, 8])
tensor([[-0.9323,  0.0361,  0.6241, -0.4999, -1.1119, -2.3558,  1.6778,  0.5107],
        [ 0.0895,  1.2031,  0.2117,  1.2669, -0.5290,  0.5751, -0.2783,  0.3618]])

And numpy

• Convert a torch sensor to numpy array

a = torch.ones(5)
print(a)
b = a.numpy()
print(b)

out:
tensor([1., 1., 1., 1., 1.])
[1. 1. 1. 1. 1.]

• Convert a numpy array to a torch sensor

import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
print(b)

out:
tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

• For the relationship with numpy, see the official website tutorial

Numpy is a great framework, but it cannot utilize GPUs to accelerate its numerical computations. For modern deep neural networks, GPUs often provide speedups of 50x or greater, so unfortunately numpy won't be enough for modern deep learning.

Here we introduce the most fundamental PyTorch concept: the Tensor. A PyTorch Tensor is conceptually identical to a numpy array: a Tensor is an n-dimensional array, and PyTorch provides many functions for operating on these Tensors. Behind the scenes, Tensors can keep track of a computational graph and gradients, but they're also useful as a generic tool for scientific computing.

Also unlike numpy, PyTorch Tensors can utilize GPUs to accelerate their numeric computations. To run a PyTorch Tensor on GPU, you simply need to cast it to a new datatype.

The following are the general meanings:

• numpy can't use GPUs to speed up computation. GPUs can speed up by more than 50 times. pytorch can be accelerated using GPUs.
• Python sensor is conceptually the same as numpy. Many similar operations are also provided.
• The sensor can track calculation graphs and gradients (which will be used later in the introduction of gradients)

CUDA tensor

• Move the sensor to any device: use the. To method

if torch.cuda.is_available():
    device = torch.device("cuda")          # a CUDA device object
    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU
    x = x.to(device)                       # or just use strings ``.to("cuda")``
    z = x + y
    print(z)
    print(z.to("cpu", torch.double))       # ``.to`` can also change dtype together!

out:
tensor([0.3882], device='cuda:0')
tensor([0.3882], dtype=torch.float64)

(CUDA is not supported on this machine, so this example cannot be tested...)