# [in depth learning] 60 questions PyTorch simple introduction guide to be a trendsetter of Technology

Keywords: Python Pytorch Deep Learning

# 1. Get to know PyTorch

## 1.1 tensor

1. Import pytorch package

```import torch
```

2. Create an empty 5x3 tensor

```x = torch.empty(5, 3)
print(x)
```

3. Create a randomly initialized 5x3 tensor

```x = torch.rand(5, 3)
print(x)
```

4. Create a 5x3 0 tensor of type long

```x = torch.zeros(5, 3, dtype=torch.long)
print(x)
```

5. Create tensors directly from the array

```x = torch.tensor([5.5, 3])
print(x)
```

6. Create a 5x3 unit tensor of type double

```x = torch.ones(5, 3, dtype=torch.double)
print(x)
```

7. Create a new tensor with the same dimension from the existing tensor, and redefine the type as float

```x = torch.randn_like(x, dtype=torch.float)
print(x)
```

8. Print a tensor dimension

```print(x.size())
```

```y = torch.rand(5, 3)
print(x + y)

# Method 2

# Method 3
# result = torch.empty(5, 3)
# print(result)

# Method 4
# print(y)
```

10. Take the first column of tensor

```print(x[:, 1])
```

11. resize a 4x4 tensor into a one-dimensional tensor

```x = torch.randn(4, 4)
y = x.view(16)
print(x.size(),y.size())
```

12. resize a 4x4 tensor into a 2x8 tensor

```y = x.view(2, 8)
print(x.size(),y.size())

# Method 2
z = x.view(-1, 8) # Determine a dimension, and the dimension of - 1 will be calculated automatically
print(x.size(),z.size())
```

13. Take the number from the tensor

```x = torch.randn(1)
print(x)
print(x.item())
```

## 1.2 operation of numpy

14. Convert tensor to numpy array

```a = torch.ones(5)
print(a)

b = a.numpy()
print(b)
```

15. Add tensor + 1 and observe the change of numpy array in the above question

```a.add_(1)
print(a)
print(b)
```

16. Create tensor from numpy array

```import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
print(a)
print(b)
```

17. Add numpy array + 1 and observe the change of tensor in the above question

```np.add(a, 1, out=a)
print(a)
print(b)
```

# 2 automatic differentiation

## 2.1 automatic differentiation of tensors

18. Create a new tensor and set requirements_ grad=True

```x = torch.ones(2, 2, requires_grad=True)
print(x)
```

19. Perform any operation on the tensor (y = x + 2)

```y = x + 2
print(y)
```

20. Perform any operation on y

```z = y * y * 3
out = z.mean()

print(z) # z more MulBackward
print(out) # out more MeanBackward
```

21. Back propagation of out

```out.backward()
```

```print(x.grad) #out=0.25*Σ3(x+2)^2
```

23. Create a calculation process with the result of vector (y=x*2^n)

```x = torch.randn(3, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
y = y * 2

print(y)
```

24. Calculate the gradient at v = [0.1, 1.0, 0.0001]

```v = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float)
y.backward(v)

```

25. Turn off the gradient function

```print(x.requires_grad)

# Method 2
# y = x.detach()
# print(x.eq(y).all())
```

# 3 neural network

This part will implement LeNet5, and the structure is as follows

## 3.1 defining networks

```import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):

def __init__(self):
super(Net, self).__init__()
# 26. Define the convolution layer of ①, input 32x32 image, convolution kernel size 5x5, convolution kernel type 6
self.conv1 = nn.Conv2d(3, 6, 5)
# 27. Define the convolution layer of ③. The input is 6 features of the previous layer, the size of convolution kernel is 5x5, and the type of convolution kernel is 16
self.conv2 = nn.Conv2d(6, 16, 5)
# 28. Define the full link layer of ⑤. The input is 16 * 5 * 5 and the output is 120
self.fc1 = nn.Linear(16 * 5 * 5, 120)  # 6*6 from image dimension
# 29. Define the full connection layer of ⑥, with input of 120 and output of 84
self.fc2 = nn.Linear(120, 84)
# 30. Define the full connection layer of ⑥. The input is 84 and the output is 10
self.fc3 = nn.Linear(84, 10)

def forward(self, x):
# 31. Complete input-S2, convolute + relu first, and then 2x2 down sampling
x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
# 32. Complete S2-S4, convolute + relu first, and then 2x2 down sampling
x = F.max_pool2d(F.relu(self.conv2(x)), 2) #When the convolution kernel is square, only one dimension can be written
# 33. Flatten the feature vector into a row vector
x = x.view(-1, 16 * 5 * 5)
# 34. Use fc1+relu
x = F.relu(self.fc1(x))
# 35. Use fc2+relu
x = F.relu(self.fc2(x))
# 36. Use fc3
x = self.fc3(x)
return x

net = Net()
print(net)
```

37. Print network parameters

```params = list(net.parameters())
# print(params)
print(len(params))
```

38. Print the shape of a layer parameter

```print(params.size())
```

39. Input a vector randomly to view the forward propagation output

```input = torch.randn(1, 1, 32, 32)
out = net(input)
print(out)
```

```net.zero_grad()
```

41. Back propagation with a random gradient

```out.backward(torch.randn(1, 10))
```

## 3.2 loss function

42. Define the loss function with the built-in mselos()

```criterion = nn.MSELoss()
```

43. Random a true value and calculate the loss with random input

```target = torch.randn(10)  # Random truth value
target = target.view(1, -1)  # Become a row vector

output = net(input)  # Calculate output with random input

loss = criterion(output, target)  # Calculate loss
print(loss)
```

44. Initialize the gradient and calculate the back propagation of loss in the previous step

```net.zero_grad()

```

45. Calculate the back propagation of loss in 43

```loss.backward()

```

## 3.3 update weights

46. Define the SGD optimizer algorithm and set the learning rate to 0.01

```import torch.optim as optim
optimizer = optim.SGD(net.parameters(), lr=0.01)
```

47. Update weights using optimizer

```optimizer.zero_grad()
output = net(input)
loss = criterion(output, target)
loss.backward()

# Update weight
optimizer.step()
```

# 4 training a classifier

## 4.1 read CIFAR10 data and standardize it

48. Construct a transform to convert the data of three channel (0,1) interval into (- 1,1) data

```import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
```

```trainset = cifar(root = './input/cifar10', segmentation='train', transforms=transform)
testset = cifar(root = './input/cifar10', segmentation='test', transforms=transform)

classes = ('plane', 'car', 'bird', 'cat',
'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
```

## 4.2 network establishment

This part follows the previous network

```net2 = Net()
```

## 4.3 defining loss functions and optimizers

49. Define the cross entropy loss function

```criterion2 = nn.CrossEntropyLoss()
```

50. Define the SGD optimizer algorithm, and set the learning rate to 0.001, momentum=0.9

```optimizer2 = optim.SGD(net2.parameters(), lr=0.001, momentum=0.9)
```

## 4.4 training network

```for epoch in range(2):

running_loss = 0.0
for i, data in enumerate(trainloader, 0):
# Get X,y pairs
inputs, labels = data

# 52. Feedforward
outputs = net2(inputs)
# 53. Calculation of losses
loss = criterion2(outputs, labels)
loss.backward()
# 55. Update weights
optimizer2.step()

# Print the average cost function value every 2000 data
running_loss += loss.item()
if i % 2000 == 1999:    # print every 2000 mini-batches
print('[%d, %5d] loss: %.3f' %
(epoch + 1, i + 1, running_loss / 2000))
running_loss = 0.0

print('Finished Training')
```

## 4.5 prediction using model

Take some data

```dataiter = iter(testloader)
images, labels = dataiter.next()

# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))
```

56. Prediction using models

```outputs = net2(images)

_, predicted = torch.max(outputs, 1)

print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]
for j in range(4)))

```

57. Score on the test set

```correct = 0
total = 0
images, labels = data
outputs = net2(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()

print('Accuracy of the network on the 10000 test images: %d %%' % (
100 * correct / total))
```

## 4.6 access model

58. Save the trained model

```PATH = './cifar_net.pth'
torch.save(net.state_dict(), PATH)
```

```pretrained_net = torch.load(PATH)
```

```net3 = Net() 