# 2021-11-05 creation and common methods of pytorch Tensor

Keywords: Python Pytorch Deep Learning

# Creation and common methods of pytorch Tensor

• Import pytorch package
```import torch
```
• View version number
```torch.__version__
'1.7.1'
```

## 1, Basic creation and types of tensors

### 1. Tensor creation method

Use the pytorch tensor creation function: torch.tensor():

```# Create tensor from list
t = torch.tensor([1, 2])
t
tensor([1,2])
```
```# Creating tensors from tuples
torch.tensor((1,2))
tensor([1,2])
```
```# Creating tensors from numpy arrays
import numpy as np
a = np.array((1,2))
a
array([1,2])
# Creating tensors from numpy arrays
t1 = torch.tensor(a)
t1
tensor([1,2],dtype=torch.int32)
```

Note: it can be seen from the above return results that the tensor also has dtype type

### 2. Type of tensor

Like arrays, tensors have dtype methods that return tensor types

```a.type
dtype('int32')
```
```t.dtype
torch.int64
```
```t1.dtype
torch.int32
```

From the above output, we can see that the array creates int32 (integer) type by default, while the tensor creates int64 (long integer) type by default

```np.array([1.1, 2.2]).dtype
dtype('float64')
```
```torch.tensor(np.array([1.1, 2.2])).dtype
torch.float64
```
```torch.tensor([1.1,2.2]).dtype
torch.float32
```

In contrast, when creating a floating-point Array, the tensor defaults to float32 (single precision floating-point), while the Array defaults to float64 (double precision floating-point).
In addition to numerical tensors, Boolean tensors are commonly used as constant types

```t2 = torch.tensor([True, False])
t2
tensor([True, False])
```
```t2.dtype
torch.bool
```

tensor type in pytorch:

data typedtype
32bit floating point numbertorch.float32 or torch.float
64bit floating point numbertorch.float64 or torch.double
16bit floating point numbertorch.float16 or torch.half
8bit unsigned integertorch.uint8
8bit signed integertorch.int8
16bit signed integertorch.int16 or torch.short
32bit signed integertorch.int32 or torch.int
64bit signed integertorch.int64 or torch.long
Booleantorch.bool
Plural formtorch.complex64

You can specify dtype during tensor creation:

```# Create int64 integer tensor
torch.tensor([1.1, 2.7], dtype = torch.int16)
tensor([1.1, 2.7], dtype=torch.int16)
```

Complex type object creation is also supported in pytoch

```a = torch.tensor([1 + 2j])
a
tensor([1.+2.j])
```

### 3. Transformation of tensor type

• Implicit transformation of tensor types
When the tensor elements belong to different types, the system will automatically perform implicit conversion
```# Implicit conversion of floating point and integer types
torch.tensor([1.1, 2]).dtype
torch.float32
```
```# Implicit transformation of Boolean and numeric types
torch.tensor([Truen, 2.0])
tensor([1., 2.])
```
• Transformation method of tensor type
You can use. float(),. int() and other methods to convert the tensor type
```t
tensor([1, 2])
```
```# Convert to default floating point (32-bit)
t.float()
tensor([1., 2.])
```
```# Convert to double precision floating point
t.double()
tensor([1., 2.], dtype=torch.float64)
t.dtype
torch.int64
```
```# Convert to 16 bit integer
t.short()
tensor([1,2],dtype=torch.int16)
```

Note:

• When dtype parameter is used in torch function, torch.float needs to be entered to indicate precision;
• When using methods for type conversion, the method name is double.

## 2, Dimension and deformation of tensor

### 1. Create high-dimensional tensor

• Create a one-dimensional array with a simple sequence
Sequences containing "simple" elements can create one-dimensional arrays
```t1 = torch.tensor([1, 2])
t1
tensor([1, 2])
```
```# Viewing tensor dimensions using the ndim attribute
t1.ndim
1
```
```# Use shape to view shapes
t1.shape
torch.Size()
```
```# Same as size function
t1.size()
torch.Size()
```

Note: unlike numpy, the size method in pytorch returns the same result as the shape attribute
In addition, it should be noted that there are two common functions / methods to view the tensor shape

```# Returns an element that has several dimensions
len(t1)
2
```
```# How many numbers are returned in total
t1.numel()
2
```

**Note: * * one dimensional tensors len and numel return the same results, but higher dimensional tensors do not

• Create a 2D tensor
```# Creating two-dimensional tensor with composite list
t2 = torch.tensor([[1, 2], [3, 4]])
t2
tensor([[1, 2],
[3, 4]])
```
```t2.ndim
2
```
```t2.shape
torch.Size([2,2])
```
```t2.size()
torch.Size([2, 2])
```
```len(t2)
2
```

Note: 2 here represents that t2 is generated by two one-dimensional tensors

```t2.numel()
4
```

Note: 4 here represents t2, which is composed of 4 numbers

• 0-dimensional tensor
```t = torch.tensor(1)
t
tensor(1)
```
```t.ndim
0
```
```t.shape
torch.Size([])
```
```t.numel()
1
```

Note: the native value object of Python cannot exist on the GPU, but the 0-bit tensor can. From the academic name, the single number in Python is scalars, and the zero dimensional tensor is tensor.

• High dimensional tensor
Generally speaking, tensors in three dimensions and above are called high-dimensional tensors. The most commonly used high-dimensional tensor is the three-dimensional tensor, which can be regarded as the collection of two-dimensional arrays or matrices
```a1 = np.array([[1, 2, 2], [3, 4, 4]])
a1
array([[1, 2, 2],
[3, 4, 4]])
```
```a2 = np.array([[5, 6, 6], [7, 8, 8]])
a2
array([[5, 6, 6],
[7, 8, 8]])
```
```# A three-dimensional tensor is created from two two-dimensional arrays of the same shape
t3 = torch.tensor([a1, a2])
t3

tensor([[[1, 2, 2],
[3, 4, 4]],
[[5, 6, 6],
[7, 8, 8]]]), dtype=torch.int32)
```
```t3.ndim
3
```
```t3.shape
torch.Size([2,2,3])
```
```len(t3)
2
```
```t3.numel()
12
```

### 2. Tensor deformation

#### 2.1 flatten flattening: transform any dimensional tensor into one-dimensional tensor

```t2
tensor([[1, 2],
[3, 4]])
```
```t2.flatten()
tensor([1, 2, 3, 4])
```

Arrange and flatten in rows

```t3
tensor([[[1, 2, 2],
[3, 4, 4]],

[[5, 6, 6],
[7, 8, 8]]], dtype=torch.int32)
```
```t3.flatten()
tensor([1, 2, 2, 3, 4, 4, 5, 6, 6, 7, 8, 8], dtype=torch.int32)
```

Note: if flatten is used for 0-dimensional tensor, it will be transformed into one-dimensional tensor

```t
tensor(1)
```
```t.flatten()
tensor()
```
```t.flatten().ndim
1
```

#### 2.2 reshape method: arbitrary deformation

```t1
tensor([1, 2])
```
```# Into two rows, a column of vectors
t1.reshape(2, 1)
tensor([,
])
```

**Note: * * the dimension after reshape conversion is determined by the number of parameters entered in this method

• One dimensional tensor is generated after transformation
```t1.reshape(2)
tensor([1, 2])
```
```t1.reshape(2).ndim
1
```
```# Another form of expression
t1.reshape(2, )
tensor([1, 2])
```
• Two dimensional tensor is generated after transformation
```t1.reshape(1, 2)           # Generate a two-dimensional tensor containing one or two elements

tensor([[1, 2]])
```
```t1.reshape(1, 2).ndim
2
```
• Three dimensional tensor is generated after transformation
```t1.reshape(1, 1, 2)
tensor([[[1, 2]]])
```
```t1.reshape(1, 2, 1)
tensor([[,
]])
```
```# Pay attention to the change of dimension in the process of transformation
t1.reshape(1, 2, 1).ndim
3
```

## 3, Creation method of special tensor

### 1. Tensor creation method with special value

• Total zero tensor
```# Create two row, three column tensors that are all 0
torch.zeros([2, 3])
tensor([[0., 0., 0.],
[0., 0., 0.]])
```

Note: since zeros has determined the value of tensor element, the parameters passed in by this function actually determine the shape of tensor

• Total 1 tensor
```torch.ones([2, 3])
tensor([[1., 1., 1.],
[1., 1., 1.]])
```
• Identity matrix
```torch.eye(5)
tensor([[1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
```
• Diagonal matrix
Note: in pytorch, we need to use one-dimensional tensor to create diagonal matrix
```t1
tensor([1, 2])
```
```torch.diag(t1)
tensor([[1, 0],
[0, 2]])
```
```# You cannot use list to create diagonal matrices directly
torch.diag([1, 2])
```
```---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-237-314c189cb631> in <module>
----> 1 torch.diag([1, 2])             # You cannot use list to create diagonal matrices directly

TypeError: diag(): argument 'input' (position 1) must be Tensor, not list
```
• rand: tensor obeying 0-1 uniform distribution
```torch.rand(2, 3)
tensor([[0.9223, 0.9948, 0.2804],
[0.8130, 0.2890, 0.5319]])
```
• randn: tensor obeying standard normal distribution
```torch.randn(2, 3)
tensor([[-1.2513,  0.6465, -2.3011],
[ 0.8447,  1.6856,  1.3615]])
```
• Normal: a tensor that obeys a specified normal distribution
```torch.normal(2, 3, size=(2, 2))      #Tensor with mean value of 2 and standard deviation of 3
tensor([[2.4660, 1.4952],
[6.0202, 0.7525]])
```
• randint: integer random sampling result
```torch.randint(1, 10, [2, 4])                       # Randomly select integers between 1-10 to form two rows and four columns
tensor([[5, 8, 8, 3],
[6, 1, 4, 2]])
```
• Range / linespace: generate sequence
```torch.arange(5)                        # Same as range
tensor([0, 1, 2, 3, 4])
```
```torch.arange(1, 5, 0.5)                             # From 1 to 5 (left closed and right open), take one value every 0.5
tensor([1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000, 4.0000, 4.5000])
```
```torch.linspace(1, 5, 3)                              # From 1 to 5 (both left and right), take three numbers at an equal distance
tensor([1., 3., 5.])
```
• empty: generates an uninitialized specified shape matrix
```torch.empty(2, 3)
tensor([[0.0000e+00, 1.7740e+28, 1.8754e+28],
[1.0396e-05, 1.0742e-05, 1.0187e-11]])
```
• full: fills the specified value according to the specified shape
```torch.full([2, 4], 2)
tensor([[2, 2, 2, 2],
[2, 2, 2, 2]])
```

### 2. Create an array of specified shapes

Use "_like" to create an array of specified shapes:

```t1
tensor([1, 2])
```
```t2
tensor([[1, 2],
[3, 4]])
```
```torch.full_like(t1, 2)                        # Fill quantity 2 according to t1 shape
tensor([2, 2])
```
```torch.randint_like(t2, 1, 10)
tensor([[4, 8],
[5, 8]])
```
```torch.zeros_like(t1)
tensor([0, 0])
```

Note:
For the "_like" type conversion, attention should be paid to the consistency of data types before and after conversion

```torch.randn_like(t1)								# t1 is an integer and will become a floating point number after conversion, so the code reports an error
```
```RuntimeError                              Traceback (most recent call last)

<ipython-input-285-82e06104e8a8> in <module>
----> 1 torch.randn_like(t1)

RuntimeError: "normal_kernel_cpu" not implemented for 'Long'
```
```t10 = torch.tensor([1.1, 2.2])             # Regenerate a new floating-point tensor
t10
tensor([1.1000, 2.2000])
```
```torch.randn_like(t10)
tensor([1.0909, 0.0379])
```

## 4, Transformation method between Tensor and other related types

• . numpy method: convert tensor to array
```t1
tensor([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
```
```t1.numpy()
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10], dtype=int64)
```
```# Of course, it can also be directly converted to array through the np.array function
np.array(t1)
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10], dtype=int64)
```
• . tolist method: convert tensor to list
```t1.tolist()
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
```
• List function: Transform tensor into list
```list(t1)
[tensor(1),
tensor(2),
tensor(3),
tensor(4),
tensor(5),
tensor(6),
tensor(7),
tensor(8),
tensor(9),
tensor(10)]
```

**Note: * * at this time, the converted list is a list composed of 0-bit tensors, rather than a list composed of tensor values.

• . item() method: convert to numeric value

In many cases, the tensor of the final calculation result needs to be converted into a separate value for output. At this time, the. item method needs to be used

```n = torch.tensor(1)
n
tensor(1)
```
```n.item()
1
```

## 5, Deep copy of tensor

Deep copy using clone method

```t1
tensor([ 1, 2,  3,  4,  5,  6,  7,  8,  9, 10 ])
```
```t11 = t1                                 # t11 is a shallow copy of t1
t11
tensor([ 1, 2,  3,  4,  5,  6,  7,  8,  9, 10])
```
```t1
tensor(2)
```
```t1 = 10
t1
tensor([1, 10,  3,  4,  5,  6,  7,  8,  9, 10])
```
```t11                                                         # t11 is modified synchronously
tensor([ 1, 10,  3,  4,  5,  6,  7,  8,  9, 10])
```

t1 and t11 here point to the same object. If you want t11 not to change with t1, you need to make a deep copy of t11 so that t11 has a separate object

```t11 = t1.clone()
t1
tensor([1, 10,  3,  4,  5,  6,  7,  8,  9, 10])
```
```t11
tensor([ 1, 10,  3,  4,  5,  6,  7,  8,  9, 10])
```
```t1
tensor(1)
```
```t1 = 100
t1
tensor([100,  10,   3,   4,   5,   6,   7,   8,   9,  10])
```
```t11
tensor([ 1, 10,  3,  4,  5,  6,  7,  8,  9, 10])
```

Posted by benrussell on Sat, 06 Nov 2021 11:45:54 -0700