Creation and common methods of pytorch Tensor
- Import pytorch package
import torch
- View version number
torch.__version__ '1.7.1'
1, Basic creation and types of tensors
1. Tensor creation method
Use the pytorch tensor creation function: torch.tensor():
# Create tensor from list t = torch.tensor([1, 2]) t tensor([1,2])
# Creating tensors from tuples torch.tensor((1,2)) tensor([1,2])
# Creating tensors from numpy arrays import numpy as np a = np.array((1,2)) a array([1,2]) # Creating tensors from numpy arrays t1 = torch.tensor(a) t1 tensor([1,2],dtype=torch.int32)
Note: it can be seen from the above return results that the tensor also has dtype type
2. Type of tensor
Like arrays, tensors have dtype methods that return tensor types
a.type
dtype('int32')
t.dtype torch.int64
t1.dtype torch.int32
From the above output, we can see that the array creates int32 (integer) type by default, while the tensor creates int64 (long integer) type by default
np.array([1.1, 2.2]).dtype
dtype('float64')
torch.tensor(np.array([1.1, 2.2])).dtype torch.float64
torch.tensor([1.1,2.2]).dtype torch.float32
In contrast, when creating a floating-point Array, the tensor defaults to float32 (single precision floating-point), while the Array defaults to float64 (double precision floating-point).
In addition to numerical tensors, Boolean tensors are commonly used as constant types
t2 = torch.tensor([True, False]) t2 tensor([True, False])
t2.dtype torch.bool
tensor type in pytorch:
| data type | dtype |
|---|---|
| 32bit floating point number | torch.float32 or torch.float |
| 64bit floating point number | torch.float64 or torch.double |
| 16bit floating point number | torch.float16 or torch.half |
| 8bit unsigned integer | torch.uint8 |
| 8bit signed integer | torch.int8 |
| 16bit signed integer | torch.int16 or torch.short |
| 32bit signed integer | torch.int32 or torch.int |
| 64bit signed integer | torch.int64 or torch.long |
| Boolean | torch.bool |
| Plural form | torch.complex64 |
You can specify dtype during tensor creation:
# Create int64 integer tensor torch.tensor([1.1, 2.7], dtype = torch.int16) tensor([1.1, 2.7], dtype=torch.int16)
Complex type object creation is also supported in pytoch
a = torch.tensor([1 + 2j]) a tensor([1.+2.j])
3. Transformation of tensor type
- Implicit transformation of tensor types
When the tensor elements belong to different types, the system will automatically perform implicit conversion
# Implicit conversion of floating point and integer types torch.tensor([1.1, 2]).dtype torch.float32
# Implicit transformation of Boolean and numeric types torch.tensor([Truen, 2.0]) tensor([1., 2.])
- Transformation method of tensor type
You can use. float(),. int() and other methods to convert the tensor type
t tensor([1, 2])
# Convert to default floating point (32-bit) t.float() tensor([1., 2.])
# Convert to double precision floating point t.double() tensor([1., 2.], dtype=torch.float64) t.dtype torch.int64
# Convert to 16 bit integer t.short() tensor([1,2],dtype=torch.int16)
Note:
- When dtype parameter is used in torch function, torch.float needs to be entered to indicate precision;
- When using methods for type conversion, the method name is double.
2, Dimension and deformation of tensor
1. Create high-dimensional tensor
- Create a one-dimensional array with a simple sequence
Sequences containing "simple" elements can create one-dimensional arrays
t1 = torch.tensor([1, 2]) t1 tensor([1, 2])
# Viewing tensor dimensions using the ndim attribute t1.ndim 1
# Use shape to view shapes t1.shape torch.Size([2])
# Same as size function t1.size() torch.Size([2])
Note: unlike numpy, the size method in pytorch returns the same result as the shape attribute
In addition, it should be noted that there are two common functions / methods to view the tensor shape
# Returns an element that has several dimensions len(t1) 2
# How many numbers are returned in total t1.numel() 2
**Note: * * one dimensional tensors len and numel return the same results, but higher dimensional tensors do not
- Create a 2D tensor
# Creating two-dimensional tensor with composite list t2 = torch.tensor([[1, 2], [3, 4]]) t2 tensor([[1, 2], [3, 4]])
t2.ndim 2
t2.shape torch.Size([2,2])
t2.size() torch.Size([2, 2])
len(t2) 2
Note: 2 here represents that t2 is generated by two one-dimensional tensors
t2.numel() 4
Note: 4 here represents t2, which is composed of 4 numbers
- 0-dimensional tensor
t = torch.tensor(1) t tensor(1)
t.ndim 0
t.shape torch.Size([])
t.numel() 1
Note: the native value object of Python cannot exist on the GPU, but the 0-bit tensor can. From the academic name, the single number in Python is scalars, and the zero dimensional tensor is tensor.
- High dimensional tensor
Generally speaking, tensors in three dimensions and above are called high-dimensional tensors. The most commonly used high-dimensional tensor is the three-dimensional tensor, which can be regarded as the collection of two-dimensional arrays or matrices
a1 = np.array([[1, 2, 2], [3, 4, 4]]) a1 array([[1, 2, 2], [3, 4, 4]])
a2 = np.array([[5, 6, 6], [7, 8, 8]]) a2 array([[5, 6, 6], [7, 8, 8]])
# A three-dimensional tensor is created from two two-dimensional arrays of the same shape t3 = torch.tensor([a1, a2]) t3 tensor([[[1, 2, 2], [3, 4, 4]], [[5, 6, 6], [7, 8, 8]]]), dtype=torch.int32)
t3.ndim 3
t3.shape torch.Size([2,2,3])
len(t3) 2
t3.numel() 12
2. Tensor deformation
2.1 flatten flattening: transform any dimensional tensor into one-dimensional tensor
t2 tensor([[1, 2], [3, 4]])
t2.flatten() tensor([1, 2, 3, 4])
Arrange and flatten in rows
t3
tensor([[[1, 2, 2],
[3, 4, 4]],
[[5, 6, 6],
[7, 8, 8]]], dtype=torch.int32)
t3.flatten() tensor([1, 2, 2, 3, 4, 4, 5, 6, 6, 7, 8, 8], dtype=torch.int32)
Note: if flatten is used for 0-dimensional tensor, it will be transformed into one-dimensional tensor
t tensor(1)
t.flatten() tensor([1])
t.flatten().ndim 1
2.2 reshape method: arbitrary deformation
t1 tensor([1, 2])
# Into two rows, a column of vectors
t1.reshape(2, 1)
tensor([[1],
[2]])
**Note: * * the dimension after reshape conversion is determined by the number of parameters entered in this method
- One dimensional tensor is generated after transformation
t1.reshape(2) tensor([1, 2])
t1.reshape(2).ndim 1
# Another form of expression t1.reshape(2, ) tensor([1, 2])
- Two dimensional tensor is generated after transformation
t1.reshape(1, 2) # Generate a two-dimensional tensor containing one or two elements tensor([[1, 2]])
t1.reshape(1, 2).ndim 2
- Three dimensional tensor is generated after transformation
t1.reshape(1, 1, 2) tensor([[[1, 2]]])
t1.reshape(1, 2, 1) tensor([[[1], [2]]])
# Pay attention to the change of dimension in the process of transformation t1.reshape(1, 2, 1).ndim 3
3, Creation method of special tensor
1. Tensor creation method with special value
- Total zero tensor
# Create two row, three column tensors that are all 0
torch.zeros([2, 3])
tensor([[0., 0., 0.],
[0., 0., 0.]])
Note: since zeros has determined the value of tensor element, the parameters passed in by this function actually determine the shape of tensor
- Total 1 tensor
torch.ones([2, 3]) tensor([[1., 1., 1.], [1., 1., 1.]])
- Identity matrix
torch.eye(5)
tensor([[1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
- Diagonal matrix
Note: in pytorch, we need to use one-dimensional tensor to create diagonal matrix
t1 tensor([1, 2])
torch.diag(t1)
tensor([[1, 0],
[0, 2]])
# You cannot use list to create diagonal matrices directly torch.diag([1, 2])
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-237-314c189cb631> in <module> ----> 1 torch.diag([1, 2]) # You cannot use list to create diagonal matrices directly TypeError: diag(): argument 'input' (position 1) must be Tensor, not list
- rand: tensor obeying 0-1 uniform distribution
torch.rand(2, 3)
tensor([[0.9223, 0.9948, 0.2804],
[0.8130, 0.2890, 0.5319]])
- randn: tensor obeying standard normal distribution
torch.randn(2, 3)
tensor([[-1.2513, 0.6465, -2.3011],
[ 0.8447, 1.6856, 1.3615]])
- Normal: a tensor that obeys a specified normal distribution
torch.normal(2, 3, size=(2, 2)) #Tensor with mean value of 2 and standard deviation of 3
tensor([[2.4660, 1.4952],
[6.0202, 0.7525]])
- randint: integer random sampling result
torch.randint(1, 10, [2, 4]) # Randomly select integers between 1-10 to form two rows and four columns
tensor([[5, 8, 8, 3],
[6, 1, 4, 2]])
- Range / linespace: generate sequence
torch.arange(5) # Same as range tensor([0, 1, 2, 3, 4])
torch.arange(1, 5, 0.5) # From 1 to 5 (left closed and right open), take one value every 0.5 tensor([1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000, 4.0000, 4.5000])
torch.linspace(1, 5, 3) # From 1 to 5 (both left and right), take three numbers at an equal distance tensor([1., 3., 5.])
- empty: generates an uninitialized specified shape matrix
torch.empty(2, 3)
tensor([[0.0000e+00, 1.7740e+28, 1.8754e+28],
[1.0396e-05, 1.0742e-05, 1.0187e-11]])
- full: fills the specified value according to the specified shape
torch.full([2, 4], 2)
tensor([[2, 2, 2, 2],
[2, 2, 2, 2]])
2. Create an array of specified shapes
Use "_like" to create an array of specified shapes:
t1 tensor([1, 2])
t2 tensor([[1, 2], [3, 4]])
torch.full_like(t1, 2) # Fill quantity 2 according to t1 shape tensor([2, 2])
torch.randint_like(t2, 1, 10) tensor([[4, 8], [5, 8]])
torch.zeros_like(t1) tensor([0, 0])
Note:
For the "_like" type conversion, attention should be paid to the consistency of data types before and after conversion
torch.randn_like(t1) # t1 is an integer and will become a floating point number after conversion, so the code reports an error
RuntimeError Traceback (most recent call last) <ipython-input-285-82e06104e8a8> in <module> ----> 1 torch.randn_like(t1) RuntimeError: "normal_kernel_cpu" not implemented for 'Long'
t10 = torch.tensor([1.1, 2.2]) # Regenerate a new floating-point tensor t10 tensor([1.1000, 2.2000])
torch.randn_like(t10) tensor([1.0909, 0.0379])
4, Transformation method between Tensor and other related types
- . numpy method: convert tensor to array
t1 tensor([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
t1.numpy() array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int64)
# Of course, it can also be directly converted to array through the np.array function np.array(t1) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], dtype=int64)
- . tolist method: convert tensor to list
t1.tolist() [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
- List function: Transform tensor into list
list(t1)
[tensor(1),
tensor(2),
tensor(3),
tensor(4),
tensor(5),
tensor(6),
tensor(7),
tensor(8),
tensor(9),
tensor(10)]
**Note: * * at this time, the converted list is a list composed of 0-bit tensors, rather than a list composed of tensor values.
- . item() method: convert to numeric value
In many cases, the tensor of the final calculation result needs to be converted into a separate value for output. At this time, the. item method needs to be used
n = torch.tensor(1) n tensor(1)
n.item() 1
5, Deep copy of tensor
Deep copy using clone method
t1 tensor([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ])
t11 = t1 # t11 is a shallow copy of t1 t11 tensor([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
t1[1] tensor(2)
t1[1] = 10 t1 tensor([1, 10, 3, 4, 5, 6, 7, 8, 9, 10])
t11 # t11 is modified synchronously tensor([ 1, 10, 3, 4, 5, 6, 7, 8, 9, 10])
t1 and t11 here point to the same object. If you want t11 not to change with t1, you need to make a deep copy of t11 so that t11 has a separate object
t11 = t1.clone() t1 tensor([1, 10, 3, 4, 5, 6, 7, 8, 9, 10])
t11 tensor([ 1, 10, 3, 4, 5, 6, 7, 8, 9, 10])
t1[0] tensor(1)
t1[0] = 100 t1 tensor([100, 10, 3, 4, 5, 6, 7, 8, 9, 10])
t11 tensor([ 1, 10, 3, 4, 5, 6, 7, 8, 9, 10])