linear regression

Linear regression formula:
y^=∑i=1p(wixi+bi) \hat{y} = \sum ^{p}_{i=1}(w_{i}x_{i}+b_{i}) y^​=i=1∑p​(wi​xi​+bi​)
Loss function:
l(i)(w,b)=12(y^(i)−y(i))L(w,b)=1n∑i=1nl(i)(w,b)=1n∑i=1n12(wTx(i)+b−y(i))2 l^{(i)}(w,b) = \frac{1}{2}(\hat y^{(i)}-y^{(i)}) \\ L(w,b) =\frac{1}{n}\sum ^{n}_{i=1}l^{(i)}(w,b)=\frac{1}{n}\sum ^{n}_{i=1}\frac{1}{2}(w^{T}x^{(i)}+b-y^{(i)})^{2} l(i)(w,b)=21​(y^​(i)−y(i))L(w,b)=n1​i=1∑n​l(i)(w,b)=n1​i=1∑n​21​(wTx(i)+b−y(i))2
Random gradient descent:
(w,b)←(w,b)−ηB∑i∈B∂(w,b)l(i)(w,b) (w,b)\leftarrow(w,b)-\frac{\eta}{\mathcal B}\sum_{i\in \mathcal B}\partial_{(w,b)}l^{(i)}(w,b) (w,b)←(w,b)−Bη​i∈B∑​∂(w,b)​l(i)(w,b)
B is the batchsize, η is the learning rate.

Dataset read:

Image universal data loader ImageFolder

Data organization form: root / [category name] / xxx.png

dset.ImageFolder(root="root folder path", [transform, target_transform])

He has the following member variables:

• self.classes - save the class name with a list
• Self.class'to'idx - the index corresponding to the class name
• self.imgs - save the list of (IMG path, class) tuples

It is summarized as the following steps:

• Read data list
• Scramble list order
• Open data and label production iterator from list

def data_iter(batch_size, features, labels):
    num_examples = len(features)   #Sample size
    indices = list(range(num_examples)) #Generate number
    random.shuffle(indices)  # random read 10 samples
    for i in range(0, num_examples, batch_size): #i increase bachsize every time
        j = torch.LongTensor(indices[i: min(i + batch_size, num_examples)]) # the last time may be not enough for a whole batch
        yield  features.index_select(0, j), labels.index_select(0, j) #Data return

Model definition and training process:

#########################Read data#################################
dataset = Data.TensorDataset(features, labels)

# put dataset into DataLoader
data_iter = Data.DataLoader(
    dataset=dataset,            # torch TensorDataset format
    batch_size=batch_size,      # mini batch size
    shuffle=True,               # whether shuffle the data or not
    num_workers=2,              # read data in multithreading
)

#############################Definition model##################################
class LinearNet(nn.Module):
    def __init__(self, n_feature):
        super(LinearNet, self).__init__()      # call father function to init 
        self.linear = nn.Linear(n_feature, 1)  # function prototype: `torch.nn.Linear(in_features, out_features, bias=True)`
        #You can also initialize each layer here

    def forward(self, x):
        y = self.linear(x)
        return y
    
##########################Instantiation model, optimizer, loss function##########################
net = LinearNet(num_inputs)
loss = nn.MSELoss() 
optimizer = optim.SGD(net.parameters(), lr=0.03) 

################################Training network#######################################
for epoch in range(1, num_epochs + 1):   #Batch cycle of training
    for X, y in data_iter:          #Training data batchsize reading
        output = net(X)             #Forward propagation, get prediction results
        l = loss(output, y.view(-1, 1))   #Calculate loss
        optimizer.zero_grad() # reset gradient, equal to net.zero_grad()
        l.backward()         #Backward propagation, gradient
        optimizer.step()     #Optimization according to gradient and optimization strategy 
    print('epoch %d, loss: %f' % (epoch, l.item()))

Softmax and classification model

Formula:
y^1,y^2,y^3=softmax(o1,o2,o3)y^k=exp(ok)∑i=13exp(oi)         (k∈1,2,3) \hat y_1,\hat y_2,\hat y_3 = softmax(o_1,o_2,o_3) \\ \hat y_k = \frac {exp(o_k)} {\sum ^3 _{i=1}exp(o_i)} \ \ \ \ \ \ \ \ \ (k\in{1,2,3)} y^​1​,y^​2​,y^​3​=softmax(o1​,o2​,o3​)y^​k​=∑i=13​exp(oi​)exp(ok​)​         (k∈1,2,3)
Meaning: transform the output value into a probability distribution with a positive sum of 1 to facilitate classification and judgment

Use together: cross entropy as loss function or softmaxloss

cross entropy
L=∑k=1Nyklog1y^k L = \sum^{N}_{k = 1}y_klog\frac{1}{\hat y_k} L=k=1∑N​yk​logy^​k​1​
softmaxloss
L=−∑j=1Tyilogy^j L = -\sum ^T _{j=1} y_ilog\hat y_j L=−j=1∑T​yi​logy^​j​
It can be seen that the two loss functions are equivalent.

torchvision.transforms

Basic operation:

Center cut the square of class torchvision.transforms.centercrop (size) 255 - > 255, (128,96) - > rectangle

Then cut class torchvision.transforms.random crop (size, padding = 0) the size is the same as above, and whether padding is filled and then cross

class torchvision.transforms.RandomSizedCrop(size, interpolation=2)

Fill class torchvision.transforms.Pad(padding, fill=0) padding size, fill number

Class torch vision. Transforms. Normalize (mean, STD) mean and variance

Convert to Tensor class torchvision.transforms.ToTensor

Use examples with datloader

import torchvision
import torch
train_augmentation = torchvision.transforms.Compose([torchvision.transforms.Resize(256),                                    torchvision.transforms.RandomCrop(224),                         torchvision.transofrms.RandomHorizontalFlip(),
                               torchvision.transforms.ToTensor(),
                               torch vision.Normalize([0.485, 0.456, -.406],[0.229, 0.224, 0.225])
                                                    ])

Class custom_dataread(torch.utils.data.Dataset):
    def __init__():
        ...
    def __getitem__():
        # use self.transform for input image
    def __len__():
        ...

train_loader = torch.utils.data.DataLoader(
    custom_dataread(transform=train_augmentation),
    batch_size = batch_size, shuffle = True,
    num_workers = workers, pin_memory = True)

Multilayer perceptron

The direct superposition of multi-layer linear network layer can't increase the network's representation ability, so it is necessary to add a non-linear layer between network layer and layer - activation function and whitening function to remove the linearity.

[failed to transfer the pictures in the external chain. The source station may have anti-theft chain mechanism. It is recommended to save the pictures and upload them directly (img-vUlpAp6p-1581683875728)(C:\Users\fan\Desktop \ typing notes \ activation function summary. png))

= workers, pin_memory = True)

#Multilayer perceptron

The direct superposition of multi-layer linear network layer can't increase the network's representation ability, so it is necessary to add a non-linear layer between network layer and layer - activation function and whitening function to remove the linearity.

! [insert picture description here] (https://img-blog.csdnimg.cn/20200214203826104.png? X-oss-process = image / watermark, type ﹐ zmfuz3pozw5nagvpdgk, shadow ﹐ 10, text ﹐ ahr0chm6ly9ibg9nlmnzg4ubmv0l2zhbjexmdi5ntgxnte =, size ﹐ 16, color ﹐ ffff, t ﹐ 70)