Tip 1: Learning Address Point Here
Tip 1: Word embeddings: how to transform text into numbers

Preface

_Task01 mainly uses RNN framework (note: this paper does not give a detailed explanation of RNN principles), IMDB dataset to build a Baseline model of text affective analysis tasks.

1. Model building process

1.1 Data Preprocessing

_TorchText data preprocessing consists of the following steps:

Step 1: Use Field to process data;
Step 2: Use Datasets to load the IMDB dataset you want to use, set up the training and test sets;
Step 3: You must build a vocabulary because our model can't manipulate strings, it can only manipulate numbers.
Step 4: To create an iterator, you need to create an iterator for a validation set, a test set, and a training set, each iteration returns one batch of data.

_Note 1: For Step 2, you can set the ratio of training set to validation set by setting the split_ratio parameter, that is, split_ratio of 0.8 means 80% of the samples form the training set, 20% form the validation set, and by default, the data is divided into training set and validation set by 70% and 30%.
_Note 2: The random seed SEED is passed to the random_state parameter to ensure that we get the same training set and verification set each time.

1.2 Model Construction and Training

_1. The model building process is as follows:
_Attention to gradient zeroing

1. Select a model, using RNN here; use torch.nn for model loading;
2. Define parameters;
3. Calculate the number of parameters to be trained (optional);
4. Model Training
_4.1, Set up Optimizer, SGD is selected here;
_4.2, Define the loss function, BCEWithLogitsLoss is used here;
_4.3, with [to], the tensor can be calculated on the gpu; (optional)
_4.4, calculation accuracy;
_4.5, train() function iterates over all samples, one batch at a time, [training function];
_4.6, Model evaluation;
_4.7, Calculate the consumption time of each epoch; (optional)
_4.8, model training [attention to model preservation] and model validation;

_2. Tips for model building functions:

# Building models
class RNN(nn.Module):
    def __init__(self, input_dim, embedding_dim, hidden_dim, output_dim):
    	...   
    def forward(self, text):
    	...
------------------------------------------------------
 # Parameter calculation
def count_parameters(model):
 	...
------------------------------------------------------
 # Optimizer, loss function, gpu calculation
import torch.optim as optim
optimizer = optim.SGD(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
model = model.to(device)
criterion = criterion.to(device)
------------------------------------------------------
# accuracy rate
def binary_accuracy(preds, y):
	...
------------------------------------------------------
# Training function
def train(model, iterator, optimizer, criterion):
	...
------------------------------------------------------
# Model evaluation
def evaluate(model, iterator, criterion):
	...
------------------------------------------------------
# Training time consumed
import time
def epoch_time(start_time, end_time):
	...
------------------------------------------------------
# Model training and validation
N_EPOCHS = 5
best_valid_loss = float('inf')
for epoch in range(N_EPOCHS):
    start_time = time.time()
    train_loss, train_acc = train(model, train_iterator, optimizer, criterion)
    valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)
    end_time = time.time()
    epoch_mins, epoch_secs = epoch_time(start_time, end_time)
    if valid_loss < best_valid_loss:
        best_valid_loss = valid_loss
        torch.save(model.state_dict(), 'tut1-model.pt')
    print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')
    print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')
    print(f'\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc*100:.2f}%')
    
# Verification  
model.load_state_dict(torch.load('tut1-model.pt'))
test_loss, test_acc = evaluate(model, test_iterator, criterion)
print(f'Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:.2f}%')

II. EXPERIMENTAL SUMMARY

_Experiments found that the accuracy rate is very low, and during the training process of five epoch s, training loss and verification loss are basically unchanged. Combining with the characteristics of the model itself, guess that the model is not fit enough, and subsequent learning will optimize the model. There are many factors that cause the Loss of in-depth learning not to decline, here is an article to guide you to read.
Reasons why Loss of in-depth learning does not fall here

summary

_That's all about Task01. If you encounter a situation where the original link cannot be loaded, you can contact the blogger to ask for it.