Based on PaddleHub to develop a registration system for students' wearing of masks

Keywords: Qt Mobile Python pip

Introduction to PaddleHub

PaddleHub is a domestic in-depth learning framework launched by Baidu - "flying oar" pre training model management and migration learning tool. Through PaddleHub developers can use high-quality pre training model and fine tune API to quickly complete the whole process from migration learning to application deployment. It provides a high-quality pre training model under the scull ecology, including image classification, target detection, lexical analysis, semantic model, emotional analysis, video classification, image generation, image segmentation, text review, key point detection and other mainstream models. For more model details, please refer to the official website: https://www.paddlepaddle.org.cn/hub

Based on the pre training model, PaddleHub supports the following functions:

1. Model is software. It can realize fast prediction through Python API or command line, and it is more convenient to use PaddlePaddle model library.
2. Migration learning: users can complete natural language processing and in-depth migration learning of computer vision scene with a small amount of code through the fine tune API.
3. Service based deployment, a simple command can build the API service of its own model.
4. Super parameter optimization, automatic search for the best super parameter, get better model effect.

Through PaddleHub, Using the high-quality pre training model provided by PaddleHub (including image classification, target detection, lexical analysis, semantic model, emotional analysis, video classification, image generation, image segmentation, text review, key point detection and other mainstream models), you can easily carry out migration learning, train your own model, apply your model to various scenes, and deploy it to migration Mobile, server and so on. For more details, please refer to the official website and courses https://aistudio.baidu.com/aistudio/education/group/info/1070

Next, this paper will introduce how to use PaddleHub's pre training model: Mask detection to develop a student's mask wearing detection registration system

The first thing to do is to install the flyer module and the PaddleHub
paddle installation tutorial https://www.paddlepaddle.org.cn/documentation/docs/zh/1.7/install/index_cn.html
Then install PaddleHub:

pip install paddlehub

After installation, test the prediction process and results of PaddleHub for a single picture:
Import the paddlehub module:

import paddlehub as hub
import cv2

1. Prepare the picture to be tested:

ima = cv2.imread('D:\\kz2.jpg')
image = cv2.resize(ima,(320,240))


2. Load paddlehub mask pre training model:

PaddleHub mask detection provides two kinds of pre training models, pyreidbox_ lite_ mobile_ Mask and pyreidbox_ lite_ server_ mask. Both of them are lightweight models developed based on pyramid box, a paper published by Baidu in ECCV 2018, a top computer vision conference in 2018. The model is based on the main network FaceBoxes, and has strong robustness to common problems such as light, mask occlusion, expression change, scale change, etc. The difference is that, for example, the pyrimidbox_ lite_ mobile_ Mask is an optimized model for mobile terminal, which is suitable for deployment on devices with limited computing power such as mobile terminal or edge detection.

module = hub.Module(name="pyramidbox_lite_mobile_mask")

Prepare forecast data:

input_dict = {"data": [image]}
#Or input_dict = {"image": image_filepath}, either way

3. Check the mask:

# Mask detection prediction, module.face_detection(data=input_dict) returns the predicted result of the picture
results = module.face_detection(data=input_dict)

4. Display forecast results:

result = results[0]#Since only one picture has been entered, batch_size is 1, so results has only one element
for item in result['data']:#Frame face annotation
    x1 = item['left']
    y1 = item['top']
    x2 = item['right']
    y2 = item['bottom']
    kz = item['label']

    image = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
    #And show the wearing condition of the mask on the picture
    cv2.putText(image, kz, (x1 - 5, y1 - 10), cv2.FONT_HERSHEY_PLAIN, 1.0, (0, 0, 255), 1)
GrayImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
draw_1 = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.imshow("draw_0", draw_1)  # Show a picture with a rectangular box drawn
cv2.waitKey(0)
cv2.destroyWindow("draw_0")


This is the process of predicting a picture. Next, we will use PaddleHub to develop a simple GUI interface for video detection of the wearing of masks

Main functions: input student information, open the camera to check whether to wear the mask, record the wearing information, and then export the information registration file

First, the main window interface design:

import time
from PyQt5.QtWidgets import *
from PyQt5 import QtCore, QtGui, QtWidgets


class Ui_Form(object):
    def setupUi(self, Form):
        Form.setObjectName("Form")
        Form.resize(1000, 800)

        self.label5 = QtWidgets.QLabel(Form)
        self.label5.setGeometry(QtCore.QRect(0, 240, 1000, 560))
        self.label5.setObjectName("label5")
        self.label5.setStyleSheet(
            '''QLabel{background:#76789E}''')



        '''tableView Design'''
        self.tableView = QtWidgets.QTableView(Form)
        self.tableView.setGeometry(QtCore.QRect(180, 290, 820, 510))
        self.tableView.setObjectName("tableView")
        # Set the data hierarchy, specify n rows and N columns, and pass in (n,n)
        self.model = QtGui.QStandardItemModel()
        # Set the text content of five head labels in horizontal direction
        self.model.setHorizontalHeaderLabels(['full name', 'Student number', 'college', 'Wearing condition of mask','date'])
        self.tableView.setModel(self.model)
        # Expand the remaining window part of the horizontal direction label and fill in the table
        self.tableView.horizontalHeader().setStretchLastSection(True)
        self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch)
        #Style tableView
        self.tableView.setStyleSheet(
            '''QTableView{
                    border:1px solid #74E9FF;
                    border-radius:10px;
                    padding:2px 4px;
            }''')
        self.tableView.horizontalHeader().setStyleSheet("QHeaderView::section{background:#87C9FF;}")#Set header color
        self.tableView.horizontalHeader().setSectionsClickable(False)#Set header not clickable
        self.tableView.verticalHeader().setSectionsClickable(False)#Set header not clickable
        self.tableView.setEditTriggers(QAbstractItemView.NoEditTriggers)#Settings are not editable
        #Add a sample at initialization
        self.model.appendRow([
            QtGui.QStandardItem('Forest xx'),
            QtGui.QStandardItem('U2018XXXXX'),
            QtGui.QStandardItem('College of artificial intelligence and automation'),
            QtGui.QStandardItem('No mask'),
            QtGui.QStandardItem(time.strftime('%Y.%m.%d', time.localtime(time.time()))),
        ])
        '''tableView End of design'''



        '''LineEdit Input box design'''
        self.name = QtWidgets.QLineEdit(Form)
        self.name.setGeometry(QtCore.QRect(40, 430, 100, 28))
        self.name.setObjectName("name")
        self.name.setPlaceholderText("Enter name")

        self.ID = QtWidgets.QLineEdit(Form)
        self.ID.setGeometry(QtCore.QRect(40, 490, 100, 28))
        self.ID.setObjectName("ID")
        self.ID.setPlaceholderText("Enter student ID")

        self.xueyuan = QtWidgets.QLineEdit(Form)
        self.xueyuan.setGeometry(QtCore.QRect(40, 550, 100, 28))
        self.xueyuan.setObjectName("xueyuan")
        self.xueyuan.setPlaceholderText("Enter college")

        self.name.setStyleSheet(
            '''QLineEdit{
                    border:1px solid gray;
                    width:300px;
                    border-radius:10px;
                    padding:2px 4px;
            }''')
        self.ID.setStyleSheet(
            '''QLineEdit{
                    border:1px solid gray;
                    width:300px;
                    border-radius:10px;
                    padding:2px 4px;
            }''')
        self.xueyuan.setStyleSheet(
            '''QLineEdit{
                    border:1px solid gray;
                    width:300px;
                    border-radius:10px;
                    padding:2px 4px;
            }''')

        '''LineEdit End of input box design'''

        '''QLabel Design'''
        self.labelplay = QtWidgets.QLabel(Form)
        self.labelplay.setGeometry(QtCore.QRect(0, 0, 320, 240))
        self.labelplay.setObjectName("labelplay")
        self.labeldete = QtWidgets.QLabel(Form)
        self.labeldete.setGeometry(QtCore.QRect(680, 0, 320, 240))
        self.labeldete.setObjectName("labeldete")

        self.labelname = QtWidgets.QLabel(Form)
        self.labelname.setGeometry(QtCore.QRect(5, 430, 30, 28))
        self.labelname.setObjectName("labelname")
        self.labelid = QtWidgets.QLabel(Form)
        self.labelid.setGeometry(QtCore.QRect(5, 490, 30, 28))
        self.labelid.setObjectName("labelid")
        self.labelxy = QtWidgets.QLabel(Form)
        self.labelxy.setGeometry(QtCore.QRect(5, 550, 30, 28))
        self.labelxy.setObjectName("labelxueyuan")

        self.labelinfo = QtWidgets.QLabel(Form)
        self.labelinfo.setGeometry(QtCore.QRect(180, 260, 300, 28))
        self.labelinfo.setObjectName("labelinfo")




        font = QtGui.QFont()
        font.setFamily("Algerian")
        font.setPointSize(16)

        self.labelpaddle = QtWidgets.QLabel(Form)
        self.labelpaddle.setGeometry(QtCore.QRect(320, 0, 360, 240))
        self.labelpaddle.setObjectName("labelpaddle")

        self.labell = QtWidgets.QLabel(Form)
        self.labell.setGeometry(QtCore.QRect(320, 0, 180, 30))
        self.labell.setObjectName("labell")
        self.labelr = QtWidgets.QLabel(Form)
        self.labelr.setGeometry(QtCore.QRect(500, 0, 180, 30))
        self.labelr.setObjectName("labell")


        self.labell.setFont(font)
        self.labelr.setFont(font)
        self.labell.setAlignment(QtCore.Qt.AlignLeading|QtCore.Qt.AlignLeft|QtCore.Qt.AlignTop)       #Text display location
        self.labelr.setAlignment(QtCore.Qt.AlignRight | QtCore.Qt.AlignTop | QtCore.Qt.AlignTrailing) #Text display location

        self.labelplay.setStyleSheet(
            '''QLabel{
                    border:1px solid red;
            }''')
        self.labeldete.setStyleSheet(
            '''QLabel{
                    border:1px solid red;
            }''')
        self.labelpaddle.setStyleSheet(
            '''QLabel{
                    border:1px solid #E8B735;
            }''')
        self.labell.setStyleSheet("color:white")#Set font color
        self.labelr.setStyleSheet("color:white")#Set font color

        '''QLabel End of design'''


        '''Button Design'''
        font = QtGui.QFont()
        font.setFamily("Algerian")
        font.setPointSize(12)
        self.play = QtWidgets.QPushButton(Form)
        self.play.setGeometry(QtCore.QRect(10, 260, 100, 28))
        self.play.setObjectName("play")
        self.dete = QtWidgets.QPushButton(Form)
        self.dete.setGeometry(QtCore.QRect(10, 310, 100, 28))
        self.dete.setObjectName("dete")
        self.quit = QtWidgets.QPushButton(Form)
        self.quit.setGeometry(QtCore.QRect(10, 360, 100, 28))
        self.quit.setObjectName("quit")
        self.daochu = QtWidgets.QPushButton(Form)
        self.daochu.setGeometry(QtCore.QRect(5, 650, 150, 28))
        self.daochu.setObjectName("daochu")

        self.play.setFont(font)
        self.dete.setFont(font)
        self.quit.setFont(font)
        #self.daochu.setFont(font)

        self.play.setStyleSheet(
            '''QPushButton{background:#87C9FF;border-radius:5px;}QPushButton:hover{background:blue;}''')
        self.dete.setStyleSheet(
            '''QPushButton{background:#F7D674;border-radius:5px;}QPushButton:hover{background:yellow;}''')
        self.quit.setStyleSheet(
            '''QPushButton{background:#6DDF6D;border-radius:5px;}QPushButton:hover{background:green;}''')
        self.daochu.setStyleSheet(
            '''QPushButton{background:#F4605F;border-radius:5px;}QPushButton:hover{background:red;}''')

        self.retranslateUi(Form)
        self.play.clicked.connect(Form.shoot_play)
        self.dete.clicked.connect(Form.frame_detect)
        self.quit.clicked.connect(Form.detect_quit)
        self.daochu.clicked.connect(Form.getinfo)
        QtCore.QMetaObject.connectSlotsByName(Form)
        '''Button End of design'''

    def retranslateUi(self, Form):
        _translate = QtCore.QCoreApplication.translate
        Form.setWindowTitle(_translate("Form", "The registration system of student mask detection(be based on PaddleHub Mask detection system of)"))
        self.labelplay.setText(_translate("Form", "TextLabel"))
        self.labeldete.setText(_translate("Form", "TextLabel"))
        self.labell.setText(_translate("Form", "Camera area"))
        self.labelr.setText(_translate("Form", "Detection area"))
        self.labelname.setText(_translate("Form", "full name"))
        self.labelid.setText(_translate("Form", "Student number"))
        self.labelxy.setText(_translate("Form", "college"))
        self.labelinfo.setText(_translate("Form", "Records of wearing of student masks:"))
        self.labelpaddle.setText(_translate("Form", "paddle"))
        self.play.setText(_translate("Form", "Turn on camera"))
        self.dete.setText(_translate("Form", "Test mask"))
        self.quit.setText(_translate("Form", "End detection"))
        self.daochu.setText(_translate("Form", "Export test information table"))

Test module package:

import paddlehub as hub
import cv2

def Mask_detect(image):
    state = 'No mask'
    module = hub.Module(name="pyramidbox_lite_mobile_mask")  # Load paddlehub pre training model
    input_dict = {"data": [image]}

    results = module.face_detection(data=input_dict)  # Mask detection prediction
    result = results[0]
    for item in result['data']:#The mask detection of paddlehub can detect multiple faces. By default, only one student appears in each video in this project, so the for loop is only executed once
        x1 = item['left']
        y1 = item['top']
        x2 = item['right']
        y2 = item['bottom']
        kz = item['label']

        image = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
        cv2.putText(image, str(kz), (x1 - 5, y1 - 10), cv2.FONT_HERSHEY_COMPLEX, 1.0, (0, 0, 255), 1)
        if kz == 'MASK':
            state = 'Mask on'

    return image,len(result['data']),state#Return the detection results, the number of faces detected, and the wearing of masks

Finally, the logic function design of the detection system:
Multithreaded QTread class is used in video detection

from form2 import Ui_Form
import numpy
import cv2
import csv
import sys, time
from detection import Mask_detect
from PyQt5.QtCore import *
from PyQt5.QtWidgets import *
from PyQt5.QtGui import *

class MyDesiger(QMainWindow, Ui_Form):
    def __init__(self, parent=None):
        super(MyDesiger, self).__init__(parent)
        self.setupUi(self)
        self.star_show()
        self.info = [["full name","Student number","college","Wearing condition of mask","date"]]
        self.VideoTimer = Video()
        self.VideoTimer.changePixmap.connect(self.setImage)
        self.VideoTimer.detectPixmap.connect(self.setDetect)
        self.VideoTimer.run_over.connect(self.finish_detect)
        self.VideoTimer.update_data.connect(self.add_data)

    def shoot_play(self):#Camera display
        if self.name.text()=='' or self.ID.text()=='' or self.xueyuan.text()=='':
            QMessageBox.information(self, 'Incomplete information', 'Please fill in the complete information correctly',QMessageBox.Yes)
        else:
            self.VideoTimer.working = True  # Enable camera and detection threads to work
            self.VideoTimer.start()

    def frame_detect(self):#Detect every frame
        if self.VideoTimer.working:
            self.VideoTimer.setDetect()

    def detect_quit(self):#End detection
        self.close()

    def getinfo(self):#export file
        with open('resource\\info.csv', 'w', newline='') as csvfile:
            writer = csv.writer(csvfile)
            for row in self.info:
                writer.writerow(row)
        QMessageBox.information(self, 'Export successful', 'File save path is:resource\\info.csv', QMessageBox.Yes)


    def star_show(self):#Initialize display interface
        frame = QPixmap('resource\\sample.jpg').scaled(self.labelplay.width(), self.labelplay.height())
        self.labelplay.setPixmap(frame)
        image = QPixmap('resource\\sample_detection.jpg').scaled(self.labeldete.width(), self.labeldete.height())
        self.labeldete.setPixmap(image)
        paddlehub = QPixmap('resource\\paddlehub.png').scaled(self.labelpaddle.width(), self.labelpaddle.height())
        self.labelpaddle.setPixmap(paddlehub)

    def finish_detect(self,kz):#detection complete
        self.star_show()
        QMessageBox.information(self, 'detection result :', kz+'!!', QMessageBox.Yes)
        self.name.clear()
        self.ID.clear()
        self.xueyuan.clear()



    def add_data(self,kz):#Log table information update
        self.model.appendRow([
            QStandardItem(self.name.text()),
            QStandardItem(self.ID.text()),
            QStandardItem(self.xueyuan.text()),
            QStandardItem(kz),
            QStandardItem(time.strftime('%Y.%m.%d',time.localtime(time.time()))),
        ])
        self.info.append([self.name.text(), self.ID.text(), self.xueyuan.text(),kz,time.strftime('%Y.%m.%d',time.localtime(time.time()))])



    def setImage(self, frame):
        self.labelplay.setPixmap(QPixmap.fromImage(QImage(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB),640/2,480/2,13)))

    def setDetect(self, image):
        self.labeldete.setPixmap(QPixmap.fromImage(QImage(cv2.cvtColor(image, cv2.COLOR_BGR2RGB),640/2,480/2,13)))


class Video(QThread):
    changePixmap = pyqtSignal(numpy.ndarray)
    detectPixmap = pyqtSignal(numpy.ndarray)
    update_data = pyqtSignal(str)
    run_over = pyqtSignal(str)

    def __init__(self):
        QThread.__init__(self)
        self.detect = False   #Whether it can be detected
        self.working = False  #Whether it can work
        self.count = 0        #Number of face detection

    def run(self):
        cap = cv2.VideoCapture(0)
        while self.working:
            ret, frame = cap.read()
            frame = cv2.resize(frame,(320,240))
            self.changePixmap.emit(frame)
            if self.detect:
                img = frame.copy()
                detected_image,num,kz=Mask_detect(img)#Mask detection
                self.detectPixmap.emit(detected_image)
                if num > 0:
                    self.count += 1
            if self.count >= 6:#Detection stops when face is detected more than 6 frames
                self.detect = False
                self.working = False
                self.count = 0  #Count clear
                self.run_over.emit(kz)
                self.update_data.emit(kz)
        cap.release()

    def setDetect(self):
        self.detect = True
    def setquit(self):
        self.detect = False
        self.working = False


if __name__ == "__main__":
    app = QApplication(sys.argv)
    ui = MyDesiger()
    ui.show()
    sys.exit(app.exec_())

Results:

This project has been uploaded to github
github project address

Posted by excence on Tue, 23 Jun 2020 22:53:42 -0700