Introduction to PaddleHub
PaddleHub is a domestic in-depth learning framework launched by Baidu - "flying oar" pre training model management and migration learning tool. Through PaddleHub developers can use high-quality pre training model and fine tune API to quickly complete the whole process from migration learning to application deployment. It provides a high-quality pre training model under the scull ecology, including image classification, target detection, lexical analysis, semantic model, emotional analysis, video classification, image generation, image segmentation, text review, key point detection and other mainstream models. For more model details, please refer to the official website: https://www.paddlepaddle.org.cn/hub
Based on the pre training model, PaddleHub supports the following functions:
1. Model is software. It can realize fast prediction through Python API or command line, and it is more convenient to use PaddlePaddle model library.
2. Migration learning: users can complete natural language processing and in-depth migration learning of computer vision scene with a small amount of code through the fine tune API.
3. Service based deployment, a simple command can build the API service of its own model.
4. Super parameter optimization, automatic search for the best super parameter, get better model effect.
Through PaddleHub, Using the high-quality pre training model provided by PaddleHub (including image classification, target detection, lexical analysis, semantic model, emotional analysis, video classification, image generation, image segmentation, text review, key point detection and other mainstream models), you can easily carry out migration learning, train your own model, apply your model to various scenes, and deploy it to migration Mobile, server and so on. For more details, please refer to the official website and courses https://aistudio.baidu.com/aistudio/education/group/info/1070
Next, this paper will introduce how to use PaddleHub's pre training model: Mask detection to develop a student's mask wearing detection registration system
The first thing to do is to install the flyer module and the PaddleHub
paddle installation tutorial https://www.paddlepaddle.org.cn/documentation/docs/zh/1.7/install/index_cn.html
Then install PaddleHub:
pip install paddlehub
After installation, test the prediction process and results of PaddleHub for a single picture:
Import the paddlehub module:
import paddlehub as hub import cv2
1. Prepare the picture to be tested:
ima = cv2.imread('D:\\kz2.jpg') image = cv2.resize(ima,(320,240))
2. Load paddlehub mask pre training model:
PaddleHub mask detection provides two kinds of pre training models, pyreidbox_ lite_ mobile_ Mask and pyreidbox_ lite_ server_ mask. Both of them are lightweight models developed based on pyramid box, a paper published by Baidu in ECCV 2018, a top computer vision conference in 2018. The model is based on the main network FaceBoxes, and has strong robustness to common problems such as light, mask occlusion, expression change, scale change, etc. The difference is that, for example, the pyrimidbox_ lite_ mobile_ Mask is an optimized model for mobile terminal, which is suitable for deployment on devices with limited computing power such as mobile terminal or edge detection.
module = hub.Module(name="pyramidbox_lite_mobile_mask")
Prepare forecast data:
input_dict = {"data": [image]} #Or input_dict = {"image": image_filepath}, either way
3. Check the mask:
# Mask detection prediction, module.face_detection(data=input_dict) returns the predicted result of the picture results = module.face_detection(data=input_dict)
4. Display forecast results:
result = results[0]#Since only one picture has been entered, batch_size is 1, so results has only one element for item in result['data']:#Frame face annotation x1 = item['left'] y1 = item['top'] x2 = item['right'] y2 = item['bottom'] kz = item['label'] image = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2) #And show the wearing condition of the mask on the picture cv2.putText(image, kz, (x1 - 5, y1 - 10), cv2.FONT_HERSHEY_PLAIN, 1.0, (0, 0, 255), 1) GrayImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) draw_1 = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2) cv2.imshow("draw_0", draw_1) # Show a picture with a rectangular box drawn cv2.waitKey(0) cv2.destroyWindow("draw_0")
This is the process of predicting a picture. Next, we will use PaddleHub to develop a simple GUI interface for video detection of the wearing of masks
Main functions: input student information, open the camera to check whether to wear the mask, record the wearing information, and then export the information registration file
First, the main window interface design:
import time from PyQt5.QtWidgets import * from PyQt5 import QtCore, QtGui, QtWidgets class Ui_Form(object): def setupUi(self, Form): Form.setObjectName("Form") Form.resize(1000, 800) self.label5 = QtWidgets.QLabel(Form) self.label5.setGeometry(QtCore.QRect(0, 240, 1000, 560)) self.label5.setObjectName("label5") self.label5.setStyleSheet( '''QLabel{background:#76789E}''') '''tableView Design''' self.tableView = QtWidgets.QTableView(Form) self.tableView.setGeometry(QtCore.QRect(180, 290, 820, 510)) self.tableView.setObjectName("tableView") # Set the data hierarchy, specify n rows and N columns, and pass in (n,n) self.model = QtGui.QStandardItemModel() # Set the text content of five head labels in horizontal direction self.model.setHorizontalHeaderLabels(['full name', 'Student number', 'college', 'Wearing condition of mask','date']) self.tableView.setModel(self.model) # Expand the remaining window part of the horizontal direction label and fill in the table self.tableView.horizontalHeader().setStretchLastSection(True) self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch) #Style tableView self.tableView.setStyleSheet( '''QTableView{ border:1px solid #74E9FF; border-radius:10px; padding:2px 4px; }''') self.tableView.horizontalHeader().setStyleSheet("QHeaderView::section{background:#87C9FF;}")#Set header color self.tableView.horizontalHeader().setSectionsClickable(False)#Set header not clickable self.tableView.verticalHeader().setSectionsClickable(False)#Set header not clickable self.tableView.setEditTriggers(QAbstractItemView.NoEditTriggers)#Settings are not editable #Add a sample at initialization self.model.appendRow([ QtGui.QStandardItem('Forest xx'), QtGui.QStandardItem('U2018XXXXX'), QtGui.QStandardItem('College of artificial intelligence and automation'), QtGui.QStandardItem('No mask'), QtGui.QStandardItem(time.strftime('%Y.%m.%d', time.localtime(time.time()))), ]) '''tableView End of design''' '''LineEdit Input box design''' self.name = QtWidgets.QLineEdit(Form) self.name.setGeometry(QtCore.QRect(40, 430, 100, 28)) self.name.setObjectName("name") self.name.setPlaceholderText("Enter name") self.ID = QtWidgets.QLineEdit(Form) self.ID.setGeometry(QtCore.QRect(40, 490, 100, 28)) self.ID.setObjectName("ID") self.ID.setPlaceholderText("Enter student ID") self.xueyuan = QtWidgets.QLineEdit(Form) self.xueyuan.setGeometry(QtCore.QRect(40, 550, 100, 28)) self.xueyuan.setObjectName("xueyuan") self.xueyuan.setPlaceholderText("Enter college") self.name.setStyleSheet( '''QLineEdit{ border:1px solid gray; width:300px; border-radius:10px; padding:2px 4px; }''') self.ID.setStyleSheet( '''QLineEdit{ border:1px solid gray; width:300px; border-radius:10px; padding:2px 4px; }''') self.xueyuan.setStyleSheet( '''QLineEdit{ border:1px solid gray; width:300px; border-radius:10px; padding:2px 4px; }''') '''LineEdit End of input box design''' '''QLabel Design''' self.labelplay = QtWidgets.QLabel(Form) self.labelplay.setGeometry(QtCore.QRect(0, 0, 320, 240)) self.labelplay.setObjectName("labelplay") self.labeldete = QtWidgets.QLabel(Form) self.labeldete.setGeometry(QtCore.QRect(680, 0, 320, 240)) self.labeldete.setObjectName("labeldete") self.labelname = QtWidgets.QLabel(Form) self.labelname.setGeometry(QtCore.QRect(5, 430, 30, 28)) self.labelname.setObjectName("labelname") self.labelid = QtWidgets.QLabel(Form) self.labelid.setGeometry(QtCore.QRect(5, 490, 30, 28)) self.labelid.setObjectName("labelid") self.labelxy = QtWidgets.QLabel(Form) self.labelxy.setGeometry(QtCore.QRect(5, 550, 30, 28)) self.labelxy.setObjectName("labelxueyuan") self.labelinfo = QtWidgets.QLabel(Form) self.labelinfo.setGeometry(QtCore.QRect(180, 260, 300, 28)) self.labelinfo.setObjectName("labelinfo") font = QtGui.QFont() font.setFamily("Algerian") font.setPointSize(16) self.labelpaddle = QtWidgets.QLabel(Form) self.labelpaddle.setGeometry(QtCore.QRect(320, 0, 360, 240)) self.labelpaddle.setObjectName("labelpaddle") self.labell = QtWidgets.QLabel(Form) self.labell.setGeometry(QtCore.QRect(320, 0, 180, 30)) self.labell.setObjectName("labell") self.labelr = QtWidgets.QLabel(Form) self.labelr.setGeometry(QtCore.QRect(500, 0, 180, 30)) self.labelr.setObjectName("labell") self.labell.setFont(font) self.labelr.setFont(font) self.labell.setAlignment(QtCore.Qt.AlignLeading|QtCore.Qt.AlignLeft|QtCore.Qt.AlignTop) #Text display location self.labelr.setAlignment(QtCore.Qt.AlignRight | QtCore.Qt.AlignTop | QtCore.Qt.AlignTrailing) #Text display location self.labelplay.setStyleSheet( '''QLabel{ border:1px solid red; }''') self.labeldete.setStyleSheet( '''QLabel{ border:1px solid red; }''') self.labelpaddle.setStyleSheet( '''QLabel{ border:1px solid #E8B735; }''') self.labell.setStyleSheet("color:white")#Set font color self.labelr.setStyleSheet("color:white")#Set font color '''QLabel End of design''' '''Button Design''' font = QtGui.QFont() font.setFamily("Algerian") font.setPointSize(12) self.play = QtWidgets.QPushButton(Form) self.play.setGeometry(QtCore.QRect(10, 260, 100, 28)) self.play.setObjectName("play") self.dete = QtWidgets.QPushButton(Form) self.dete.setGeometry(QtCore.QRect(10, 310, 100, 28)) self.dete.setObjectName("dete") self.quit = QtWidgets.QPushButton(Form) self.quit.setGeometry(QtCore.QRect(10, 360, 100, 28)) self.quit.setObjectName("quit") self.daochu = QtWidgets.QPushButton(Form) self.daochu.setGeometry(QtCore.QRect(5, 650, 150, 28)) self.daochu.setObjectName("daochu") self.play.setFont(font) self.dete.setFont(font) self.quit.setFont(font) #self.daochu.setFont(font) self.play.setStyleSheet( '''QPushButton{background:#87C9FF;border-radius:5px;}QPushButton:hover{background:blue;}''') self.dete.setStyleSheet( '''QPushButton{background:#F7D674;border-radius:5px;}QPushButton:hover{background:yellow;}''') self.quit.setStyleSheet( '''QPushButton{background:#6DDF6D;border-radius:5px;}QPushButton:hover{background:green;}''') self.daochu.setStyleSheet( '''QPushButton{background:#F4605F;border-radius:5px;}QPushButton:hover{background:red;}''') self.retranslateUi(Form) self.play.clicked.connect(Form.shoot_play) self.dete.clicked.connect(Form.frame_detect) self.quit.clicked.connect(Form.detect_quit) self.daochu.clicked.connect(Form.getinfo) QtCore.QMetaObject.connectSlotsByName(Form) '''Button End of design''' def retranslateUi(self, Form): _translate = QtCore.QCoreApplication.translate Form.setWindowTitle(_translate("Form", "The registration system of student mask detection(be based on PaddleHub Mask detection system of)")) self.labelplay.setText(_translate("Form", "TextLabel")) self.labeldete.setText(_translate("Form", "TextLabel")) self.labell.setText(_translate("Form", "Camera area")) self.labelr.setText(_translate("Form", "Detection area")) self.labelname.setText(_translate("Form", "full name")) self.labelid.setText(_translate("Form", "Student number")) self.labelxy.setText(_translate("Form", "college")) self.labelinfo.setText(_translate("Form", "Records of wearing of student masks:")) self.labelpaddle.setText(_translate("Form", "paddle")) self.play.setText(_translate("Form", "Turn on camera")) self.dete.setText(_translate("Form", "Test mask")) self.quit.setText(_translate("Form", "End detection")) self.daochu.setText(_translate("Form", "Export test information table"))
Test module package:
import paddlehub as hub import cv2 def Mask_detect(image): state = 'No mask' module = hub.Module(name="pyramidbox_lite_mobile_mask") # Load paddlehub pre training model input_dict = {"data": [image]} results = module.face_detection(data=input_dict) # Mask detection prediction result = results[0] for item in result['data']:#The mask detection of paddlehub can detect multiple faces. By default, only one student appears in each video in this project, so the for loop is only executed once x1 = item['left'] y1 = item['top'] x2 = item['right'] y2 = item['bottom'] kz = item['label'] image = cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2) cv2.putText(image, str(kz), (x1 - 5, y1 - 10), cv2.FONT_HERSHEY_COMPLEX, 1.0, (0, 0, 255), 1) if kz == 'MASK': state = 'Mask on' return image,len(result['data']),state#Return the detection results, the number of faces detected, and the wearing of masks
Finally, the logic function design of the detection system:
Multithreaded QTread class is used in video detection
from form2 import Ui_Form import numpy import cv2 import csv import sys, time from detection import Mask_detect from PyQt5.QtCore import * from PyQt5.QtWidgets import * from PyQt5.QtGui import * class MyDesiger(QMainWindow, Ui_Form): def __init__(self, parent=None): super(MyDesiger, self).__init__(parent) self.setupUi(self) self.star_show() self.info = [["full name","Student number","college","Wearing condition of mask","date"]] self.VideoTimer = Video() self.VideoTimer.changePixmap.connect(self.setImage) self.VideoTimer.detectPixmap.connect(self.setDetect) self.VideoTimer.run_over.connect(self.finish_detect) self.VideoTimer.update_data.connect(self.add_data) def shoot_play(self):#Camera display if self.name.text()=='' or self.ID.text()=='' or self.xueyuan.text()=='': QMessageBox.information(self, 'Incomplete information', 'Please fill in the complete information correctly',QMessageBox.Yes) else: self.VideoTimer.working = True # Enable camera and detection threads to work self.VideoTimer.start() def frame_detect(self):#Detect every frame if self.VideoTimer.working: self.VideoTimer.setDetect() def detect_quit(self):#End detection self.close() def getinfo(self):#export file with open('resource\\info.csv', 'w', newline='') as csvfile: writer = csv.writer(csvfile) for row in self.info: writer.writerow(row) QMessageBox.information(self, 'Export successful', 'File save path is:resource\\info.csv', QMessageBox.Yes) def star_show(self):#Initialize display interface frame = QPixmap('resource\\sample.jpg').scaled(self.labelplay.width(), self.labelplay.height()) self.labelplay.setPixmap(frame) image = QPixmap('resource\\sample_detection.jpg').scaled(self.labeldete.width(), self.labeldete.height()) self.labeldete.setPixmap(image) paddlehub = QPixmap('resource\\paddlehub.png').scaled(self.labelpaddle.width(), self.labelpaddle.height()) self.labelpaddle.setPixmap(paddlehub) def finish_detect(self,kz):#detection complete self.star_show() QMessageBox.information(self, 'detection result :', kz+'!!', QMessageBox.Yes) self.name.clear() self.ID.clear() self.xueyuan.clear() def add_data(self,kz):#Log table information update self.model.appendRow([ QStandardItem(self.name.text()), QStandardItem(self.ID.text()), QStandardItem(self.xueyuan.text()), QStandardItem(kz), QStandardItem(time.strftime('%Y.%m.%d',time.localtime(time.time()))), ]) self.info.append([self.name.text(), self.ID.text(), self.xueyuan.text(),kz,time.strftime('%Y.%m.%d',time.localtime(time.time()))]) def setImage(self, frame): self.labelplay.setPixmap(QPixmap.fromImage(QImage(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB),640/2,480/2,13))) def setDetect(self, image): self.labeldete.setPixmap(QPixmap.fromImage(QImage(cv2.cvtColor(image, cv2.COLOR_BGR2RGB),640/2,480/2,13))) class Video(QThread): changePixmap = pyqtSignal(numpy.ndarray) detectPixmap = pyqtSignal(numpy.ndarray) update_data = pyqtSignal(str) run_over = pyqtSignal(str) def __init__(self): QThread.__init__(self) self.detect = False #Whether it can be detected self.working = False #Whether it can work self.count = 0 #Number of face detection def run(self): cap = cv2.VideoCapture(0) while self.working: ret, frame = cap.read() frame = cv2.resize(frame,(320,240)) self.changePixmap.emit(frame) if self.detect: img = frame.copy() detected_image,num,kz=Mask_detect(img)#Mask detection self.detectPixmap.emit(detected_image) if num > 0: self.count += 1 if self.count >= 6:#Detection stops when face is detected more than 6 frames self.detect = False self.working = False self.count = 0 #Count clear self.run_over.emit(kz) self.update_data.emit(kz) cap.release() def setDetect(self): self.detect = True def setquit(self): self.detect = False self.working = False if __name__ == "__main__": app = QApplication(sys.argv) ui = MyDesiger() ui.show() sys.exit(app.exec_())
Results:
This project has been uploaded to github
github project address