I. Preface

Similar projects have been done many years ago. It's just to upload the local pictures to the server. It's so simple. In fact, it's easy to upload with http post. You don't need to customize the protocol. You can directly set the binary data. If you use TCP or UDP communication, you must customize the protocol, because you don't know when the data will be received is the complete picture data , may be sending a lot of picture data at the same time, and can't distinguish which client sent the received picture. If TCP has a long connection, you need to have a heart to jump to detect the connection, so you must customize a set of protocol to support the communication. This protocol adopts the communication protocol format of Shanghai regulatory platform, which has strong expansibility. The header information includes type + current completion The data length of the whole package, which is the identification of the communication protocol, so that another type of building intercom can be called IDOOR next time. According to this identification, the server can know what kind of parsing algorithm is used to process the later data. However, the data length of the current complete package can be used to process the received data. Only the data of this length can indicate that the receiving is completed Complete picture data, and then decode it. When the transmitted pictures reach a certain speed, for example, 20 pictures are transmitted in one second, which is equivalent to the transmission of video. Generally, people's naked eyes recognize that 20 pictures in one second are basically video.

In theory, TCP is a stable connection, which will not lose packets or insert any packet into the middle of a packet. It is sure to ensure the integrity of a packet. There are two types of TCP connections. One is a long connection. Once connected, it always communicates. It is mainly used in the scene of frequent communication, such as real-time upload. The other is a short connection. The client sends the data or the server connects Disconnect the connection immediately after receiving the data, which is mainly used in infrequent communication scenarios such as alarm upload. After all, alarm rarely occurs in a day. It is better to use short connection, which can save a lot of system overhead. QT is also a good encapsulation for TCP communication. In a few small concurrent projects with dozens of connections, the efficiency is OK. It is said that qnetw of Qt5 The bottom layer of the ork component has been rewritten, which is more efficient than Qt4. I didn't need to check the corresponding source code in detail, just heard about it. QT network communication class, we usually use three: QTcpSocket client class, QTcpServer server class, QUdpSocket communication class, why there is no QUdpServer class? In fact, UDP is a connectionless communication, which takes up a small amount of resources. It can be either a client or a server. If it is to be a server, specify the port to call the bind method. This program supports both TCP mode and UDP mode. It is recommended to use TCP mode after actual test. UDP mode will lose packets when a large number of packets are sent in a short time due to no connection, and the packet size is limited, which is 65507 bytes, about 64K. Therefore, the resolution of pictures transmitted in real time in UDP mode cannot be too large. The video file measured at 640 * 480 is still good. 7 20p is basically a bit miserable, with a lot of packet loss. It may need to be improved from the protocol later.

The pictures in this program and protocol are transmitted by Base64 encoding. After receiving, the base64 string will be decoded to generate pictures. The QByteArray built-in class toBase64 method will be converted into Base64 encoded strings, and the QByteArray::fromBase64 method will restore the base64 string to data. After many experiments, the statistical data shows that the speed of encoding and decoding is still OK, including 25 ms-30 MS for 720P pictures, 15 MS-20 MS for decoding, 35 ms-40 MS for 1080P pictures and 25 ms-30 MS for decoding. Generally speaking, there is no problem in transmitting 25-30 pictures and decoding 25-30 pictures in a second. It's just CPU coding and decoding. If the number of channels is large, it still consumes CPU. However, it's no pressure to deal with some simple application scenarios.

Experience address: https://pan.baidu.com/s/1bbL2ZughZAgfIGrexyN-9g Extraction code: zkeh file name: Bin? Video? Image.zip.

II. Functional features

1. Multithread sending and receiving picture data and analyzing picture data, no card in the main interface.
2. At the same time, it supports both TCP and UDP modes, encapsulating the client and server classes of TCP mode and UDP mode.
3. The image transmission client supports sending to multiple servers at the same time, which can be used as a teacher machine to send to multiple student machines on the same screen.
4. At the same time, it supports multiple clients to send pictures to the server at the same time, and each connection of the server will automatically open a thread to send, receive and analyze picture data.
5. The signal slot mechanism of the custom label control draws pictures without the main interface.
6. It has its own heartbeat mechanism to judge whether it is offline and automatically reconnects the server. The timeout can be set.
7. Each message has a unique message ID uuid. After receiving, the server will return the corresponding uuid message to indicate receipt. The client can judge that the resolution of the server is successful based on the returned message, and it does not need to be sent again, so as to ensure that the data server sent receives and analyzes successfully.
8. Each message has a unique picture ID flag, which is equivalent to ID number. According to this ID, judge which interface needs to be displayed.
9. The picture is sent in the format of base64 string, and the receiving end receives the picture data of base64 string, decodes and regenerates the picture.
10. All data are sent and received by signal, which is convenient for output and check.
11. Single instance classes are provided, which is convenient to use directly when there is only one without new.
12. Using the self defined xml protocol, you can freely expand other attribute fields, such as the content with pictures and so on.

III. communication protocol

1. TCP long connection and UDP protocol are optional. The default communication port is 6000.
2. The self defined xml communication protocol is adopted.
3. All transfers are added with 20 byte headers: IIMAGE:0000000000000, IIMAGE: fixed headers followed by 13 byte content length (including 20 header length) strings.
4. The following protocol section omits the header bytes.
5. The uuid in the data returned by the server is the uuid corresponding to the received message.
6. Each time the server returns, it brings the current time, which can be used for client timing.

Client sends heartbeat
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" Flag="SHJC00000001">
	<ClientHeart/>
</ImageClient>

//Server receives heartbeat return
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" NowTime="2019-12-05 16:37:47">
	Ok
</ImageServer>

//Client sends picture
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" Flag="SHJC00000001">
	<ClientImage>picture base64 Encoded string/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAJAAtADASIAAhEBAxEB/8QAHwAAAQUBAQEB...nvWsQRlXA61mTjmtWcdazLgcmrQ0U2plSMKjpDE7UtFFAwxRRRQAUuKWigQlFFFLcD//2Q==</ClientImage>
</ImageClient>

//The server receives the picture and returns it
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" NowTime="2019-12-05 16:38:47">
	Ack
</ImageServer>

IV. renderings

V. core code

//Image to base64 string
QByteArray DeviceFun::getImageData2(const QImage &image)
{
    QByteArray imageData;
    QBuffer buffer(&imageData);
    image.save(&buffer, "jpg");
    imageData = imageData.toBase64();
    return imageData;
}

//base64 string to picture
QImage DeviceFun::getImage(const QString &data)
{
    QByteArray imageData = QByteArray::fromBase64(data.toLatin1());
    QImage image;
    image.loadFromData(imageData);
    return image;
}

//Client thread sends picture
void TcpImageClient::run()
{
    while(!stopped) {
        //In fact, it can be done by timer. After all, the write of tcp is asynchronous and the operating system automatically schedules
        //For later expansibility, for example, it needs to judge whether the transmission is successful or not, and it needs to be processed synchronously, so the changed thread will process it
        //It also takes time for image data to be converted into base64 encoded data. The main time is transcoding
        //Take out the data and send it. Lock it here to avoid inserting data
        if (isOk && images.count() > 0) {
            QMutexLocker locker(&mutexImage);
            QImage image = images.takeFirst();
            QString imageData = DeviceFun::getImageData(image);
            emit readyWrite(imageData);
        }

        //Take a little rest, or the CPU will be occupied all the time
        msleep(1);
    }

    stopped = false;
}

//Server thread parsing picture
void TcpImageSocket::run()
{
    while(!stopped) {
        //In fact, it can be done by timer. After all, the write of tcp is asynchronous and the operating system automatically schedules
        //For later expansibility, for example, it needs to judge whether the transmission is successful or not, and it needs to be processed synchronously, so the changed thread will process it
        //base64 encoding data to image data also takes time, the main time is transcoding
        //Take out the data and send it. Lock it here to avoid inserting data
        if (imageFlags.count() > 0) {
            QMutexLocker locker(&mutexImage);
            QString imageFlag = imageFlags.takeFirst();
            QString imageData = imageDatas.takeFirst();
            QImage image = DeviceFun::getImage(imageData);
            emit receiveImage(imageFlag, image);
        }

        //Take a little rest, or the CPU will be occupied all the time
        msleep(1);
    }

    stopped = false;
}

//Client class usage
TcpImageClient *client = new TcpImageClient(this);
UdpImageClient *client = new UdpImageClient(this);
connect(widget, SIGNAL(receiveImage(QImage)), client, SLOT(append(QImage)));                
client->setServerIP("127.0.0.1");
client->setServerPort(6000);
client->start();

//How to use server class
TcpImageServer *tcpServer = new TcpImageServer(this);
UdpImageServer *tcpServer = new UdpImageServer(this);
connect(tcpServer, SIGNAL(receiveImage(QString, QImage)), this, SLOT(receiveImage(QString, QImage)));