Process IoT upload data

LwIP

LwIP, the Light Weight IP protocol, can be run without operating system or RTOS. The resource occupation is only tens of KB RAM and 40KB ROM. it is suitable for running in embedded devices. Now general Internet of things devices are using LwIP protocol. Its basic functions are as follows:

• Support multi network port IP forwarding
• Support ICMP and DHCP
• Support extended UDP
• TCP supporting blocking control, RTT estimation and fast forwarding
• Provide callback interface (RAW API)
• Support for optional Berkeley interface API

The implementation of LwIP is very complex, but at present, the code is open source and can be easily transplanted in various devices. Many device manufacturers also provide LwIP templates and libraries. Therefore, this paper mainly introduces the driver writing and application of LwIP

An LwIP device usually needs the following hierarchy support:

• physical layer

  It includes PHY chip, interface and basic circuit to realize network connection

• data link layer

  • MAC layer: Ethernet controller of hardware equipment
  • Driver layer: Ethernet device driver

• Network IP layer and transport layer

  It is mainly composed of LwIP core code

  LwIP will build its own TCP/IP protocol API based on hardware API and implement general application API

• application layer

  Web application

Basic API of LwIP

LwIP provides three API s

• RAW API: an API that can be used without operating system. This interface puts the protocol stack and application program in the same process and uses state machine and callback function for control. The application program using this interface does not need continuous operation

  SoC like ESP32 and STM32 can realize network communication based on RAW API by using built-in / external network card driver (WiFi baseband)

• NETCONN API: it requires the support of the operating system. It puts the receiving and processing on the same thread. The efficiency is slightly lower than that of RAW API, but the software compatibility is better

• BSD API: a standard AP based on UNIX POSIX and the idea of "everything is a file", which is convenient for application transplantation, but it is less used because it is inefficient and occupies more resources in embedded systems. Now RT thread can provide software migration package compatible with the API, but it needs to run on medium and high-performance embedded devices such as STM32H750 to run smoothly

RAW API

The API is characterized by high efficiency but high programming difficulty. Developers need to be familiar with the function callback mechanism

Its basic unit is PCB (Protocol Control Block, note not Print Circuit Board!), which is similar to socket in function

According to the transmission protocol, PCB is divided into TCP PCB and UDP PCB

Its corresponding API is as follows

//Create a new TCP PCB
struct tcp_pcb* tcp_new(void)
{
    return tcp_alloc(TCP_PRIO_NORMAL);
}

//Bind PCB to local port number and IP address
err_t tcp_bind(struct tcp_pcb * pcb, struct ip_addr * ipaddr, u16_t port);
//Use IP_ADDR_ANY can bind the PCB to any local address    

//Monitor PCB
#define tcp_listen(pcb) tcp_listen_with_backlog(pcb,TCP_DEFAULT_LISTEN_BACKLOG)
struct tcp_pcb * tcp_listen_with_backlog(struct tcp_pcb *pcb, u8_t backlog);
//This function will return a new TCP PCB that is already listening, and the original PCB will be released
//So it must be like tcpb=tcp_listen(tcpb); Use this function this way

//Connect server
err_t tcp_connect(struct tcp_pcb * pcb, struct ip_addr * ipaddr, u16_t port,
                 err_t (* connected)(void* arg, struct tcp_pcb * tpcb, err_t err));
/*
struct tcp_pcb * pcb Indicates the TCP PCB to be set
struct ip_addr * ipaddr Represents the IP address of the server. You can set the IP address using the following methods:
	1. Define a struct ip_addr * ipaddr structure
	2. Use IP4_ Addr (& IPADDR, a, b, c, d) sets the IP, where a, b, c, D are the middle digits of the IPv4 address
	For example, 192.168.1.2 corresponds to a=192,b=168,c=1,d=2
u16_t port Indicates the server port number
err_t (* connected)(void* arg, struct tcp_pcb * tpcb, err_t err)Indicates the callback function when the connection is successful. The developer should use the state machine to control the connection state
*/

//Callback function when connection request is set
void tcp_accept(struct tcp_pcb * pcb,
               err_t (* accept)(void* arg, struct tcp_pcb * newpcb, err_t err))
{
    pcb->accept = accept;
}
/* accept represents the callback function. The callback function should be like the following */
static err_t tcp_server_accept_handler(void* arg, struct tcp_pcb * newpcb, err_t err)
{
    //Setting the callback function priority is very important when there are multiple connections, and the function must be called
    tcp_setprio(pcb, TCP_PRIO_MIN);
    //Set the TCP data receiving callback function. When there is network data, tcp_server_recv will be called
    tcp_recv(pcb,tcp_server_recv);
    err = ERR_OK; //Here is the execution result of the function. The if statement is generally used to judge whether the above function is successfully executed
    return err;
}
//Where tcp_recv is used to set the TCP data receiving callback function
void tcp_recv(struct tcp_pcb * pcb,
             err_t (* recv)(void * arg, struct tcp_pcb * tpcb, struct pbuf * p, err_t err))
{
    pcb->recv = recv;
}
//Callback functions are mostly used to cache network data. Typical TCP data receiving callback functions are as follows
static err_t tcp_server_recv(void * arg, struct tcp_pcb * tpcb, struct pbuf * p, err_t err)
{
    //Define a pbuf pointer to the incoming parameter p, which is used to receive network data and cache it locally
    struct pbuf * p_temp = p;
    
    //If the data is not empty
    if(p_temp != NULL)
    {
        tcp_recved(pcb, p_temp->tot_len);//Read data
        //If the data is not empty, traverse the data by traversing the linked list (in fact, the bottom layer is a linked list)
        while(p_temp != NULL)
        {
            //Resend the received data to the client
            tcp_write(pcb, p_temp->payload, p_temp->len, TCP_WRITE_FLAG_COPY);
            //Start sending
            tcp_output(pcb);
            //Get next packet
            p_temp = p_temp->next;
		}
	}
    else //If the data is empty, the reception fails
    {
        tcl_close(pcb); //Close connection
    }
    pbuf_free(p); //Free memory
    err = ERR_OK;
    return err;
}

The programming idea of RAW API is very similar to Socket, but it uses callback function instead of sequential structure to control data flow

tcp_new,tcp_bind,tcp_ The use method of listen is basically consistent with the corresponding function of socket; But tcp_accept needs to set a corresponding tcp_server_accept_handler callback function, and then through tcp_recv setting TCP_ server_ recv_ The handler callback function can complete the functions of accept and receive in the socket. This is because LwIP puts all tasks in the same process, and no operating system transmits data in the to be received and received FIFO through event set, semaphore and other methods.

NETCONN API

In this interface, netconn is used as the connection structure without distinguishing between TCP and UDP, which is convenient for applications to use a unified connection structure and programming functions

Referring to the code, we can see that netconn connection structure is actually the upper packaging of PCB, abstracting the lower PCB structure into an "object"

struct netconn{
    /* netconn Type, TCP, UDP, RAW can be used */
    enum netconn_type type;
    /* netconn current state */
    enum netconn_state state;
    /* LwIP The internal protocol control block is essentially encapsulated on the basis of PCB */
    union {
        struct ip_pcb * ip;
        struct tcp_pcb * tcp;
        struct udp_pcb * udp;
        struct raw_pcb * raw;
    } pcb;
    /* netconn Last error */
    err_t last_err;
    
#if !LWIP_NETCONN_SEM_PER_THREAD
	/* Functions for synchronous execution in the kernel context */
    sys_sem_t op_completed;
#endif
    
    /* mbox:Receive the mbox of the packet until they are fetched by the netconn application thread (which can become very large) */
    sys_mbox_t recvmbox;
    
#if LWIP_TCP
    sys_mbox_t acceptmbox;
#endif /* LWIP_TCP */
    
/* Encapsulation for socket layer only, not normally used */
#if LWIP_SOCKET
    int socket;
#endif /* LWIP_SOCKET */
    
#if LWIP_SO_SNDTIMEO
    /* Timeout waiting to send data, in milliseconds (send data as an internal buffer) */
    s32_t send_timeout;
#endif /* LWIP_SO_SNDTIMEO */
#if LWIP_SO_RCVTIMEO
    /* Timeout waiting to receive new data, in milliseconds (or connect to listen on netconns) */
    int recv_timeout;
#endif /* LWIP_SO_RCVTIMEO */
    
#if LWIP_SO_RCVBUF
    /* recvmbox The maximum number of bytes queued in. If it is not used for TCP, it should be adjusted for TCP_WND */
    int recv_bufsize;
    /* The current number of bytes to be received in the recvmbox, for recv_bufsize test to limit bytes on recvmbox; For UDP and RAW, for FIONREAD */
    int recv_avail;
#endif /* LWIP_SO_RCVBUF */

#if LWIP_S_LINGER
	/* A value less than 0 disables the delay, and a value greater than 0 indicates the number of seconds to delay */
    s16_t linger;
#endif /* LWIP_S_LINGER */
    
    u8_t flags;//More flags for netconn internal status
    
#if LWIP_TCP
    /* TCP:When passed to netconn_ When the write data is not suitable for the sending buffer, the sent quantity is temporarily stored */
    size_t write_offset;
    /* TCP:When passed to netconn_ When the write data is not suitable for sending the buffer, the message is temporarily stored and used during connection and shutdown */
    struct api_msg * current_msg;
#endif /* LWIP_TCP */
    netconn_callback callback;//Callback function that notifies this netconn event
}

The API s available to users are as follows:

1. netconn_new

   #define netconn_new(t) netconn_new_with_proto_and_callback(t, 0, NULL)
   
   struct netconn * netconn_new_with_proto_and_callback(enum netconn_type t,
                                                        u8_t proto,
                                                        netconn_callback callback);
   

   Where netconn_type is the connection type created. TCP or UDP is usually used. Its value can be any one of the enumerations

   NETCONN_INVALID = 0
   NETCONN_TCP = 0x10
   NETCONN_UDP = 0x20
   NETCONN_UDPLITE = 0x21
   NETCONN_UDPNOCHKSUM = 0x22
   NETCONN_RAW = 0x40
   

   proto represents the IP of the original RAW IP pcb, usually written as 0

   netconn_cakkback is a callback function when the setting state changes. Usually, you can fill in NULL directly

   This API returns a netconn structure

2. netconn_delete

   err_t netconn_delete(struct netconn * conn)
   

   This function deletes the netconn structure and frees up memory

   Note: when the client is disconnected, the user must call this function and release netconn resources, otherwise memory leakage may occur

3. netconn_bind

   It is used to bind the IP address and port number of netconn structure. The use method is similar to that of socket bind

   err_t netconn_bind(struct netconn * conn, ip_addr_t * addr, u16_t port)
   

4. netconn_listen

   #define netconn_listen(conn) netconn_listen_with_backlog(conn,TCP_DEFAULT_LISTEN_BACKLOG)
   
   err_t netconn_listen_with_backlog(struct netconn * conn, u8_t backlog)
   

   backlog indicates the maximum limit of the connection queue

   This function is used to listen for client connections

5. netconn_connect

   This function is used by the client to initiate a connection to the server

   err_t netconn_connect(struct netconn * conn, const ip_addr_t * adr, u16_t port)
   

   const ip_addr_t * adr indicates the IP address of the server, using IP4_ The addr function sets the IP. This function is described in the RAW API above and will not be repeated

6. netconn_accept

   err_t netconn_accept(struct netconn * conn, struct netconn ** new_conn)
   

   This function assigns a new netconn structure pointer to new_conn parameter, the new structure pointer shall be used for subsequent client data sending and receiving

   The server uses this function to accept new client connection requests

7. netconn_recv

   Used to receive data from the network

   err_t netconn_recv(struct netconn * conn, struct netbuf ** new_buf)
   

   Where new_buf is a pointer to the network buffer used to receive data. This pointer points to the struct netbuf structure

   When using, you should first initialize a pointer and put it in new_ The position of the buf parameter. After using this function, all received network data will be placed in this pointer. As long as this pointer (commonly referred to as handle) is used with the relevant function, data reception can be realized

8. netbuf_data

   Use the following function to get data of the specified length from the netbuf structure

   err_t netbuf_data(struct netbuf ** buf, void ** dataptr, u16_t * len)
   

   buf indicates the handle to the netbuf structure that specifies the data to be obtained

   dataptr represents the cache stored after obtaining data

   len indicates the length of data to be obtained

9. netconn_write

   #define netconn_write(conn, dataptr, size, apiflags) \
   		netconn_write_partly(conn, dataptr, size, apiflags, NULL)
   
   err_t netconn_write_partly(struct netconn * conn, const void * dataptr, size_t size, u8_t apiflags, size_t * bytes_written)
   

   Use this function to send data to the network

   The following parameters can be selected for apiflags:

   • NETCNN_COPY: the data will be copied to the memory belonging to the stack
   • NETCONN_MORE: for TCP connections, the PSH flag will be set on the last data segment sent
   • NETCONN_DONTBLOCK: data is written only when all data can be written at once

   Dataptr indicates the data buffer to be sent, and size indicates the data length. You can use sizeof(dataptr)

   bytes_writing indicates the pointer to the position where the number of bytes to be written is received, which is usually set to NULL

10. netconn_close

    This function is used to close the connection and will not be repeated

    err_t netconn_close(struct netconn * conn)
    

BSD API

LwIP provides a set of UNIX standard APIs based on open read / write close model, and implements socket, bind, recv, send and other APIs. However, BSD API interfaces need to occupy too many resources and are rarely used in embedded devices

The basic usage is the same as the Socket standard API, so I won't repeat it