Recently learning the basic knowledge of python, I find it interesting to go to the network programming section. Write a blog to record, first you can know the basic knowledge of computer network, just refer to relevant books, this article only makes a simple introduction!
1.OSI seven-layer protocol, TCP, UDP protocol
First of all, the computers we use today are all connected to the Internet. There is very little single machine going around the world. Then how does our computer communicate through the network? Let's first learn some basic knowledge about the network. Then we can start to learn some about network programming. The first term to be interpreted is protocol. Do we only know what protocol is?Looking back on the various communication rules will allow you to see the diversity.
Mandarin: A network protocol is a set of agreements that both sides of a communication computer must follow together.How to establish connections, how to identify each other, and so on.Computers can communicate with each other only by following this Convention
Putonghua: The agreement is made between the two computers. Is the format of the data I send? When you receive the data, use the same format to get it
Lao Zi: You are communicating with a Korean. You speak Chinese and he speaks Korean. You two can't understand what the other side is saying. What should you do? You both agreed to speak English. To communicate, this is called an agreement.
Network Protocol: A unified system of conventions used to communicate messages between the Internet
In today's Internet data transmission, the OSI seven-layer protocol is commonly used. It is also referred to as the five-layer, four-layer protocol. However, the definitions of different network layers are different.
The internal origin and function are the same.
Here is a brief description of the basic functions of the OSI Layer 7 protocol
First, the physical layer: it is the data transmission medium for network communication, consisting of cables and devices that connect different nodes.The main functions are to provide physical connection for data link layer using transmission media, handle data transmission and monitor data error rate for transparent data flow transmission
Data Link Layer: This layer is responsible for assembling the MAC addresses of its own and its counterpart hosts. MAC addresses: unique encoding for each network device. Unique globally. Burned directly on the network card by different vendors.
Role: In a large network system, who is sending the data to and from whom. This is equivalent to an envelope you write a letter to. It must have the addresses of the recipients clearly written on it.
Network layer: In fact, with a MAC address, our computer can start to communicate. However, at this time, the communication mode is broadcasting. This is equivalent to the basic roar of communication. You send a data out. It will automatically be sent to all computers under the current network. Then each computer's network card will see that the data is sent to itself. Communication like this, if you calculateThere is no problem with a small amount of data on your computer. However, if all computers worldwide transmit messages in this way. That's not just an issue of efficiency. It's absolutely catastrophic. What can you do? You'll think of a new scheme called the IP protocol. Using the IP protocol, computers in different regions are divided into subnets. Communication within subnets is used.Broadcasting delivers messages. Outside the broadcast, messages are delivered in and out of the air. You can interpret it as a distribution center with a courier company. I'll send you a courier. First check that it's not your own region. Yes, your region looks directly from door to door for OK s. But if it's not my region, you'll find your regional distribution center (Freezer Gateway) through the distribution center (Freezer Gateway).This IP protocol is designed to divide subnets. When transmitting data, you must carry the IP address of the other party. With this IP and subnet mask, you can determine which subnet the data belongs to.
IP address: denoted by decimal at 4 sites. Each person is 255. So the range of IP address is 0.0.0-255.255.255. Why is 255? Answer:
28 bits are represented in 8-bit binary, 32 bits together represent a computer's ip address
Subnet mask: A 4-point decimal system used to divide subnets.
Gateway: An interface for data transmission from ip. within a subnet to a local area network (Distribution Center)
The process of calculating the subnet:
1 ip1: 192.168.123.16 2 ip2: 192.168.123.45 3 Subnet mask: 255.255.255.0 4 Convert all to binary 4 ip1: 11000000 10101000 01111011 00010000 5 ip2: 11000000 10101000 01111011 00101101 6 subnets: 11111111 11111 11111 11111 00000000 7 Enter ip1 and ip2 into AND operation separately and subnets 8 IP1 &subnet: 11000000 10101000 01111011 00000000 9 IP2 &subnet: 11000000 10101000 01111011 00000000 10 equals. OK these two IP s are the same subnet
Transport Layer: We are now solving external data transfer problems. Using MAC addresses and IP addresses, we can only locate one computer. Then there is another problem that remains unsolved. We know that it is very possible to transport multiple network applications within a computer. For example, if you are driving a LOL, hanging a DNF, chatting a QQ, and watching John. Then your computer network cardReceived a piece of data from a distance. So what exactly is this data for that application? Say Bai, the courier is delivered to your company. The address is OK. But there are so many people in your company. Who is this courier for? What can I do? Internet guys think of a new term called port.
Transport Layer Provisions: Assign each application a unique port number. When data is sent, the specific application that sends the data is determined by the port number. However, depending on the requirements of the application for the network (some are fast and some are reliable), the transport layer is divided into two protocols. One is TCP and the other is UDP. So, weThe most important of the commonly used TCP/IP protocols is actually IP and port. Because of these two, we can actually locate a network application on a computer. We can also send messages to him.
Number of ports on a 32-bit computer:
TCP: 65536
UDP: 65536
Differences between TCP and UDP:
- TCP, which is connection-based, is continuous, reliable and inefficient. It does not seem to be a phone call. It cannot be interrupted during chat.
- UDP, it is not connection-based. It is not continuous, it is not reliable. It is efficient. It is not like sending a letter. One today, one tomorrow. It's what you want.
Application layer: TCP+IP can be located in an application on your computer. But the data format may be different from that of the same transport. It's like courier. Some are large packages. Some are small files. One is to be packed in a large courier bag, and one is to be packed in a small courier bag. To the application layer, we usually re-packed according to different types of applications. For example, HTTP Protocol, SMTP protocol, FTP protocol, etc.
2. First-time Socket-TCP programming
In almost all programming languages, we use socket. socket to translate sockets when we write network programs. We also learned above that data from a network communication needs to be wrapped with information such as mac, ip, port, etc. But if we need programmers to prepare data one by one each time we develop it, the job is absolutely hopeless. So computers.Sockets are proposed to help us accomplish most of the operations in network communication. We just need to tell sockets which computer (ip, port) I want to send data to. All the rest is done by sockets. So it's very convenient to use sockets for data transfer.
Say nothing but practice~~
server side:
import socket
#Create socket channel
sk = socket.socket()
sk.bind(("127.0.0.1", 8088)) # Bind ip and port
sk.listen() # Start listening
print("Server ready to connect")
conn, address = sk.accept() # Program blocked, waiting for connection
print("With client connections, ip Yes:", address) # address is the client's ip and port
while 1: #Ability to continuously deliver messages to clients
conn.send(input(">>>").encode("utf-8")) # Send content can only be bytes
print(conn.recv(1024).decode("utf-8")) #Show received messagesclient side:
import socket
sk = socket.socket() # Create Channel
print("Client Initialization Completed")
sk.connect(("127.0.0.1", 8088)) # Set up a connection
print("Client Connection Successful")
while 1: #Continuous message sending to server
print(sk.recv(1024).decode("utf-8")) # Accept up to 1024 bytes of content
sk.send(input(">>>").encode("utf-8")) #Show received messages3. First-time Socket-UDP programming
server side:
import socket
sk = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
sk.bind(("127.0.0.1", 8089))
msg, address = sk.recvfrom(1024)
print(msg)
sk.sendto(b'hi', address)client side:
import socket
sk = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
sk.sendto(b'hello', ("127.0.0.1", 8089))
msg, address = sk.recvfrom(1024)
print(msg)Similar to TCP programming, except that UDP does not need to establish a connection for a long time, send a package, wait for a reply, or ignore whether the other party receives or confiscates it. There is no strict sense of server and client
4. sticking phenomenon
When using TCP protocol to transfer data, the following problems occur.
client side:
import socket
sk = socket.socket() # Create Channel
print("Client Initialization Completed")
sk.connect(("127.0.0.1", 8088)) # Set up a connection
print("Client Connection Successful")
sk.send("Ha-ha".encode("utf-8"))
sk.send("Ha-ha".encode("utf-8")) #Send twice
print("Send complete")
sk.close()server side:
import socket
# Create socket channel
sk = socket.socket()
sk.bind(("127.0.0.1", 8088)) # Bind ip and port
sk.listen() # Start listening
print("Server ready to connect")
conn, address = sk.accept() # Program blocked, waiting for connection
print("With client connections, ip Yes:", address) # address is the client's ip and port
msg1 = conn.recv(1024)
print(msg1.decode("utf-8"))
msg2 = conn.recv(1024)
print(msg2.decode("utf-8"))
sk.close()What the server side receives:
You can see that the content sent twice, glued together, becomes a package, which is a typical glue phenomenon.
So how do I solve the sticky problem? Simple. The sticky stuff happens because the data has no boundaries. Mix the two packages directly into one package. Then I can specify the boundaries when sending data. Tell the other party how big the next packet is. When receiving data to the other side, read the size of the packet first. Then read the data. This will happen.Sticky stubble.
Putonghua: Format data when sending data: Length + Receive data to know how much of the current packet size is. This is equivalent to defining a delimited boundary.
Show a wave of actions:
client side:
import socket
sk = socket.socket() # Create Channel
print("Client Initialization Completed")
sk.connect(("127.0.0.1", 8088)) # Set up a connection
print("Client Connection Successful")
s = "Ha-ha"
bs = sk.send(s.encode("utf-8"))
# Calculate data length. Format into four digits
bs_len = format(len(bs), "04d").encode("utf-8")
# Send the length of the data before sending it
sk.send(bs_len)
sk.send(bs)
# Send a second time
sk.send(bs_len)
sk.send(bs)
print("Send complete")
sk.close()server side:
import socket
# Create socket channel
sk = socket.socket()
sk.bind(("127.0.0.1", 8088)) # Bind ip and port
sk.listen() # Start listening
print("Server ready to connect")
conn, address = sk.accept() # Program blocked, waiting for connection
print("With client connections, ip Yes:", address) # address is the client's ip and port
# Receive data 4 bytes long, convert to number
bs_len = int(conn.recv(4).decode("utf-8"))
# Read data
msg1 = conn.recv(1024)
print(msg1.decode("utf-8"))
# Receive data length, read data
bs_len = int(conn.recv(4).decode("utf-8"))
msg2 = conn.recv(1024)
print(msg2.decode("utf-8"))
sk.close()However, you should find that although this method of transmission can solve the sticky problem, the length of each receive should be defined first. Is it too tired?python also has his own attitude on this issue. The code is determined to make it simple for you. It introduces a new module struct, which is the key point. Take a small book and write it down for reference!!
Show me the usage of struct:
import struct
ret = struct.pack("i", 123456789) # Package numbers into bytes
print(ret)
print(len(ret)) # 4 Four bytes dead, no matter the size of the number
# Restore bytes back to numbers
ds = b'\x15\xcd[\x07'
num = struct.unpack("i", ds)[0] # num returns a meta-ancestor, the zero-indexed element, which is the first element, the number we transfer
print(num) # 123456789Okay, let's go back to the sticky stuff and talk about how to solve it gracefully:
client side:
import socket
import struct
sk = socket.socket() # Create Channel
print("Client Initialization Completed")
sk.connect(("127.0.0.1", 8088)) # Set up a connection
print("Client Connection Successful")
msg_bs = "How do you do".encode("utf-8")
msg_struct_len = struct.pack("i", len(msg_bs))
sk.send(msg_struct_len)
sk.send(msg_bs)
print("Send complete")
# Send a second time
sk.send(msg_struct_len)
sk.send(msg_bs)
print("Send complete")
sk.close()server side:
import socket
import struct
# Create socket channel
sk = socket.socket()
sk.bind(("127.0.0.1", 8088)) # Bind ip and port
sk.listen() # Start listening
print("Server ready to connect")
conn, address = sk.accept() # Program blocked, waiting for connection
print("With client connections, ip Yes:", address) # address is the client's ip and port
msg_struct_len = conn.recv(4)
msg_len = struct.unpack("i", msg_struct_len)[0]
data = conn.recv(msg_len)
print(data.decode("utf-8"))
print("Received")
# Receive the second packet
msg_struct_len = conn.recv(4)
msg_len = struct.unpack("i", msg_struct_len)[0]
data = conn.recv(msg_len)
print(data.decode("utf-8"))
print("Received")
sk.close()However, this is a bit cumbersome. If you use it many times, you can also encapsulate the struct module you use and import this custom module when you need it:
Encapsulated module my_struct_util.py:
import struct
def my_send(sk, msg):
msg_len = msg.encode("utf-8")
msg_struct_len = struct.pack("i", len(msg_len))
sk.send(msg_struct_len)
sk.send(msg_len)
def my_recv(sk):
msg_struct_len = sk.recv(4)
msg_len = struct.unpack("i", msg_struct_len)[0]
data = sk.recv(msg_len)
print(data.decode("utf-8"))client side:
import socket
import my_socket_util as msu
sk = socket.socket() # Create Channel
print("Client Initialization Completed")
sk.connect(("127.0.0.1", 8088)) # Set up a connection
print("Client Connection Successful")
msu.my_send(sk, "How are you")
msu.my_send(sk, "How are you")
sk.close()server side:
import socket
import my_socket_util as msu
# Create socket channel
sk = socket.socket()
sk.bind(("127.0.0.1", 8088)) # Bind ip and port
sk.listen() # Start listening
print("Server ready to connect")
conn, address = sk.accept() # Program blocked, waiting for connection
print("With client connections, ip Yes:", address) # address is the client's ip and port
msu.my_recv(conn)
msu.my_recv(conn)
sk.close()In this way, is it much easier ~~