1. HTTP Protocol
Hypertext is the abbreviation of hypertext, which refers to the transcendence of text restrictions or hyperlinks, such as pictures, music, videos, hyperlinks and so on. Transferring data in HTTP protocol format is based on TCP transport protocol, and a connection needs to be established before sending data.
Role:
It specifies the format of data that browsers and Web servers communicate with, that is to say, browsers and web servers communicate using http protocol.
Summary:
* HTTP Protocol is a Hypertext Transfer Protocol
* HTTP protocol is a data transmission protocol based on TCP
* HTTP protocol specifies the format of communication data between browser and Web server
The communication process of browser accessing web server is as follows:
URLs (Uniform Resoure Locator)
URL s: Unified Resource Locator, commonly understood as the address of network resources, that is, what we often call the address of network resources.
The look of the URL: https://news.163.com/18/1122/10/E178J2O4000189FH.html
Components of the URL:
Protocol part: https://, http://, ftp://.//
Domain Name: news.163.com
Resource Path: 18/1122/10/E178J2O4000189FH.html
Domain name: Domain name is an alias of IP address. It is a name divided by dots and composed of letters and numbers. The purpose of using domain name is to remember the IP address of a host conveniently.
Extension of URL: https://news.163.com/hello.html? Page = 1 & count = 10
Query parameter section (optional): page = 1 & count = 10
Parametric Description: The following page represents the first parameter, and the latter parameters are connected using &
3. Using Google to View the Communication Process of HTTP Protocol
Uses of Google Browser Developer Tools: Windows and Linux platforms call out developer tools by F12, mac OS Selection View - > Developer - > Developer Tools or use alt+command+i shortcut directly, and a common multi-platform operation is to right-click on the page to check.
Label options for developer tools:
Elements: Used to view or modify HTML Tags
Console: Execute js code
Sources: View static resource files and debug JS code at breakpoints
Network: View the communication process of http protocol
Instructions for the use of developer tools:
Click on the Network tab option
Enter Baidu's address in the browser's address bar, and you can see the communication process of http requesting Baidu's home page.
Each record here is a request + response process
Summary:
The developer's tool of Google Browser is a sharp tool for viewing the communication process of http protocol. It can view the communication process of every request and response through the Network label option. The common method to call out the developer's tool is to right-click on the page to select and check.
The Headers option for developer tools consists of three parts:
General: Main Information
Response Headers: Response Headers
Request Headers: Request Headers
Response option is to view response volume information
IV. HTTP Request Message
Request message in two ways:
GET Request Message: Getting web Server Data
POST-style request message: submitting data to the web server
2. HTTP GET Request Message Analysis
GET Request Message Description:
1 ---- Request line ---- 2 GET / HTTP/1.1 # GET Request Mode Request Resource Path HTTP Protocol version 3 ---- Request header ----- 4 Host: www.itcast.cn # Host address and port number of server,The default is 80. 5 Connection: keep-alive # Keep a long connection with the server 6 Upgrade-Insecure-Requests: 1 # Let browsers upgrade unsafe requests and use https request 7 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 # User Agent, which is the name of the client 8 Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 # Acceptable data types 9 Accept-Encoding: gzip, deflate # Acceptable compression format 10 Accept-Language: zh-CN,zh;q=0.9 #Acceptable language 11 Cookie: pgv_pvi=1246921728; # The identity of the logged-in user; cookie It can record some information of browser, put advertisements, display history browsing records. 12 ---- Blank line ----
GET requests original message description:
1 GET / HTTP/1.1\r\n 2 Host: www.itcast.cn\r\n 3 Connection: keep-alive\r\n 4 Upgrade-Insecure-Requests: 1\r\n 5 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36\r\n 6 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8\r\n 7 Accept-Encoding: gzip, deflate\r\n 8 Accept-Language: zh-CN,zh;q=0.9\r\n 9 Cookie: pgv_pvi=1246921728; \r\n 10 \r\n (There's a separate request header behind it.'\r\n'Can not omit) 11
Note: Use between each data: rn
3.HTTP POST Request Message Analysis
POST Request Message Description:
1 ---- Request line ---- 2 POST /xmweb?host=mail.itcast.cn&_t=1542884567319 HTTP/1.1 # POST Request Mode Request Resource Path HTTP Protocol version 3 ---- Request header ---- 4 Host: mail.itcast.cn # Host address and port number of server,The default is 80. 5 Connection: keep-alive # Keep a long connection with the server 6 Content-Type: application/x-www-form-urlencoded # Tell the server the type of data requested 7 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 # The name of the client 8 ---- Blank line ---- 9 ---- Request body ---- 10 username=hello&pass=hello # Request parameters
POST Request Original Message Description:
1 POST /xmweb?host=mail.itcast.cn&_t=1542884567319 HTTP/1.1\r\n 2 Host: mail.itcast.cn\r\n 3 Connection: keep-alive\r\n 4 Content-Type: application/x-www-form-urlencoded\r\n 5 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36\r\n 6 \r\n(There's a separate request header behind it.'\r\n'Can not omit) 7 username=hello&pass=hello
4. Summary
An HTTP request message can be composed of four parts: the request line, the request header, the blank line and the request body.
The requesting line consists of three parts:
1. Request mode
2. Request resource path
3.HTTP Protocol Version
GET request message has no request body, only request line, request header and blank line.
Request message in POST mode can be composed of four parts: request line, request header, blank line and request body. Note: POST mode can allow no request body, but this format is rare.
5. HTTP response message analysis
1. Message description:
An HTTP response message consists of four parts: response line, response header, blank line and response body.
The response line consists of three parts: the HTTP version status code status description, and the most common status code is 200.
Response message description:
1 --- Response line/Status line --- 2 HTTP/1.1 200 OK # HTTP State description of protocol version status code 3 --- Response head --- 4 Server: Tengine # Server name 5 Content-Type: text/html; charset=UTF-8 # Content type 6 Transfer-Encoding: chunked # The length of content sent to the client is uncertain, and the end of sending is marked 0.\r\n, Content-Length Represents that the server determines the size of the content sent to the client, but only one of them can be used. 7 Connection: keep-alive # Keep a long connection with the client 8 Date: Fri, 23 Nov 2018 02:01:05 GMT # Response time of server 9 --- Blank line --- 10 --- Response body --- 11 <!DOCTYPE html><html lang="en"> …</html> # Data responded to the client
The original response message description:
1 HTTP/1.1 200 OK\r\n 2 Server: Tengine\r\n 3 Content-Type: text/html; charset=UTF-8\r\n 4 Transfer-Encoding: chunked\r\n 5 Connection: keep-alive\r\n 6 Date: Fri, 23 Nov 2018 02:01:05 GMT\r\n 7 \r\n(There is a separate response header message behind it.'\r\n'Can not omit) 8 <!DOCTYPE html><html lang="en"> …</html>
2.HTTP status code: 3-digit digital code used to represent the response status of the web server.
6. Build Python Static Web Server
Definition: A program that provides static documentation to the browser that makes the request. Usually when we browse Baidu news data, the daily news data will change. The page we visited is dynamic, while the data we developed is static, and the page data will not change.
Develop your own static Web server -- return fixed page data
Implementation steps:
Write a TCP Server Program
Get http request message data sent by browser
Read the fixed page data, assemble the page data into HTTP response message data and send it to the browser.
When HTTP response message data is sent, the socket serving the client is closed.
Return fixed page data instance code
1 import socket 2 3 if __name__ == '__main__': 4 # Establish tcp serversocket 5 tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 6 # Setting Port Number Multiplexing, Program Exit Port Released Immediately 7 tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) 8 # Binding port number 9 tcp_server_socket.bind(("", 9000)) 10 # Set up monitoring 11 tcp_server_socket.listen(128) 12 while True: 13 # Waiting to Accept Connection Requests from Clients 14 new_socket, ip_port = tcp_server_socket.accept() 15 # This is where the code executes, indicating that the connection was successfully established 16 recv_client_data = new_socket.recv(4096) 17 # Decoding binary data 18 recv_client_content = recv_client_data.decode("utf-8") 19 print(recv_client_content) 20 with open("static/index.html", "rb") as file: 21 # Read file data 22 file_data = file.read() 23 24 # Response line 25 response_line = "HTTP/1.1 200 OK\r\n" 26 # Response head 27 response_header = "Server: PWS1.0\r\n" 28 # Response body 29 response_body = file_data 30 # Stitching response message 31 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 32 # send data 33 new_socket.send(response_data) 34 # Close the socket between the service and the client 35 new_socket.close()
2. Return specified page data
Implementation steps:
Getting the Path of User Request Resources
Read the data of the specified file according to the path of the requested resource
Assemble the response message of the specified file data and send it to the browser
Determine that the requested file does not exist on the server, assemble the 404 status response message and send it to the browser
1 import socket 2 3 4 def main(): 5 # Establish tcp serversocket 6 tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 7 # Setting Port Number Multiplexing, Program Exit Port Released Immediately 8 tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) 9 # Binding port number 10 tcp_server_socket.bind(("", 9000)) 11 # Set up monitoring 12 tcp_server_socket.listen(128) 13 while True: 14 # Waiting to Accept Connection Requests from Clients 15 new_socket, ip_port = tcp_server_socket.accept() 16 # This is where the code executes, indicating that the connection was successfully established 17 recv_client_data = new_socket.recv(4096) 18 if len(recv_client_data) == 0: 19 print("Close the browser") 20 new_socket.close() 21 return 22 # Decoding binary data 23 recv_client_content = recv_client_data.decode("utf-8") 24 print(recv_client_content) 25 # Divide according to the specified string and take the request line 26 request_list = recv_client_content.split('\n')[0].split() 27 # Get the request resource path and cut the request line 28 request_path = request_list[1] 29 print(request_path) 30 # Determine whether the request is the root directory (which is relatively running at the root directory)?.py Document), if the condition holds, specify the home page data return 31 if request_path == "/": 32 request_path = "/index.html" 33 try: 34 # Open the specified file dynamically 35 with open("static" + request_path, "rb") as file: 36 # Read file data 37 file_data = file.read() 38 except Exception as e: 39 # Request resource does not exist, return 404 data 40 # Response line 41 response_line = "HTTP/1.1 404 Not Found\r\n" 42 # Response head 43 response_header = "Server: PWS1.0\r\n" 44 with open("static/error.html", "rb") as file: 45 file_data = file.read() 46 # Response body 47 response_body = file_data 48 # Stitching response message 49 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 50 # send data 51 new_socket.send(response_data) 52 else: 53 # Response line 54 response_line = "HTTP/1.1 200 OK\r\n" 55 # Response head 56 response_header = "Server: PWS1.0\r\n" 57 # Response body 58 response_body = file_data 59 # Stitching response message 60 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 61 # send data 62 new_socket.send(response_data) 63 finally: 64 # Close the socket between the service and the client 65 new_socket.close() 66 67 if __name__ == '__main__': 68 main() 69
3. Multitask Edition
Implementation steps:
When the connection between client and server is successful, create sub-threads, and use sub-threads to process client requests to prevent blocking of the main thread.
Set the created sub-threads to the daemon main threads to prevent the main threads from being able to exit.
1 import socket 2 import threading 3 4 # Processing client requests 5 def handle_client_request(new_socket): 6 # This is where the code executes, indicating that the connection was successfully established 7 recv_client_data = new_socket.recv(4096) 8 if len(recv_client_data) == 0: 9 print("Close the browser") 10 new_socket.close() 11 return 12 13 # Decoding binary data 14 recv_client_content = recv_client_data.decode("utf-8") 15 print(recv_client_content) 16 # Segmentation based on specified strings 17 request_list = recv_client_content.split("\n")[0].split() 18 19 # Get the request resource path 20 request_path = request_list[1] 21 print(request_path) 22 23 # Determine whether the request is the root directory, and if the condition holds, specify the home page data return? 24 if request_path == "/": 25 request_path = "/index.html" 26 27 try: 28 # Open the specified file dynamically 29 with open("static" + request_path, "rb") as file: 30 # Read file data 31 file_data = file.read() 32 except Exception as e: 33 # Request resource does not exist, return 404 data 34 # Response line 35 response_line = "HTTP/1.1 404 Not Found\r\n" 36 # Response head 37 response_header = "Server: PWS1.0\r\n" 38 with open("static/error.html", "rb") as file: 39 file_data = file.read() 40 # Response body 41 response_body = file_data 42 43 # Stitching response message 44 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 45 # send data 46 new_socket.send(response_data) 47 else: 48 # Response line 49 response_line = "HTTP/1.1 200 OK\r\n" 50 # Response head 51 response_header = "Server: PWS1.0\r\n" 52 53 # Response body 54 response_body = file_data 55 56 # Stitching response message 57 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 58 # send data 59 new_socket.send(response_data) 60 finally: 61 # Close the socket between the service and the client 62 new_socket.close() 63 64 65 # Program Entry Function 66 def main(): 67 # Establish tcp serversocket 68 tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 69 # Setting Port Number Multiplexing, Program Exit Port Released Immediately 70 tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) 71 # Binding port number 72 tcp_server_socket.bind(("", 9000)) 73 # Set up monitoring 74 tcp_server_socket.listen(128) 75 76 while True: 77 # Waiting to Accept Connection Requests from Clients 78 new_socket, ip_port = tcp_server_socket.accept() 79 print(ip_port) 80 # When client and server establish connection, create sub-threads 81 sub_thread = threading.Thread(target=handle_client_request, args=(new_socket,)) 82 # Setting up daemon main thread 83 sub_thread.setDaemon(True) 84 # Starter threads perform corresponding tasks 85 sub_thread.start() 86 87 88 if __name__ == '__main__': 89 main()
4. Object-Oriented Edition
Implementation steps:
Abstract the Web server that provides the service into a class (HTTP Web Server)
Provide initialization methods for Web servers to create socket objects in initialization methods
Provides a way to turn on the Web server and let the Web server handle client requests.
1 import socket 2 import threading 3 4 # Definition web Server class 5 class HttpWebServer(object): 6 def __init__(self): 7 # Establish tcp serversocket 8 tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 9 # Setting Port Number Multiplexing, Program Exit Port Released Immediately 10 tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) 11 # Binding port number 12 tcp_server_socket.bind(("", 9100)) 13 # Set up monitoring 14 tcp_server_socket.listen(128) 15 # Save the successfully created server socket 16 self.tcp_server_socket = tcp_server_socket 17 18 19 # Processing client requests 20 @staticmethod 21 def handle_client_request(new_socket): 22 # This is where the code executes, indicating that the connection was successfully established 23 recv_client_data = new_socket.recv(4096) 24 if len(recv_client_data) == 0: 25 print("Close the browser") 26 new_socket.close() 27 return 28 29 30 # Decoding binary data 31 recv_client_content = recv_client_data.decode("utf-8") 32 print(recv_client_content) 33 # Segmentation based on specified strings 34 request_list = recv_client_content.split("\n")[0].split() 35 36 37 # Get the request resource path 38 request_path = request_list[1] 39 print(request_path) 40 41 42 # Determine whether the request is the root directory, and if the condition holds, specify the home page data return? 43 if request_path == "/": 44 request_path = "/index.html" 45 46 47 try: 48 # Open the specified file dynamically 49 with open("python" + request_path, "rb") as file: 50 # Read file data 51 file_data = file.read() 52 except Exception as e: 53 # Request resource does not exist, return 404 data 54 # Response line 55 response_line = "HTTP/1.1 404 Not Found\r\n" 56 # Response head 57 response_header = "Server: PWS1.0\r\n" 58 with open("python/error.html", "rb") as file: 59 file_data = file.read() 60 # Response body 61 response_body = file_data 62 63 64 # Stitching response message 65 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 66 # send data 67 new_socket.send(response_data) 68 else: 69 # Response line 70 response_line = "HTTP/1.1 200 OK\r\n" 71 # Response head 72 response_header = "Server: PWS1.0\r\n" 73 74 75 # Response body 76 response_body = file_data 77 78 79 # Stitching response message 80 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 81 # send data 82 new_socket.send(response_data) 83 finally: 84 # Close the socket between the service and the client 85 new_socket.close() 86 87 # start-up web The server works 88 def start(self): 89 while True: 90 # Waiting to Accept Connection Requests from Clients 91 new_socket, ip_port = self.tcp_server_socket.accept() 92 # When client and server establish connection, create sub-threads 93 sub_thread = threading.Thread(target=self.handle_client_request, args=(new_socket,)) 94 # Setting up daemon main thread 95 sub_thread.setDaemon(True) 96 # Starter threads perform corresponding tasks 97 sub_thread.start() 98 99 # Program Entry Function 100 def main(): 101 # Establish web Server object 102 web_server = HttpWebServer() 103 # start-up web The server works 104 web_server.start() 105 106 if __name__ == '__main__': 107 main() 108
5. (Terminal) Develop a command line to start a static web server with dynamically bound port numbers
Implementation steps:
Obtain terminal command-line parameters for executing python programs
Determine the type of parameter and set the port number as an integer.
Add a port number parameter to the initialization method of the Web server class to bind the port number
Note: Get a list of terminal command-line parameters sys.argv that execute Python programs as strings [python,'8080']
1 import socket 2 import threading 3 import sys 4 5 6 # Definition web Server class class HttpWebServer(object): 7 def __init__(self, port): 8 # Establish tcp serversocket 9 tcp_server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 10 # Setting Port Number Multiplexing, Program Exit Port Released Immediately 11 tcp_server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) 12 # Binding port number 13 tcp_server_socket.bind(("", port)) 14 # Set up monitoring 15 tcp_server_socket.listen(128) 16 # Save the successfully created server socket 17 self.tcp_server_socket = tcp_server_socket 18 19 # Processing client requests @staticmethod 20 def handle_client_request(new_socket): 21 # This is where the code executes, indicating that the connection was successfully established 22 recv_client_data = new_socket.recv(4096) 23 if len(recv_client_data) == 0: 24 print("Close the browser") 25 new_socket.close() 26 return 27 28 # Decoding binary data 29 recv_client_content = recv_client_data.decode("utf-8") 30 print(recv_client_content) 31 # Segmentation based on specified strings 32 request_list = recv_client_content.split(" ", maxsplit=2) 33 34 # Get the request resource path 35 request_path = request_list[1] 36 print(request_path) 37 38 # Determine whether the request is the root directory, and if the condition holds, specify the home page data return? 39 if request_path == "/": 40 request_path = "/index.html" 41 42 try: 43 # Open the specified file dynamically 44 with open("static" + request_path, "rb") as file: 45 # Read file data 46 file_data = file.read() 47 except Exception as e: 48 # Request resource does not exist, return 404 data 49 # Response line 50 response_line = "HTTP/1.1 404 Not Found\r\n" 51 # Response head 52 response_header = "Server: PWS1.0\r\n" 53 with open("static/error.html", "rb") as file: 54 file_data = file.read() 55 # Response body 56 response_body = file_data 57 58 # Stitching response message 59 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 60 # send data 61 new_socket.send(response_data) 62 else: 63 # Response line 64 response_line = "HTTP/1.1 200 OK\r\n" 65 # Response head 66 response_header = "Server: PWS1.0\r\n" 67 68 # Response body 69 response_body = file_data 70 71 # Stitching response message 72 response_data = (response_line + response_header + "\r\n").encode("utf-8") + response_body 73 # send data 74 new_socket.send(response_data) 75 finally: 76 # Close the socket between the service and the client 77 new_socket.close() 78 79 # start-up web The server works 80 def start(self): 81 while True: 82 # Waiting to Accept Connection Requests from Clients 83 new_socket, ip_port = self.tcp_server_socket.accept() 84 # When client and server establish connection, create sub-threads 85 sub_thread = threading.Thread(target=self.handle_client_request, args=(new_socket,)) 86 # Setting up daemon main thread 87 sub_thread.setDaemon(True) 88 # Starter threads perform corresponding tasks 89 sub_thread.start() 90 91 92 # Program Entry Function 93 def main(): 94 95 96 print(sys.argv) 97 # Determine whether the command line parameter is equal to 2, 98 if len(sys.argv) != 2: 99 print("The execution order is as follows: python3 xxx.py 8000") 100 return 101 102 103 # Determine whether strings are made up of numbers 104 if not sys.argv[1].isdigit(): 105 print("The execution order is as follows: python3 xxx.py 8000") 106 return 107 108 109 # Get terminal command line parameters 110 port = int(sys.argv[1]) 111 # Establish web Server object 112 web_server = HttpWebServer(port) 113 # start-up web The server works 114 web_server.start() 115 116 117 118 119 if __name__ == '__main__': 120 main()