python basic protocol-microthreaded greenlet gevent

Keywords: network Python REST

The python base series is being continuously updated:)

Article Directory

Recall process and thread introduction

We said earlier that threads are responsible for time-consuming operations, because processes are called synchronously, blocked, and executed serially, so we need to use threads, asynchronous calls, non-blocked, and execute in parallel to increase efficiency.
The problem is, suppose there are also several subtasks in the thread, which is time consuming. Then our programs are all serial, so run () inside a Thread is all serial, and a subtask, such as slow download and other tasks blocking there, is inefficient.
Threads are as embarrassing as previous processes.
We python come out with a micro-thread that can operate in parallel, the subtasks of a thread, also known as
Ctrip
Coroutine cor - routine can also see what parallelism means

For example, we have a lot of tasks to do, play games, learn python, and make friends. These are three processes.In the game, I have Dark Soul 1, 2, 3, Ancient Scroll 5, which is 4 threads, of course, Ancient Scroll 5 has 100 mods, I need to update 100 mods, there are 100 coordinations.Because updates are time consuming, the task of updating mod must be invoked in parallel.

greenlet

python encapsulates the library greenlet to implement the protocol, let's take an example:

#-*- utf-8 -*-
from time import sleep,time
from greenlet import greenlet
def taskA():
    for i in range(5):
        print('A'+str(i))
        gB.switch()
        sleep(1)  # Simulate network blocking time-consuming operations


def taskB():
    for i in range(5):
        print('B' + str(i))
        gC.switch()
        sleep(1)  # Simulate network blocking time-consuming operations



def taskC():
    for i in range(5):
        print('C' + str(i))
        gA.switch()
        sleep(1)  # Simulate network blocking time-consuming operations


if __name__ == "__main__":
    t1 = time()
    gA = greenlet(run = taskA)
    gB = greenlet(run = taskB)
    gC = greenlet(run = taskC)

    gA.switch()
    
    t2 = time()
    print("time consume",t2-t1)

Like process thread, greenlet must also pass a target parameter (which he named run has the opposite meaning)
This is mainly to implement A B C parallel polling, but we can see that writing like this is silly, requires manual switching, and is not very efficient. So the official greenlet is not very useful. There are big guys who come out with the gevent library (third party) and encapsulate a wave on the basis of greenlet to achieve automatic switching.

gevent

To do with the spawn function (taskA B C), spawn spawns eggs; triggers; causes these meanings, a little run.
Note that we have encountered in the process before, the main process runs after start ing the child process, causing the child process to gg, so we can block the main process with join, and this is the same thing.

#-*- utf-8 -*-
from time import sleep,time
from gevent import monkey,spawn

def taskA():
    for i in range(3):
        print('A'+str(i))
        sleep(1)  # Simulate network blocking time-consuming operations


def taskB():
    for i in range(3):
        print('B' + str(i))
        sleep(1)  # Simulate network blocking time-consuming operations



def taskC():
    for i in range(3):
        print('C' + str(i))
        sleep(1)  # Simulate network blocking time-consuming operations


if __name__ == "__main__":
    t1 = time()
    gA = spawn(run = taskA)
    gB = spawn(run = taskB)
    gC = spawn(run = taskC)

    gA.join()
    gB.join()
    gC.join()
    t2 = time()
    print("time consume",t2-t1)

Result:

A0
A1
A2
B0
B1
B2
C0
C1
C2
time consume 9.041131258010864

We found that,
1:Didn't poll ABC ABC like this
2: No parallelism Ah 2333 In fact, the greenlet above also does not implement parallelism???Or 9s

monkey

Our time is not appropriate for this occasion, so we use monkey.patch_all()

#-*- utf-8 -*-
import time
from gevent import monkey,spawn
monkey.patch_all()

def taskA():
    for i in range(3):
        print('A'+str(i))
        time.sleep(1)  # Simulate network blocking time-consuming operations


def taskB():
    for i in range(3):
        print('B' + str(i))
        time.sleep(1)  # Simulate network blocking time-consuming operations



def taskC():
    for i in range(3):
        print('C' + str(i))
        time.sleep(1)  # Simulate network blocking time-consuming operations


if __name__ == "__main__":
    t1 = time.time()
    gA = spawn(run = taskA)
    gB = spawn(run = taskB)
    gC = spawn(run = taskC)

    gA.join()
    gB.join()
    gC.join()
    t2 = time.time()
    print("time consume",t2-t1)

time.sleep is required for this to work because monkey stealthily replaces the original sleep with an overload behind it. You import sleep directly from time. monkey does not match
Result:

A0
B0
C0
A1
B1
C1
A2
B2
C2
time consume 3.036893606185913

Process finished with exit code 0

Visible, more perfect parallel implementation, time consume time consumption from 9s parallel to 3s

summary

We just use gevent monkey and the rest of the execution is too cumbersome. It's like recreating a wheel:)

18 original articles published, acclaimed 13, visited 2854
Private letter follow

Posted by think-digitally on Thu, 30 Jan 2020 19:41:17 -0800