Article directory
1, Sharing global variable resource competition
One thread can write and one thread can read. If two threads write to a resource at the same time, resource competition may occur.
import threading num = 0 def demo1(count): global num for i in range(count): num += 1 print('demo1--%d' % num) def demo2(count): global num for i in range(count): num += 1 print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(100,)) t2 = threading.Thread(target=demo2, args=(100,)) t1.start() t2.start() print('main--%d' % num) if __name__ == '__main__': main()
Printing
demo1--100 demo2--200 main--200
When the parameters are further expanded:
import threading num = 0 def demo1(count): global num for i in range(count): num += 1 print('demo1--%d' % num) def demo2(count): global num for i in range(count): num += 1 print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(1000000,)) t2 = threading.Thread(target=demo2, args=(1000000,)) t1.start() t2.start() print('main--%d' % num) if __name__ == '__main__': main()
Printing
main--181091 demo2--1209597 demo1--1410717
Interpretation:
num is a global variable, starting with 0. There is a for loop in demo1() and demo2(). In the loop of each sub thread:
(1) First, get the value of num;
(2) Then add 1 to the obtained num;
(3) And save the results to num.
When executing, when one sub thread executes to step (2), it has not been saved. At this time, turn to another sub thread. At this time, because the first sub thread has not yet executed to step (3), in order to save num, the value of num is 0. When executing to step (2), add 1. At this time, go back to step (3) of the first sub thread to save num, which is 1, and then to step 3 of the second sub thread Step is also saved, so it is also 1.
The above phenomenon is a probability problem. When the number of cycles is more, the probability of occurrence is greater. Therefore, when the number of cycles is 100, this phenomenon does not occur. When the number of cycles is 1000000, there is a more obvious phenomenon.
At this point, resource competition occurs.
Verify execution (view bytecode):
import dis def add_num(a): a += 1 print(dis.dis(add_num))
Printing
36 0 LOAD_FAST 0 (a) 2 LOAD_CONST 1 (1) 4 INPLACE_ADD 6 STORE_FAST 0 (a) 8 LOAD_CONST 0 (None) 10 RETURN_VALUE None
Explain:
(1) LOAD_FAST: load a;
(2) LOAD_CONST: load constant value 1;
(3) Replace? Add: add;
(4) STORE_FAST: assign to a.
Obviously, this process is similar to the one that explained the execution of a child thread when competing for resources.
2, Mutex and deadlock
1. mutual exclusive lock
When multiple threads modify a shared data almost at the same time, synchronization control is needed. When a thread wants to change the shared data, it first locks it. At this time, the resource status is locked. Other threads cannot change it. Only when the thread releases the resource and turns the resource status into non locked, can other threads lock the resource again.
The mutex ensures that only one thread writes at a time, thus ensuring the correctness of data in the case of multithreading.
import threading num = 0 # Create a mutex mutex = threading.Lock() def demo1(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo1--%d' % num) def demo2(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(1000000,)) t2 = threading.Thread(target=demo2, args=(1000000,)) t1.start() t2.start() print('main--%d' % num) if __name__ == '__main__': main()
Printing
main--166589 demo1--1000000 demo2--2000000
At this time, the printing of sub thread 1 and 2 is normal, but the printing results of the main thread are still irregular.
Interpretation:
After t1.start() and t2.start() are executed, the two sub threads will run. At this time, they will continue to run downward. The main thread will print the value of num at this time. Because the sub thread may not finish running, that is, the value of num has not been added to 2000000, it will print the main first, wait until the sub thread 1 finishes printing 1000000, release the lock, run the sub thread 2, and print 2000000.
Further improvement:
import threading import time num = 0 # Create a mutex mutex = threading.Lock() def demo1(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo1--%d' % num) def demo2(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(1000000,)) t2 = threading.Thread(target=demo2, args=(1000000,)) t1.start() t2.start() time.sleep(2) print('main--%d' % num) if __name__ == '__main__': main()
Printing
demo1--1000000 demo2--2000000 main--2000000
At this time, pause for 2 seconds after executing t1.start() and t2.start(), which is enough for sub thread 1 and sub thread 2 to finish execution, so print first, and then the main thread prints out the last value 2000000 of num.
2. deadlock
When sharing multiple resources between online programs, if two threads occupy part of the resources respectively and wait for the resources of each other at the same time, it will cause deadlock.
import threading import time class MyThread1(threading.Thread): def run(self): # Lock mutexA mutexA.acquire() # After mutex A is locked, delay 1 second and wait for another thread to lock mutex B print(self.name+'----do1---up----') time.sleep(1) # At this time, it will be blocked because mutexB has been locked by another thread mutexB.acquire() print(self.name+'----do1---down----') mutexB.release() # Unlock mutexA mutexA.release() class MyThread2(threading.Thread): def run(self): # Lock mutexB mutexB.acquire() # After mutex B is locked, delay for 1 second and wait for another thread to lock mutex a print(self.name+'----do2---up----') time.sleep(1) # At this time, it will be blocked, because this mutex a has been locked by another thread mutexA.acquire() print(self.name+'----do2---down----') mutexA.release() # Unlock mutexB mutexB.release() mutexA = threading.Lock() mutexB = threading.Lock() if __name__ == '__main__': t1 = MyThread1() t2 = MyThread2() t1.start() t2.start()
Show:
Obviously, the two threads are locked in a deadlock waiting for each other, and the program cannot run normally until the program is stopped manually or the program with full memory crashes.
Avoid deadlock:
- Try to avoid it in programming
- Add timeout
At the same time, the created lock can only be locked again after it is released. Otherwise, it will fall into deadlock waiting, as follows
import threading import time num = 0 # Create a mutex mutex = threading.Lock() def demo1(count): global num # Lock up mutex.acquire() mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() mutex.acquire() print('demo1--%d' % num) def demo2(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(100,)) t2 = threading.Thread(target=demo2, args=(100,)) t1.start() t2.start() time.sleep(2) print('main--%d' % num) if __name__ == '__main__': main()
Show:
It's easy to know that the main thread has been waiting for the execution of the sub thread to end, but the sub thread demo1() has been locked twice.
To reuse a lock object, you need to use RLock:
import threading import time num = 0 # Create a reentrant lock mutex = threading.RLock() def demo1(count): global num # Lock up mutex.acquire() mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() mutex.release() print('demo1--%d' % num) def demo2(count): global num # Lock up mutex.acquire() for i in range(count): num += 1 # Unlock mutex.release() print('demo2--%d' % num) def main(): t1 = threading.Thread(target=demo1, args=(100,)) t2 = threading.Thread(target=demo2, args=(100,)) t1.start() t2.start() time.sleep(2) print('main--%d' % num) if __name__ == '__main__': main()
Show:
Deadlock extension banker algorithm
Background:
How can a banker safely lend a certain number of funds to several customers so that these customers can not only borrow money to complete their tasks, but also recover all the funds without bankruptcy? This is the banker's problem. This problem is very similar to the resource allocation problem in the operating system: the banker is like an operating system, the customer is like a running process, and the banker's funds are the system's resources.
Problem Description:
A banker has a certain amount of money and several customers want to borrow money. Each customer must declare at the beginning the total amount of the loan he needs. If the total amount of the customer's loan does not exceed the total amount of the banker's funds, the banker can accept the customer's request. Customer loans are made in the form of one fund unit at a time (such as RMB 10000, etc.), and customers may wait until they have fully borrowed the required amount of all units, but bankers must ensure that such waiting is limited and can be completed.
For example, there are three customers C1, C2 and C3, who borrow money from a banker. The total amount of the banker's funds is 10 fund units, among which, C1 customer needs to borrow 9 fund units, C2 customer needs to borrow 3 fund units, C3 customer needs to borrow 8 fund units, a total of 20 fund units. The state of a certain moment is as shown in the figure:
For the state of a-chart, according to the requirements of the security sequence, the first customer we choose should meet the customer's needs of loan less than or equal to the money left by the banker at present. It can be seen that only C2 customers can be satisfied: C2 customers need 1 fund unit, and small bankers have 2 fund units, so the banker lends 1 fund unit to C2 customers to complete the work and return it to them The money borrowed from three fund units is shown in Figure b. Similarly, the banker lends 4 fund units to C3 customers to complete the work. In Figure c, there is only one customer C1, which needs 7 fund units. At this time, the banker has 8 fund units, so C1 can borrow money and complete the work smoothly. Finally (see Figure d) the banker recovers all 10 units and guarantees no loss. Then the customer sequence {C1, C2, C3} is a secure sequence, according to which the banker is secure. Otherwise, if the banker lends four fund units to C1 in the state of figure b, there will be an unsafe state: at this time, C1 and C3 can not complete the work, and the banker has no money in his hand, the system is in a stalemate situation, and the banker can not recover the investment.
To sum up, the banker algorithm starts from the current state, checks each customer who can complete their work one by one according to the security sequence, then assumes that they can complete their work and repay all loans, and then checks the next customer who can complete their work. If all the clients can do the work, then find a security sequence, and the banker is safe.
3, Thread synchronization Condition
Background:
Xiaodu: I love my classmates;
Little love classmate: I am here;
Xiaodu: what time is it now?
Xiaoai: guess what time it is
Code implementation attempt:
import threading class XiaoAi(threading.Thread): def __init__(self): super().__init__(name='Little love classmates') def run(self): print('{}:Where am I?'.format(self.name)) class XiaoDu(threading.Thread): def __init__(self): super().__init__(name='Xiaodu') def run(self): print('{}:Little love classmates'.format(self.name)) if __name__ == '__main__': xiaoai = XiaoAi() xiaodu = XiaoDu() xiaodu.start() xiaoai.start()
Printing
Xiaodu: Xiaoai Xiaoai: where am I
Further expansion attempts:
import threading class XiaoAi(threading.Thread): def __init__(self): super().__init__(name='Little love classmates') def run(self): print('{}:Where am I?'.format(self.name)) print('{}:Guess what time it is'.format(self.name)) class XiaoDu(threading.Thread): def __init__(self): super().__init__(name='Xiaodu') def run(self): print('{}:Little love classmates'.format(self.name)) print('{}:What time is it now?'.format(self.name)) if __name__ == '__main__': xiaoai = XiaoAi() xiaodu = XiaoDu() xiaodu.start() xiaoai.start()
Printing
Xiaodu: Xiaoai Xiaodu: what time is it now? Xiaoai: where am I Xiaoai: guess what time it is
Obviously, it didn't achieve the effect we wanted.
Lock again:
import threading class XiaoAi(threading.Thread): def __init__(self,lock): super().__init__(name='Little love classmates') self.lock = lock def run(self): self.lock.acquire() print('{}:Where am I?'.format(self.name)) self.lock.release() self.lock.acquire() print('{}:Guess what time it is'.format(self.name)) self.lock.release() class XiaoDu(threading.Thread): def __init__(self,lock): super().__init__(name='Xiaodu') self.lock = lock def run(self): self.lock.acquire() print('{}:Little love classmates'.format(self.name)) self.lock.release() self.lock.acquire() print('{}:What time is it now?'.format(self.name)) self.lock.release() if __name__ == '__main__': mutex = threading.Lock() xiaoai = XiaoAi(mutex) xiaodu = XiaoDu(mutex) xiaodu.start() xiaoai.start()
Printing
Xiaodu: Xiaoai Xiaodu: what time is it now? Xiaoai: where am I Xiaoai: guess what time it is
The result is the same as before, but the desired effect is still not achieved, and the effect cannot be achieved with the lock.
Try to synchronize with thread:
There are two methods in the Condition class:
__enter? Calls Lock's acquire() method;
__exit () calls the release() method of Lock.
It can be seen that the Condition object can be processed by the context handler.
import threading class XiaoAi(threading.Thread): def __init__(self,cond): super().__init__(name='Little love classmates') self.cond = cond def run(self): with self.cond: self.cond.wait() print('{}:Where am I?'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:Guess what time it is'.format(self.name)) self.cond.notify() class XiaoDu(threading.Thread): def __init__(self,cond): super().__init__(name='Xiaodu') self.cond = cond def run(self): self.cond.acquire() print('{}:Little love classmates'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:What time is it now?'.format(self.name)) self.cond.notify() self.cond.wait() self.cond.release() if __name__ == '__main__': cond = threading.Condition() xiaoai = XiaoAi(cond) xiaodu = XiaoDu(cond) xiaodu.start() xiaoai.start()
Show:
Because the Condition class implements the \\\\\\\\\\\\\\.
Obviously, the execution of the program is blocked, because there is a problem with the calling order of the sub threads. Debug:
import threading class XiaoAi(threading.Thread): def __init__(self,cond): super().__init__(name='Little love classmates') self.cond = cond def run(self): with self.cond: print(4) self.cond.wait() print(5) print('{}:Where am I?'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:Guess what time it is'.format(self.name)) self.cond.notify() class XiaoDu(threading.Thread): def __init__(self,cond): super().__init__(name='Xiaodu') self.cond = cond def run(self): self.cond.acquire() print('{}:Little love classmates'.format(self.name)) print(1) self.cond.notify() print(2) self.cond.wait() print(3) print('{}:What time is it now?'.format(self.name)) self.cond.notify() self.cond.wait() self.cond.release() if __name__ == '__main__': cond = threading.Condition() xiaoai = XiaoAi(cond) xiaodu = XiaoDu(cond) xiaodu.start() xiaoai.start()
Show:
It is easy to know that the order of execution is:
First, execute the print('{}: xiaoaiclassmate'. format(self.name)) and cond.notify(), and Xiaodu always cond.wait(), then execute the cond.acquire(), and cond.wait(), both of which are waiting and locked.
Exchange the thread creation and execution order of xiaodu and xiaoai
import threading class XiaoAi(threading.Thread): def __init__(self,cond): super().__init__(name='Little love classmates') self.cond = cond def run(self): with self.cond: self.cond.wait() print('{}:Where am I?'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:Guess what time it is'.format(self.name)) self.cond.notify() class XiaoDu(threading.Thread): def __init__(self,cond): super().__init__(name='Xiaodu') self.cond = cond def run(self): self.cond.acquire() print('{}:Little love classmates'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:What time is it now?'.format(self.name)) self.cond.notify() self.cond.wait() self.cond.release() if __name__ == '__main__': cond = threading.Condition() xiaoai = XiaoAi(cond) xiaodu = XiaoDu(cond) xiaoai.start() xiaodu.start()
Printing
Xiaodu: Xiaoai Xiaoai: where am I Xiaodu: what time is it now? Xiaoai: guess what time it is
Three sub thread attempts:
import threading class XiaoAi(threading.Thread): def __init__(self,cond): super().__init__(name='Little love classmates') self.cond = cond def run(self): with self.cond: self.cond.wait() print('{}:Where am I?'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:Guess what time it is'.format(self.name)) self.cond.notify() class XiaoDu(threading.Thread): def __init__(self,cond): super().__init__(name='Xiaodu') self.cond = cond def run(self): self.cond.acquire() print('{}:Love classmates, tmall'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:What time is it now?'.format(self.name)) self.cond.notify() self.cond.wait() self.cond.release() class Jingling(threading.Thread): def __init__(self,cond): super().__init__(name='Tmall Elf') self.cond = cond def run(self): with self.cond: self.cond.wait() print('{}:I am here too.'.format(self.name)) self.cond.notify() self.cond.wait() print('{}:Guess what time it is'.format(self.name)) self.cond.notify() if __name__ == '__main__': cond = threading.Condition() xiaoai = XiaoAi(cond) xiaodu = XiaoDu(cond) jingling = Jingling(cond) xiaoai.start() jingling.start() xiaodu.start()
Printing
Xiaodu: Xiaoai, tmall Xiaoai: where am I Tmall: me too Xiaodu: what time is it now? Xiaoai: guess what time it is Tmall: guess what time it is
4, Multi task UDP chat
Implementation ideas:
- Create socket
- Bind local information
- Get the IP and port of the opposite party
- Sending and receiving data
- Create two threads and perform functions
NetAssist is configured as follows:
Code implementation:
import socket import threading def recv_msg(udp_socket): '''receive data''' while True: recv_data = udp_socket.recvfrom(1024) print(recv_data[0].decode('gbk')) def send_msg(udp_socket, dest_ip, dest_port): '''send data''' while True: send_data = input('Inoput data to send:') udp_socket.sendto(send_data.encode('gbk'), (dest_ip, dest_port)) def main(): #Create socket udp_socket = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) #binding udp_socket.bind(('',7890)) # Get the IP and port of the other party dest_ip = input('Input IP:') dest_port = int(input('Input Port:')) t_recv = threading.Thread(target=recv_msg, args=(udp_socket, )) t_send = threading.Thread(target=send_msg, args=(udp_socket, dest_ip, dest_port)) t_recv.start() t_send.start() if __name__ == '__main__': main()
Demonstration effect:
5, Process introduction
1. Concept analysis
- Process:
Running code + resources used; - Procedure:
No code is executed, it is static.
2. Status of the process
It mainly includes three states: ready, running and waiting, which is a cyclic process.
3. Use process to realize multitask
Multiprocessing module is a cross platform multiprocessing module. It provides a Process class to represent a Process object. This object can be understood as an independent Process and can perform other things.
Thread test:
import threading import time def demo1(): for _ in range(5): print('--1--') time.sleep(1) def demo2(): for _ in range(5): print('--2--') time.sleep(1) def main(): t1 = threading.Thread(target=demo1) t2 = threading.Thread(target=demo2) t1.start() t2.start() if __name__ == '__main__': main()
Print:
--1-- --2-- --1-- --2-- --1-- --2-- --1-- --2-- --1-- --2--
Change to process:
import multiprocessing import time def demo1(): for _ in range(5): print('--1--') time.sleep(1) def demo2(): for _ in range(5): print('--2--') time.sleep(1) def main(): t1 = multiprocessing.Process(target=demo1) t2 = multiprocessing.Process(target=demo2) t1.start() t2.start() if __name__ == '__main__': main()
Print:
--1-- --2-- --1-- --2-- --1-- --2-- --1-- --2-- --1-- --2--
Obviously, the code implementation of process and thread is similar, and the result is very similar.
But threads and processes are very different:
Watch Task Manager:
When it's a thread,
There is only one Python process at this time.
When it's a process,
At this time, there are two Python processes, representing two subprocesses in the code.
One main process and two subprocesses, subprocesses copy and execute the code of the main process, which will cause waste of resources, but it will be faster than single process.
Run in Ubuntu:
import os import time #Create child thread pid = os.fork() print('Hello World') #Judge is a child thread if pid == 0: print('s_fork:{},f_fork:{}'.format(os.getpid(),os.getppid())) else: print('f_fork:{}'.format(os.getpid()))
The result of execution (error will be reported by Windows operation) is
Hello World f_fork:2180 Hello World s_fork:2181,f_fork:2180
Printed Corley twice.
Interpretation:
os.fork() copies the code of the parent process to create a child process. The parent process and the child process execute once respectively, print out Hello World successively, enter if judgment respectively, and print id twice.
