Multi-threading in Python is not really multi-threading. If you want to make full use of multi-core CPU resources, in most cases you need to use multi-processes in python.Python provides a very useful multiprocess package, Multiprocessing, that just defines a function and python does everything else.With this package, the conversion from single process to concurrent execution can be easily completed.Multprocessing supports subprocesses, communicating and sharing data, performing different forms of synchronization, and provides Process, Queue, Pipe, LocK, and other components
1. Process
Syntax: Process([group[,target[,name[,args[,kwargs]]]])
Parameter meanings: target denotes the calling object; args denotes the location parameter meta-ancestor of the calling object; kwargs denotes the dictionary of the calling object.The name is an alias and groups will not actually be called.
Method: is_alive():
join(timeout):
run():
start():
terminate():
Attributes: authkey, daemon (to be set via start(), exitcode (process is None at run time, if -N, it is ended by signal N), name, pid.Where daemon is automatically terminated after the parent process terminates and cannot generate a new process by itself, it must be set before start().
1. Create a function as a single process
from multiprocessing import Process def func(name): print("%s Once a good person"%name) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) p.start() #start() tells the system to start the process
2. Create functions and use them as multiple processes
from multiprocessing import Process import random,time def hobby_motion(name): print('%s Like Sports'% name) time.sleep(random.randint(1,3)) def hobby_game(name): print('%s Like games'% name) time.sleep(random.randint(1,3)) if __name__ == "__main__": p1 = Process(target=hobby_motion,args=('Fu Tingting',)) p2 = Process(target=hobby_game,args=('Kobe',)) p1.start() p2.start()
Execution results:
Fu Tingting likes sports Kobe likes games
3. Define a process as a class (another way to start a process, which is not very common)
from multiprocessing import Process class MyProcess(Process): def __init__(self,name): super().__init__() self.name = name def run(self): #When start(), run is called automatically, and can only be defined here as run. print("%s Once a good person"%self.name) if __name__ == "__main__": p = MyProcess('kebi') p.start() #Process is treated as a parent class and a function is customized.
4. Comparison of daemon programs
Without daemon attribute
import time def func(name): print("work start:%s"% time.ctime()) time.sleep(2) print("work end:%s"% time.ctime()) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) p.start() print("this is over") #results of enforcement this is over work start:Thu Nov 30 16:12:00 2017 work end:Thu Nov 30 16:12:02 2017
Add the daemon attribute
from multiprocessing import Process import time def func(name): print("work start:%s"% time.ctime()) time.sleep(2) print("work end:%s"% time.ctime()) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) p.daemon = True #Parent process terminates automatically after termination, no new process can be generated, must be set before start() p.start() print("this is over") #results of enforcement this is over
The method you want to execute once you have set the daemon property:
import time def func(name): print("work start:%s"% time.ctime()) time.sleep(2) print("work end:%s"% time.ctime()) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) p.daemon = True p.start() p.join() #Execute the previous code before executing the following print("this is over") #results of enforcement work start:Thu Nov 30 16:18:39 2017 work end:Thu Nov 30 16:18:41 2017 this is over
5.join(): The code above will not execute until it is executed.
Let's start with an example:
from multiprocessing import Process import time,os,random def func(name,hour): print("A lifelong friend:%s,%s"% (name,os.getpid())) time.sleep(hour) print("Good bother:%s"%name) if __name__ == "__main__": p = Process(target=func,args=('kebi',2)) p1 = Process(target=func,args=('maoxian',1)) p2 = Process(target=func,args=('xiaoniao',3)) p.start() p1.start() p2.start() print("this is over")
Execution results:
this is over #Last executed, first printed, indicating that start() only starts the process, not necessarily completes it A lifelong friend:kebi,12048 A lifelong friend:maoxian,8252 A lifelong friend:xiaoniao,6068 Good bother:maoxian #Print first, execute second Good bother:kebi Good bother:xiaoniao
Add join ()
from multiprocessing import Process import time,os,random def func(name,hour): print("A lifelong friend:%s,%s"% (name,os.getpid())) time.sleep(hour) print("Good bother:%s"%name) start = time.time() if __name__ == "__main__": p = Process(target=func,args=('kebi',2)) p1 = Process(target=func,args=('maoxian',1)) p2 = Process(target=func,args=('xiaoniao',3)) p.start() p.join() #After executing the above code, execute the following p1.start() p1.join() p2.start() p2.join() print("this is over") print(time.time() - start) #results of enforcement A lifelong friend:kebi,14804 Good bother:kebi A lifelong friend:maoxian,11120 Good bother:maoxian A lifelong friend:xiaoniao,10252 #Each process is finished before the next one is executed Good bother:xiaoniao this is over 6.497815370559692 #2+1+3+Main Program Execution Time
Change location
from multiprocessing import Process import time,os,random def func(name,hour): print("A lifelong friend:%s,%s"% (name,os.getpid())) time.sleep(hour) print("Good bother:%s"%name) start = time.time() if __name__ == "__main__": p = Process(target=func,args=('kebi',2)) p1 = Process(target=func,args=('maoxian',1)) p2 = Process(target=func,args=('xiaoniao',3)) p.start() p1.start() p2.start() p.join() #2 seconds required p1.join() #Execution is now complete p2.join() #Executed for 2 seconds, 1 second print("this is over") print(time.time() - start) #results of enforcement A lifelong friend:kebi,13520 A lifelong friend:maoxian,11612 A lifelong friend:xiaoniao,17064 #Almost simultaneously open execution Good bother:maoxian Good bother:kebi Good bother:xiaoniao this is over 3.273620367050171 #Give priority to the longest time
6. Other attributes and methods
from multiprocessing import Process import time def func(name): print("work start:%s"% time.ctime()) time.sleep(2) print("work end:%s"% time.ctime()) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) p.start() p.terminate() #Kill the process and must be placed behind start(), similar to what daemon does #results of enforcement this is over
from multiprocessing import Process import time def func(name): print("work start:%s"% time.ctime()) time.sleep(2) print("work end:%s"% time.ctime()) if __name__ == "__main__": p = Process(target=func,args=('kebi',)) # p.daemon = True print(p.is_alive()) p.start() print(p.name) #Get the name of the process print(p.pid) #Get pid of process print(p.is_alive()) #Determine whether a process exists print("this is over")