8. Tuesday class (December 12)

14.6 multi process Manager

Powerful Manage
The above implementation of data sharing only has two structures: Value and Array. Python provides a powerful Manage for data sharing, which supports many types, including Value, Array, list, dict, Queue, Lock, etc.
Here is an example:

#!/usr/bin/env python# 
-*- coding:utf-8 -*-
from multiprocessing import Process, 
Managerdef func(dt, lt):    
for i in range(10):       
key = 'arg' + str(i)        
dt[key] = i * i    
lt += range(11, 16)

Example 2

import multiprocessing

def worker(d,l):
    l += range(11,16)
    for i in range(1,6):
        key = "key{0}".format(i)
        val = "val{0}".format(i)
        d[key] = val
if __name__ == "__main__":
    manager = multiprocessing.Manager()
    d = manager.dict()
    l = manager.list()
    p = multiprocessing.Process(target=worker,args=(d,l))
    p.start()
    p.join()
    print(d)
    print(l)
    print("mian end")

14.7 process pool

Process pool
Pool can provide a specified number of processes for users to call. When a new request is submitted to the pool, if the pool is not full, a new process will be created to execute the request. However, if the number of processes in the pool has reached the specified maximum, the request will wait until the end of a process in the pool, a new process will be created.
Let's take a non blocking example of a process pool:

#!/usr/bin/env python#
 -*- coding:utf-8 -*-       
import multiprocessing  
import time
def fun(msg):      
print("######start######  {0}".format(msg)) 
time.sleep(3)   
print("######end######  {0}".format(msg))

if __name__ == "__main__": 
pool = multiprocessing.Pool(processes=3) 
for i in xrange(1, 6):      
 msg = "hello {0}".format(i)       
pool.apply_async(fun, (msg,))   
 print("##########start main#########")
pool.close()
pool.join() 
#Call the close function before calling the join, otherwise an error will occur. After executing close, no new processes will be added to the pool. The join function waits for all subprocesses to finish
    
print("##########end main#########")