Eight five classes (December 15)

16.1 multi process message queue

Message queuing is a container that holds messages during their transmission.
The most classic use of message queuing is to pass messages through message channels between consumers and producers. Consumers and producers are different processes. Producers write messages to the pipeline, and consumers read messages from the pipeline.
The operating system provides many mechanisms to realize the communication between processes. The multiprocessing module provides two methods, Queue and Pipe.

Using Queue in multiprocessing to implement message Queue

from multiprocessing import Queue
q = Queue
q.put(data)
data = q.get(data)


#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @Time    : 2017/6/6 17:36
# @Author  : lingxiangxiang
# @File    : demon8.py
from multiprocessing import Process, Queue
import os, time, random
# Write the code of data process execution:
def write(q):
    print('Process to write: %s' % os.getpid())
    for value in ['A', 'B', 'C']:
        print('Put %s to queue...' % value)
        q.put(value)
        time.sleep(random.random())
        # Read the code executed by the data process:
def read(q):
    print('Process to read: %s' % os.getpid())
    while True:
        value = q.get(True)
        print('Get %s from queue.' % value)
if __name__=='__main__':
# The parent process creates a Queue and passes it to each child process:
    q = Queue()
    pw = Process(target=write, args=(q,))
    pr = Process(target=read, args=(q,))
    # Start the sub process pw, write: 
    pw.start()
    # Start subprocess pr, read: 
    pr.start()  
    # Wait for pw to end:
    pw.join()
    # pr process is a dead cycle. It cannot wait for it to end. It can only be forcibly terminated:    
    pr.terminate()

16.2 message queue pipe

The message queue is implemented through the Pipe in Mutiprocess:
1. The Pipe method returns (conn1, conn2) representing two ends of a Pipe. The Pipe method has a duplex parameter. If the duplex parameter is true (the default value), the Pipe is in full duplex mode, that is, both conn1 and conn2 can be sent and received. Duplex is False, conn1 is only responsible for receiving messages, and conn2 is only responsible for sending messages.
2. The send and recv methods are the methods of sending and receiving messages respectively. The close method means to close the pipeline. When the message is accepted, close the pipeline.

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @Time    : 2017/6/6 16:32
# @Author  : lingxiangxiang
# @File    : demon6.py
import multiprocessing
import time
 
def proc1(pipe):
    for i in xrange(5):
        print "send: %s" %(i)
        pipe.send(i)
        # print(dir(pipe))
        time.sleep(1)
 
def proc2(pipe):
    n = 5
    while n:
        print "proc2 rev:", pipe.recv()
        time.sleep(1)
        n -= 1
 
if __name__ == "__main__":
    pipe = multiprocessing.Pipe(duplex=False)
    print(type(pipe))
    print(pipe)
    p1 = multiprocessing.Process(target=proc1, args=(pipe[1],))
    p2 = multiprocessing.Process(target=proc2, args=(pipe[0],))
 
    p1.start()
    p2.start()
 
    p1.join()
    p2.join()
    pipe[0].close()    pipe[1].close()