Using swoole_process to implement PHP process pool

Keywords: Linux PHP Programming

swoole_process is mainly used to replace the pcntl extension of PHP. We know that pcntl is used for multi-process programming, while pcntl only provides the original interface such as fork, which is easy to use errors, and does not provide inter-process communication and redirection of standard input and output functions.

swoole_process, on the other hand, provides a more powerful and easier-to-use API than pcntl, making PHP easier in multi-process programming.

This article uses swoole_process and EventLoop to complete a php process pool, and supports the dynamic creation of new processes.

EventLoop

swoole has a Reactor thread, which can be said to encapsulate the epoll model and set the listener callback functions for read and write events.

A function is used below:

bool swoole_event_add(mixed $sock, mixed $read_callback, mixed $write_callback = null, int $flags = null);
  • Parameter 1 is a file descriptor, including swoole_client-> $sock, swoole_process-> $pipe or other fd (resources created by socket_create, resources created by stream_socket_client/fsockopen)
  • Parametric 2 is a readable event callback function
  • Parameter 3 is a writable event callback function

Multiprocess programming requires communication between processes. There are two ways of communication between swoole processes, one is queue, the other is pipe. So this article uses the pipe approach.

Here is an example of delivering tasks to the process pool on a regular basis.

Code:

<?php
class ProcessPool{

    private $process;

    /**
     * Worker Process array
     * @var array
     */
    private $process_list = [];

    /**
     * Processes in use
     * @var array
     */
    private $process_use = [];

    /**
     * Minimum number of processes
     * @var int
     */
    private $min_worker_num = 3;

    /**
     * Maximum number of processes
     * @var int
     */
    private $max_worker_num = 6;

    /**
     * Number of current processes
     * @var int
     */
    private $current_num;


    public function __construct()
    {
        $this->process = new swoole_process(array($this, 'run'), false, 2);
        $this->process->start();
        swoole_process::wait();
    }

    public function run()
    {
        $this->current_num = $this->min_worker_num;
        //Create all worker processes
        for($i = 0; $i < $this->current_num; $i++){
            $process = new swoole_process(array($this, 'task_run'), false, 2);
            $pid = $process->start();
            $this->process_list[$pid] = $process;
            $this->process_use[$pid] = 0;
        }

        foreach($this->process_list as $process){
            swoole_event_add($process->pipe, function ($pipe) use ($process){
                $data = $process->read();
                var_dump($data . 'free');
                //Receive the information processed by the child process and reset it to idle
                $this->process_use[$data] = 0;
            });
        }

        //Delivery of tasks to worker pipeline at regular intervals per second
        swoole_timer_tick(1000 ,function ($timer_id){
            static $index = 0;
            $index = $index + 1;
            $flag = true; //Whether to create a new worker
            foreach ($this->process_use as $pid => $used){
                if($used == 0){
                    $flag = false;
                    //Mark positive in use
                    $this->process_use[$pid] = 1;
                    // Call write within the parent process, and the child process can call read to receive this data
                    $this->process_list[$pid]->write($index. "hello");
                    break;
                }
            }

            if($flag && $this->current_num < $this->max_worker_num){
                //No idle worker, new worker to deal with
                $process = new swoole_process(array($this, 'task_run'), false, 2);
                $pid = $process->start();
                $this->process_list[$pid] = $process;
                $this->process_use[$pid] = 1;
                $this->process_list[$pid]->write($index. "hello");
                $this->current_num++;
            }
            var_dump('The first' .$index. 'Task');
            if($index == 10){
                foreach($this->process_list as $process){
                    $process->write("exit");
                }
                swoole_timer_clear($timer_id);
                $this->process->exit();
            }

        });
    }

    /**
     * Subprocess processing
     * @param $worker
     */
    public function task_run($worker)
    {
        swoole_event_add($worker->pipe, function($pipe)use($worker){
            $data = $worker->read();
            var_dump($worker->pid . ':' . $data);
            if($data == 'exit'){
                $worker->exit();
                exit;
            }
            //Simulating time-consuming tasks
            sleep(5);
            //Tell the main process that processing is complete
            //Call write within a child process, and the parent process can call read to receive this data
            $worker->write($worker->pid);
        });
    }

}

new ProcessPool();

Firstly, several important attributes are defined:

  • $process_list: Worker process array
  • $process_use: The process being used
  • $min_worker_num: Minimum number of processes
  • $max_worker_num: Maximum number of processes
  • Curr_num: Number of current processes
  • $process: main process

When instantiating, create the main process and run the run method. In the run method, create all the worker processes first and set them to idle state.

It then traverses all worker processes and joins EventLoop to set up readable events for receiving idle signals from subprocesses.

Finally, tasks are delivered to the worker process every second. The dynamic expansion process pool is implemented here. If there are no idle processes and there are new tasks at this time, a new process needs to be created dynamically and kept busy. Since only ten tasks are simulated, exits are sent in the parent process after the tenth task is completed to exit all the child processes.

Operation effect and illustration:

Reference link:

https://wiki.swoole.com/wiki/...
https://opso.coding.me/2018/0...

Posted by DarrenL on Mon, 21 Jan 2019 11:48:13 -0800