The technologies used are docker + redis + beanstalkd + swoole
#Download redis and beanstalkd from the warehouse docker pull redis:5.0.7 docker pull schickling/beanstalkd #View image list docker images #Running beanstalkd in the docker container and mapping to the local host port 11300 docker run --name beanstalkd -d -it -p 11300:11300 428 docker run --name redis01 -d -it -p 6380:6379 redis:5.0.7 d: cd D:\phpstudy_pro\WWW\test\beanstalkd #install composer require pda/pheanstalk:3.1 composer require predis/predis:1.1
| performance | Reliability (ack response) | Extensibility | |
|---|---|---|---|
| kafka | 8w/s | Unreliable | colony |
| rabitMQ | 4w/s | reliable | colony |
| redis | 8w/s | Unreliable | colony |
| beanstalk | 8w/s | reliable | Manual construction |
beanstalkd is a high performance and lightweight memory queue system
beanstalkd feature
1. Support priority (support team to jump in line)
2. Delay (to achieve timed tasks)
3. Persistence (regularly brush the data in memory to the binlog log)
4. Reservation (set the task as reservation, the consumer cannot take out the task, and then take it out for processing at a proper time)
5. Task timeout resend (the consumer must process the task within the specified time, if not, the task will be deemed as failed and re entered the queue)
beanstalkd core elements
Producer - > tube - > job - > consumer
job: a task to be processed asynchronously needs to be placed in a tube
tube: a task queue with a name, used to store unified types of job s, and can create multiple channels
Producer: the producer of a job
Consumer: the consumer of a job
Process: the producer generates a task job, pushes the task job into a tube, and then the consumer takes the job out of the tube for execution
composer.json
"require":{ "pda/pheanstalk":"^3.1", "predis/predis": "^1.1", }
Producer generated task
<?php //producer.php require "vendor/autoload.php"; try{ $p = new \Pheanstalk\Pheanstalk('127.0.0.1',11300); //Swoole? Timer? Tick timer swoole_timer_tick(10,function() use ($p)){ $data = [ 'msg_id' => session_create_id(),//php7.1 new generated random id 'tid'=>time().uniqid(), ]; $id = $p->putInTube('task',json_encode($data));//Put into pipeline var_dump($id); } }catch(Exception $e){ }
Consumer performs tasks
<?php //consumer.php require "vendor/autoload.php"; //If it takes 5 seconds to execute a task, then it takes 50 seconds for 10 tasks. How to improve efficiency //Get task data and execute business (multi process programming) according to task type $workerNum = 5; $pool = new Swoole\Process\Pool($workerNum); //Process start up $pool->on('WorderStart',function($pool,$workerId){ echo 'Worder#'.$workderId.'is started'; try{ $p = new \Pheanstalk\Pheanstalk('127.0.0.1',11300); $redis = new Predis\Client('tcp://127.0.0.1:6380'); //Remove task from pipeline //If there is no data, it will suspend waiting for data, so while will not enter the loop while(true){ $job = $p->watch('task')->reserve(); if(!$empty($job)){ $json = $job->getData(); $data = json_decode($json,true); $state = $redis->get('job:'.$data["msg_id"]); if($state == 1){//There are consumers in the process of implementation $p->release($job,0,5);//Delay the delivery task for 5 seconds }elseif($state == 2){//It's done continue; }else{ //setex(key,seconds,value) $redis->setex('job:'.$data['msg_id'],6,1); //Do other operations such as sending SMS and adding points sleep(5); $redis->set('job:'.$data['msg_id'],2); //ack response mechanism is designed for message reliability $p->delete($job); } } } }catch(Exception $e){ } }); $pool->on('WorderStop',function($pool,$workerId){ echo 'Worder#'.$workderId.'is stopped'; });