think-queue 3.0 Practical Course: Creating a Timing Deduction System

Preface

By the time of writing, ThinkPHP 6.0 had entered RC3. ThinkPHP 6.0 is expected to be released this autumn as the last or penultimate RC version before the official version, which means that ThinkPHP 6.0 is becoming more and more perfect and stable, and is a candidate version worth trying.

As a result, the thought-queue extension, which has undergone significant changes, has also been upgraded and released close to the official version of 3.0.2. So the little partners who have already used think-queue in the production environment are sure to want to know, think-queue.

Is 3.0 ready to be tasted? This edition will show you how to use think-queue 3.0.2 timed queue to build a timed deduction system to tell you the answer.

PS: If there are any mistakes or omissions in my article, please also Haihan, welcome Daniel to criticize and guide you at any time.

Dead work

Using think-queue queues requires the following conditions:

1: A server based on liunx system, windows is also not recommended.

2: redis server. Recommendation 5.0,

Reference article: https://www.jianshu.com/p/fe6...；

Beginners recommend pagoda panels. https://www.bt.cn/download/li... To save energy and reduce common problems on the server side caused by configuration compilation.

3: Compooser package manager

Reference: https://www.phpcomposer.com；

Recommended mirror sources: https://mirrors.aliyun.com/co... Thank you for Ali Yun's contribution.

4: A redis client, recommended by Windows developers for the following projects

https://github.com/uglide/Red...；

https://github.com/qishibo/An...；

https://github.com/cinience/R...；

And Windows PowerShell comes with windows.

For those who are not familiar with think-queue, be sure to read the following tutorial before returning to continue reading:

https://github.com/coolseven/...

Install thinkhp and think-queue

1: Create Thinkphp 6.0 new project as the first choice. https://www.kancloud.cn/manua...

composer create-project topthink/think=6.0.x-dev tp

2: Use think-queue 3.0.2

https://packagist.org/package...

composer require topthink/think-queue

You can also add configuration items to the composer.json file in the project root directory

    "require": {
        "php": ">=7.1.0",
        "topthink/framework": "6.0.*-dev",
        "topthink/think-view": "^1.0",
        "symfony/var-dumper":"^4.2",
        "topthink/think-queue": "^3.0"
    },

3: Check if the installation is successful

Run under the project root directory

php think

notice

That means think-queue has been installed successfully.

The next step is to create the database of the project, and I'm ready for the structure.

User membership form

DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  `username` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL DEFAULT '' COMMENT 'User name',
  `nickname` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Nickname?',
  `realname` varchar(30) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Full name',
  `password` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL DEFAULT '' COMMENT 'Password',
  `create_time` int(10) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Registration time',
  `update_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Update time',
  `login_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Landing time',
  `login_count` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Number of landings',
  `login_ip` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Sign in ip',
  `vip` tinyint(2) UNSIGNED NULL DEFAULT 0 COMMENT 'vip Grade',
  `vip_join` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'vip Join time',
  `vip_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'vip Expiration time',
  `ip` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'register ip',
  `status` tinyint(1) UNSIGNED NOT NULL DEFAULT 1 COMMENT 'State 1:Normal 2:Ban 3:temporary',
  `lock_uid` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Banned persons',
  `lock_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Closure time',
  `lock_tips` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Causes of blockade',
  `back_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Unsealing time',
  `group` tinyint(1) UNSIGNED NOT NULL DEFAULT 1 COMMENT 'Identity 1:Ordinary 2:Administrator 3:Agent 4:Partner 5:Channel merchants',
  `group_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Identity expiration',
  `safe_level` tinyint(1) UNSIGNED NOT NULL DEFAULT 1 COMMENT 'Safety Level',
  `safe_code` char(8) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Secret Security Code',
  `safe_token` tinyint(1) UNSIGNED NULL DEFAULT 0 COMMENT 'Secret Security Token 1:No. 2:yes',
  `safe_device` tinyint(1) UNSIGNED NOT NULL DEFAULT 1 COMMENT 'Encryption device 1:No. 2:yes',
  `safe_phone` bigint(11) UNSIGNED NULL DEFAULT 0 COMMENT 'Secret Security Mobile Phone',
  `safe_email` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Secret Mailbox',
  `verify_code` char(8) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Retrieving Check Code',
  `verify_lock` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Lock-back',
  `verify_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Check code expiration',
  `money` decimal(10, 3) UNSIGNED NOT NULL DEFAULT 0.000 COMMENT 'balance',
  `give` decimal(10, 3) UNSIGNED NOT NULL DEFAULT 0.000 COMMENT 'Additional delivery',
  `brokerage` decimal(10, 3) UNSIGNED NOT NULL DEFAULT 0.000 COMMENT 'Commission',
  `server` int(10) UNSIGNED NOT NULL DEFAULT 10 COMMENT 'The server',
  `gold` tinyint(8) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Gold coins',
  `credits` tinyint(8) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'integral',
  `union_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'WeChat',
  `unionid` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'QQ',
  `inviters` tinyint(5) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Number of invitations',
  `inviter_id` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Inviter',
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `id`(`id`, `username`, `realname`, `vip`, `group`, `money`, `create_time`) USING BTREE COMMENT 'Joint Index'
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8 COLLATE = utf8_general_ci COMMENT = 'User table' ROW_FORMAT = Dynamic;

Log table

DROP TABLE IF EXISTS `logs`;
CREATE TABLE `logs`  (
  `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  `uid` int(10) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'user id',
  `subid` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Host id',
  `op_id` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Operator id',
  `type` tinyint(2) UNSIGNED NOT NULL DEFAULT 1 COMMENT 'Type 1:Membership 2:Management 3:System 5:Finance',
  `time` int(10) UNSIGNED NOT NULL DEFAULT 0 COMMENT 'time',
  `ip` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL DEFAULT '' COMMENT 'Sign in ip',
  `content` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL DEFAULT '0' COMMENT 'Log content',
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `idx_uid_type_time`(`uid`, `type`, `time`, `op_id`, `subid`) USING BTREE COMMENT 'Joint Index'
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8 COLLATE = utf8_general_ci COMMENT = 'Log table' ROW_FORMAT = Dynamic;

User Host Table

DROP TABLE IF EXISTS `server`;
CREATE TABLE `server`  (
  `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  `uid` int(10) UNSIGNED NOT NULL COMMENT 'Subordinate users',
  `status` int(2) UNSIGNED NOT NULL DEFAULT 2 COMMENT 'State 1:Stopped 2:Operating 3:Overdue 4:Renewal fee 5:Deleted 6:abnormal',
  `time` int(10) UNSIGNED NOT NULL COMMENT 'Creation time',
  `op` int(4) UNSIGNED NULL DEFAULT 0 COMMENT 'Operator',
  `op_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Operating time',
  `subid` bigint(11) UNSIGNED NULL DEFAULT 0 COMMENT 'Example id',
  `ip_address` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'IPv4',
  `password` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT '' COMMENT 'root Password',
  `snapshotid` varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT '' COMMENT 'snapshot id',
  `port` int(10) UNSIGNED NULL DEFAULT 22 COMMENT 'port',
  `ips` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'High prevention IP',
  `enable_ipv6` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'IPv6',
  `ipv6` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'IPv6 address',
  `dcid` int(5) UNSIGNED NULL DEFAULT 0 COMMENT 'position',
  `osid` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'operating system',
  `arch` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'System type',
  `vpsplanid` double(32, 0) UNSIGNED NULL DEFAULT 0 COMMENT 'Configuration Specification',
  `hostname` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'Custom Name',
  `ddos` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT '' COMMENT 'DDOS',
  `appid` int(6) UNSIGNED NULL DEFAULT 0 COMMENT 'Pre-installed application',
  `destroy` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Delete time',
  `month` int(3) UNSIGNED NULL DEFAULT 0 COMMENT 'Purchase time',
  `deduction` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Number of deductions',
  `money` decimal(10, 3) UNSIGNED NULL DEFAULT 0.000 COMMENT 'Cost',
  `deduction_time` int(10) UNSIGNED NULL DEFAULT 0 COMMENT 'Deduction time',
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8 COLLATE = utf8_general_ci COMMENT = 'Host table' ROW_FORMAT = Compact;

The registration code for system members is omitted. Please supplement it by yourself. You can also download the corresponding example code of this tutorial at the end of the article.

Start using queues

In this tutorial, think-queue queues are redis-driven. Be sure to configure redis first. Configuration files are queue.php under the config folder. Make no mistake.

return [
    'default' => 'redis',
    'connections' => [
        'sync' => [
            'driver' => 'sync',
        ],
        'database' => [
            'driver' => 'database',
            'queue' => 'default',
            'table' => 'jobs',
        ],
        'redis' => [
            'driver' => 'redis',
            'queue' => 'default',
            'host' => '127.0.0.1',
            'port' => 6379,
            'password' => 'Xun166123',
            'select' => 1,
            'timeout' => 0,
            'persistent' => false,
        ],
    ],
    'failed' => [
        'type' => 'none',
        'table' => 'failed_jobs',
    ],
];

Next add the queue

/*
* Queue task
* @author zakeear <zakeear@86dede.com>
* @version v0.1.5
* @time 2019-06-10
*/
namespace app\queue\controller;
use think\Exception;
use think\facade\Db;
use think\facade\Queue;
class Host{
    /**
     * Adding queues
     * @access public
     * @param int $subid Host id
     * @param string $type Task Name
     * @param int $times Delay seconds
     * @throws \think\Exception
     */
    public function addTask(int $subid=0,string $type='server',int $times=0){
        $server=Db::name('server')->where(['subid'=>$subid])->find();
        if(!$server){
            exit;
        }
        switch($type){
            case 'server':
                $jobHandlerClassName='app\queue\job\Money@fire';
                $jobDataArr=['submit'=>time(),'doit'=>time()+$times,'subid'=>$server['subid'],'hostname'=>$server['hostname']];
                $jobQueueName="Money";
                break;
            case 'destroy':
                $jobHandlerClassName='app\queue\job\Destroy@fire';
                $jobDataArr=['submit'=>time(),'doit'=>time()+$times,'subid'=>$server['subid'],'hostname'=>$server['hostname']];
                $jobQueueName="Destroy";
                break;
            default:
                break;
        }
        if($times==0){
            $isPushed=Queue::push($jobHandlerClassName,$jobDataArr,$jobQueueName);
        }else{
            $isPushed=Queue::later($times,$jobHandlerClassName,$jobDataArr,$jobQueueName);
        }
    }
}

Then there's the consumer class.

/*
* Host Deduction Category
* @author zakeear <zakeear@86dede.com>
* @version v0.2.0
* @time 2019-06-13
*/
namespace app\queue\job;
use think\queue\Job;
use think\facade\Db;
class Money{
    public function fire(Job $job,$data){
        //job
        $isJobDone=$this->doJob($job,$data);
        $attempts=$job->attempts()+1;
        if($isJobDone){
            print('<info>['.date('Y-m-d H:i:s',time())."] Host".$data['hostname']."Deduction task completed, task destroyed</info>\n");
            $job->delete();
        }else{
            $release=strtotime(date('Y-m-d H:',time()).'00')+3599+date('i',$data['submit'])*60+date('s',$data['submit'])-time();
            print('<info>['.date('Y-m-d H:i:s',time())."] ".$release."Execute the host in seconds".$data['hostname']."First".$attempts."Sub-deduction task</info>\n");
            $job->release($release);
        }
    }
    private function doJob($job,$data){
        //job
        $attempts=$job->attempts();
        print('<info>['.date('Y-m-d H:i:s',time())."] Host".$data['hostname']."First".$attempts."Secondary deduction</info>\n");
        //Host
        $server=Db::name('server')->field('id,uid,subid,month,money,hostname,deduction')->where(['subid'=>$data['subid'],'status'=>2])->find();
        if(!$server){
            print('<info>['.date('Y-m-d H:i:s')."] Host".$data['hostname']."No longer exists or deleted!"."\n");
            return true;
        }
        //Journal
        $logs=new \app\common\logic\Logs();
        //To configure
        $this->config = Db::name('config')->field('rate,month,is_buy,vultr_api,vultr_keys,web_name,web_icon,time')->where(['status'=>1])->order('time','desc')->find();
        if($server['deduction']==$this->config['month']){
            //Deduction
            $logs->database($server['uid'],5,'','Host['.$server['hostname'].']If the monthly payment limit is reached, no deduction will be made this month.',1,$server['subid']);
            //count
            Db::name('server')->where(['subid'=>$data['subid']])->update(['deduction'=>0,'deduction_time'=>0]);
            //job
            $attempts=$attempts-1;
            print('<info>['.date('Y-m-d H:i:s')."] Host".$data['hostname']."The monthly payment ceiling has been reached.".$attempts."second\n");
            //queue
            $job=new \app\queue\controller\Host();
            //delete
            $job->addTask($data['subid'],'destroy',0);
            //Establish
            $release=\app\Timer::nextMonth()[0]+date('i',$data['submit'])*60+date('s',$data['submit'])-time();//Recalculation next month
            $job->addTask($data['subid'],'server',$release);
            //Return
            return true;
        }
        //user
        $user=Db::name('user')->field('id,money')->where(['id'=>$server['uid']])->find();
        if($user['money']<$server['money']){
            //Journal
            $logs->database($server['uid'],5,'','Host['.$server['hostname'].']Not enough to pay:['.$server['money'].']',1,$server['subid']);
            //delete
            Db::name('server')->where(['subid'=>$data['subid']])->update(['status'=>5,'destroy'=>time()]);
            //Journal
            $logs->database($server['uid'],5,'','Host['.$server['hostname'].']delete',1,$server['subid']);
            //queue
            $job=new \app\queue\controller\Host();
            //delete
            $job->addTask($data['subid'],'destroy',0);
            //job
            print('<info>['.date('Y-m-d H:i:s')."] User balance insufficient to pay".$user['money']."element\n");
            //Return
            return true;
        }
        //Cost
        $money=new \app\common\logic\Money();
        //Deduction
        $money->hostDec($server['uid'],1,$server['money'],1,$server['subid'],'Host['.$server['hostname'].']Payment');
        //Journal
        $logs->database($server['uid'],5,'','Host['.$server['hostname'].']Payment fees:['.$server['money'].']',1,$server['subid']);
        //count
        Db::name('server')->where(['subid'=>$data['subid']])->inc('deduction',1)->update(['deduction_time'=>time()]);
    }
}

After the creation, the corresponding structure of the file directory is as follows:

Guarding Consumer Process

Refer to this article

https://segmentfault.com/a/11...

Or use the pagoda for planning tasks

Add the following script

#Check whether the php Money queue script is started
php_count=`ps -ef | grep Money | grep -v "grep" | wc -l`
if [ $php_count == 0 ];then
    echo '----php Money queue start'
    `sudo -H -u www bash -c 'nohup php /www/wwwroot/www.demo.com/think queue:listen --queue Money > /www/wwwroot/www.demo.com/logs/Money.txt 2>&1 &'`
else
    echo '----php Money queue ok'
fi

#Check whether the php DestroyQueue queue script is started
php_count=`ps -ef | grep Destroy | grep -v "grep" | wc -l`
if [ $php_count == 0 ];then
    echo '----php Destroy queue start'
    `sudo -H -u www bash -c 'nohup php /www/wwwroot/www.demo.com/think queue:listen --queue Destroy > /www/wwwroot/www.demo.com/logs/Destroy.txt 2>&1 &'`
else
    echo '----php Destroy queue ok'
fi

As shown in the figure:

Code Interpretation

Think-queue has not yet implemented subscribe function. Here, think-queue delay queue is used to achieve the timing task. When the tasks in the consumer class are completed, no return true, just throw the delay back to the queue. Whether the queue will always exist or not will be deleted, it will be disguised to achieve the timing task.

But there's a question here, how can you tell the queue exactly how long it will take?

The code is as follows:

            $release=strtotime(date('Y-m-d H:',time()).'00')+3599+date('i',$data['submit'])*60+date('s',$data['submit'])-time();
            print('<info>['.date('Y-m-d H:i:s',time())."] ".$release."Execute the host in seconds".$data['hostname']."First".$attempts."Sub-deduction task</info>\n");
            $job->release($release);

Here's the key. You need to figure it out by yourself. If you do it once a minute, you can do it according to the general principle of $job - > release (60), but forget that it takes time for the consumer class to run itself, at least hundreds of milliseconds. When there are many tasks, it may take 1-2 seconds to complete. Join the first task and start the consumption task at 15:00:01. It takes another second to complete the consumption. After the completion of the consumption, you will pay $job - > release (60). Then the next consumption queue will be 15:01:02. Then the third execution will be 15:01:03. By analogy, after 60 tasks, in the middle. There has been no deduction for up to a minute, which is a bug or disaster for projects requiring regular deduction. Here we use dynamic calculation to determine how many seconds the delay is, and then we solve this problem. We often observe the production environment for up to two months, the error is about two seconds. Let's leave a lead here. What if it doesn't exceed 1 second before and after control? If there are too many task queues at this time, and thousands of queues are piled up and then queued up, what should we do? In the next issue, we will probably show you how to use think-queue to achieve task scheduling to build an order placing system that supports high concurrency.

The missing code in this article, please go to https://github.com/zakeear/man Find or modify it in combination with your own business.

think-queue 3.0 and think-queue 2.0 changes

stay https://github.com/coolseven/... thinkphp's queue core is written by itself, and laravel's queue core relies on the composer package symfony/process. Look through the source code of think-queue 3.0 and find out

Just in line with the idea of Thinkphp 6.0, embrace composer in an all-round way! The biggest difference between think-queue 3.0 and think-queue 2.0 is that think-queue 3.0 requires registration services, and think-queue 2.0 does not. This difference can cause extreme problems encountered by a small editor in the process of using. Think-queue 3.0 installed under windows is on liunx, and php think queue is invalid. If there are similar problems, the solution is to install Thinkphp 6.0 and think-queue 3.0 on liunx and then download them to Windows for use, which is customary for small partners to pay attention to.

In addition, students who have used think-queue 2.0 will find that to use it properly, think-queue 3.0 must depend on two built-in functions of php. These two built-in functions are so sensitive that operation and maintenance are generally disabled. The pagoda recommended by the edition will also disable them by default. Think-queue 3.0 will be replaced by symfony/process package. After that, we no longer rely on these two built-in functions, so 3.0 is safer than 2.0.

epilogue

https://www.kancloud.cn/think... ThinkPHP Developer Weekly is an important part of Thinkphp ecology. It has been updated and maintained for more than a year by itself. It provides a channel for phper to learn and understand excellent projects, books and developers in php circle. At present, the weekly has been maintained by volunteers. In the near future, it will be maintained by the community. This is the case. At present, there are good articles, projects, books and cases. You are welcome to contribute. Contribution Address: QQ Group: 780179357

This tutorial is in a hurry. If there are any omissions or errors in this tutorial, you are welcome to correct them. The full example source code for this tutorial has been hosted to giehub: https://github.com/zakeear/man

Posted by candle21428 on Sun, 21 Jul 2019 03:14:47 -0700

Programmer Group