Use of ssdb queues

Keywords: SQL Redis JSON

Recently contacted with ssdb, I feel that it is similar to redis. For the time being, I haven't studied too much. Anyway, even the command is almost used.

The current framework is yii

Requirements:
Due to the large amount of data exported by users, in order to monitor the memory utilization of the server, the task is queued in the form of queue.

I won't introduce the configuration, just post a code.

When performing the controller operation of the Export button:

        //Add a record to the queue
        $uid = Yii::app()->user->id;
        $createTime = Helper::zeqiiTime('u');
        $filename = 'Order statistics report';
        Yii::app()->ssdb->_cache->qpush('export_csv',$uid.'|'.$createTime);
        Yii::app()->ssdb->_cache->hset('export_csv',$uid.'|'.$createTime,json_encode(array(
            'uid' => $uid,
            'createTime' => Helper::zeqiiTime('l'),
            'sql' => $sql,
            'status' => 0,
            'count' => $count,
            'type' => 'order_export',
            'filename' => $filename.''.date('YmdHis',time()) . '.csv'
        )));

Explain the use of ssdb in the above code:
Add a piece of data to the export ﹣ CSV queue. The name is export ﹣ CSV. The value is a field spliced with uid and timestamp to avoid repetition.
Then set a hashmap name in ssdb, also known as export_csv. The value of key is the value in the current queue, which is easy to use when fetching out.
The third parameter is a json format data. Here, arbitrary data is placed according to its own business logic.

Then the data is written to the export ﹣ CSV queue and hashmap, and then the server makes a scheduled task to execute the queue operation.

The code is as follows:

    public function run( $args ){
        if( Yii::app()->ssdb->_cache->qsize('export_csv')->data == 0 ){
            exit;
        }

        $fp = fopen(dirname(__FILE__).'/../../../runtime/keys/'.md5('export_csv_running'), 'w+');
        //Nonblocking
        if (!flock($fp, LOCK_EX | LOCK_NB)) {
            exit;
        }

        try {
            while ($hashMapKey = Yii::app()->ssdb->_cache->qfront('export_csv')->data) {
                $content = Yii::app()->ssdb->_cache->hget('export_csv', $hashMapKey)->data;
                if ($content) {
                    $data = json_decode($content, true);
                    $dir = Html::realStaticPath() . "files/export/" . date('Ymd', strtotime($data['createTime'])) . '/';
                    if (!is_dir($dir)) {
                        mkdir($dir, 0777, true);
                    }
                    $filename = $dir . $data['filename'];
                    if (file_exists($filename)) {
                        continue;
                    }

                    //Add a type to expand the export of other data
                    switch ($data['type']) {
                        case 'order_export':
                            $this->orderExpoer($filename, $data);
                            break;
                        default:
                            break;
                    }

                    //Change the status of the current task to completed
                    $data['status'] = 1;
                    Yii::app()->ssdb->_cache->hset('export_csv', $hashMapKey, json_encode($data));
                }

                //Loop one at a time and automatically pop one
                Yii::app()->ssdb->_cache->qpop('export_csv', 1);
            }

            flock($fp, LOCK_UN);
            fclose($fp);
        }catch (Exception $e){
            Yii::log('Running exportCsv error:'.$e->getMessage(), 'error', 'info');
            flock($fp, LOCK_UN);
            fclose($fp);
        }

    }

Explain the logic of the above code:
First, judge the length of the queue in export ﹣ CSV. If it is 0, it will prove that there is no data and exit directly.
Then add a file lock judgment, because it's possible that the previous task hasn't finished executing, and the later scheduled task starts executing the script again.
After that, loop out the data and do what you want to do.

Posted by pulsedriver on Sat, 02 May 2020 01:31:41 -0700