[rookie light series] a brief discussion on the software collaboration

Keywords: PHP

Reading this article requires the following knowledge points

  • Understand the process, thread related basis
  • Proficient in hello world output of php
  • Can spell the word swoole

Introduction to the program

What is a collaborative process?

A coroutine is a function that can suspend its execution (yield) until the given given YieldInstruction finishes.

In short, coprogramming is a kind of cooperative lightweight thread that is implemented by the programmer under the thread with lighter concurrency

With the increase of the number of programmers, the number of bigwigs is also growing explosively. Of course, some people begin to think that threads are not easy to use. What should be done? Of course, it is based on the idea of thread to realize a set of lighter threads that are more light-weight and better cheat star s (in fact, coprocessors can not be regarded as threads, because a thread can have multiple coroutines)

The difference between coroutines and threads

essence

Thread kernel state
Coprocess user state

Dispatching mode

The thread scheduling method is system scheduling, and the common scheduling strategies are time-sharing scheduling and preemptive scheduling. In other words, the scheduling of threads is completely out of their control

The scheduling mode of coprocess is cooperative scheduling, which is not controlled by the kernel and is switched by free policy scheduling

wait

Collaborative scheduling?

As mentioned above, coprogramming is user mode, so the so-called collaborative scheduling can be directly understood as the scheduling method written by programmers, that is, I can schedule as I want, instead of being scheduled through the system kernel.

Deep.... Shallow understanding of the coroutine of swote

Since we intend to understand the coroutine of swoole, we must know its coprocess model.
The coroutine of droole is based on single thread. It can be understood that the switch of coprogram is serial, and only one coprocessor runs at the same time point

At this point, someone must have asked. go is based on multithreading. Of course, each has its own advantages, you can use the search engine to understand

We can directly copy & paste the following code and run the demo run in the local environment

<?php

$func = function ($index, $isCorotunine = true) {
    $isCorotunine && \Swoole\Coroutine::sleep(2);
    echo "index:" . $index . ", value:" . (++$count) . PHP_EOL;
    echo "is corotunine:" . intval($isCorotunine) . PHP_EOL;
};

$func(1, false);
go($func, 2, true);
go($func, 3, true);
go($func, 4, true);
go($func, 5, true);
go($func, 6, true);
$func(7, false);    

You get the following results

index:1, value:1
is corotunine:0
index:7, value:2
is corotunine:0
index:2, value:3
is corotunine:1
index:6, value:4
is corotunine:1
index:5, value:5
is corotunine:1
index:4, value:6
is corotunine:1
index:3, value:7
is corotunine:1

Some people will think, wow, it's all done in one second, there's no jam at all!!

Well, in fact, you can go back and look at the concept of coroutines when you're done in one second.
We can focus on the execution order. 1 and 7 are non coprocessors. The execution can immediately return the results as expected.
On scheduling order of coroutines
Why is 26543 not an ordered return of 65432 or 23456

In order to find our answer, we can only know something through the source code

Analysis of source code

Picture from https://segmentfault.com/a/11...

If you don't have a strong foundation and a gnawing apcu (of course, I don't! T_T)
We need to be concerned about the following two
yield is the coprocessor's CPU
Resume resume collaboration

We can think for ourselves, if we let you design the collaborative process, what aspects need to be considered?

Coprocess structure diagram

Creation of a program

<?php
go (function(){
echo "swoole fantastic";
});

The swoole is encapsulated in the go function of PHP to create a coroutine

We expand the source code according to the

Most of the PHP extension functions and parameter declarations of extension methods are placed in swote_ *.cc, swoole.cc Inside.

PHP_FALIAS(go, swoole_coroutine_create, arginfo_swoole_coroutine_create)

You can know go > swote_ coroutine_ create

In swote_ coroutine.cc Found in the file

PHP_FUNCTION(swoole_coroutine_create)
{
    ....
    // Key points for examination
    long cid = PHPCoroutine::create(&fci_cache, fci.param_count, fci.params);
    ....
}

long PHPCoroutine::create(zend_fcall_info_cache *fci_cache, uint32_t argc, zval *argv)
{
    if (sw_unlikely(Coroutine::count() >= config.max_num))
    {
        php_swoole_fatal_error(E_WARNING, "exceed max number of coroutine %zu", (uintmax_t) Coroutine::count());
        return SW_CORO_ERR_LIMIT;
    }

    if (sw_unlikely(!active))
    {
        // Key points for examination
        activate();
    }

    // Save callback function
    php_coro_args php_coro_args;
    //Function information
    php_coro_args.fci_cache = fci_cache;
    //parameter
    php_coro_args.argv = argv;
    php_coro_args.argc = argc;
    // Key points for examination
    save_task(get_task());

    // Key points for examination
    return Coroutine::create(main_func, (void*) &php_coro_args);
}
// Save stack
void PHPCoroutine::save_task(php_coro_task *task)
{
    save_vm_stack(task);
    save_og(task);
}
// Event to initialize reactor
inline void PHPCoroutine::activate()
{
    if (sw_unlikely(active))
    {
        return;
    }

    /* init reactor and register event wait */
    php_swoole_check_reactor();

    /* replace interrupt function */
    orig_interrupt_function = zend_interrupt_function;
    zend_interrupt_function = coro_interrupt_function;
    
    /* replace the error function to save execute_data */
    orig_error_function = zend_error_cb;
    zend_error_cb = error;

    if (config.hook_flags)
    {
        enable_hook(config.hook_flags);
    }

    if (SWOOLE_G(enable_preemptive_scheduler) || config.enable_preemptive_scheduler)
    {
        /* create a thread to interrupt the coroutine that takes up too much time */
        interrupt_thread_start();
    }

    if (!coro_global_active)
    {
        if (zend_hash_str_find_ptr(&module_registry, ZEND_STRL("xdebug")))
        {
            php_swoole_fatal_error(E_WARNING, "Using Xdebug in coroutines is extremely dangerous, please notice that it may lead to coredump!");
        }

        /* replace functions that can not work correctly in coroutine */
        inject_function();

        coro_global_active = true;
    }
    /**
     * deactivate when reactor free.
     */
    swReactor_add_destroy_callback(SwooleG.main_reactor, deactivate, nullptr);
    active = true;
}

Continue to jump down according to Coroutine::create

    static inline long create(coroutine_func_t fn, void* args = nullptr)
    {
        return (new Coroutine(fn, args))->run();
    }

It will be executed immediately after creating the cooperation program
Let's look at the construction method

    Coroutine(coroutine_func_t fn, void *private_data) :
            ctx(stack_size, fn, private_data)
    {
        cid = ++last_cid;
        coroutines[cid] = this;
        if (sw_unlikely(count() > peak_num))
        {
            peak_num = count();
        }
    }

In the above code, I can find that there is also a Context class. We can guess that three things have been done

  1. Assign the corresponding coroutine id (each coroutine has its own id)
  2. Save context
  3. Update the current number of coroutines

The coroutine library used by droole is boost.context Self Searching
The main exposed function interface is jump_fcontext and make_fcontext
The specific function is to save the context of the current execution state, pause the current execution state, and jump to another location to continue execution

Implement the process

inline long run()
    {
        long cid = this->cid;
        origin = current;
        current = this;
        // rely on boost.context  Cutting stack
        ctx.swap_in();
        // Judge whether the execution is finished
        check_end();
        return cid;
    }

Judge whether it is over or not

inline void check_end()
    {
        if (ctx.is_end())
        {
            close();
        }
        else if (sw_unlikely(on_bailout))
        {
            SW_ASSERT(current == nullptr);
            on_bailout();
            // expect that never here
            exit(1);
        }
    }

according to ctx.is_end() function found

    inline bool is_end()
    {
        return end_;
    }
bool Context::swap_in()
{
    jump_fcontext(&swap_ctx_, ctx_, (intptr_t) this, true);
    return true;
}

Let's summarize what swoole did when creating the coprocess

  1. Detection environment
  2. Analytic parameters
  3. Save context
  4. Switch C stack
  5. Implementation of the program

yield of a coroutine

For the above demo, we use the tool / coroutine:: sleep (2)
According to the above, the function states that we are in swote_ corotunine_ System. H found that the corresponding file is swote_ coroutine_ sleep function of system

PHP_METHOD(swoole_coroutine_system, sleep)
{
    double seconds;

    ZEND_PARSE_PARAMETERS_START(1, 1)
        Z_PARAM_DOUBLE(seconds)
    ZEND_PARSE_PARAMETERS_END_EX(RETURN_FALSE);

    if (UNEXPECTED(seconds < SW_TIMER_MIN_SEC))
    {
        php_swoole_fatal_error(E_WARNING, "Timer must be greater than or equal to " ZEND_TOSTR(SW_TIMER_MIN_SEC));
        RETURN_FALSE;
    }
    System::sleep(seconds);
    RETURN_TRUE;
}

After calling the sleep function, three things are done to the current coroutine
timer increased by 1
2. Register the callback function and delay the resume procedure
3. Dispatch through yield

int System::sleep(double sec)
{
// Gets the current coroutine
    Coroutine* co = Coroutine::get_current_safe();
   //swTimer_add register timer sleep_ Function of timeout callback
   if (swTimer_add(&SwooleG.timer, (long) (sec * 1000), 0, co, sleep_timeout) == NULL)
    {
        return -1;
    }
    // Give up the current cpu
    co->yield();
    return 0;
}

// Callback function
static void sleep_timeout(swTimer *timer, swTimer_node *tnode)
{
   // Resume scheduling
    ((Coroutine *) tnode->data)->resume();
}
swTimer_node* swTimer_add(swTimer *timer, long _msec, int interval, void *data, swTimerCallback callback)
{
    ....
    // Saves the current context and the corresponding expiration time
    tnode->data = data;
    tnode->type = SW_TIMER_TYPE_KERNEL;
    tnode->exec_msec = now_msec + _msec;
    tnode->interval = interval ? _msec : 0;
    tnode->removed = 0;
    tnode->callback = callback;
    tnode->round = timer->round;
    tnode->dtor = NULL;

    // A kind of next_msec saves the fastest expired events
    if (timer->_next_msec < 0 || timer->_next_msec > _msec)
    {
        timer->set(timer, _msec);
        timer->_next_msec = _msec;
    }

    tnode->id = timer->_next_id++;
    if (sw_unlikely(tnode->id < 0))
    {
        tnode->id = 1;
        timer->_next_id = 2;
    }

    tnode->heap_node = swHeap_push(timer->heap, tnode->exec_msec, tnode);
    ....
    timer->num++;
    return tnode;
}

Switch of coprogram

We

void Coroutine::resume()
{
    SW_ASSERT(current != this);
    if (sw_unlikely(on_bailout))
    {
        return;
    }
    state = SW_CORO_RUNNING;
    if (sw_likely(on_resume))
    {
        on_resume(task);
    }
    // Save the current process as origin > understand procedure previous
    origin = current;
    // The coroutine that needs to be executed becomes current
    current = this;
    // Stack execution
    ctx.swap_in();
    check_end();
}

By this time, the answer to the sequence of calls to the coroutines has come out

When the coroutine is created (New coroutine (FN, args)) - > run(); and sleep trigger yield are interleaved with the time when php code creates the coroutine when the current and origin of Corotunine are constantly changing, instead of the stack or queue that we imagine to execute in order
For example, when there are only two collaborators to create

<?php

$func = function ($index, $isCorotunine = true) {
    $isCorotunine && \Swoole\Coroutine::sleep(2);
    echo "index:" . $index . PHP_EOL;
    echo "is corotunine:" . intval($isCorotunine) . PHP_EOL;
};

$func(1, false);

go($func, 2, true);
go($func, 3, true);

Return the output, because the execution time of continuous creation of coroutines is small, and it is not disturbed

php swoole_go_demo1.php
index:1
is corotunine:0
index:2
is corotunine:1
index:3
is corotunine:1

When you create it in succession, when you have 200 coprocesses
The index, which becomes disordered after returning, conforms to the expected conjecture

index:1,index:202,index:1,index:2,index:4,index:8,index:16,index:32,index:64,index:128,index:129,index:65,index:130,index:131,index:33,index:66,index:132,index:133,index:67,index:134,index:135,index:17,index:34,index:68,index:136,index:137,index:69,index:138,index:139,index:35,index:70,index:140,index:141,index:71,index:142,index:143,index:9,index:18,index:36,index:72,index:144,index:145,index:73,index:146,index:147,index:158,index:157,index:156,index:155,index:154,index:153,index:152,index:151,index:37,index:74,index:148,index:149,index:75,index:150,index:19,index:38,index:76,index:77,index:39,index:78,index:79,index:5,index:10,index:20,index:40,index:80,index:81,index:41,index:82,index:83,index:21,index:127,index:126,index:125,index:124,index:123,index:122,index:121,index:120,index:119,index:118,index:117,index:116,index:115,index:114,index:113,index:112,index:111,index:110,index:109,index:108,index:107,index:106,index:105,index:104,index:103,index:102,index:101,index:100,index:99,index:98,index:97,index:96,index:95,index:94,index:93,index:92,index:91,index:90,index:89,index:88,index:87,index:42,index:84,index:85,index:43,index:86,index:11,index:22,index:44,index:45,index:23,index:46,index:47,index:3,index:6,index:12,index:24,index:48,index:49,index:25,index:50,index:51,index:13,index:26,index:63,index:62,index:61,index:60,index:59,index:58,index:57,index:56,index:55,index:52,index:53,index:27,index:54,index:7,index:14,index:28,index:29,index:15,index:30,index:31,index:200,index:199,index:192,index:185,index:175,index:168,index:161,index:163,index:172,index:179,index:187,index:194,index:174,index:160,index:173,index:176,index:198,index:195,index:180,index:167,index:169,index:184,index:197,index:193,index:177,index:162,index:171,index:186,index:182,index:164,index:191,index:183,index:166,index:196,index:178,index:170,index:189,index:188,index:165,index:181,index:190,index:159

Last egg

We use GO's coroutine to achieve the above demo

package main

import (
    "fmt"
    "time"
)

var count int = 0

func main() {
    output(false, 1)

    go output(true, 2)
    go output(true, 3)
    go output(true, 4)
    go output(true, 5)
    go output(true, 6)

    output(false, 7)

    time.Sleep(time.Second)
}

func output(isCorotunine bool, index int) {
    time.Sleep(time.Second)
    count = count + 1
    fmt.Println(count, isCorotunine, index)
}

Guess what the return result is. You can study it based on the multithreading method of go coroutine

At the end of the article, I understand it completely according to the code and data. If there is any mistake, I will feel very sorry. If I am misled by some wrong views, I can only say that

Posted by solarith on Mon, 29 Jun 2020 19:56:35 -0700