memcached Source Code Analysis--item Overdue Failure Processing and LRU Reptiles

Keywords: less REST

Reprinted: http://blog.csdn.net/luotuo44/article/details/42963793

Warm Tip: This article uses some global variables that can be set at the start of memcached. The meaning of these global variables can be referred to.< Detailed explanation of start-up parameters of memcached " For these global variables, the approach is like< How to read memcached source code > Take it directly as you say Default value . In addition, LRU queues are mentioned in this article, and an introduction to LRU queues can be consulted.< LRU Queues and item Structures>.



Overdue Failure Treatment:

An item expires in two cases: 1. The item's exptime stamp arrives. 2. Users use the flush_all command to make all items expired. Readers may say that the touch command can also expire an item, which is actually the first case mentioned earlier.


Overtime failure:

For the first type of expiration, memcached uses lazy processing: it does not actively detect whether an item is expired or not. When the worker thread accesses the item, it detects whether the item's exptime stamp has arrived. It's relatively simple. We don't paste the code here, but we will paste it later.


The flush_all command:

The second expiration is set by the user flush_all command. Flush_all will render all items expired. What items do all items refer to? Because multiple clients are constantly inserting items into memcached, it is important to understand what all items refer to. Is it bounded by the moment the worker thread receives this command or by the moment it deletes it?

When the worker thread receives the flush_all command, the oldest_live member of the global variable settings stores the time at which it receives the command (to be precise, the worker thread parses that it knows this is a flush_all command, minus one at the moment), the code is settings.oldest_live= current_time-1; then the item_flush_expired function is called to lock the cache_lock, and then The do_item_flush_expired function is then called to complete the work.

  1. void do_item_flush_expired(void) {  
  2.     int i;  
  3.     item *iter, *next;  
  4.     if (settings.oldest_live == 0)  
  5.         return;  
  6.     for (i = 0; i < LARGEST_ID; i++) {  
  7.         for (iter = heads[i]; iter != NULL; iter = next) {  
  8.             if (iter->time != 0 && iter->time >= settings.oldest_live) {  
  9.                 next = iter->next;  
  10.                 if ((iter->it_flags & ITEM_SLABBED) == 0) {  
  11.                     do_item_unlink_nolock(iter, hash(ITEM_key(iter), iter->nkey));  
  12.                 }  
  13.             } else {  
  14.                 /* We've hit the first old item. Continue to the next queue. */  
  15.                 break;  
  16.             }  
  17.         }  
  18.     }  
  19. }  

The do_item_flush_expired function traverses all LRU queues and detects the time members of each item. It is reasonable to detect time members. If the time member is less than settings.oldest_live, the item already exists when the worker thread receives the flush_all command (the time member represents the last access time of the item). Then it's time to delete the item.

It seems that memcached is bounded by the moment the worker thread receives the flush_all command. Wait a minute, see clearly!! In the do_item_flush_expired function, the item is not deleted when the time member of the item is less than settings.oldest_live, but when it is larger. In the sense of time member variables, what is greater than that? Do you have anything larger than that? Strange! "T" & T $


In fact, memcached is bounded by the moment of deletion. Why does settings.oldest_live store the timestamp when the worker thread receives the flush_all command? Why judge whether ITER - > time is larger than settings.oldest_live?

In general, delete all items on the hash table and LRU directly in the do_item_flush_expired function. That's really what we can achieve. However, during the processing of this worker thread, other worker threads can't work at all (because the caller of do_item_flush_expired has locked cache_lock). There may be a lot of data in the LRU queue, and the process of overdue processing may be very long. Other worker threads are totally unacceptable.

The author of memcached must be aware of this problem, so he wrote a strange do_item_flush_expired function to accelerate it. Do_item_flush_expired deletes only a few special items. How to use the special method will be explained in the following code comments. For many other items, memcached handles them lazily. Only when the worker thread attempts to access the item does it detect whether the item has been set to expire. In fact, you can detect whether the item is expired without any settings, using the settings.oldest_live variable. This laziness is the same as the first item that failed to expire.


Now let's look at the do_item_flush_expired function and see the special item.

  1. void do_item_flush_expired(void) {  
  2.     int i;  
  3.     item *iter, *next;  
  4.     if (settings.oldest_live == 0)  
  5.         return;  
  6.     for (i = 0; i < LARGEST_ID; i++) {  
  7.         for (iter = heads[i]; iter != NULL; iter = next) {  
  8.             //ITER - > time = 0 is lru crawler item, which is ignored directly  
  9.             //In general, ITER - > time is less than settings.oldest_live. But in this case  
  10.             //It's possible that ITER - > time >= settings.oldest_live: > worker1 receives  
  11.             //flush_all command and assign settings.oldest_live to current_time-1.  
  12.             //The worker1 thread is called by worker2 before it has time to call the item_flush_expired function.  
  13.             //The cpu is preempted, and worker2 inserts an item into the lru queue. time of this item  
  14.             //Members will satisfy ITER - > time >= settings.oldest_live  
  15.             if (iter->time != 0 && iter->time >= settings.oldest_live) {  
  16.                 next = iter->next;  
  17.                 if ((iter->it_flags & ITEM_SLABBED) == 0) {  
  18.                     //Although nolock is called, item_flush_expired, the caller of this function  
  19.                     //The cache_lock has been locked before calling this function.  
  20.                     do_item_unlink_nolock(iter, hash(ITEM_key(iter), iter->nkey));  
  21.                 }  
  22.             } else {  
  23.                 //Because items in the lru queue are sorted in descending order of time, when there is a time member of an item  
  24.                 //Less than settings.oldest_live, the rest of the item s do not need to be compared.  
  25.                 break;  
  26.             }  
  27.         }  
  28.     }  
  29. }  


Laziness delete:

Now read item's lazy deletion. Note the comments in the code.

  1. item *do_item_get(const char *key, const size_t nkey, const uint32_t hv) {  
  2.     item *it = assoc_find(key, nkey, hv);  
  3.     ...  
  4.   
  5.     if (it != NULL) {  
  6.         //settings.oldest_live initialization value is 0  
  7.         //Check whether the user has used the flush_all command and delete all item s.  
  8.         //It - > time <= settings.oldest_live indicates when the user uses the flush_all command  
  9.         //The item already exists. Then the item is to be deleted.  
  10.         //The flush_all command can have parameters to set all item s at some point in the future  
  11.         //To expire, settings.oldest_live is a flush_all received by a worker.  
  12.         //The command takes a long time, so you need to judge settings.oldest_live <= current_time  
  13.         if (settings.oldest_live != 0 && settings.oldest_live <= current_time &&  
  14.             it->time <= settings.oldest_live) {  
  15.             do_item_unlink(it, hv);  
  16.             do_item_remove(it);  
  17.             it = NULL;  
  18.    
  19.         } else if (it->exptime != 0 && it->exptime <= current_time) {//The item has expired  
  20.             do_item_unlink(it, hv);//The number of references will be reduced by one.  
  21.             do_item_remove(it);//Subtract the number of references by one, and delete if the number of references equals zero  
  22.             it = NULL;  
  23.   
  24.         } else {  
  25.             it->it_flags |= ITEM_FETCHED;  
  26.         }  
  27.     }  
  28.   
  29.   
  30.     return it;  
  31. }  

As you can see, after finding an item, you need to check whether it has expired. If it fails, delete it.

In addition to the do_item_get function, the do_item_alloc function also handles expired items. The do_item_alloc function does not delete the expired item, but takes it for its own use. Because the function of this function is to apply for an item, if an item expires, it will occupy the memory of the item directly. Let's take a look at the code.

  1. item *do_item_alloc(char *key, const size_t nkey, const int flags,  
  2.                     const rel_time_t exptime, const int nbytes,  
  3.                     const uint32_t cur_hv) {  
  4.     uint8_t nsuffix;  
  5.     item *it = NULL;  
  6.     char suffix[40];  
  7.     //The total space required to store this item  
  8.     size_t ntotal = item_make_header(nkey + 1, flags, nbytes, suffix, &nsuffix);  
  9.     if (settings.use_cas) {  
  10.         ntotal += sizeof(uint64_t);  
  11.     }  
  12.   
  13.     //Judging which slab to belong to according to size  
  14.     unsigned int id = slabs_clsid(ntotal);  
  15.   
  16.     /* do a quick check if we have any expired items in the tail.. */  
  17.     int tries = 5;  
  18.     item *search;  
  19.     item *next_it;  
  20.     rel_time_t oldest_live = settings.oldest_live;  
  21.   
  22.     search = tails[id];  
  23.     for (; tries > 0 && search != NULL; tries--, search=next_it) {  
  24.         next_it = search->prev;  
  25.         ...  
  26.   
  27.         if (refcount_incr(&search->refcount) != 2) {//References, and other threads are referencing, so you can't override this item  
  28.             //Refresh the access time of this item and its location in the LRU queue  
  29.             do_item_update_nolock(search);  
  30.             tries++;  
  31.             refcount_decr(&search->refcount);  
  32.             //At this point, the number of references >= 2  
  33.       
  34.             continue;  
  35.         }  
  36.   
  37.         //The refcount of the item pointed to by search is equal to 2, which means that the item at this time is excluded from the worker.  
  38.         //There are no other worker threads indexing it outside the thread. You can safely release and reuse this item  
  39.           
  40.          //Because this loop starts at the back of the lru list. So from the beginning, search points to  
  41.          //The least commonly used item, if the item has not expired. So others are more common than that.  
  42.         //Don't delete item s (even if they expire). Only memory can be requested from slabs at this time  
  43.         if ((search->exptime != 0 && search->exptime < current_time)  
  44.             || (search->time <= oldest_live && oldest_live <= current_time)) {  
  45.             //The item that search points to is an expired item that can be used  
  46.             it = search;  
  47.             //Recalculate the memory allocated by this slabclass_t  
  48.             //Directly occupying old item s requires recalculation  
  49.             slabs_adjust_mem_requested(it->slabs_clsid, ITEM_ntotal(it), ntotal);  
  50.             do_item_unlink_nolock(it, hv);//Delete from hash and lru lists  
  51.             /* Initialize the item block: */  
  52.             it->slabs_clsid = 0;  
  53.         }  
  54.           
  55.   
  56.         //Reference count minus one. At this point, the item is no longer indexed by any worker threads, and the hash table is  
  57.         //No longer indexed  
  58.         refcount_decr(&search->refcount);  
  59.         break;  
  60.     }  
  61.   
  62.     ...  
  63.     return it;  
  64. }  
  65.   
  66.   
  67. //This function is called when the new item takes over the old item directly.  
  68. void slabs_adjust_mem_requested(unsigned int id, size_t old, size_t ntotal)  
  69. {  
  70.     pthread_mutex_lock(&slabs_lock);  
  71.     slabclass_t *p;  
  72.     if (id < POWER_SMALLEST || id > power_largest) {  
  73.         fprintf(stderr, "Internal error! Invalid slab class\n");  
  74.         abort();  
  75.     }  
  76.   
  77.     p = &slabclass[id];  
  78.     //Recalculate the memory allocated by this slabclass_t  
  79.     p->requested = p->requested - old + ntotal;  
  80.     pthread_mutex_unlock(&slabs_lock);  
  81. }  

The flush_all command can have time parameters. This time, like other times, ranges from 1 to REALTIME_MAXDELTA(30 days). If the command is flush_all 100, then all items expire after 99 seconds. The settings.oldest_live value is current_time+100-1, and the do_item_flush_expired function is useless (it will never be preempted for 99 seconds). It is for this reason that we need to add the judgment of settings.oldest_live<= current_time in do_item_get to prevent premature deletion of items.

There is obviously a bug here. Suppose client A submits the flush_all10 command to the server. After five seconds, client B submits the command flush_all100 to the server. Client A's commands will fail and do nothing.



LRU crawler:

As mentioned earlier, memcached is lazy to delete expired items. So even if the user uses the flush_all command on the client to expire all items, these items still occupy the hash table and LRU queues and are not returned to the slab allocator.


LRU crawler threads:

Is there any way to force the removal of these expired item s, no longer occupy the hash table and LRU queue space, and return them to slabs? Of course there is. memcached provides LRU crawlers for this purpose.

To use LRU crawlers, you must use the lru_crawler command on the client side. The memcached server processes according to the specific command parameters.

Memcached is a special thread responsible for clearing these expired item s. This article will call this thread LRU crawler thread. By default, memcached does not start this thread, but you can start this thread by adding the parameter - o lru_crawler when starting memcached. It can also be started by client command. Even if the LRU crawler thread is started, it will not work. Additional commands are required to indicate which LRU queue to clear. Now let's see what parameters lru_crawler has.



LRU crawler command:


  • Lru_crawler <enable | disable> starts or stops an LRU crawler thread. At most one LRU crawler thread at any time. This command assigns settings.lru_crawler to true or false
  • Lru_crawler crawl <classid, classid, CLassID | all> can use lists like 2,3,6 to indicate which LRU queue to clear. You can also use all to process all LRU queues
  • Lru_crawler sleep < microseconds > LRU crawler thread will occupy lock when cleaning item, which will hinder the normal business of worker thread. So LRU crawlers need to sleep from time to time when they are dealing with it. The default dormancy time is 100 microseconds. This command assigns settings.lru_crawler_sleep
  • Lru_crawler to crawl <32u> An LRU queue may have many expired items. If it is checked and cleaned up all the time, it will inevitably hinder the normal business of worker threads. This parameter is used to specify how many items are checked for each LRU queue at most. The default value is 0, so it won't work unless specified. This command assigns settings.lru_crawler_tocrawl

If you want to start an LRU crawler to actively delete expired items, you need to do this: First start an LRU crawler thread using the lru_crawler enable command. Then use the lru_crawler to crawl num command to determine that each LRU queue checks num-1 items at most. Finally, use the command lru_crawler Crawl < classid, classid, CLassID | all > specifies the LRU queue to be processed. lru_crawler sleep may not be set, but if it is to be set, it can be set before the lru_crawler crawl command.


Start the LRU crawler thread:

Now let's see how LRU reptiles work. Let's first look at what global variables memcached defines for LRU crawlers.

  1. static volatile int do_run_lru_crawler_thread = 0;  
  2. static int lru_crawler_initialized = 0;  
  3. static pthread_mutex_t lru_crawler_lock = PTHREAD_MUTEX_INITIALIZER;  
  4. static pthread_cond_t  lru_crawler_cond = PTHREAD_COND_INITIALIZER;  
  5.   
  6.   
  7. int init_lru_crawler(void) {//The main function calls the function  
  8.     if (lru_crawler_initialized == 0) {  
  9.         if (pthread_cond_init(&lru_crawler_cond, NULL) != 0) {  
  10.             fprintf(stderr, "Can't initialize lru crawler condition\n");  
  11.             return -1;  
  12.         }  
  13.         pthread_mutex_init(&lru_crawler_lock, NULL);  
  14.         lru_crawler_initialized = 1;  
  15.     }  
  16.     return 0;  
  17. }  

The code is relatively simple, let's not talk about it here. Let's look at the lru_crawler enable and disable commands. The enable command will start an LRU crawler thread, and disable will stop the LRU crawler thread, not call pthread_exit directly to stop the thread. The pthread_exit function is a dangerous function and should not appear in the code.

  1. static pthread_t item_crawler_tid;  
  2.   
  3. //The worker thread calls this function when it receives the "lru_crawler enable" command  
  4. //This function is also called when memcached is started with the - o lru_crawler parameter  
  5. int start_item_crawler_thread(void) {  
  6.     int ret;  
  7.   
  8.     //You can see the pthread_join function in the stop_item_crawler_thread function  
  9.     //settings.lru_crawler is set to false only after pthread_join returns.  
  10.     //So there won't be two crawler threads at the same time  
  11.     if (settings.lru_crawler)  
  12.         return -1;  
  13.       
  14.     pthread_mutex_lock(&lru_crawler_lock);  
  15.     do_run_lru_crawler_thread = 1;  
  16.     settings.lru_crawler = true;  
  17.     //Create an LRU crawler thread with the thread function item_crawler_thread. LRU crawler threads are entering  
  18.     //After the item_crawler_thread function, pthread_cond_wait is called, waiting for the worker thread to specify  
  19.     //LRU queues to be processed  
  20.     if ((ret = pthread_create(&item_crawler_tid, NULL,  
  21.         item_crawler_thread, NULL)) != 0) {  
  22.         fprintf(stderr, "Can't create LRU crawler thread: %s\n",  
  23.             strerror(ret));  
  24.         pthread_mutex_unlock(&lru_crawler_lock);  
  25.         return -1;  
  26.     }  
  27.     pthread_mutex_unlock(&lru_crawler_lock);  
  28.   
  29.     return 0;  
  30. }  
  31.   
  32.   
  33. //The worker thread executes this function when it receives the command "lru_crawler disable"  
  34. int stop_item_crawler_thread(void) {  
  35.     int ret;  
  36.     pthread_mutex_lock(&lru_crawler_lock);  
  37.     do_run_lru_crawler_thread = 0;//Stop LRU threads  
  38.     //LRU crawler threads may sleep on wait condition variables and need to wake up to stop LRU crawler threads  
  39.     pthread_cond_signal(&lru_crawler_cond);  
  40.     pthread_mutex_unlock(&lru_crawler_lock);  
  41.     if ((ret = pthread_join(item_crawler_tid, NULL)) != 0) {  
  42.         fprintf(stderr, "Failed to stop LRU crawler thread: %s\n", strerror(ret));  
  43.         return -1;  
  44.     }  
  45.     settings.lru_crawler = false;  
  46.     return 0;  
  47. }  

You can see that the worker thread starts an LRU crawler thread after receiving the "lru_crawler enable" command. This LRU crawler thread has not yet performed the task, because no task has been specified. The command "lru_crawler to crawlnum" does not start a task. For this command, the worker thread simply assigns settings.lru_crawler_tocrawl to num.


Clear the invalid item:

The command "lru_crawler crawl < classid, classid, CLassID | all >" is the specified task. This command specifies which LRU queue to clean up. If all is used, then all RU queues are cleaned up.

Before looking at memcached's cleanup code, consider one question: How to clean up an LRU queue?

The most intuitive approach is to first lock (lock cache_lock) and then traverse an entire LRU queue. Judge each item in the LRU queue directly. Obviously, there is a problem with this method. If memcached has a large number of items, traversing an LRU queue will take too long. This hinders the normal business of worker threads. Of course, we can consider using divide and conquer method, only a few items at a time, many times, and ultimately achieve the goal of processing the entire LRU queue. However, LRU queues are linked lists and do not support random access. Processing an item in the middle of the queue requires sequential access from the head or tail of the list, with time complexity of O(n).


Pseudo item:

memcached uses a clever method to achieve random access. It inserts a pseudo-item at the end of the LRU queue, and then drives the pseudo-item to the head of the queue, one at a time.

This pseudoitem is a global variable, and LRU crawler threads can access the pseudoitem directly without traversing the head or tail of the LRU queue. With the next and prev pointers of this pseudoitem, you can access the real item. Thus, the LRU crawler thread can directly access an item in the middle of the LRU queue without traversing it.

Let's look at the lru_crawler_crawl function, where memcached inserts pseudoitems into the end of the LRU queue. This function is called when the worker thread receives the lru_crawler crawl < classid, classid, CLassID | all > command. Because users may require LRU crawler threads to clean up expired items from multiple LRU queues, a pseudo-item array is required. The size of pseudoitem arrays is equal to the number of LRU queues, and they correspond one to one.

  1. //This structure is similar to the item structure. It is a pseudo-item structure for LRU reptiles.  
  2. typedef struct {  
  3.     struct _stritem *next;  
  4.     struct _stritem *prev;  
  5.     struct _stritem *h_next;    /* hash chain next */  
  6.     rel_time_t      time;       /* least recent access */  
  7.     rel_time_t      exptime;    /* expire time */  
  8.     int             nbytes;     /* size of data */  
  9.     unsigned short  refcount;  
  10.     uint8_t         nsuffix;    /* length of flags-and-length string */  
  11.     uint8_t         it_flags;   /* ITEM_* above */  
  12.     uint8_t         slabs_clsid;/* which slab class we're in */  
  13.     uint8_t         nkey;       /* key length, w/terminating null and padding */  
  14.     uint32_t        remaining;  /* Max keys to crawl per slab per invocation */  
  15. } crawler;  
  16.   
  17.   
  18.   
  19. static crawler crawlers[LARGEST_ID];  
  20. static int crawler_count = 0;//How many LRU queues will be processed for this task  
  21.   
  22.   
  23. //When the client uses the command lru_crawler crawl < classid, classid, CLassID | all>,  
  24. //The worker thread calls the function and takes the second parameter of the command as the parameter of the function.  
  25. enum crawler_result_type lru_crawler_crawl(char *slabs) {  
  26.     char *b = NULL;  
  27.     uint32_t sid = 0;  
  28.     uint8_t tocrawl[POWER_LARGEST];  
  29.   
  30.     //When the LRU crawler thread cleans up, it locks lru_crawler_lock. Until all is done  
  31.     //The clean-up task will be unlocked. So before the client's previous clean-up task is over, no  
  32.     //Submit another clean-up task.  
  33.     if (pthread_mutex_trylock(&lru_crawler_lock) != 0) {  
  34.         return CRAWLER_RUNNING;  
  35.     }  
  36.     pthread_mutex_lock(&cache_lock);  
  37.   
  38.     //Parse commands. If the command requires cleaning up an LRU queue, it's in the tocrawl array  
  39.     //The corresponding element assigns 1 as a mark  
  40.     if (strcmp(slabs, "all") == 0) {//Processing all lru queues  
  41.         for (sid = 0; sid < LARGEST_ID; sid++) {  
  42.             tocrawl[sid] = 1;  
  43.         }  
  44.     } else {  
  45.         for (char *p = strtok_r(slabs, ",", &b);  
  46.              p != NULL;  
  47.              p = strtok_r(NULL, ",", &b)) {  
  48.   
  49.             //Resolve one sid after another  
  50.             if (!safe_strtoul(p, &sid) || sid < POWER_SMALLEST  
  51.                     || sid > POWER_LARGEST) {//sid cross-border  
  52.                 pthread_mutex_unlock(&cache_lock);  
  53.                 pthread_mutex_unlock(&lru_crawler_lock);  
  54.                 return CRAWLER_BADCLASS;  
  55.             }  
  56.             tocrawl[sid] = 1;  
  57.         }  
  58.     }  
  59.   
  60.     //crawlers are an array of pseudo item types. If the user wants to clean up an LRU queue, then  
  61.     //Insert a pseudo-item in this LRU queue  
  62.     for (sid = 0; sid < LARGEST_ID; sid++) {  
  63.         if (tocrawl[sid] != 0 && tails[sid] != NULL) {  
  64.   
  65.             //For pseudo-item and real item, you can use the values of nkey and time to distinguish between them.  
  66.             crawlers[sid].nbytes = 0;  
  67.             crawlers[sid].nkey = 0;  
  68.             crawlers[sid].it_flags = 1; /* For a crawler, this means enabled. */  
  69.             crawlers[sid].next = 0;  
  70.             crawlers[sid].prev = 0;  
  71.             crawlers[sid].time = 0;  
  72.             crawlers[sid].remaining = settings.lru_crawler_tocrawl;  
  73.             crawlers[sid].slabs_clsid = sid;  
  74.             //Insert this pseudo item at the end of the corresponding lru queue  
  75.             crawler_link_q((item *)&crawlers[sid]);  
  76.             crawler_count++;//The number of LRU queues to be processed plus one  
  77.         }  
  78.     }  
  79.     pthread_mutex_unlock(&cache_lock);  
  80.     //When you have a task, wake up the LRU crawler thread and let it perform the cleanup task  
  81.     pthread_cond_signal(&lru_crawler_cond);  
  82.     STATS_LOCK();  
  83.     stats.lru_crawler_running = true;  
  84.     STATS_UNLOCK();  
  85.     pthread_mutex_unlock(&lru_crawler_lock);  
  86.     return CRAWLER_OK;  
  87. }  


Now let's look at how pseudo-item moves forward in the LRU queue. Let's first look at a pseudo-item forward diagram.

        


As can be seen from the figure above, pseudoitem advances by exchanging positions with the precursor nodes. If the pseudoitem is the head node of the LRU queue, then move the pseudoitem out of the LRU queue. The function crawler_crawl_q completes this exchange operation and returns the precursor node of the pseudo item before the exchange (which, of course, becomes the precursor node of the pseudo item after the exchange). If the pseudoitem is at the head of the LRU queue, it returns to NULL (there are no precursor nodes at this time). The pointers in the crawler_crawl_q function are flying all over the sky, so no code is posted here.

The above figure, although pseudoitem traverses the LRU queue, does not delete an item. The first is to look good, and the second is to traverse the LRU queue without necessarily deleting items (items will not be deleted if they expire).


Clean up item:

As mentioned earlier, the command lru_crawler to crawl num can be used to specify that each LRU queue can only check num-1 items at most. Look clearly, it's the number of checks, not deletions, and it's num-1. First, call the item_crawler_evaluate function to check if an item is expired, and if so, delete it. If the num-1 is checked and the pseudo-item has not reached the head of the LRU queue, then the pseudo-item is deleted from the LRU queue directly. Let's look at the item_crawler_thread function.

  1. static void *item_crawler_thread(void *arg) {  
  2.     int i;  
  3.   
  4.     pthread_mutex_lock(&lru_crawler_lock);  
  5.     while (do_run_lru_crawler_thread) {  
  6.     //The lru_crawler_crawl function and stop_item_crawler_thread function will signal this conditional variable.  
  7.     pthread_cond_wait(&lru_crawler_cond, &lru_crawler_lock);  
  8.   
  9.     while (crawler_count) {//crawler_count indicates how many LRU queues to process  
  10.         item *search = NULL;  
  11.         void *hold_lock = NULL;  
  12.   
  13.         for (i = 0; i < LARGEST_ID; i++) {  
  14.             if (crawlers[i].it_flags != 1) {  
  15.                 continue;  
  16.             }  
  17.             pthread_mutex_lock(&cache_lock);  
  18.             //Returns the precursor node of crawlers[i] and exchanges the location of crawlers[i] and precursor nodes  
  19.             search = crawler_crawl_q((item *)&crawlers[i]);  
  20.             if (search == NULL || //crawlers[i] is the header node, there is no precursor node  
  21.                 //The value of remaining is settings.lru_crawler_tocrawl. Start LRU each time  
  22.                 //The crawler thread checks the number of item s per lru queue.  
  23.                 (crawlers[i].remaining && --crawlers[i].remaining < 1)) {  
  24.   
  25.                 //Check enough times to exit checking the lru queue  
  26.                 crawlers[i].it_flags = 0;  
  27.                 crawler_count--;//Clean up an LRU queue and subtract the number of tasks by one  
  28.                 crawler_unlink_q((item *)&crawlers[i]);//Delete this pseudo item from the LRU queue  
  29.                 pthread_mutex_unlock(&cache_lock);  
  30.                 continue;  
  31.             }  
  32.             uint32_t hv = hash(ITEM_key(search), search->nkey);  
  33.             //Attempt to lock hash table segment level locks that control this item  
  34.             if ((hold_lock = item_trylock(hv)) == NULL) {  
  35.                 pthread_mutex_unlock(&cache_lock);  
  36.                 continue;  
  37.             }  
  38.   
  39.   
  40.             //At this point, other worker threads are referencing this item  
  41.             if (refcount_incr(&search->refcount) != 2) {  
  42.                 refcount_decr(&search->refcount);//The lru crawler thread discards referencing the item  
  43.                 if (hold_lock)  
  44.                     item_trylock_unlock(hold_lock);  
  45.                 pthread_mutex_unlock(&cache_lock);  
  46.                 continue;  
  47.             }  
  48.   
  49.             //If the item expires, delete the item  
  50.             item_crawler_evaluate(search, hv, i);  
  51.   
  52.             if (hold_lock)  
  53.                 item_trylock_unlock(hold_lock);  
  54.             pthread_mutex_unlock(&cache_lock);  
  55.   
  56.             //The lru crawler can't stop crawling the lru queue, which can hinder the normal business of worker threads.  
  57.             //So you need to suspend the lru crawler thread for a period of time. In the default settings, it hibernates for 100 microseconds  
  58.             if (settings.lru_crawler_sleep)  
  59.                 usleep(settings.lru_crawler_sleep);//microsecond  
  60.         }  
  61.     }  
  62.     STATS_LOCK();  
  63.     stats.lru_crawler_running = false;  
  64.     STATS_UNLOCK();  
  65.     }  
  66.     pthread_mutex_unlock(&lru_crawler_lock);  
  67.   
  68.     return NULL;  
  69. }  
  70.   
  71.   
  72.  //If the item expires, delete it  
  73. static void item_crawler_evaluate(item *search, uint32_t hv, int i) {  
  74.     rel_time_t oldest_live = settings.oldest_live;  
  75.   
  76.     //The item's exptime timestamp has expired  
  77.     if ((search->exptime != 0 && search->exptime < current_time)  
  78.         //Because the client sends the flush_all command, the item fails.  
  79.         || (search->time <= oldest_live && oldest_live <= current_time)) {  
  80.         itemstats[i].crawler_reclaimed++;  
  81.   
  82.         if ((search->it_flags & ITEM_FETCHED) == 0) {  
  83.             itemstats[i].expired_unfetched++;  
  84.         }  
  85.   
  86.         //Delete item from the LRU queue  
  87.         do_item_unlink_nolock(search, hv);  
  88.         do_item_remove(search);  
  89.         assert(search->slabs_clsid == 0);  
  90.     } else {  
  91.         refcount_decr(&search->refcount);  
  92.     }  
  93. }  


Real LRU phasing out:

Although the word LRU has been used many times before in this article, the function names in memcached code also use the LRU prefix, especially the lru_crawler command. But in fact, it has nothing to do with the elimination of LRU!!

Be deceived and scold: & & *%... %%,%... #%@%...


Readers can recall operating system LRU inside algorithm . The item s deleted in this article are expired and should have been deleted. Overdue still dominate the position, a bit like the Mao pit not shit. The LRU algorithm in the operating system is kicked because of insufficient resources, and the kicked is helpless. It's not the same, so it's not LRU.

Where does the RU of memcached reflect? do_item_alloc function!! Previous blogs have always mentioned this godlike function, but never given a complete version. Of course, the full version will not be given here. Because there are still some things in this function that can't be explained to readers for the time being. Now it is estimated that readers will appreciate it.< How to read memcached source code > Written in: There is too much correlation between memcached modules.

  1. item *do_item_alloc(char *key, const size_t nkey, const int flags,  
  2.                     const rel_time_t exptime, const int nbytes,  
  3.                     const uint32_t cur_hv) {  
  4.     uint8_t nsuffix;  
  5.     item *it = NULL;  
  6.     char suffix[40];  
  7.     //The total space required to store this item  
  8.     size_t ntotal = item_make_header(nkey + 1, flags, nbytes, suffix, &nsuffix);  
  9.     if (settings.use_cas) {  
  10.         ntotal += sizeof(uint64_t);  
  11.     }  
  12.   
  13.     //Judging which slab to belong to according to size  
  14.     unsigned int id = slabs_clsid(ntotal);  
  15.   
  16.     int tries = 5;  
  17.     item *search;  
  18.     item *next_it;  
  19.     rel_time_t oldest_live = settings.oldest_live;  
  20.     search = tails[id];  
  21.   
  22.     for (; tries > 0 && search != NULL; tries--, search=next_it) {  
  23.         next_it = search->prev;  
  24.        
  25.         uint32_t hv = hash(ITEM_key(search), search->nkey);  
  26.           
  27.         /* Now see if the item is refcount locked */  
  28.         if (refcount_incr(&search->refcount) != 2) {//Reference number >=3  
  29.             refcount_decr(&search->refcount);  
  30.             continue;  
  31.         }  
  32.   
  33.         //The refcount of the item pointed to by search is equal to 2, which means that the item at this time is excluded from the worker.  
  34.         //There are no other worker threads indexing it outside the thread. You can safely release and reuse this item  
  35.           
  36.          //Because this loop starts at the back of the lru list. So from the beginning, search points to  
  37.          //The least commonly used item, if the item has not expired. So others are more common than that.  
  38.         //Don't delete item s (even if they expire). Only memory can be requested from slabs at this time  
  39.         if ((search->exptime != 0 && search->exptime < current_time)  
  40.             || (search->time <= oldest_live && oldest_live <= current_time)) {  
  41.   
  42.             ..  
  43.         } else if ((it = slabs_alloc(ntotal, id)) == NULL) {//Failed to apply for memory  
  44.             //At this point, the expired item was not found, and the application memory failed again. It seems that it can only be used.  
  45.             //LRU phases out an item (even if the item does not expire)  
  46.               
  47.             if (settings.evict_to_free == 0) {//Set no LRU elimination item  
  48.                 //At this point, you can only reply to the client for errors.  
  49.                 itemstats[id].outofmemory++;  
  50.             } else {  
  51.                 //Even if an item exptime member is set to never timeout (0), it will still be kicked.  
  52.       
  53.                 it = search;  
  54.                 //Recalculate the memory allocated by this slabclass_t  
  55.                 //Directly occupying old item s requires recalculation  
  56.                 slabs_adjust_mem_requested(it->slabs_clsid, ITEM_ntotal(it), ntotal);  
  57.                 do_item_unlink_nolock(it, hv);//Delete from hash and lru lists  
  58.                 /* Initialize the item block: */  
  59.                 it->slabs_clsid = 0;  
  60.   
  61.             }  
  62.         }  
  63.   
  64.         //Reference count minus one. At this point, the item is no longer indexed by any worker threads, and the hash table is  
  65.         //No longer indexed  
  66.         refcount_decr(&search->refcount);  
  67.         break;  
  68.     }  
  69.   
  70.     ...  
  71.   
  72.     return it;  
  73. }  

Posted by benn600 on Tue, 26 Mar 2019 06:51:30 -0700