Linux CPU context switching performance monitoring

Keywords: Linux

Context switching

In short, it is to save the resources of the register and program counter of the previous task, then load the register and program counter of the next task, and finally jump to the address of the new task.

The context switching of CPU can be divided into:

  • System call context switching
  • Process context switching
  • Thread context switching
  • Interrupt context switching

1. System call context switching

Processes can enter the kernel state from the user state through system calls. There is context switching between user space and kernel space in this process. And it needs to switch back to the user state after the system call, so there will be two context switches. This process has no process switching, also known as "privileged mode switching".

2 process context switching

The process scheduling is completed by the kernel, so the process context switching only exists in the kernel space. The resources to be switched include user space resources such as virtual memory, stack and global variables, as well as kernel space resources such as kernel stack and register.

The switching of virtual memory must involve the operation of TLB fast table in MMU, which will affect the memory access speed. Frequent process context switching may consume a lot of CPU time on resource protection and recovery, affecting CPU utilization.

3 thread context switching

All threads in a process share resources such as virtual memory and global variables. Therefore, when context switching occurs between two threads in the same process, they only need to switch their own private data, such as stacks and registers. However, if the two switched threads are not in the same process, their context switching is the same as the process context switching.

4 interrupt context switching

When the hardware interrupt signal is generated, the running process will be interrupted and enter the interrupt handler in kernel state. At this time, the interrupt handler has no right to operate the user space memory of the current user state process. Therefore, the context switching at this time does not involve the resource switching in user space, but only includes the resources necessary for the execution of the interrupt handler, including registers, kernel stack Interrupt parameters, etc.

vmstat

Frequent context switching causes system performance degradation. At this time, you can use the command vmstat to query the context switching of the system.

vmstat - Report virtual memory statistics.
vmstat  reports  information  about processes, memory, paging, block IO, traps, disks and cpu activity.
SYNOPSIS: vmstat [options] [delay [count]]

Common options:

  • -d, --disk - Report disk statistics
  • -p, --partition <device> - Detailed statistics about partition
  • -t -- output on time

Example:

 $ vmstat -t 5 2
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- -----timestamp-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st                 EDT
 2  0 202496 158400  83312 400100    0    2    29    11   68   95  1  1 98  0  0 2021-11-02 14:40:59
 0  0 202496 158400  83320 400100    0    0     0     2  303  473  0  1 99  0  0 2021-11-02 14:41:04

Definition of output parameters:

  • Process related
    r -- running or waiting for run time
    b -- number of processes in non interruptible sleep
  • Memory related
    swpd -- virtual memory used
    Free -- free memory remaining
    buffer and cache -- memory used as buff er and cache
  • Exchange space correlation
    si and so -- the amount of memory swapped in and out of disk per second. When si and so are large, it indicates that the system frequently uses the swap area, and it is necessary to determine whether the memory is sufficient.
  • I/O related
    bi and bo -- the number of data blocks read and written from the block device per second. Both represent I/O activity, and the load of disk I/O can be understood according to its size.
  • System related
    in -- number of interrupts per second
    cs -- the number of thread context switches per second. When this data is too large, it indicates that there is a problem with the synchronization mechanism of the thread

Test tool sysbench

Run with 10 threads for 5 minutes to simulate the problem of multi-threaded switching

sudo sysbench --threads=10 --time=300 --events=10000000 --test=threads run

Use vmstat to view:

 $ vmstat -t 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- -----timestamp-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st                 EDT
 9  0 238848 145288  60544 460356    0    2    30    12   72   50  1  1 98  0  0 2021-11-02 15:05:50
 7  0 238848 145288  60552 460384    0    0     0    36 6753 1235883 18 82  0  0  0 2021-11-02 15:05:51
 7  0 238848 145288  60552 460392    0    0     0     0 4971 1206581 18 81  1  0  0 2021-11-02 15:05:52
 9  0 238848 145288  60560 460384    0    0     0    12 6051 1141920 18 83  0  0  0 2021-11-02 15:05:53
 5  0 238848 145288  60560 460392    0    0     0     0 6480 1208340 17 83  1  0  0 2021-11-02 15:05:54
 8  0 238848 145288  60560 460392    0    0     0     0 6330 1159581 16 84  0  0  0 2021-11-02 15:05:55
 8  0 238848 145288  60560 460392    0    0     0     0 5007 1213778 19 81  0  0  0 2021-11-02 15:05:56
 7  0 238848 145288  60560 460392    0    0     0     0 6823 1139003 17 83  1  0  0 2021-11-02 15:05:57
^C 		# ctrl+c to stop

It is not difficult to see from the parameter cs that context switching is very frequent at this time.

Posted by RobinTibbs on Tue, 02 Nov 2021 14:07:31 -0700