Yesterday, we introduced the key components of resource plug-in mechanism in k8s. Today, we will continue to see how each component communicates and the key design behind event handling in k8s

1.PluginManager

PluginManager is a top-level component, which contains the key components in the previous article, coordinates its internal data flow, and provides specific controllers for different plug-ins

1.1 core data structure

The core structure is actually designed according to the data flow. First, a perception plug-in, the desiredStateOfWorldPopulator, is needed to sense the creation or deletion of back-end services. Then, the perceived events are added to the desiredStateOfWorld's expectation state cache. The adjuster is responsible for the underlying registration and logoff, and the results are stored in the actual stateofworld Interstate cache

type pluginManager struct {
	// Plug in awareness
	desiredStateOfWorldPopulator *pluginwatcher.Watcher

    // Coordinator plug in
	reconciler reconciler.Reconciler

    // Actual state cache
	actualStateOfWorld cache.ActualStateOfWorld
	// Expected state cache
	desiredStateOfWorld cache.DesiredStateOfWorld
}

1.2 initialization

In initialization, both dsw and asw will be given to the adjuster for event awareness and updating the corresponding cache

func NewPluginManager(
	sockDir string,
	recorder record.EventRecorder) PluginManager {
	asw := cache.NewActualStateOfWorld()
	dsw := cache.NewDesiredStateOfWorld()
    // Here, the expected state cache and the actual state cache will be handed over to the reconciler
	reconciler := reconciler.NewReconciler(
		operationexecutor.NewOperationExecutor(
			operationexecutor.NewOperationGenerator(
				recorder,
			),
		),
		loopSleepDuration,
		dsw,
		asw,
	)

	pm := &pluginManager{
        //  Start a watcher and store the desired state cache of dsw, then the reconciler can sense the new state through dsw
		desiredStateOfWorldPopulator: pluginwatcher.NewWatcher(
			sockDir,
			dsw,
		),
		reconciler:          reconciler,
		desiredStateOfWorld: dsw,
		actualStateOfWorld:  asw,
	}
	return pm
}

1.3 starting plug-in manager

When the plug-in manager is started, it is actually to start the internal desiredStateOfWorldPopulator, which will talk about the events sensed by the watcher, and constantly modify its own internal cache, so that the reconciler can continuously call the grpc through the expected state cache to meet the expected state

func (pm *pluginManager) Run(sourcesReady config.SourcesReady, stopCh <-chan struct{}) {
	defer runtime.HandleCrash()

    // Running the expected state cache is mainly to modify its own cache through the events perceived by the watcher
    // The subsequent reconciler will acquire periodically
	pm.desiredStateOfWorldPopulator.Start(stopCh)
	klog.V(2).Infof("The desired_state_of_world populator (plugin watcher) starts")

	klog.Infof("Starting Kubelet Plugin Manager")
    // Periodic operation verification data
	go pm.reconciler.Run(stopCh)

	metrics.Register(pm.actualStateOfWorld, pm.desiredStateOfWorld)
	<-stopCh
	klog.Infof("Shutting down Kubelet Plugin Manager")
}

1.4 controller registration

In fact, the controller mainly refers to the reconciler. After generating the corresponding event by comparing the difference between the expected cache and the actual cache, what is the subsequent processing flow for this type of plug-in, such as the specific grpc interface for registration / logoff and the processing mechanism for the corresponding plug-in type

func (pm *pluginManager) AddHandler(pluginType string, handler cache.PluginHandler) {
	pm.reconciler.AddHandler(pluginType, handler)
}

1.5 CSI and general equipment

There are two types of plug-in controllers registered in the current kubelet, CSI and deviceplugin. You can also know the general meaning from the name

	kl.pluginManager.AddHandler(pluginwatcherapi.CSIPlugin, plugincache.PluginHandler(csi.PluginHandler))
	kl.pluginManager.AddHandler(pluginwatcherapi.DevicePlugin, kl.containerManager.GetPluginRegistrationHandler())

2. PluginHandler

Here we only introduce the core implementation mechanism of DevicePlugin

2.1 Endpoint

Endpoint actually refers to a service that provides extended resources. In the previous reconciler, the address of its corresponding grpc service will be obtained, and then it will directly call grpc for communication

Endpoint needs to sense the change of the corresponding resource device, and notify the current

2.2 Manager

Manager is mainly responsible for the implementation of the real Register/UnRegister of the back-end. It will create an Endpoint for each Device internally and collect the information reported by the resource service provided by the back-end. Finally, it will send the corresponding information to kubelet, and then kubelet will transfer the information to APIServer when it is responsible for updating the node information

2.3 Checkpoint

Checkpoint mechanism is commonly used in many systems. It is mainly used to serialize and store the data in memory to the local disk periodically. During subsequent recovery, the previous data will be reloaded through the disk, so as to realize the rapid recovery of memory resources

The overall implementation process of expanding resources is like this. From how to perceive data, register resource services, obtain resource information of resource services, and finally report to kubelet, and land on local disk at the same time, the whole process of detecting resources from perception to reporting is realized. The main deficiency is the description of resource entities, which leads to resource allocation and reporting There are relatively large scalability restrictions on the submission of resources. For example, to realize the fine resource allocation extension, it is not possible to achieve it

k8s source reading e-book address: https://www.yuque.com/baxiaoshi/tyado3

>Wechat: baxiaoshi2020 >Pay attention to the bulletin number to read more source code analysis articles >More articles www.sreguide.com >This article is based on the platform of blog one article multiple sending OpenWrite Release