Source Code Analysis of Kubernetes Replication Controller

Keywords: Kubernetes snapshot

Although in Kubernetes v1.2, Kubernetes introduced the Deployments feature, Deployment manages Pod s by creating ReplicaSets, which are considered the next generation of Replication Controller. But in fact, the difference between ReplicaSet and Replication Controller is simply the type of support that its Selector supports.

  • ReplicaSet supports both equality-based selector requirements and set-based selector requirements.
  • Replication Controller only supports equality-based selector requirements.

Deployments, of course, is useful in supporting users'rolling deployment needs. An analysis of Deployments will be followed by a separate blog post. In this article, we only look at Replication Controller.

> It needs to be clarified that the Replication Controller we are talking about here is the RC controller, not the RC Resource. > > This paper is based on the code of kubernetes v1.5 for analysis.

ReplicationManager

Replication Manager is the Replication Controller Controller Controller Controller object, which facilitates the distinction between the Replication Controller Resource API Object and the code. The following code is the structural definition of Replication Manager.

pkg/controller/replication/replication_controller.go:75

// ReplicationManager is responsible for synchronizing ReplicationController objects stored in the system with actual running pods.
type ReplicationManager struct {
    kubeClient clientset.Interface
    podControl controller.PodControlInterface

    // internalPodInformer is used to hold a personal informer.  If we're using
    // a normal shared informer, then the informer will be started for us.  If
    // we have a personal informer, we must start it ourselves.   If you start
    // the controller using NewReplicationManager(passing SharedInformer), this
    // will be null
    internalPodInformer cache.SharedIndexInformer

    // An rc is temporarily suspended after creating/deleting these many replicas.
    // It resumes normal action after observing the watch events for them.
    burstReplicas int
    // To allow injection of syncReplicationController for testing.
    syncHandler func(rcKey string) error

    // A TTLCache of pod creates/deletes each rc expects to see.
    expectations *controller.UIDTrackingControllerExpectations

    // A store of replication controllers, populated by the rcController
    rcStore cache.StoreToReplicationControllerLister
    // Watches changes to all replication controllers
    rcController *cache.Controller
    // A store of pods, populated by the podController
    podStore cache.StoreToPodLister
    // Watches changes to all pods
    podController cache.ControllerInterface
    // podStoreSynced returns true if the pod store has been synced at least once.
    // Added as a member to the struct to allow injection for testing.
    podStoreSynced func() bool

    lookupCache *controller.MatchingCache

    // Controllers that need to be synced
    queue workqueue.RateLimitingInterface

    // garbageCollectorEnabled denotes if the garbage collector is enabled. RC
    // manager behaves differently if GC is enabled.
    garbageCollectorEnabled bool
}

Emphasis is laid on the following objects:

  • podControl: Provides the operation interface of Create/Delete Pod.
  • Burst Replicas: The maximum number of concurrent allowed for each batch of Create/Delete Pods.
  • syncHandler: A function that actually executes Replica Sync.
  • Expect: Maintains the Uid Cache of the Pod in the expected state, and provides an interface to modify the Cache.
  • RcStore: Indexer of the Replication Controller Resource object, data provided and maintained by rcController.
  • rcController: Used for watching all Replication Controller Resources. The change from watches is updated to rcStore.
  • podStore: Pod's Indexer, data provided and maintained by podController.
  • podController: Used for watching all Pod Resource s. The change from watches is updated to the podStore.
  • Queue: The RC used to store sync is a RateLimit type queue.
  • lookupCache: A cache that provides Pod and RC matching information to improve query efficiency.

Where did Replication Controller start?

I read my blog: Internal Implementation Principle and Source Code Analysis of Kubernetes ResourceQuota Controller It also mentions how the controller manager started ResourceQuota Controller, as well as Replication Controller. When kube-controller-manager calls newController Initializers to initialize the controller, the startReplication Controller is registered to start the Replication Controller controller.

cmd/kube-controller-manager/app/controllermanager.go:224

func newControllerInitializers() map[string]InitFunc {
    controllers := map[string]InitFunc{}
    controllers["endpoint"] = startEndpointController
    controllers["replicationcontroller"] = startReplicationController
    controllers["podgc"] = startPodGCController
    controllers["resourcequota"] = startResourceQuotaController
    controllers["namespace"] = startNamespaceController
    controllers["serviceaccount"] = startServiceAccountController
    controllers["garbagecollector"] = startGarbageCollectorController
    controllers["daemonset"] = startDaemonSetController
    controllers["job"] = startJobController
    controllers["deployment"] = startDeploymentController
    controllers["replicaset"] = startReplicaSetController
    controllers["horizontalpodautoscaling"] = startHPAController
    controllers["disruption"] = startDisruptionController
    controllers["statefuleset"] = startStatefulSetController
    controllers["cronjob"] = startCronJobController
    controllers["certificatesigningrequests"] = startCSRController

    return controllers
}

The code continues to follow the startReplication Controller. Simply, start a goroutine, call the replication controller. NewReplication Manager to create a Replication Manager and execute the Run method to start working.

cmd/kube-controller-manager/app/core.go:55

func startReplicationController(ctx ControllerContext) (bool, error) {
    go replicationcontroller.NewReplicationManager(
        ctx.InformerFactory.Pods().Informer(),
        ctx.ClientBuilder.ClientOrDie("replication-controller"),
        ResyncPeriod(&ctx.Options),
        replicationcontroller.BurstReplicas,
        int(ctx.Options.LookupCacheSizeForRC),
        ctx.Options.EnableGarbageCollector,
    ).Run(int(ctx.Options.ConcurrentRCSyncs), ctx.Stop)
    return true, nil
}

Create Replication Manager

From the above analysis, controller-manager creates a Replication Manager object through New Replication Manager, which is actually the Replication Controller controller.

pkg/controller/replication/replication_controller.go:122

// NewReplicationManager creates a replication manager
func NewReplicationManager(podInformer cache.SharedIndexInformer, kubeClient clientset.Interface, resyncPeriod controller.ResyncPeriodFunc, burstReplicas int, lookupCacheSize int, garbageCollectorEnabled bool) *ReplicationManager {
    eventBroadcaster := record.NewBroadcaster()
    eventBroadcaster.StartLogging(glog.Infof)
    eventBroadcaster.StartRecordingToSink(&v1core.EventSinkImpl{Interface: kubeClient.Core().Events("")})
    return newReplicationManager(
        eventBroadcaster.NewRecorder(v1.EventSource{Component: "replication-controller"}),
        podInformer, kubeClient, resyncPeriod, burstReplicas, lookupCacheSize, garbageCollectorEnabled)
}

pkg/controller/replication/replication_controller.go:132
// newReplicationManager configures a replication manager with the specified event recorder
func newReplicationManager(eventRecorder record.EventRecorder, podInformer cache.SharedIndexInformer, kubeClient clientset.Interface, resyncPeriod controller.ResyncPeriodFunc, burstReplicas int, lookupCacheSize int, garbageCollectorEnabled bool) *ReplicationManager {
    if kubeClient != nil && kubeClient.Core().RESTClient().GetRateLimiter() != nil {
        metrics.RegisterMetricAndTrackRateLimiterUsage("replication_controller", kubeClient.Core().RESTClient().GetRateLimiter())
    }

    rm := &ReplicationManager{
        kubeClient: kubeClient,
        podControl: controller.RealPodControl{
            KubeClient: kubeClient,
            Recorder:   eventRecorder,
        },
        burstReplicas: burstReplicas,
        expectations:  controller.NewUIDTrackingControllerExpectations(controller.NewControllerExpectations()),
        queue:         workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "replicationmanager"),
        garbageCollectorEnabled: garbageCollectorEnabled,
    }

    rm.rcStore.Indexer, rm.rcController = cache.NewIndexerInformer(
        &cache.ListWatch{
            ListFunc: func(options v1.ListOptions) (runtime.Object, error) {
                return rm.kubeClient.Core().ReplicationControllers(v1.NamespaceAll).List(options)
            },
            WatchFunc: func(options v1.ListOptions) (watch.Interface, error) {
                return rm.kubeClient.Core().ReplicationControllers(v1.NamespaceAll).Watch(options)
            },
        },
        &v1.ReplicationController{},
        // TODO: Can we have much longer period here?
        FullControllerResyncPeriod,
        cache.ResourceEventHandlerFuncs{
            AddFunc:    rm.enqueueController,
            UpdateFunc: rm.updateRC,
            // This will enter the sync loop and no-op, because the controller has been deleted from the store.
            // Note that deleting a controller immediately after scaling it to 0 will not work. The recommended
            // way of achieving this is by performing a `stop` operation on the controller.
            DeleteFunc: rm.enqueueController,
        },
        cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc},
    )

    podInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc: rm.addPod,
        // This invokes the rc for every pod change, eg: host assignment. Though this might seem like overkill
        // the most frequent pod update is status, and the associated rc will only list from local storage, so
        // it should be ok.
        UpdateFunc: rm.updatePod,
        DeleteFunc: rm.deletePod,
    })
    rm.podStore.Indexer = podInformer.GetIndexer()
    rm.podController = podInformer.GetController()

    rm.syncHandler = rm.syncReplicationController
    rm.podStoreSynced = rm.podController.HasSynced
    rm.lookupCache = controller.NewMatchingCache(lookupCacheSize)
    return rm
}

Replication Manager is mainly configured in the new Replication Manager, such as:

  • Configure queue through workqueue.NewNamedRateLimitingQueue.
  • Configure expectations through controller. NewUIDTracking Controller Expectations.
  • Configure rcStore, podStore, rcController, podController.
  • Configuring syncHandler as rm. syncReplication Controller is important, so I'll list it separately. As will be mentioned later, sync Replication Controller is the way to do the core work. It can be said that the automatic maintenance of Replica is accomplished by it.

Execute ReplicationManger.Run to get started

Replication Manager has been created, and now we have to work. Run method is the starting point of work, starting to watch and syncing.

pkg/controller/replication/replication_controller.go:217

// Run begins watching and syncing.
func (rm *ReplicationManager) Run(workers int, stopCh <-chan struct{}) {
    defer utilruntime.HandleCrash()
    glog.Infof("Starting RC Manager")
    go rm.rcController.Run(stopCh)
    go rm.podController.Run(stopCh)
    for i := 0; i < workers; i++ {
        go wait.Until(rm.worker, time.Second, stopCh)
    }

    if rm.internalPodInformer != nil {
        go rm.internalPodInformer.Run(stopCh)
    }

    <-stopCh
    glog.Infof("Shutting down RC Manager")
    rm.queue.ShutDown()
}
  • watching
    • go rm.rcController.Run(stopCh) is responsible for watching all rc.
    • go rm.podController.Run(stopCh) is responsible for watching all pod.
  • syncing
    • Start the goroutine for the number of workers.
    • Each goroutine iterates through rm.worker, staying one second between each cycle. rm.worker is responsible for getting rc from queue and calling syncHandler for synchronization.
    • Each goroutine does not end until a stop Ch signal is received.

The following is the implementation of the Run method of rcController and podController. The function is to complete the watch of rc/pod.

pkg/client/cache/controller.go:84

// Run begins processing items, and will continue until a value is sent down stopCh.
// It's an error to call Run more than once.
// Run blocks; call via go.
func (c *Controller) Run(stopCh <-chan struct{}) {
    defer utilruntime.HandleCrash()
    r := NewReflector(
        c.config.ListerWatcher,
        c.config.ObjectType,
        c.config.Queue,
        c.config.FullResyncPeriod,
    )

    c.reflectorMutex.Lock()
    c.reflector = r
    c.reflectorMutex.Unlock()

    r.RunUntil(stopCh)

    wait.Until(c.processLoop, time.Second, stopCh)
}

The key implementation of sync, in the Replication Manager worker method, is as follows.

pkg/controller/replication/replication_controller.go:488

// worker runs a worker thread that just dequeues items, processes them, and marks them done.
// It enforces that the syncHandler is never invoked concurrently with the same key.
func (rm *ReplicationManager) worker() {
    workFunc := func() bool {
        key, quit := rm.queue.Get()
        if quit {
            return true
        }
        defer rm.queue.Done(key)

        err := rm.syncHandler(key.(string))
        if err == nil {
            rm.queue.Forget(key)
            return false
        }

        rm.queue.AddRateLimited(key)
        utilruntime.HandleError(err)
        return false
    }
    for {
        if quit := workFunc(); quit {
            glog.Infof("replication controller worker shutting down")
            return
        }
    }
}

The main logic in worker is:

  • Get a key of rc from RatLimited Queue of rm.
  • Call syncHandler Interface to sync the rc.

In the new Replication Manager, syncHandler is registered as syncReplication Controller by rm. syncHandler = rm. syncReplication Controller. So the logic of sync rc is in sync Replication Controller.

pkg/controller/replication/replication_controller.go:639

// syncReplicationController will sync the rc with the given key if it has had its expectations fulfilled, meaning it did not expect to see any more of its pods created or deleted. This function is not meant to be invoked concurrently with the same key.

func (rm *ReplicationManager) syncReplicationController(key string) error {
    trace := util.NewTrace("syncReplicationController: " + key)
    defer trace.LogIfLong(250 * time.Millisecond)

    startTime := time.Now()
    defer func() {
        glog.V(4).Infof("Finished syncing controller %q (%v)", key, time.Now().Sub(startTime))
    }()

    if !rm.podStoreSynced() {
        // Sleep so we give the pod reflector goroutine a chance to run.
        time.Sleep(PodStoreSyncedPollPeriod)
        glog.Infof("Waiting for pods controller to sync, requeuing rc %v", key)
        rm.queue.Add(key)
        return nil
    }

    obj, exists, err := rm.rcStore.Indexer.GetByKey(key)
    if !exists {
        glog.Infof("Replication Controller has been deleted %v", key)
        rm.expectations.DeleteExpectations(key)
        return nil
    }
    if err != nil {
        return err
    }
    rc := *obj.(*v1.ReplicationController)

    trace.Step("ReplicationController restored")
    rcNeedsSync := rm.expectations.SatisfiedExpectations(key)
    trace.Step("Expectations restored")

    // NOTE: filteredPods are pointing to objects from cache - if you need to
    // modify them, you need to copy it first.
    // TODO: Do the List and Filter in a single pass, or use an index.
    var filteredPods []*v1.Pod
    if rm.garbageCollectorEnabled {
        // list all pods to include the pods that don't match the rc's selector
        // anymore but has the stale controller ref.
        pods, err := rm.podStore.Pods(rc.Namespace).List(labels.Everything())
        if err != nil {
            glog.Errorf("Error getting pods for rc %q: %v", key, err)
            rm.queue.Add(key)
            return err
        }
        cm := controller.NewPodControllerRefManager(rm.podControl, rc.ObjectMeta, labels.Set(rc.Spec.Selector).AsSelectorPreValidated(), getRCKind())
        matchesAndControlled, matchesNeedsController, controlledDoesNotMatch := cm.Classify(pods)
        // Adopt pods only if this replication controller is not going to be deleted.
        if rc.DeletionTimestamp == nil {
            for _, pod := range matchesNeedsController {
                err := cm.AdoptPod(pod)
                // continue to next pod if adoption fails.
                if err != nil {
                    // If the pod no longer exists, don't even log the error.
                    if !errors.IsNotFound(err) {
                        utilruntime.HandleError(err)
                    }
                } else {
                    matchesAndControlled = append(matchesAndControlled, pod)
                }
            }
        }
        filteredPods = matchesAndControlled
        // remove the controllerRef for the pods that no longer have matching labels
        var errlist []error
        for _, pod := range controlledDoesNotMatch {
            err := cm.ReleasePod(pod)
            if err != nil {
                errlist = append(errlist, err)
            }
        }
        if len(errlist) != 0 {
            aggregate := utilerrors.NewAggregate(errlist)
            // push the RC into work queue again. We need to try to free the
            // pods again otherwise they will stuck with the stale
            // controllerRef.
            rm.queue.Add(key)
            return aggregate
        }
    } else {
        pods, err := rm.podStore.Pods(rc.Namespace).List(labels.Set(rc.Spec.Selector).AsSelectorPreValidated())
        if err != nil {
            glog.Errorf("Error getting pods for rc %q: %v", key, err)
            rm.queue.Add(key)
            return err
        }
        filteredPods = controller.FilterActivePods(pods)
    }

    var manageReplicasErr error
    if rcNeedsSync && rc.DeletionTimestamp == nil {
        manageReplicasErr = rm.manageReplicas(filteredPods, &rc)
    }
    trace.Step("manageReplicas done")

    newStatus := calculateStatus(rc, filteredPods, manageReplicasErr)

    // Always updates status as pods come up or die.
    if err := updateReplicationControllerStatus(rm.kubeClient.Core().ReplicationControllers(rc.Namespace), rc, newStatus); err != nil {
        // Multiple things could lead to this update failing.  Returning an error causes a requeue without forcing a hotloop
        return err
    }

    return manageReplicasErr
}

The main logic of syncReplication Controller is:

  1. If the podStore has not been synchronized once, the key of the rc is re-added to the queue to wait for the podStore to synchronize and the process ends, otherwise the subsequent process continues.
  2. According to the key value of the rc, the corresponding rc object is obtained from the RC Store. If the rc object does not exist, it indicates that the RC has been deleted. Then the RC is deleted from the epectations according to the key and returned to the end of the process. If the rc object exists, the subsequent process continues.
  3. The add s and del s in expectations and whether a timestamp on the distance exceeds 5 minutes are detected to determine whether the rc needs sync.
  4. If GC is started, the pods under the entire namespace in the podStore are retrieved, and then matches AndControlled and matches Needs Controller pods are filtered as filtered filtered pods to be synchronized. If the GC is not started, the pods matching the Active state of rc.Spec.Selector under the namespace in the podStore are obtained directly as filtered and synchronized filtered pods. (For an understanding of matches AndControlled and matches Needs Controller, refer to the PodController RefManager. Classify function defined in pkg/controller/controller_ref_manager.go:57)
  5. If it is detected in step 3 that the rc requires sync and the Deletion Timestamp timestamp is nil, the manageReplicas method is called so that the number of pods in the active state managed by the rc is the same as the expected value.
  6. After executing manageReplicas, you need to recalculate the status of rc immediately, update Conditions, Replicas, Fully Labeled Replicas, Ready Replicas, Available Replicas information in status.
  7. By calling the interface of kube-api-server through the updateReplication Controller Status method, the status of the rc is updated to the new status after recalculating in the previous step, and the process ends.

A key step in the syncReplication Controller process described above is the manageReplicas method called in step 5, which is responsible for the rc replicas repair (add or delete).

pkg/controller/replication/replication_controller.go:516

// manageReplicas checks and updates replicas for the given replication controller.
// Does NOT modify <filteredPods>.
func (rm *ReplicationManager) manageReplicas(filteredPods []*v1.Pod, rc *v1.ReplicationController) error {
    diff := len(filteredPods) - int(*(rc.Spec.Replicas))
    rcKey, err := controller.KeyFunc(rc)
    if err != nil {
        return err
    }
    if diff == 0 {
        return nil
    }

    if diff < 0 {
        diff *= -1
        if diff > rm.burstReplicas {
            diff = rm.burstReplicas
        }
        // TODO: Track UIDs of creates just like deletes. The problem currently
        // is we'd need to wait on the result of a create to record the pod's
        // UID, which would require locking *across* the create, which will turn
        // into a performance bottleneck. We should generate a UID for the pod
        // beforehand and store it via ExpectCreations.
        errCh := make(chan error, diff)
        rm.expectations.ExpectCreations(rcKey, diff)
        var wg sync.WaitGroup
        wg.Add(diff)
        glog.V(2).Infof("Too few %q/%q replicas, need %d, creating %d", rc.Namespace, rc.Name, *(rc.Spec.Replicas), diff)
        for i := 0; i < diff; i++ {
            go func() {
                defer wg.Done()
                var err error
                if rm.garbageCollectorEnabled {
                    var trueVar = true
                    controllerRef := &metav1.OwnerReference{
                        APIVersion: getRCKind().GroupVersion().String(),
                        Kind:       getRCKind().Kind,
                        Name:       rc.Name,
                        UID:        rc.UID,
                        Controller: &trueVar,
                    }
                    err = rm.podControl.CreatePodsWithControllerRef(rc.Namespace, rc.Spec.Template, rc, controllerRef)
                } else {
                    err = rm.podControl.CreatePods(rc.Namespace, rc.Spec.Template, rc)
                }
                if err != nil {
                    // Decrement the expected number of creates because the informer won't observe this pod
                    glog.V(2).Infof("Failed creation, decrementing expectations for controller %q/%q", rc.Namespace, rc.Name)
                    rm.expectations.CreationObserved(rcKey)
                    errCh <- err
                    utilruntime.HandleError(err)
                }
            }()
        }
        wg.Wait()

        select {
        case err := <-errCh:
            // all errors have been reported before and they're likely to be the same, so we'll only return the first one we hit.
            if err != nil {
                return err
            }
        default:
        }

        return nil
    }

    if diff > rm.burstReplicas {
        diff = rm.burstReplicas
    }
    glog.V(2).Infof("Too many %q/%q replicas, need %d, deleting %d", rc.Namespace, rc.Name, *(rc.Spec.Replicas), diff)
    // No need to sort pods if we are about to delete all of them
    if *(rc.Spec.Replicas) != 0 {
        // Sort the pods in the order such that not-ready < ready, unscheduled
        // < scheduled, and pending < running. This ensures that we delete pods
        // in the earlier stages whenever possible.
        sort.Sort(controller.ActivePods(filteredPods))
    }
    // Snapshot the UIDs (ns/name) of the pods we're expecting to see
    // deleted, so we know to record their expectations exactly once either
    // when we see it as an update of the deletion timestamp, or as a delete.
    // Note that if the labels on a pod/rc change in a way that the pod gets
    // orphaned, the rs will only wake up after the expectations have
    // expired even if other pods are deleted.
    deletedPodKeys := []string{}
    for i := 0; i < diff; i++ {
        deletedPodKeys = append(deletedPodKeys, controller.PodKey(filteredPods[i]))
    }
    // We use pod namespace/name as a UID to wait for deletions, so if the
    // labels on a pod/rc change in a way that the pod gets orphaned, the
    // rc will only wake up after the expectation has expired.
    errCh := make(chan error, diff)
    rm.expectations.ExpectDeletions(rcKey, deletedPodKeys)
    var wg sync.WaitGroup
    wg.Add(diff)
    for i := 0; i < diff; i++ {
        go func(ix int) {
            defer wg.Done()
            if err := rm.podControl.DeletePod(rc.Namespace, filteredPods[ix].Name, rc); err != nil {
                // Decrement the expected number of deletes because the informer won't observe this deletion
                podKey := controller.PodKey(filteredPods[ix])
                glog.V(2).Infof("Failed to delete %v due to %v, decrementing expectations for controller %q/%q", podKey, err, rc.Namespace, rc.Name)
                rm.expectations.DeletionObserved(rcKey, podKey)
                errCh <- err
                utilruntime.HandleError(err)
            }
        }(i)
    }
    wg.Wait()

    select {
    case err := <-errCh:
        // all errors have been reported before and they're likely to be the same, so we'll only return the first one we hit.
        if err != nil {
            return err
        }
    default:
    }

    return nil

}

The main logic of the manageReplicas code above is:

  • Firstly, the difference diff between the number of Pods in filteredPods and the expected number defined in rc.Spec.Replicas is calculated.
  • If the difference diff is 0, it means that the current state is the same as the expected state, and returns directly to the end of the process.
  • If the difference diff is negative, indicating that the number of Pods in the current Active state is insufficient, start the following process:
    • Compare the values of | diff | and burstReplicas to ensure that only a maximum number of pods are created for burstReplicas this time.
    • Call the expectations.ExpectCreations interface to set the value of add size | diff | in expectations, indicating that a new number of pods | diff | will be created to achieve the desired state.
    • sync.WaitGroup starts the | diff | number of goroutine protocols, each of which is responsible for calling the podControl.CreatePods interface to create a corresponding spec Template pod managed by the namespace.rc.
    • When all goroutine s are executed, if one or more pod s fail to create, err is returned, otherwise nil is returned, and the process ends.
  • If the difference diff is positive, indicating that the number of Pods in the current Active state exceeds the expected value, start the following process:
    • Compare the values of | diff | and burstReplicas to ensure that only the number of pods of burstReplicas is deleted at most this time.
    • The purpose of sorting pods in filtered Pods is: not-ready < ready, unscheduled < scheduled, and pending < running, so that pods with earlier stages are delete d first.
    • After sorting, select the first | diff | pods as the pods to be delete d.
    • Call the expectations.ExpectDeletions interface to set the value of del size | diff | in expectations to delete the number of pods | diff | to achieve the desired state.
    • sync.WaitGroup starts the | diff | number of goroutine protocols, each of which is responsible for calling the podControl.DeletePod interface to delete a Pod in the delete Pods.
    • When all goroutine s are executed, if one or more pod deletions fail, err is returned, otherwise nil is returned, and the process ends.

At this point, I think the key code has been analyzed, calculateStatus, updateReplication Controller Status method is relatively simple, interested in their own targeting.

summary

  • Replication Manager is the code implementation of the Replication Controller controller to distinguish Replication Controller Resource Object.
  • When kube-controller-manager calls newController Initializers to initialize the controller, the startReplication Controller is registered to start the Replication Controller controller.
  • When the Replication Manager is mainly configured in the new Replication Manager, the following key configurations are made:

    • Configure queue through workqueue.NewNamedRateLimitingQueue.
    • Configure expectations through controller. NewUIDTracking Controller Expectations.
    • Configure rcStore, podStore, rcController, podController.
    • Configuring syncHandler as rm. syncReplication Controller, syncReplication Controller is the way to do the core work. It can be said that the automatic maintenance of Replica is accomplished by it.
  • Run method is the starting point of work, starting watching and syncing:
    • watching
      • go rm.rcController.Run(stopCh) is responsible for watching all rc.
      • go rm.podController.Run(stopCh) is responsible for watching all pod.
    • syncing
      • Start the goroutine for the number of workers.
      • Each goroutine iterates through rm.worker, staying one second between each cycle. rm.worker is responsible for getting rc from queue and calling syncHandler for synchronization.
      • Each goroutine does not end until a stop Ch signal is received.
  • In the syncReplication Controller process, the core step is to call the manageReplicas method, which is responsible for the rc replicas repair (add or delete).

Posted by bogdani on Sat, 20 Apr 2019 12:00:35 -0700