The Yurt-Manager component consists of several controllers and webhooks, which are used to provide abilities to ensure that Kubernetes can work as it would in a normal data center in a cloud-edge collaboration scenario. such as easily managing multi-regions workloads and providing AdvancedRollingUpdate and OTA upgrade capabilities for edge workloads(DaemonSet and static pods).
The Yurt-Manager component is recommended to co-located with Kubernetes control plane components such as Kube-Controller-Manager. and Yurt-Manager is deployed as a
Deployment, usually consists of
two instances, one leader and one backup.
Each controller and webhook is described as follows.
Due to the stringent approval policy for CSR(Certificate Signing Request) in Kubernetes, the CSRs generated by OpenYurt components, such as YurtHub, are not automatically approved by theKube-Controller-Manager.
As a result, A controller named
csrapprover within Yurt-Manager is responsible for approving the CSRs of OpenYurt components.
The traditional RollingUpdate strategy for DaemonSet can become easily obstructed when NotReady nodes are present in a Cloud-Edge collaboration scenario. To address this issue, the
daemonpodupdater controller employs two upgrade models: AdvancedRollingUpdate and OTA (Over-the-Air) upgrade.
The AdvancedRollingUpdate strategy initially upgrades pods on Ready nodes while skipping NotReady nodes. When a node transitions from NotReady to Ready, the daemon pod on that node is automatically upgraded.
The OTA strategy is utilized in scenarios where the decision to upgrade a workload falls under the purview of the edge node owner, rather than the cluster owner. This approach is particularly relevant in cases such as electric vehicles, where the edge node owner has greater control over the upgrade process.
delegatelease controller is designed to work in conjunction with pool-coordinator component. When a node becomes disconnected from the cloud, the lease reported through
pool-coordinator component will have the
openyurt.io/delegate-heartbeat=true annotation. Upon detecting a lease with this annotation, the
delegatelease controller will apply the
openyurt.io/unschedulable taint to the node, ensuing that newly created pods cannot be scheduled on such nodes.
Certain edge services require that Pods not be evicted in the event of node failure; instead, they demand a specific Pod to be bound to a particular node.
For instance, image processing applications need to be bound to the machine connected to a camera, while intelligent transportation applications must be fixed to a machine located at a specific intersection.
Users can add the
apps.openyurt.io/binding=true annotation to nodes to enable the Pods Binding feature, ensuring that all Pods on that node are bound to it and remain unaffected by the cloud-edge network.
The podbinding controller oversees the management of pod tolerations when the
apps.openyurt.io/binding annotation of a node is modified. If the
apps.openyurt.io/binding node annotation is set to true,
the TolerationSeconds for
node.kubernetes.io/unreachable tolerations in pods will be set to 0, preventing the eviction of pods even when the cloud-edge network is offline.
Conversely, if the annotation is not set to true, the TolerationSeconds for
node.kubernetes.io/unreachable tolerations in pods will be set to 300 seconds.
ravenl3 controller serves as the controller for the Gateway Custom Resource Definition (CRD). This controller is intended to be used in conjunction with the Raven Agent component,
which facilitates Level3 network connectivity among pods located in different physical or network regions.
NodePool abstracts the concept of a node pool based on specific node attributes, such as region, CPU architecture, cloud provider, and more, allowing for unified management of nodes at the pool level.
We are accustomed to grouping and managing nodes using various Kubernetes Labels. However, as the number of nodes and labels increases, the operation and maintenance of nodes (e.g., batch configuration of scheduling policies, taints, etc.) become increasingly complex, as illustrated in the following figure:
nodepool controller/webhook can manage nodes across different edge regions from the perspective of a node pool, as depicted in the subsequent figure:
poolcoordinatorcert controller is responsible for preparing certificates and kubeconfig file for the pool-coordinator component. All certificates and kubeconfig files are stored as Secret resources within the system.
The servicetopology controller is used to assist servicetopology filter in YurtHub to provide service topology routing capabilities for the cluster. When topology annotation of service is modified, the servicetopology controller updates the corresponding endpoints and EndpointSlices, triggering a service topology update on the node side.
Owing to the vast number and distributed nature of edge devices, manually deploying and upgrading Static pods in cloud-edge collaboration scenarios can result in substantial operational challenges and increased risk of errors. To address this issue, OpenYurt has introduced a new Custom Resource Definition (CRD) called YurtStaticSet to improve the management of Static pods. The yurtstaticset controller/webhook offers features such as AdvancedRollingUpdate and Over-The-Air (OTA) upgrade capabilities for static pods.
In native Kubernetes, managing the same type of applications across multiple nodepools requires creating a Deployment for each node pool, leading to higher management costs and potential risks of errors. To address this issue, the YurtAppSet CRD provides a way to define an application template (supporting both Deployment and StatefulSet) and is responsible for managing workloads across multiple nodepools.
YurtAppSet requires users to explicitly specify the node pools to which the workloads should be deployed by configuring its
Spec.Topology field. This approach simplifies the application deployment and management process, making it easier to scale, upgrade, and maintain applications in a multiple nodepools environment. By using YurtAppSet, users can centrally manage application deployments across multiple nodepools, thereby reducing management complexity and potential risks of errors.
In native Kubernetes, DaemonSet is used to run a daemon Pod on each node. When a node is added or removed, the corresponding daemon Pod on that node is automatically created or removed. However, when workloads need to be adjusted automatically based on the creation and removal of nodepools, DaemonSet cannot meet our needs.
YurtAppDaemon aims to ensure that workloads specified in the template (Spec.WorkloadTemplate) are automatically deployed in all nodepools or those selected by
Spec.NodePoolSelector. When a nodepool is added or removed from the cluster, the YurtAppDaemon controller and Webhook create or remove workloads for the corresponding nodepool, ensuring that the required nodepools always contain the expected Pods.
In YurtAppDaemon and YurtAppSet above, workloads are distributed based on a uniform template. However, when workloads need to be customized based on node pool or region, YurtAppDaemon does not satisfy our needs. Although YurtAppSet can be personalized to a certain extent by the Topology field, we introduced YurtAppOverrider as the multi-region workloads rendering engine in order to reduce coupling and achieve backward compatibility.
YurtAppOverrider is designed to ensure that all YurtAppDaemon and YurtAppSet bindings are rendered by the corresponding Webhook when dispatching workloads. When there are any changes to the templates of YurtAppDaemon and YurtAppSet or to the personalization configuration of YurtAppOverrider, the YurtAppOverrider controller triggers a recalculation to ensure customized configuration in a multi-region scenario.
1.13 platformadmin controller/webhook
PlatformAdmin evolved from previous versions of EdgeX CRD and serves as an abstraction for an edge device management platform. Users only need to input settings such as the connected platform, the name of the NodePool to be deployed, the desired version, and more. This allows them to deploy an edge device management platform on a node pool.
The platformadmin-controller is integrated within yurt-manager and is responsible for parsing the PlatformAdmin CR into corresponding config maps, services, and yurtappsets. This enables the deployment of the edge device management platform.
Additionally, it is responsible for distributing yurt-iot-dock to the corresponding node pool to facilitate edge device synchronization. Users can select optional EdgeX components to be deployed through the components field in the PlatformAdmin CR. For more information on this aspect, please refer to the Cloud-Native Device Management section.