K8s Affinity亲和力详解(调度策略)
作者:大新屋
文章介绍了Kubernetes中亲和力和反亲和力的调度策略,包括节点和Pod的硬性、软性规则,用于实现Pod与节点的标签匹配、区域均衡负载及避免单节点过载等场景,强调标签匹配和副本数限制对调度结果的影响
K8s Affinity亲和力
提示:Kubernetes 官网Affinity说明文档 https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/
kubernetes Affinity亲和力是一种调度策略,用于确保Pod被调度到具有特定标签(label)或字段的节点上。
亲和力和反亲和力:
- 亲和力和反亲和力可以用来控制 Pod 如何调度到节点上。
- 亲和力表示 Pod 要求与特定的节点调度在一起,反亲和力表示 Pod 不希望与特定的节点调度在一起。
Affinity亲和力分为两类:
- 节点亲和力:NodeAffinity (包括节点亲和力和节点反亲和力)
- Pod亲和力:PodAffinity/PodAntiAffinity(PodAffinity Pod亲和力和PodAntiAffinity Pod反亲和力)
亲和力和反亲和力有两种性:
- requiredDuringSchedulingIgnoredDuringExecution(硬亲和性):这种类型亲和力是硬性,表示Pod必须被调度到满足指定条件的节点上。如果不满足条件,Pod将不会被调度。
- preferredDuringSchedulingIgnoredDuringExecution(软亲和性):这种类型亲和力是软性,表示Pod更倾向于被调度到满足指定条件的节点上,但如果无法满足,Pod仍会被调度到其他节点。
一、Affinity参数说明
提示:节点反亲和力通过operator参数设置成Notln值实现
1、节点nodeAffinity(硬性)亲和力
spec: affinity: # 定义Affinity亲和力配置 nodeAffinity: # 定义节点Affinity亲和力配置 requiredDuringSchedulingIgnoredDuringExecution: # 定义硬性Affinity亲和力配置 nodeSelectorTerms: # 定义节点选择器配置 - matchExpressions: # 定义匹配节点标签键值对key=value - key: disktype # 设置节点标签键名key operator: In # 节点标签匹配方式,In等于 values: # 设置节点标签值value - ssd
2、节点nodeAffinity(软性)亲和力
spec: affinity: # 定义Affinity亲和力配置 nodeAffinity: # 定义节点Affinity亲和力配置 preferredDuringSchedulingIgnoredDuringExecution: # 定义软性Affinity亲和力配置 - weight: 10 # 软亲和力的权重,权重越高优先级越大,范围1-100 preference: # 软亲和力配置项 matchExpressions: # 定义匹配节点标签键值对key=value - key: disktype # 设置节点标签键名key operator: In # 节点标签匹配方式,In等于 values: # 设置节点标签值value - ssd
3、Pod podAffinity(硬性)亲和力
spec: affinity: # 定义Affinity亲和力配置 podAffinity: # 定义Pod Affinity亲和力配置 requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置 - labelSelector: # 定义Pod选择器配置 matchExpressions: # 定义匹配Pod标签键值对key=value - key: disktype i # 设置Pod标签键名key operator: In # Pod标签规则匹配条件,In等于 values: # 设置Pod标签值value - ssd topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
4、Pod podAffinity(软性)亲和力
spec: affinity: # 定义Affinity亲和力配置 podAffinity: # 定义Pod Affinity亲和力配置 requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置 - labelSelector: # 定义Pod选择器配置 matchExpressions: # 定义匹配Pod标签键值对key=value - key: disktype # 设置Pod标签键名key operator: In # Pod标签规则匹配条件,In等于 values: # 设置Pod标签值value - ssd topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
5、Pod podAffinity(硬性)反亲和力
spec: affinity: # 定义Affinity亲和力配置 podAffinity: # 定义Pod Affinity亲和力配置 requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置 - labelSelector: # 定义Pod选择器配置 matchExpressions: # 定义匹配Pod标签键值对key=value - key: disktype # 设置Pod标签键名key operator: In # Pod标签规则匹配条件,In等于 values: # 设置Pod标签值value - ssd topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
6、Pod podAntAffinity(软性)反亲和力
spec: affinity: # 定义Affinity亲和力配置 podAntAffinity: # 定义Pod Affinity反亲和力配置 preferredDuringSchedulingIgnoredDuringExecution: # 定义Affinity软性反亲和力配置 - weight: 10 # 反亲和力的权重,权重越高反亲和力越大,范围1-100 podAffinityTerm: # Pod反亲和力配置项 labelSelector: # 定义Pod选择器配置 matchExpressions: # 定义匹配Pod标签键值对key=value - key: security # 设置Pod标签键名key operator: In # Pod标签规则匹配条件,In等于 values: # 设置Pod标签值value - S2 topologyKey: failure-domain.beta.kubernetes.io/zone # 定义反亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。
7、operator参数规则匹配条件
operator # 规则匹配条件 --- In # 相当于key=value的形式(等于) --- Notln # 相当于key!=value的形式(不等于) --- Exists # 节点存在label的key为指定的值即可,不能配置values字段 --- DoesNotExist # 节点不存在label的key为指定的值即可,不能配置values字段 --- Gt # 大于value指定的值 --- Lt # 小于value指定的值
二、示例1:节点Affinity(硬性)亲和力(强制Pod部署到指定节点)
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数 kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints ### 选择k8s-node01和k8s-node02节点打标签 kubectl label node k8s-node01 k8s-node02 disktype=ssd kubectl get nodes --show-labels -l disktype=ssd ### 创建Deployment mkdir -p /data/yaml/affinity cat > /data/yaml/affinity/deploy-nodeaffinity-hard.yaml << 'EOF' apiVersion: apps/v1 kind: Deployment metadata: labels: app: deploy-nodeaffinity-hard name: deploy-nodeaffinity-hard namespace: default spec: replicas: 6 selector: matchLabels: app: nginx-pod template: metadata: labels: app: nginx-pod spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: disktype operator: In values: - ssd containers: - name: nginx image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26 imagePullPolicy: IfNotPresent EOF kubectl create -f /data/yaml/affinity/deploy-nodeaffinity-hard.yaml kubectl get deploy deploy-nodeaffinity-hard -n default -oyaml ### 查看Pod节点部署节点 kubectl get pods -n default -owide ### 删除Deployment和节点标签 kubectl delete -f /data/yaml/affinity/deploy-nodeaffinity-hard.yaml kubectl label nodes k8s-node01 k8s-node02 disktype-
三、示例2:节点Affinity(软性)亲和力(尽可能Pod部署到指定节点)
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数 kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints ### 选择k8s-node02节点打标签 kubectl label node k8s-node02 node=erp kubectl get nodes --show-labels -l node=erp ### 创建Deployment mkdir -p /data/yaml/affinity cat > /data/yaml/affinity/deploy-nodeaffinity-soft.yaml << 'EOF' apiVersion: apps/v1 kind: Deployment metadata: labels: app: deploy-nodeaffinity-soft name: deploy-nodeaffinity-soft namespace: default spec: replicas: 10 selector: matchLabels: app: nginx-pod template: metadata: labels: app: nginx-pod spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: node operator: In values: - erp containers: - name: nginx image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26 imagePullPolicy: IfNotPresent EOF kubectl create -f /data/yaml/affinity/deploy-nodeaffinity-soft.yaml kubectl get deploy deploy-nodeaffinity-soft -n default -oyaml ### 查看Pod节点部署节点 kubectl get pods -n default -owide ### 删除Deployment和节点标签 kubectl delete -f /data/yaml/affinity/deploy-nodeaffinity-soft.yaml kubectl label nodes k8s-node02 node-
四、示例3:Pod Affinity(硬性)反亲和力(强制Pod部署到指定的不同的节点)
示例场景描述:假如k8s-node01是深圳机房,k8s-node02是北京机房,k8s-master03是杭州机房,每个不同的机房部署一个nginx容器,实现区域性均衡负载
提示:当Deployment定义启动超过3个Pod数量,但节点只定义了3个key,当定义了3个key节点各自都只能运行一个Pod(不会运行两个以上的Pod),那剩下的Pod会在其它节点上运行
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数 kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints ### 节点创建label中键名key要一样,但value值不要一样,topologyke拓扑域参数才能区分不同的区域 kubectl get nodes kubectl label nodes k8s-master01 area=hangzhou kubectl label nodes k8s-node01 area=shenzhen kubectl label nodes k8s-node02 area=beijing kubectl get nodes --show-labels | grep area ### 创建Deployment mkdir -p /data/yaml/affinity cat > /data/yaml/affinity/deploy-podantaffinity-area.yaml << 'EOF' apiVersion: apps/v1 kind: Deployment metadata: labels: app: deploy-podantaffinity-area name: deploy-podantaffinity-area namespace: default spec: replicas: 3 selector: matchLabels: run: deploy-pod-area template: metadata: labels: run: deploy-pod-area spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: run operator: In values: - deploy-pod-area topologyKey: area containers: - name: nginx image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26 imagePullPolicy: IfNotPresent EOF kubectl create -f /data/yaml/affinity/deploy-podantaffinity-area.yaml kubectl get deploy deploy-podantaffinity-area -n default -oyaml ### 查看Pod节点部署节点 kubectl get pods -n default -owide ### 删除Deployment和节点标签 kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-area.yaml kubectl label nodes k8s-node01 k8s-node02 k8s-master01 area-
五、示例4:Pod Affinity(硬性)反亲和力(强制Pod部署到K8s集群中每一个节点)
提示:Deployment设置的Pod副本数不能超过K8s节点数,超过的Pod副本数启动Pod会失败,kubernetes.io/hostname是K8s内置的默认标签
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数 kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints ### 创建Deployment mkdir -p /data/yaml/affinity cat > /data/yaml/affinity/deploy-podantaffinity-hostname.yaml << 'EOF' apiVersion: apps/v1 kind: Deployment metadata: labels: app: deploy-podantaffinity-hostname name: deploy-podantaffinity-hostname namespace: default spec: replicas: 5 selector: matchLabels: run: deploy-pod-hostname template: metadata: labels: run: deploy-pod-hostname spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: run operator: In values: - deploy-pod-hostname topologyKey: kubernetes.io/hostname containers: - name: nginx image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26 imagePullPolicy: IfNotPresent EOF kubectl create -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml kubectl get deploy deploy-podantaffinity-hostname -n default -oyaml ### 查看Pod节点部署节点 kubectl get pods -n default -owide ### 删除Deployment kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml
六、示例5:节点Affinity(软性)亲和力(重要Pod尽可能部署在高性能节点)
测试场景描述: nginx容器不部署到master节点,nginx容器第一选择部署到高存储性能k8s-node01,如果k8s-node01节点出现故障,第二选择部署到k8s-node02节点
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数 kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints ### 节点创建label kubectl get nodes kubectl label nodes k8s-master01 k8s-master02 k8s-master03 master=manager kubectl label nodes k8s-node01 ssd=fastest kubectl label nodes k8s-node02 sata=faster kubectl get nodes --show-labels | grep master=manager kubectl get nodes --show-labels | grep ssd kubectl get nodes --show-labels | grep sata ### 创建Deployment mkdir -p /data/yaml/affinity cat > /data/yaml/affinity/nginx-deploy-select.yaml << 'EOF' apiVersion: apps/v1 kind: Deployment metadata: labels: app: nginx-deploy-select name: nginx-deploy-select namespace: default spec: replicas: 3 selector: matchLabels: app: nginx-pod template: metadata: labels: app: nginx-pod spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 preference: matchExpressions: - key: ssd operator: In values: - "fastest" - key: master operator: NotIn values: - "manager" - weight: 50 preference: matchExpressions: - key: sata operator: In values: - "faster" containers: - name: nginx image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26 imagePullPolicy: IfNotPresent EOF kubectl create -f /data/yaml/affinity/nginx-deploy-select.yaml kubectl get deploy nginx-deploy-select -n default -oyaml ### 查看Pod节点部署节点 kubectl get pods -n default -owide ### 删除Deployment kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml ### 删除Deployment和节点标签 kubectl delete -f /data/yaml/affinity/nginx-deploy-select.yaml kubectl label nodes k8s-node01 k8s-master01 k8s-master02 k8s-master03 master- kubectl label nodes k8s-node01 ssd- kubectl label nodes k8s-node02 sata-
总结
以上为个人经验,希望能给大家一个参考,也希望大家多多支持脚本之家。