K8s Affinity亲和力详解(调度策略)
作者:大新屋
文章介绍了Kubernetes中亲和力和反亲和力的调度策略,包括节点和Pod的硬性、软性规则,用于实现Pod与节点的标签匹配、区域均衡负载及避免单节点过载等场景,强调标签匹配和副本数限制对调度结果的影响
K8s Affinity亲和力
提示:Kubernetes 官网Affinity说明文档 https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/
kubernetes Affinity亲和力是一种调度策略,用于确保Pod被调度到具有特定标签(label)或字段的节点上。
亲和力和反亲和力:
- 亲和力和反亲和力可以用来控制 Pod 如何调度到节点上。
- 亲和力表示 Pod 要求与特定的节点调度在一起,反亲和力表示 Pod 不希望与特定的节点调度在一起。
Affinity亲和力分为两类:
- 节点亲和力:NodeAffinity (包括节点亲和力和节点反亲和力)
- Pod亲和力:PodAffinity/PodAntiAffinity(PodAffinity Pod亲和力和PodAntiAffinity Pod反亲和力)
亲和力和反亲和力有两种性:
- requiredDuringSchedulingIgnoredDuringExecution(硬亲和性):这种类型亲和力是硬性,表示Pod必须被调度到满足指定条件的节点上。如果不满足条件,Pod将不会被调度。
- preferredDuringSchedulingIgnoredDuringExecution(软亲和性):这种类型亲和力是软性,表示Pod更倾向于被调度到满足指定条件的节点上,但如果无法满足,Pod仍会被调度到其他节点。
一、Affinity参数说明
提示:节点反亲和力通过operator参数设置成Notln值实现
1、节点nodeAffinity(硬性)亲和力
spec:
affinity: # 定义Affinity亲和力配置
nodeAffinity: # 定义节点Affinity亲和力配置
requiredDuringSchedulingIgnoredDuringExecution: # 定义硬性Affinity亲和力配置
nodeSelectorTerms: # 定义节点选择器配置
- matchExpressions: # 定义匹配节点标签键值对key=value
- key: disktype # 设置节点标签键名key
operator: In # 节点标签匹配方式,In等于
values: # 设置节点标签值value
- ssd
2、节点nodeAffinity(软性)亲和力
spec:
affinity: # 定义Affinity亲和力配置
nodeAffinity: # 定义节点Affinity亲和力配置
preferredDuringSchedulingIgnoredDuringExecution: # 定义软性Affinity亲和力配置
- weight: 10 # 软亲和力的权重,权重越高优先级越大,范围1-100
preference: # 软亲和力配置项
matchExpressions: # 定义匹配节点标签键值对key=value
- key: disktype # 设置节点标签键名key
operator: In # 节点标签匹配方式,In等于
values: # 设置节点标签值value
- ssd
3、Pod podAffinity(硬性)亲和力
spec:
affinity: # 定义Affinity亲和力配置
podAffinity: # 定义Pod Affinity亲和力配置
requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置
- labelSelector: # 定义Pod选择器配置
matchExpressions: # 定义匹配Pod标签键值对key=value
- key: disktype i # 设置Pod标签键名key
operator: In # Pod标签规则匹配条件,In等于
values: # 设置Pod标签值value
- ssd
topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
4、Pod podAffinity(软性)亲和力
spec:
affinity: # 定义Affinity亲和力配置
podAffinity: # 定义Pod Affinity亲和力配置
requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置
- labelSelector: # 定义Pod选择器配置
matchExpressions: # 定义匹配Pod标签键值对key=value
- key: disktype # 设置Pod标签键名key
operator: In # Pod标签规则匹配条件,In等于
values: # 设置Pod标签值value
- ssd
topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
5、Pod podAffinity(硬性)反亲和力
spec:
affinity: # 定义Affinity亲和力配置
podAffinity: # 定义Pod Affinity亲和力配置
requiredDuringSchedulingIgnoredDuringExecution: # 定义Affinity硬性亲和力配置
- labelSelector: # 定义Pod选择器配置
matchExpressions: # 定义匹配Pod标签键值对key=value
- key: disktype # 设置Pod标签键名key
operator: In # Pod标签规则匹配条件,In等于
values: # 设置Pod标签值value
- ssd
topologkey: kubernetes.io/hostname # 定义亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。kubernetes.io/hostname是K8s默认节点标签
6、Pod podAntAffinity(软性)反亲和力
spec:
affinity: # 定义Affinity亲和力配置
podAntAffinity: # 定义Pod Affinity反亲和力配置
preferredDuringSchedulingIgnoredDuringExecution: # 定义Affinity软性反亲和力配置
- weight: 10 # 反亲和力的权重,权重越高反亲和力越大,范围1-100
podAffinityTerm: # Pod反亲和力配置项
labelSelector: # 定义Pod选择器配置
matchExpressions: # 定义匹配Pod标签键值对key=value
- key: security # 设置Pod标签键名key
operator: In # Pod标签规则匹配条件,In等于
values: # 设置Pod标签值value
- S2
topologyKey: failure-domain.beta.kubernetes.io/zone # 定义反亲和力的范围,匹配的拓扑域的key,也就是节点上label和key,key和value相同的为同一个域。topologyke拓扑域,主要针对宿主机,相当于对宿主机进行区域的划分。用label进行判断,不同的key和不同的value是属于不同的拓扑域。
7、operator参数规则匹配条件
operator # 规则匹配条件 --- In # 相当于key=value的形式(等于) --- Notln # 相当于key!=value的形式(不等于) --- Exists # 节点存在label的key为指定的值即可,不能配置values字段 --- DoesNotExist # 节点不存在label的key为指定的值即可,不能配置values字段 --- Gt # 大于value指定的值 --- Lt # 小于value指定的值
二、示例1:节点Affinity(硬性)亲和力(强制Pod部署到指定节点)
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数
kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
### 选择k8s-node01和k8s-node02节点打标签
kubectl label node k8s-node01 k8s-node02 disktype=ssd
kubectl get nodes --show-labels -l disktype=ssd
### 创建Deployment
mkdir -p /data/yaml/affinity
cat > /data/yaml/affinity/deploy-nodeaffinity-hard.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: deploy-nodeaffinity-hard
name: deploy-nodeaffinity-hard
namespace: default
spec:
replicas: 6
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- ssd
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26
imagePullPolicy: IfNotPresent
EOF
kubectl create -f /data/yaml/affinity/deploy-nodeaffinity-hard.yaml
kubectl get deploy deploy-nodeaffinity-hard -n default -oyaml
### 查看Pod节点部署节点
kubectl get pods -n default -owide
### 删除Deployment和节点标签
kubectl delete -f /data/yaml/affinity/deploy-nodeaffinity-hard.yaml
kubectl label nodes k8s-node01 k8s-node02 disktype-
三、示例2:节点Affinity(软性)亲和力(尽可能Pod部署到指定节点)
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数
kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
### 选择k8s-node02节点打标签
kubectl label node k8s-node02 node=erp
kubectl get nodes --show-labels -l node=erp
### 创建Deployment
mkdir -p /data/yaml/affinity
cat > /data/yaml/affinity/deploy-nodeaffinity-soft.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: deploy-nodeaffinity-soft
name: deploy-nodeaffinity-soft
namespace: default
spec:
replicas: 10
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node
operator: In
values:
- erp
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26
imagePullPolicy: IfNotPresent
EOF
kubectl create -f /data/yaml/affinity/deploy-nodeaffinity-soft.yaml
kubectl get deploy deploy-nodeaffinity-soft -n default -oyaml
### 查看Pod节点部署节点
kubectl get pods -n default -owide
### 删除Deployment和节点标签
kubectl delete -f /data/yaml/affinity/deploy-nodeaffinity-soft.yaml
kubectl label nodes k8s-node02 node-
四、示例3:Pod Affinity(硬性)反亲和力(强制Pod部署到指定的不同的节点)
示例场景描述:假如k8s-node01是深圳机房,k8s-node02是北京机房,k8s-master03是杭州机房,每个不同的机房部署一个nginx容器,实现区域性均衡负载
提示:当Deployment定义启动超过3个Pod数量,但节点只定义了3个key,当定义了3个key节点各自都只能运行一个Pod(不会运行两个以上的Pod),那剩下的Pod会在其它节点上运行
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数
kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
### 节点创建label中键名key要一样,但value值不要一样,topologyke拓扑域参数才能区分不同的区域
kubectl get nodes
kubectl label nodes k8s-master01 area=hangzhou
kubectl label nodes k8s-node01 area=shenzhen
kubectl label nodes k8s-node02 area=beijing
kubectl get nodes --show-labels | grep area
### 创建Deployment
mkdir -p /data/yaml/affinity
cat > /data/yaml/affinity/deploy-podantaffinity-area.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: deploy-podantaffinity-area
name: deploy-podantaffinity-area
namespace: default
spec:
replicas: 3
selector:
matchLabels:
run: deploy-pod-area
template:
metadata:
labels:
run: deploy-pod-area
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: run
operator: In
values:
- deploy-pod-area
topologyKey: area
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26
imagePullPolicy: IfNotPresent
EOF
kubectl create -f /data/yaml/affinity/deploy-podantaffinity-area.yaml
kubectl get deploy deploy-podantaffinity-area -n default -oyaml
### 查看Pod节点部署节点
kubectl get pods -n default -owide
### 删除Deployment和节点标签
kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-area.yaml
kubectl label nodes k8s-node01 k8s-node02 k8s-master01 area-
五、示例4:Pod Affinity(硬性)反亲和力(强制Pod部署到K8s集群中每一个节点)
提示:Deployment设置的Pod副本数不能超过K8s节点数,超过的Pod副本数启动Pod会失败,kubernetes.io/hostname是K8s内置的默认标签
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数
kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
### 创建Deployment
mkdir -p /data/yaml/affinity
cat > /data/yaml/affinity/deploy-podantaffinity-hostname.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: deploy-podantaffinity-hostname
name: deploy-podantaffinity-hostname
namespace: default
spec:
replicas: 5
selector:
matchLabels:
run: deploy-pod-hostname
template:
metadata:
labels:
run: deploy-pod-hostname
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: run
operator: In
values:
- deploy-pod-hostname
topologyKey: kubernetes.io/hostname
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26
imagePullPolicy: IfNotPresent
EOF
kubectl create -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml
kubectl get deploy deploy-podantaffinity-hostname -n default -oyaml
### 查看Pod节点部署节点
kubectl get pods -n default -owide
### 删除Deployment
kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml
六、示例5:节点Affinity(软性)亲和力(重要Pod尽可能部署在高性能节点)
测试场景描述: nginx容器不部署到master节点,nginx容器第一选择部署到高存储性能k8s-node01,如果k8s-node01节点出现故障,第二选择部署到k8s-node02节点
### 查看K8s集群节点是否有污点,如果有污点需要清除节点污点,为了不影响Affinity亲和力参数
kubectl get nodes -o=custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
### 节点创建label
kubectl get nodes
kubectl label nodes k8s-master01 k8s-master02 k8s-master03 master=manager
kubectl label nodes k8s-node01 ssd=fastest
kubectl label nodes k8s-node02 sata=faster
kubectl get nodes --show-labels | grep master=manager
kubectl get nodes --show-labels | grep ssd
kubectl get nodes --show-labels | grep sata
### 创建Deployment
mkdir -p /data/yaml/affinity
cat > /data/yaml/affinity/nginx-deploy-select.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-deploy-select
name: nginx-deploy-select
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: ssd
operator: In
values:
- "fastest"
- key: master
operator: NotIn
values:
- "manager"
- weight: 50
preference:
matchExpressions:
- key: sata
operator: In
values:
- "faster"
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/dockerghost/nginx:1.26
imagePullPolicy: IfNotPresent
EOF
kubectl create -f /data/yaml/affinity/nginx-deploy-select.yaml
kubectl get deploy nginx-deploy-select -n default -oyaml
### 查看Pod节点部署节点
kubectl get pods -n default -owide
### 删除Deployment
kubectl delete -f /data/yaml/affinity/deploy-podantaffinity-hostname.yaml
### 删除Deployment和节点标签
kubectl delete -f /data/yaml/affinity/nginx-deploy-select.yaml
kubectl label nodes k8s-node01 k8s-master01 k8s-master02 k8s-master03 master-
kubectl label nodes k8s-node01 ssd-
kubectl label nodes k8s-node02 sata-
总结
以上为个人经验,希望能给大家一个参考,也希望大家多多支持脚本之家。
