1
Cilium v1.10.6 安装部署
source link: https://blog.51cto.com/liujingyu/5260550
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Cilium v1.10.6 安装部署
推荐 原创1.系统要求
系统要求
Requirements:
- Kernel >= 5.10
- Direct-routing configuration or tunneling
- eBPF-based kube-proxy replacement
- eBPF-based masquerading
总结建议：
- 采用 Ubuntu 20.04 Desktop 版本，kernel = 5.13.0
- 安装 k8s 时，使用
--skip-phases=addon/kube-proxy 选项来跳过 kube-proxy 安装 - 默认关闭防火墙，
ufw status 进行确认是否为 disable 状态 - 默认使用kubeadm部署集群，可以忽略 Key-Value store etcd >= 3.1.0 or consul >= 0.6.4
2. kubeadm 部署 k8s 1.23.5
kubeadm 部署 kubernetes 之前需要跳过 kube-proxy 的安装
具体部署步骤： kubeadm 部署 k8s
kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --kubernetes-version=1.23.5 --apiserver-advertise-address=$(hostname -i) --skip-phases=addon/kube-proxy
cilium 部署
1.预先安装 cilium-cli 工具
cilium-cli
2.使用 Helm 部署 cilium
添加 helm repo 仓库
helm repo add cilium https://helm.cilium.io/
helm repo update
helm repo update
下载 cilium 1.10.6 包至本地，并修改如下内容
helm pull cilium/cilium --version 1.10.6
tar -xf cilium-1.10.6.tgz
# 更换镜像下载地址为 docker hub 镜像下载站
sed -i "s/quay.io/docker.io/g" cilium/values.yaml
tar -xf cilium-1.10.6.tgz
# 更换镜像下载地址为 docker hub 镜像下载站
sed -i "s/quay.io/docker.io/g" cilium/values.yaml
修改为 debug 模式
将 kubeProxyReplacement 修改为 strict 直连模式
修改 k8sServiceHost 和 k8sServicePort 参数为 master 的 nodeip 和 apiServer 的端口，默认为 6443
通过 Helm template 渲染 yaml 进行二次更改(debug 模式需要更改)
helm template --namespace kube-system cilium/ > 1.10.6.yaml
添加 debug-verbose 参数为 datapath 和修改 monitor-aggregation 参数为 none
如果就像一次进入到 BPF 的模式，则需要多添加一行参数，在 debug 或者 monitor-aggregation 下添加即可，如果未添加也无妨，下边有步骤会进行添加
enable-bpf-masquerade: "true"
部署 cilium
kubectl apply -f 1.10.6.yaml
使用 cilium 命令查看部署情况
# 进入任意一个 cilium 的 pod 中
kubectl exec -it cilium-2md6v -- bash
# 查看 cilium 的状态
# 发现为 Legacy ，说明配置还是有问题
# 依旧使用 IPTables
root@master:~/test# kubectl exec -it cilium-2md6v -- bash
Defaulted container "cilium-agent" out of: cilium-agent, mount-cgroup (init), clean-cilium-state (init)
root@master:/home/cilium# cilium status
KVStore: Ok Disabled
Kubernetes: Ok 1.23 (v1.23.5) [linux/amd64]
Kubernetes APIs: ["cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "core/v1::Namespace", "core/v1::Node", "core/v1::Pods", "core/v1::Service", "discovery/v1::EndpointSlice", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: Strict [ens33 192.168.0.110 (Direct Routing)]
Cilium: Ok 1.10.6 (v1.10.6-17d3d15)
NodeMonitor: Listening for events on 128 CPUs with 64x4096 of shared memory
Cilium health daemon: Ok
IPAM: IPv4: 4/254 allocated from 10.0.2.0/24,
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: IPTables [IPv4: Enabled, IPv6: Disabled]
Controller Status: 27/27 healthy
Proxy Status: OK, ip 10.0.2.233, 0 redirects active on ports 10000-20000
Hubble: Ok Current/Max Flows: 4095/4095 (100.00%), Flows/s: 37.29 Metrics: Disabled
Encryption: Disabled
Cluster health: 3/3 reachable (2022-04-28T06:10:11Z)
# 通过 iptables-save 查看到有许多 cilium 的链，这说明默认还是用 iptables 做的转发，并没有使用 BPF 的特性
kubectl exec -it cilium-2md6v -- bash
# 查看 cilium 的状态
# 发现为 Legacy ，说明配置还是有问题
# 依旧使用 IPTables
root@master:~/test# kubectl exec -it cilium-2md6v -- bash
Defaulted container "cilium-agent" out of: cilium-agent, mount-cgroup (init), clean-cilium-state (init)
root@master:/home/cilium# cilium status
KVStore: Ok Disabled
Kubernetes: Ok 1.23 (v1.23.5) [linux/amd64]
Kubernetes APIs: ["cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "core/v1::Namespace", "core/v1::Node", "core/v1::Pods", "core/v1::Service", "discovery/v1::EndpointSlice", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: Strict [ens33 192.168.0.110 (Direct Routing)]
Cilium: Ok 1.10.6 (v1.10.6-17d3d15)
NodeMonitor: Listening for events on 128 CPUs with 64x4096 of shared memory
Cilium health daemon: Ok
IPAM: IPv4: 4/254 allocated from 10.0.2.0/24,
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: IPTables [IPv4: Enabled, IPv6: Disabled]
Controller Status: 27/27 healthy
Proxy Status: OK, ip 10.0.2.233, 0 redirects active on ports 10000-20000
Hubble: Ok Current/Max Flows: 4095/4095 (100.00%), Flows/s: 37.29 Metrics: Disabled
Encryption: Disabled
Cluster health: 3/3 reachable (2022-04-28T06:10:11Z)
# 通过 iptables-save 查看到有许多 cilium 的链，这说明默认还是用 iptables 做的转发，并没有使用 BPF 的特性
再次修改 cilium 配置
# 通过 Falling 关键字，查看是否降级
root@master:~# kubectl logs cilium-2md6v | grep -i fall
level=info msg=" --enable-xt-socket-fallback='true'" subsys=daemon
level=debug msg="Skipping Leases support fallback discovery" subsys=k8s
level=info msg="BPF host routing requires enable-bpf-masquerade. Falling back to legacy host routing (enable-host-legacy-routing=true)." subsys=daemon
root@master:~# kubectl logs cilium-2md6v | grep -i fall
level=info msg=" --enable-xt-socket-fallback='true'" subsys=daemon
level=debug msg="Skipping Leases support fallback discovery" subsys=k8s
level=info msg="BPF host routing requires enable-bpf-masquerade. Falling back to legacy host routing (enable-host-legacy-routing=true)." subsys=daemon
在我们刚才渲染的 1.10.6 yaml 中添加如下一行语句
monitor-aggregation: "none"
enable-bpf-masquerade: "true"
enable-bpf-masquerade: "true"
# 修改完以后，需要先删除然后重新部署 cilium
kubectl delete -f 1.10.6.yaml
kubectl apply -f 1.10.6.yaml
# 查看任意一个 cilium 是否还有 Falling 的日志
<sub># kubectl logs cilium-fxdkr | grep -i fall
level=info msg=" --enable-xt-socket-fallback='true'" subsys=daemon
level=debug msg="Skipping Leases support fallback discovery" subsys=k8s
# 确认 cilium 是否使用 bpf 进行源目的地址转换
</sub># kubectl exec -it cilium-fxdkr -- bash
root@node-2:/home/cilium# cilium status
kubectl delete -f 1.10.6.yaml
kubectl apply -f 1.10.6.yaml
# 查看任意一个 cilium 是否还有 Falling 的日志
<sub># kubectl logs cilium-fxdkr | grep -i fall
level=info msg=" --enable-xt-socket-fallback='true'" subsys=daemon
level=debug msg="Skipping Leases support fallback discovery" subsys=k8s
# 确认 cilium 是否使用 bpf 进行源目的地址转换
</sub># kubectl exec -it cilium-fxdkr -- bash
root@node-2:/home/cilium# cilium status
验证 BPF
在master 节点查看有多少个 pod 被 cilium 管理
cilium status
创建一个 deployment，expose 创建一个 svc ，查看是否还有 iptables 规则生成
# 创建 deployment
kubectl create deploy --name test-nginx --image=nginx:latest --replicas=1
# 创建 svc
kubectl expose deployment test-nginx --port=80 --target-port=80 --type=NodePor
kubectl create deploy --name test-nginx --image=nginx:latest --replicas=1
# 创建 svc
kubectl expose deployment test-nginx --port=80 --target-port=80 --type=NodePor
测试已经通过
查看 pod 所在节点的iptables 规则是否含有 test-nginx 关键字
iptables-save | grep "test-nginx"
查看 cilium 管理的 pod 是否由之前的 2，变更为 3
- 打赏
- 赞
- 收藏
- 评论
- 分享
- 举报
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK