k8s部署与安装

原创已于 2026-05-19 20:17:05 修改 · 369 阅读

7 ·

本内容遵循CC 4.0 BY-SA版权协议

GEO检测

标签

#k8s

于 2026-05-19 20:15:59 首次发布

k8s 专栏收录该内容

3 篇文章

订阅专栏

一、介绍

2021 百度k8s版本1.18

2023 百度k8s版本1.2x

k8s至今都没有一个长期支持版LTS

(一) k8s有两种安装方式：

1. kubeadm工具

容器引擎、kubelet：使用yum安装

其他k8s组件：

都被做成了镜像，然后kubeadm会用这些镜像启动相关组件---》容器

kubeadm会把这些容器做成静态pod（没有控制器管理，挂掉会自动重启）

总结：这些静态pod+容器引擎+kubelet===》构成了k8s环境

2. 二进制安装

自己通过yum命令或者源码安装来安装所有组件

二、kubeadm安装k8s

(一) 准备工作

修改主机名和解析

# 1、修改主机名
hostnamectl set-hostname k8s-master-01
hostnamectl set-hostname k8s-node-01
hostnamectl set-hostname k8s-node-02

# 2、三台机器添加host解析
cat >> /etc/hosts << "EOF"
192.168.71.12 k8s-master-01 m1
192.168.71.13 k8s-node-01 n1
192.168.71.14 k8s-node-02 n2
EOF

关闭一些不要的服务

# 1、关闭selinux
sed -i 's#enforcing#disabled#g' /etc/selinux/config
setenforce 0
 
# 2、禁用防火墙，网络管理，邮箱
systemctl disable --now firewalld NetworkManager postfix
 
# 3、关闭swap分区
swapoff -a 

# 注释swap分区
cp /etc/fstab /etc/fstab_bak
sed -i '/swap/d' /etc/fstab

sshd优化

# 1、加速访问
sed -ri 's@^#UseDNS yes@UseDNS no@g' /etc/ssh/sshd_config 
sed -ri 's#^GSSAPIAuthentication yes#GSSAPIAuthentication no#g' /etc/ssh/sshd_config 
grep ^UseDNS /etc/ssh/sshd_config 
grep ^GSSAPIAuthentication /etc/ssh/sshd_config
systemctl restart sshd
 
# 2、密钥登录（主机点做）:为了让后续一些远程拷贝操作更方便
ssh-keygen
ssh-copy-id -i root@k8s-master-01
ssh-copy-id -i root@k8s-node-01
ssh-copy-id -i root@k8s-node-02

增大文件打开的数量（退出当前会话会自动生效）

cat > /etc/security/limits.d/k8s.conf <<'EOF' 
* soft nofile 65535 
* hard nofile 131070 
EOF 
 
ulimit -Sn 
ulimit -Hn

所有节点配置模块自动加载（此步骤不做，kubeadm init 会直接失败！！！）

modprobe br_netfilter
modprobe ip_conntrack
cat >>/etc/rc.sysinit<<EOF
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done
EOF
echo "modprobe br_netfilter" >/etc/sysconfig/modules/br_netfilter.modules
echo "modprobe ip_conntrack" >/etc/sysconfig/modules/ip_conntrack.modules
chmod 755 /etc/sysconfig/modules/br_netfilter.modules
chmod 755 /etc/sysconfig/modules/ip_conntrack.modules
lsmod | grep br_netfilter

同步集群时间

# =====================》chrony服务端：服务端我们可以自己搭建，也可以直接用公网上的时间服务器，所以是否部署服务端看你自己
# 1、安装
yum -y install chrony

# 2、修改配置文件
mv /etc/chrony.conf /etc/chrony.conf.bak

cat > /etc/chrony.conf << EOF
server ntp1.aliyun.com iburst minpoll 4 maxpoll 10
server ntp2.aliyun.com iburst minpoll 4 maxpoll 10
server ntp3.aliyun.com iburst minpoll 4 maxpoll 10
server ntp4.aliyun.com iburst minpoll 4 maxpoll 10
server ntp5.aliyun.com iburst minpoll 4 maxpoll 10
server ntp6.aliyun.com iburst minpoll 4 maxpoll 10
server ntp7.aliyun.com iburst minpoll 4 maxpoll 10
driftfile /var/lib/chrony/drift
makestep 10 3
rtcsync
allow 0.0.0.0/0
local stratum 10
keyfile /etc/chrony.keys
logdir /var/log/chrony
stratumweight 0.05
noclientlog
logchange 0.5
 
EOF

# 4、启动chronyd服务
systemctl restart chronyd.service # 最好重启，这样无论原来是否启动都可以重新加载配置
systemctl enable chronyd.service
systemctl status chronyd.service
 
# =====================》chrony客户端：在需要与外部同步时间的机器上安装，启动后会自动与你指定的服务端同步时间
# 下述步骤一次性粘贴到每个客户端执行即可
# 1、安装chrony
yum -y install chrony
# 2、需改客户端配置文件
mv /etc/chrony.conf /etc/chrony.conf.bak
cat > /etc/chrony.conf << EOF
server 服务端的ip地址或可解析的主机名 iburst
driftfile /var/lib/chrony/drift
makestep 10 3
rtcsync
local stratum 10
keyfile /etc/chrony.key
logdir /var/log/chrony
stratumweight 0.05
noclientlog
logchange 0.5
 
EOF
# 3、启动chronyd
systemctl restart chronyd.service
systemctl enable chronyd.service
systemctl status chronyd.service
 
# 4、验证
chronyc sources -v

更新基础yum源

# 1、清理
rm -rf /etc/yum.repos.d/*
yum remove epel-release -y
rm -rf /var/cache/yum/x86_64/6/epel/
 
# 2、安装阿里的base与epel源
curl -s -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo 
curl -s -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
yum clean all yum makecache
 
# 或者用华为的也行
# curl -o /etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo 
# yum install -y https://repo.huaweicloud.com/epel/epel-release-latest-7.noarch.rpm

更新系统软件（排除内核）

 yum update -y --exclud=kernel*

安装基础常用软件

yum -y install expect wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 git ntpdate chrony bind-utils rsync unzip git

所有节点安装IPVS

# 1、安装ipvsadm等相关工具
yum -y install ipvsadm ipset sysstat conntrack libseccomp 
 
# 2、配置加载
cat > /etc/sysconfig/modules/ipvs.modules <<"EOF" 
#!/bin/bash 
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack" 
 
for kernel_module in ${ipvs_modules}; 
do 
	/sbin/modinfo -F filename ${kernel_module} > /dev/null 2>&1 
	if [ $? -eq 0 ]; then 
		/sbin/modprobe ${kernel_module} 
	fi 
done 
EOF
 
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs

修改内核参数

cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp.keepaliv.probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp.max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp.max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.top_timestamps = 0
net.core.somaxconn = 16384
EOF
 
# 立即生效
sysctl --system

(二) 新版1.30.3：https://egonlin.com/?p=10762

规划：

master：192.168.71.111

node01：192.168.71.112

node02：192.168.71.113

每台机器内存>=2G

配置好静态ip地址

1. 需要升级内核（docker对系统内核要求较高，最好用4.4+）

wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-5.4.274-1.el7.elrepo.x86_64.rpm
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.274-1.el7.elrepo.x86_64.rpm
 
for i in n1 n2 m1 ; do scp kernel-lt-* $i:/opt; done
 
 
补充：如果下载的慢就从网盘里拿吧
链接：https://pan.baidu.com/s/1gVyeBQsJPZjc336E8zGjyQ 
提取码：Egon

三个节点做

 #安装
yum localinstall -y /opt/kernel-lt*
 
#调到默认启动
grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg 
 
#查看当前默认启动的内核
grubby --default-kernel
 
#重启系统
reboot

2. 安装containerd（所有节点）

1、#升级centos7默认的libseccomp的版本为2.3.1，不满足containerd的需求，需要下载2.4以上的版本即我这里部署2.5.1版本。
rpm -qa | grep libseccomp
rpm -e libseccomp-2.3.1-4.el7.x86_64 --nodeps
https://mirrors.aliyun.com/centos/8/Base0S/x86_64/os/Packages
https://mirrors.aliyun.com/centos/8/BaseOS/x86_64/os/Packages/libseccomp-2.5.1-1.el8.x86_64.rpm
rpm -ivh libseccomp-2.5.1-1.el8.x86_64

2、#安装containerd
wget http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install containerd* -y

3、#为container生成配置文件 
mkdir -pv /etc/containerd
containerd config default>/etc/containerd/config.toml

4、#修改配置文件的pause镜像的地址
grep sandbox_image /etc/containerd/config.toml
sed -i 's/registry.k8s.io/registry.cn-hangzhou.aliyuncs.com\/google_containers/g' /etc/containerd/config.toml
#配置加速器，否则后续安装无法从dockers.io中下载镜像
grep config_path /etc/containerd/config.toml
sed -i 's/config_path\ =.*/config_path = \"\/etc\/containerd\/certs.d\"/g' /etc/containerd/config.toml
#配置systemd作为容器的cgroup driver
grep SystemdCgroup /etc/containerd/config.toml
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/' /etc/containerd/config.toml

cat > /etc/containerd/certs.d/docker.io/hosts.toml << EOF
server = "https://docker.io"
[host."https://dockerproxy.com"]
capabilities = ["pull", "resolve"]

[host."https://docker.m.daocloud.io"]
capabilities = ["pull", "resolve"]

[host."https://docker.agsv.top"]
capabilities = ["pull", "resolve"]

[host."https://registry.docker-cn.com"]
capabilities = ["pull", "resolve"]

EOF

#5、配置containerd开机自启动
#5.1启动containerd服务并配置开机自启动
systemctl daemon-reload && systemctl restart containerd
systemctl enable --now containerd
#5.2 查看containerd状态
systemctl status containerd
#5.3 查看containerd的版本
ctr version

3. 安装k8s

安装kubelet组件,kubeadm,kubectl

#yum仓库
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.33/rpm/repodata/repomd.xml.key
 
EOF
# 第一步：在所有节点执行
yum install -y kubelet-1.33* kubeadm-1.33* kubectl-1.33*	#会安装依赖kubenets-cni,这些插件在/opt/cni/下，后面还会安装flannel，这些都是和网络有关的，rpm -qf 
systemctl enable kubelet && systemctl start kubelet && systemctl status kubelet

4. 部署，master做kubeadm init

1、#初始查看
kubeadm config images list

2、#生成配置文件安装，可以指定ipvs
kubeadm config print init-defaults > kubeadm.yaml

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.0.0.111		# 控制节点
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock	# 指定containerd容器运行时
  imagePullPolicy: IfNotPresent
  name: k8s-master-01
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers	# 指定镜像仓库地址
kind: ClusterConfiguration
kubernetesVersion: 1.30.14
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12		# 指定service的网段
  podSubnet: 10.244.0.0/16		# 指定pod的网段
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs # kube-proxy代理模式是ipvs,默认是iptables,但效率低。阿里云和华为云托管的K8s，也支持
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

# 第二步：只在master节点上执行
[root@k8s-master-01 ~]# kubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification --ignore-preflight-errors=Swap

# 去污点
kubectl taint node tool node-role.kubernetes.io/control-plane:NoSchedule-

。。。。。。。。。。。。。。。。。。。。。。。
Your Kubernetes control-plane has initialized successfully!
 
To start using your cluster, you need to run the following as a regular user:
 
  mkdir -p $HOME/.kube		# 验证信息
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
 
Alternatively, if you are the root user, you can run:
 
  export KUBECONFIG=/etc/kubernetes/admin.conf
 
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
 
Then you can join any number of worker nodes by running the following on each as root:
 
kubeadm join 10.0.0.111:6443 --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:75511c06c1a4563545edbe34b3d9d94e0a4428070ecd5316ecf81fe6ba8aaa5f

5. 把node加入master

#kubeadm身份验证信息
scp .kube/config n1:/root/.kube/
#在node用kubeadm加入master
kubeadm join 10.0.0.111:6443 --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:f918cfb87a025d365cd40e31cba327b2ea6df0d849b1292f21cf24c4d4351b02

6. 查看node节点状态

[root@k8s-master-01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master-01 NotReady control-plane 4m26s v1.30.0

kubectl get namespace

 [root@k8s-master-01 ~]# kubectl -n kube-system get pods
NAME READY STATUS RESTARTS AGE
coredns-7c445c467-mfls7 0/1 Pending 0 6m30s
coredns-7c445c467-zvkkw 0/1 Pending 0 6m30s
etcd-k8s-master-01 1/1 Running 0 6m44s
kube-apiserver-k8s-master-01 1/1 Running 0 6m44s
kube-controller-manager-k8s-master-01 1/1 Running 0 6m44s
kube-proxy-jhxrd 1/1 Running 0 109s
kube-proxy-nh7tj 1/1 Running 0 33s
kube-proxy-q92mx 1/1 Running 0 6m30s
kube-scheduler-k8s-master-01 1/1 Running 0 6m44s
 kubectl -n kube-system get pods -o wide#在哪个节点

7. 安装网络插件

 https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
# 第三步：部署网络插件，镜像要改成阿里云代理拉取国外镜像
crpi-dq0f9bstnnzg0kea.cn-chengdu.personal.cr.aliyuncs.com/yuezheng_namespace/fl:v0.27.4
pull crpi-dq0f9bstnnzg0kea.cn-chengdu.personal.cr.aliyuncs.com/yuezheng_namespace/1flannel-cni:v1.8.0-flannel1
kubectl apply -f kube-flannel.yml 			#使用yaml文件安装拉取镜像

kubectl -n kube-flannel get pods
kubectl -n kube-flannel get pods -w
[root@k8s-master-01 ~]# kubectl -n kube-flannel get pods -w # 全部ready
[root@k8s-master-01 ~]# kubectl -n kube-system get pods # 两个coredns的pod也都ready

(三) 命令补全

#所有节点
yum install bash-completion* -y
 
kubectl completion bash > ~/.kube/completion.bash.inc
echo "source '$HOME/.kube/completion.bash.inc'" >> $HOME/.bash_profile
source $HOME/.bash_profile

三、出错解决

(一) 强调：一旦kubeadm init报错，解决步骤

1、修复问题（把错误的、遗漏的地方改掉）

2、清理环境
kubeadm reset -f
rm -rf ~/.kube/
rm -rf /etc/kubernetes/
rm -rf /etc/cni
rm -rf /opt/cni
rm -rf /var/lib/etcd
rm -rf /var/etcd

3、重新执行kubeadm init命令

# 添加node节点的命令
kubeadm join 192.168.71.111:6443 --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:7caa76286b93555d25e527aa551b541ae05f2624c57c605d5dba21f1ef22fed2

registry.cn-shanghai.aliyuncs.com/egon-k8s-test/flannel:v0.25.5
registry.cn-shanghai.aliyuncs.com/egon-k8s-test/flannel-cni-plugin:v1.5.1-flannel1

(二) 部署遇到问题之后

1. 铲掉环境重新部署

# 在master节点上
kubeadm reset -f


# 在所有节点包括master节点在内上执行如下命令
cd /tmp # 有时候在当前目录下可能与要卸载的包重名的而导致卸载报错，可以切个目录
rm -rf ~/.kube/	凭证信息
rm -rf /etc/kubernetes/	#kubeadm init初始放的东西都在这下面，包括静态pod配置信息
rm -rf /etc/cni		#cni的配置文件
rm -rf /opt/cni	#cni的插件
rm -rf /var/lib/etcd
rm -rf /var/etcd

rm -rf /run/flannel		#flannel插件
rm -rf /opt/cni
rm -rf /etc/cni/net.d
rm -rf /run/xtables.lock		#锁文件


systemctl stop kubelet
yum remove kube* -y

for i in `df |grep kubelet |awk '{print $NF}'`;do umount -l $i ;done # 先卸载所有kubelet挂载否则下条命令无法删除
rm -rf /var/lib/kubelet		#他的数据目录文件
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
rm -rf /usr/bin/kube*

iptables -F

reboot # 重新启动，从头再来

2. 安装k8s

ubantu 22.0.4