Skip to main content

k3s

本文内容​

  • 通过binary方式部署轻量级kubernertes(K3S)
  • 配置K3S使用私有镜像仓库

部署背景​

什么是K3S​

K3s 是轻量级的 Kubernetes。K3s 易于安装，而且仅需要 Kubernetes 内存的一半，所有组件都在一个小于 100 MB 的二进制文件中。

安装包准备​

  1. k3s-airgap-images-amd64.tar.gz. K3S镜像包，下载地址 https://github.com/k3s-io/k3s/releases . 由于服务器的CPU是X86_64架构，所以选择 amd64版本.
  2. k3s.K3S 可执行文件. 下载地址 https://github.com/k3s-io/k3s/releases
  3. install.sh. K3S安装脚本.下载地址 https://get.k3s.io
  4. k3s-selinux-1.4-1.el7.noarch.rpm. 非必须，只有当你的服务器需要SELinux时才需要安装。下载地址https://github.com/k3s-io/k3s-selinux/releases/tag/v1.4.stable.1

K3S安装​

将安装包上传到目标服务器,并放在同一个目录
安装SELinux支持（如不需要可跳过此步骤）
假如出现缺少container-selinux 依赖的报错，下载安装 container-selinux-2.107-3.el7.noarch.rpm

$ sudo rpm -ivh container-selinux-2.107-3.el7.noarch.rpm
$ sudo rpm -ivh k3s-selinux-1.4-1.el7.noarch.rpm

关闭firewalld

$ sudo systemctl disable firewalld --now

如果你希望保持firewalld的开启状态，那需要为K3S加几条规则

$ firewall-cmd --permanent --add-port=6443/tcp #apiserver
$ firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 #pods
$ firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 #services
$ firewall-cmd --reload

安装K3S

$ sudo mkdir -p /var/lib/rancher/k3s/agent/images/
$ sudo cp ./k3s-airgap-images-amd64.tar.gz /var/lib/rancher/k3s/agent/images/
$ sudo cp ./k3s /usr/local/bin/
$ sudo chmod +x /usr/local/bin/k3s
$ INSTALL_K3S_SELINUX_WARN=true INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh --write-kubeconfig-mode=644
[INFO]  Skipping k3s download and verify
[INFO]  Skipping installation of SELinux RPM
[WARN]  Failed to find the k3s-selinux policy, please install:
    yum install -y container-selinux
    yum install -y https://rpm.rancher.io/k3s/stable/common/centos/7/noarch/

[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink from /etc/systemd/system/multi-user.target.wants/k3s.service to /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

参数说明:

INSTALL_K3S_SELINUX_WARN=true: 当缺少k3s-selinux依赖时，也会继续安装
INSTALL_K3S_SKIP_DOWNLOAD=true: 跳过下载k3s安装包
--write-kubeconfig-mode=644: 修改k3s配置文件的权限，避免普通用户无法使用kubectl的情况

验证

$ systemctl status k3s
● k3s.service - Lightweight Kubernetes
   Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2023-12-24 09:52:44 EST; 1min 4s ago
     Docs: https://k3s.io
  Process: 11405 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
  Process: 11402 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
  Process: 11399 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
 Main PID: 11408 (k3s-server)
    Tasks: 59
   Memory: 486.0M
   CGroup: /system.slice/k3s.service
           ├─11408 /usr/local/bin/k3s server
           └─11462 containerd 
$ kubectl get pod -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   local-path-provisioner-84db5d44d9-ws8wq   1/1     Running   0          19s
kube-system   metrics-server-67c658944b-xj6mr           1/1     Running   0          19s
kube-system   coredns-6799fbcd5-6k2ff                   1/1     Running   0          19s

如果pod/coredns出现报错plugin/forward: no nameservers found,编辑configmap kubectl edit cm coredns -n kube-system, 将 forward . /etc/resolv.conf这一行注释。保存退出后，删除pod/coredns

私有镜像仓库配置​

我们可以将容器配置为连接到私有镜像仓库，并在节点上使用私有镜像仓库拉取私有镜像。K3s在启动时会检查/etc/rancher/k3s/中是否存在registries.yaml文件，存在的话在启动容器的时候会使用该文件中定义的镜像仓库。

$ sudo vi /etc/rancher/k3s/registries.yaml
mirrors:
  docker.io:
    endpoint:
      - "http://127.0.0.1:5000"
$ sudo systemctl restart k3s

比如上面的配置，当我们容器需要的镜像仓库是docker.io时，k3s会从127.0.0.1:5000仓库来拉取镜像。