Kubernetes-离线部署Kubernetes 1.9.0

  1. 云栖社区>
  2. 博客>
  3. 正文

Kubernetes-离线部署Kubernetes 1.9.0

statmoon 2018-07-25 14:27:00 浏览698
展开阅读全文

准备工作


  1. 准备四台机器,基本信息如下:
IP hostname Role OS Memery
192.168.242.136 k8smaster Kubernetes master 节点 CentOS 7.2 3G
192.168.242.137 k8snode1 Kubernetes node 节点 CentOS 7.2 2G
192.168.242.138 k8snode2 Kubernetes node 节点 CentOS 7.2 2G
192.168.242.139 k8snode3 Kubernetes node 节点 CentOS 7.2 2G
  1. 设置master节点到node节点的免密登录,具体方法请参考这里
  2. 每台机器【/etc/hosts】文件需包含:
    192.168.242.136 k8smaster
    192.168.242.137 k8snode1
    192.168.242.138 k8snode2
    192.168.242.139 k8snode3
    CentOS修改机器名参考这里
  3. 每台机器预装【docker 17.03.2-ce】,安装步骤参考这里
  4. 关闭所有机器防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
img_b0fe63238743803597188734a6e4b6c4.png
关闭防火墙.png
  1. 所有机器关闭selinux,使容器能够访问到宿主机文件系统
vim /etc/selinux/config

将【SELINUX】设置为【disabled】

img_3dfbf4cd39340de21f91c93070827675.png
关闭selinux.png

临时关闭selinux

setenforce 0
  1. 配置系统路由参数,防止kubeadm报路由警告
    在【/etc/sysctl.d/】目录下新建一个Kubernetes的配置文件【kubernetes.conf】,并写入如下内容:
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
img_899d90ec8ea9b34f8bbea49b9d44a01d.png
kubernetes.conf.png

运行如下命令使配置生效

sysctl --system
img_5f8d26d99a1b77bafd0fa60c8e9e1a5f.png
生效配置.png

【注意】我这里是新增了一个配置文件,而不是直接写到文件【/etc/sysctl.conf】中,所以生效配置的命令参数是【--system】,如果是直接写到文件【/etc/sysctl.conf】中,那么生效命令的参数是【-p】。

  1. 关闭虚拟内存
    修改配置文件【/etc/fstab】
vim /etc/fstab

  注释掉swap那一行

img_3d2b6b7a21b505173865232079255e41.png
image.png

  然后通过命令临时关闭虚拟内存

swapoff -a

  如果不关闭swap,就会在kubeadm初始化Kubernetes的时候报错

[ERROR Swap]: running with swap on is not supported. Please disable swap
img_2c30f0aa8157c1cc583f2f7a30354f26.png
ERROR Swap
  1. 准备镜像
    我是参考的这篇博客进行搭建的,所以我这里的镜像都是从该博客提供的地址下载的,将镜像压缩包上传到各节点。
img_24d8471b8b610ca230c0b06264fb7074.png
镜像压缩包.png

使用解压命令解压

tar -jxvf k8s_images.tar.bz2
img_aa9801717e6d470c57319ffcd4954329.png
安装包.png

然后导入镜像

docker load -i /usr/local/k8s_images/docker_images/etcd-amd64_v3.1.10.tar
docker load -i /usr/local/k8s_images/docker_images/flannel:v0.9.1-amd64.tar
docker load -i /usr/local/k8s_images/docker_images/k8s-dns-dnsmasq-nanny-amd64_v1.14.7.tar
docker load -i /usr/local/k8s_images/docker_images/k8s-dns-kube-dns-amd64_1.14.7.tar
docker load -i /usr/local/k8s_images/docker_images/k8s-dns-sidecar-amd64_1.14.7.tar
docker load -i /usr/local/k8s_images/docker_images/kube-apiserver-amd64_v1.9.0.tar
docker load -i /usr/local/k8s_images/docker_images/kube-controller-manager-amd64_v1.9.0.tar
docker load -i /usr/local/k8s_images/docker_images/kube-proxy-amd64_v1.9.0.tar
docker load -i /usr/local/k8s_images/docker_images/kube-scheduler-amd64_v1.9.0.tar
docker load -i /usr/local/k8s_images/docker_images/pause-amd64_3.0.tar
docker load -i /usr/local/k8s_images/kubernetes-dashboard_v1.8.1.tar

路径请按照镜像解压路径填写,全部导入成功后通过命令【docker images】可查看到导入成功的镜像。

img_208004c6352999bd5981281e256c1afc.png
镜像.png

到这里,前期的准备工作就全部完成了,下面就要开始安装了。

搭建Kubernetes集群


  1. 在所有节点上部署socat、kubernetes-cni、kubelet、kubectl、kubeadm。
rpm -ivh /usr/local/k8s_images/socat-1.7.3.2-2.el7.x86_64.rpm
rpm -ivh /usr/local/k8s_images/kubernetes-cni-0.6.0-0.x86_64.rpm /usr/local/k8s_images/kubelet-1.9.9-9.x86_64.rpm
rpm -ivh /usr/local/k8s_images/kubectl-1.9.0-0.x86_64.rpm
rpm -ivh /usr/local/k8s_images/kubeadm-1.9.0-0.x86_64.rpm
img_c37d61a053d580c86cdc241fb03130b8.png
install kubelet/kubectl/kubeadm.png

  接着修改kubelet的配置文件

vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
img_4a407be9770b9c80ad6c0ac97e219e5d.png
kubelet config

  kubelet的【cgroup-driver】需要和docker的保持一致,通过命令【docker info】可以查看docker的【Cgroup Driver】属性值。

img_aba9f285a6e29486691344bfced3d1bc.png
docker info

  这里可以看到docker的【Cgroup Driver】是【cgroupfs】,所以这里需要将kubelet的【cgroup-driver】也修改为【cgroupfs】。
  修改完成后重载配置文件

systemctl daemon-reload

设置kubelet开机启动

systemctl enable kubelet
  1. 配置master节点
    2.1 初始化Kubernetes
kubeadm init --kubernetes-version=v1.9.0 --pod-network-cidr=10.244.0.0/16

kubernetes默认支持多重网络插件如flannel、weave、calico,这里使用flanne,就必须要设置【--pod-network-cidr】参数,10.244.0.0/16是kube-flannel.yml里面配置的默认网段,这里的【--pod-network-cidr】参数要和【kube-flannel.yml】文件中的【Network】参数对应。

img_ce766dc4fbb37508d21499f704f6c79f.png
kube-flannel.yml

初始化输入如下:

[root@k8smaster ~]# kubeadm init --kubernetes-version=v1.9.0 --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.9.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
    [WARNING FileExisting-crictl]: crictl not found in system path
[preflight] Starting the kubelet service
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [k8smaster kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.242.136]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.
[apiclient] All control plane components are healthy after 44.002305 seconds
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[markmaster] Will mark node k8smaster as master by adding a label and a taint
[markmaster] Master k8smaster tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: abb43a.62186b817d71bcd2
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token abb43a.62186b817d71bcd2 192.168.242.136:6443 --discovery-token-ca-cert-hash sha256:6a7625aa2928085fde84cfd918398408771dfe6af5c88c73b2d47527a00a8dad

将【 kubeadm join --token xxxx】这段记下来,加入node节点需要用到这个令牌,如果忘记了可以使用如下命令查看

kubeadm token list
img_c9ec0427de3f96de0c57bcf765b7e11f.png
查看token

令牌的时效性是24个小时,如果过期了可以使用如下命令创建

kubeadm token create

2.2 配置环境变量
此时root用户还不能使用kubelet控制集群,需要按照以下方法配置环境变量
将信息写入bash_profile文件

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile

运行命令立即生效

source ~/.bash_profile

查看版本测试下

kubectl version
img_519d8e761458e7e9ddbd9fbe5a0c45b5.png
version

2.3 安装flannel
直接使用离线包里面的【kube-flannel.yml】

kubectl create -f /usr/local/k8s_images/kube-flannel.yml
img_64a3e5098b39804536a10dfcf87a7888.png
安装flannel
  1. 配置node节点
    使用配置master节点初始化Kubernetes生成的token将3个node节点加入master,参见2.1,分别在每个node节点上运行如下命令:
kubeadm join --token abb43a.62186b817d71bcd2 192.168.242.136:6443 --discovery-token-ca-cert-hash sha256:6a7625aa2928085fde84cfd918398408771dfe6af5c88c73b2d47527a00a8dad
img_474c76ce30a20a65d62ee8719a71bc7b.png
join master

全部加入后就可以到master节点上通过如下命令查看是否加入成功

kubectl get nodes
img_298cfb5c876f48c1c4a08d558b2e2798.png
查看节点

kubernetes会在每个node节点创建flannel和kube-proxy的pod,通过如下命令查看pods

kubectl get pods --all-namespaces
img_12e7a3e2b1886f39f64f6c0f6ea65838.png
pods

查看集群信息

kubectl cluster-info
img_a926dc03bfe37b86ccd36378f9bf5011.png
kubernetes cluster info

搭建dashboard


在master节点上,直接使用离线包里面的【kubernetes-dashboard.yaml】来创建

kubectl create -f /usr/local/k8s_images/kubernetes-dashboard.yaml
img_f7e37a534969ebc1ca7ee0ed1fc9a6e6.png
安装dashboard

接着设置验证方式,默认验证方式有kubeconfig和token,这里使用basicauth的方式进行apiserver的验证。
创建【/etc/kubernetes/pki/basic_auth_file】用于存放用户名、密码、用户ID。

admin,admin,2

编辑【/etc/kubernetes/manifests/kube-apiserver.yaml】文件,添加basic_auth验证

vim /etc/kubernetes/manifests/kube-apiserver.yaml

添加一行

- --basic_auth_file=/etc/kubernetes/pki/basic_auth_file
img_734e0701264e007a6392655db9513cf7.png
kube-apiserver.yaml

重启kubelet

systemctl restart kubelet

更新kube-apiserver容器

kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.yaml
img_12c59ad80dd027aeff7cd9144f8fcb09.png
update apiserver

  接下来给admin用户授权,k8s1.6后版本都采用RBAC授权模型,默认cluster-admin是拥有全部权限的,将admin和cluster-admin bind这样admin就有cluster-admin的权限。
先查看cluster-admin

kubectl get clusterrole/cluster-admin -o yaml
img_2075f8c697ff21240151c2d8efb71fee.png
cluster-admin

将admin和cluster-admin绑定

kubectl create clusterrolebinding login-on-dashboard-with-cluster-admin --clusterrole=cluster-admin --user=admin
img_23d062587a50deed96026a052d0ce344.png
绑定

然后查看一下

kubectl get clusterrolebinding/login-on-dashboard-with-cluster-admin -o yaml
img_b4db8572734823035c70877f632b8029.png
image.png

现在可以登录试试,在浏览器中输入地址【https://192.168.242.136:32666】,这里需要用Firefox,Chrome由于安全机制访问不了。

img_f103b975e8ebba28df6f255b6fc9ada4.png
Chrome

通过Firefox可以看到如下界面

img_18785f29b21191a301f7839bf0ff6aeb.png
login

选择【Basic】认证方式,输入【/etc/kubernetes/pki/basic_auth_file】文件中配置的用户名和密码登录。

img_e1c39fd61f02f154e23bc46a5a9dd961.png
基本身份认证

登录成功可以看到如下界面

img_808e616ab4fcf2dfcfdd58873f73f2ce.png
dashboard

至此,部署全部完成。

网友评论

登录后评论
0/500
评论
statmoon
+ 关注