kubernetes 1.9.0 kubeadm方式安装
1、安装rpm包
yum localinstall -y kubeadm-1.9.0-0.x86_64.rpm kubectl-1.9.0-0.x86_64.rpm kubelet-1.9.0-0.x86_64.rpm kubernetes-cni-0.6.0-0.x86_64.rpm2、修改内核参数
修改 /etc/sysctl.conf，添加以下内容
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1修改后，及时生效
sysctl -p3、修改kubelet配置文件
kubelet和docker 的cgroup driver 有2种方式:cgroupfs和systemd.注意保持 2个应用的driver保持一致。
3.1 docker是cgroupfs的，修改kubelet
vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
#修改systemd为cgroupfs
Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"
#新加一行
Environment="KUBELET_EXTRA_ARGS=--v=2 --fail-swap-on=false --pod-infra-container-image=foxchan/google_containers/pause-amd64:3.0"修改完成后
systemctl daemon-reload3.2 docker是systemd的，kubelet无需修改
docker 启动命令添加如下内容，可以修改cgroup driver为systemd
--exec-opt native.cgroupdriver=systemd4、安装 kubernetes
kubeadm init --kubernetes-version=1.9.0 --token-ttl 0参数说明
- --kubernetes-version 最好指定版本，否则会请求 https://storage.googleapis.com/kubernetes-release/release/stable-1.9.txt ，如果没"翻""墙"，就超时报错
- --token-ttl 默认的token有效期24小时， 设置为0表示永不过期
因为镜像要"翻""墙"，可以预先下载镜像，然后更改tag
镜像列表
- gcr.io/google_containers/kube-apiserver-amd64:v1.9.0
- gcr.io/google_containers/kube-controller-manager-amd64:v1.9.0
- gcr.io/google_containers/kube-scheduler-amd64:v1.9.0
- gcr.io/google_containers/pause-amd64:3.0
- gcr.io/google_containers/etcd-amd64:3.1.10
- gcr.io/google_containers/kube-proxy-amd64:v1.9.0
- gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
- gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
- gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
- quay.io/calico/node:v3.0.1
- quay.io/coreos/etcd:v3.2.4
- quay.io/calico/cni:v2.0.0
- quay.io/calico/kube-controllers:v1.0.1
如果下载自己的或者dockerhub的镜像。可以利用脚本，批量替换镜像imagename
docker images | sed 's/foxchan/gcr.io\/google_containers/'| awk '{print "docker tag "$3" "$1":"$2}'安装信息
[root@kvm-gs242024 ~]# kubeadm init --kubernetes-version=1.9.0 --token-ttl 0 --ignore-preflight-errors=all
[init] Using Kubernetes version: v1.9.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[WARNING FileExisting-crictl]: crictl not found in system path
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [kvm-gs024 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.24]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[apiclient] All control plane components are healthy after 78.502690 seconds
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[markmaster] Will mark node kvm-gs242024 as master by adding a label and a taint
[markmaster] Master kvm-gs242024 tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: 1ac970.704ce2d03cc45382
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token 1ac970.704ce2d03cc45382 192.168.0.24:6443 --discovery-token-ca-cert-hash sha256:f70f07be83a7b2af2c41752b00def4389e3019006b3be643fe1ccf1c53368043
安装后提示如何管理集群和添加node
token要记得保存，当前版本 token 无法通过命令找回，否则无法添加node
master节点验证
[root@kvm-master ~]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health": "true"}node节点验证
[root@kvm-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kvm-node1 Ready 1h v1.9.0
kvm-master NotReady master 18h v1.9.0
kvm-node2 Ready 6m v1.9.0命令行验证
curl --cacert /etc/kubernetes/pki/ca.crt --cert /etc/kubernetes/pki/apiserver-kubelet-client.crt --key /etc/kubernetes/pki/apiserver-kubelet-client.key https://k8smaster:6443管理集群使用kubectl的报错
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
https://github.com/kubernetes/kubernetes/issues/48378
解决方式： export KUBECONFIG=/etc/kubernetes/kubelet.conf 这是普通用户，没有权限，会有报错 Error from server (Forbidden): daemonsets.extensions is forbidden: User "system:node:kvm-master" cannot list daemonsets.extensions in the namespace "default"
export KUBECONFIG=/etc/kubernetes/admin.conf 管理用户
###由于安全原因，默认情况下pod不会被schedule到master节点上，可以通过下面命令解除这种限制：
kubectl taint nodes --all node-role.kubernetes.io/master-calico网络安装
先装rbac
kubectl apply -f https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/rbac.yaml
下载calico.yaml
https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml
如果使用calico自带etcd，注意保证calico etcd的稳定
如果使用自有etcd集群，需要修改yaml里的etcd_endpoints
默认安装时，如果节点是多网卡会报错，导致网络不成功，calico 总是随机绑定网卡，导致注册失败
calico 报错日志
Skipping datastore connection test
IPv4 address 10.96.0.1 discovered on interface kube-ipvs0
No AS number configured on node resource, using global value
需要修改calico.yaml,注意顺序
- name: IP
value: "autodetect"
- name: IP_AUTODETECTION_METHOD
value: "can-reach=192.168.1.1"IP_AUTODETECTION_METHOD 参数说明
使用通过ip访问的interface
can-reach=192.168.1.1
使用通过域名访问的interface
can-reach=www.baidu.com
使用指定的interface
interface=ethx
安装dashboard
下载yaml
wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml下载完成后修改imagename
name: kubernetes-dashboard
image: foxchan/google_containers/kubernetes-dashboard-amd64:v1.8.0kubectl proxy --address=masterip --accept-hosts='^*$'访问
http://masterip:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy安装heapster
下载yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml下载完成后，修改image为自己的
#grafana.yaml
- name: grafana
image: gcr.io/google_containers/heapster-grafana-amd64:v4.4.3
#heapster.yaml
- name: heapster
image: gcr.io/google_containers/heapster-amd64:v1.4.2
#influxdb.yaml
- name: influxdb
image: gcr.io/google_containers/heapster-influxdb-amd64:v1.3.3下载镜像
docker pull foxchan/heapster-grafana-amd64:v4.4.3
docker pull foxchan/heapster-amd64:v1.4.2
docker pull foxchan/heapster-influxdb-amd64:v1.3.3如果成功，dashboard页面 会有图形
将skydns替换为coredns
下载脚本
wget https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/deploy.sh
wget https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/coredns.yaml.sed利用脚本 修改coredns.yaml.sed配置文件
deploy.sh clusterip|kubectl apply我的clusterip是10.96.0.0/12
./deploy.sh 10.96.0.0/12|kubectl apply -f请确保coredns正常运行，然后就可以删除skydns
kubectl delete --namespace=kube-system deployment kube-dns另外 kubeadm 也添加了替换dns功能，需要kubeadm 1.9版本
参数如下：
kubeadm init --feature-gates=CoreDNS=true总结
总的来说1.9 和1.8 没什么大的变化，以下是我关注的
- Bootstrap Tokens 从alpha 变为beta。
- etcd 版本从3.0 升级到3.1，因为etcd 版本数据是不兼容的，如果要升级etcd，需要提前备份数据。
- kube-proxy 的ipvs 从alpha变为beta
- coredns 比较有前景，可以持续关注
本文转自银狐博客51CTO博客，原文链接http://blog.51cto.com/foxhound/2057395如需转载请自行联系原作者
战狐
