consul实现kubernetes-1.15集群master的高可用访问实现

提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

1、准备consul环境，参考我之前的博客实现或参考consul的官网部署最新的consul。

2、本次测试使用的是kubernetes-1.15.0

3、初始化集群

1）准备初始化文件

controlPlaneEndpoint: "kubeadm-ha.service.hq:6443" ,kubeadm-ha.service.hq是注册到consul的域名。kubeadm-ha是service name，service.hq是consul的domain。

# cat kubeadm-config.yaml

---

apiVersion: kubeproxy.config.k8s.io/v1alpha1

kind: KubeProxyConfiguration

mode: "ipvs"

---

apiVersion: kubeadm.k8s.io/v1beta2

certificatesDir: /etc/kubernetes/pki

clusterName: kubernetes

controlPlaneEndpoint: "kubeadm-ha.service.hq:6443"

dns:

  type: CoreDNS

imageRepository: k8s.gcr.io

kind: ClusterConfiguration

kubernetesVersion: v1.15.0

networking:

  dnsDomain: cluster.local

  podSubnet: 192.244.0.0/

  serviceSubnet: 192.96.0.0/

apiServer:

  timeoutForControlPlane: 4m0s

  certSANs:

  - 10.4.6.7

  - kubeadm-ha.service.hq

  - ku13-

controllerManager:

  extraArgs:

    address: 0.0.0.0

scheduler:

  extraArgs:

    address: 0.0.0.0

etcd:

  external:

    endpoints:

    - https://10.4.7.10:2379

    - https://10.4.6.77:2379

    - https://10.4.8.28:2379

    caFile: /etc/kubernetes/ssl/ca.pem

    certFile: /etc/etcd/ssl/etcd.pem

    keyFile: /etc/etcd/ssl/etcd-key.pem

2）执行初始化

# kubeadm init --config kubeadm-config.yaml --upload-certs

等一段时间就会初始化完成，下面会输出一系列的信息，有两个信息非常重要，一个是加入control-plane，一个是加入worker

control-plane:

kubeadm join kubeadm-ha.service.hq: --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --experimental-control-plane --certificate-key 3640e475a8cd4a57396355gf3005dd40b44ccd8cc9dda624c7159cffdfr41989 --ignore-preflight-errors=IPVSProxierCheck

worker:

kubeadm join kubeadm-ha.service.hq: --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --ignore-preflight-errors=IPVSProxierCheck

copy 配置文件/etc/kubernetes/admin.conf 并创建~/.kube目录，

# mkdir ~/.kube && cp /etc/kubernetes/admin.conf ~/.kube/config

4、在consul里面注册apiserver

由于我们这里使用的是3个master节点，所以service是三个

# cat kubeadm-ha.json

{

    "services": [

      {

        "id": "kubeadm-ha-0",

        "name": "kubeadm-ha",

        "tags": [

          "kubeconfig-addr"

        ],

        "address": "10.4.6.77",

        "port": ,

        "check": {

          "args": ["/data/scripts/kubeadm-ha-0.sh",""],

          "interval": "10s"

     }

   },

   {

     "id": "kubeadm-ha-1",

        "name": "kubeadm-ha",

        "tags": [

          "kubeconfig-addr"

        ],

        "address": "10.4.7.10",

        "port": ,

        "check": {

          "args": ["/data/scripts/kubeadm-ha-1.sh",""],

          "interval": "10s"

     }

   },

   {

     "id": "kubeadm-ha-2",

        "name": "kubeadm-ha",

        "tags": [

          "kubeconfig-addr"

        ],

        "address": "10.4.8.28",

        "port": ,

        "check": {

          "args": ["/data/scripts/kubeadm-ha-2.sh",""],

          "interval": "10s"

     }

   }

 ]

}

#cat kubeadm-ha-0.sh

#!/bin/bash

# check kubernetes apiserver  alive

curl -k https://10.4.8.77:6443/healthz

#cat kubeadm-ha-1.sh

#!/bin/bash

# check kubernetes apiserver  alive

curl -k https://10.4.7.10:6443/healthz

#cat kubeadm-ha-2.sh

#!/bin/bash

# check kubernetes apiserver  alive

curl -k https://10.4.8.28:6443/healthz

使生效：

#consul-reload

ps：consul-reload是自己写的一个脚本，参考我的https://www.cnblogs.com/cuishuai/p/8194345.html

5、添加节点

1）control-plane

kubeadm join kubeadm-ha.service.hq: --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --experimental-control-plane --certificate-key 3640e475a8cd4a57396355gf3005dd40b44ccd8cc9dda624c7159cffdfr41989 --ignore-preflight-errors=IPVSProxierCheck

 这里会报错，找不到可执行文件ipset，所以加一个--ignore-preflight-errors=IPVSProxierCheck，保证命令顺利执行。

2）worker

kubeadm join kubeadm-ha.service.hq: --token 8snd4e.j9o0icdh1mo0ls9b     --discovery-token-ca-cert-hash sha256:4cfa22006b2be98388c14c20721005e990101d6e086ff5183644c7383149a7ed     --ignore-preflight-errors=IPVSProxierCheck

扩展集群变得非常方便。