· 6 years ago · Sep 23, 2019, 09:02 PM
1Note for k8s the hard way
2
3Tools
4 cfssl, cfssljson, and kubectl.
5
6Provision servers
7 create vpc, subnet, internet gateway, route table, security group, nlb
8 create ssh key
9 create 3 controllers
10 create 3 workers
11
12CA, TLS certificates
13 create CA
14 create client, server certificates
15 - admin client
16 - kubelet client
17 - controller manager client
18 - kube proxy client
19 - scheduler client
20 - kubernetes api server
21 - Service Account key pair
22
23Distribute Client and Server certificates
24 scp ca.pem, worker[i]-key.pem worker[i].pem to each worker
25 scp ca-key.pem, kubernetes-key.pem kubernetes.pem, service-account-key.pem, service-account.pem to each controller
26
27Kubernetes Configuration for authentication
28 generate kubeconfig files for `controller manager`, `kubelet`, `kube-proxy`, `scheduler` clients and the `admin` user
29
30 get KUBERNETES_PUBLIC_ADDRESS (loadbalancer)
31 generate worker kubeconfig using node0,1,2 private key and k8s public ip (worker-0.kubeconfig, worker-1.kubeconfig, worker-2.kubeconfig)
32 generate kube-proxy kubeconfig using kube-proxy private key and k8s public ip
33 generate kube-controller-manager using kube-controller-manager private key and k8s public ip
34 genrate kube-scheduler using kube-scheduler private key and k8s public ip
35 generate kubeconfig for `admin` user
36
37Distribute kubconfig files
38 scp kubeconfig for kubelet and kube-proxy to each worker
39 scp kubeconfig for kube-controller-manager and kube-scheduler to each controller
40
41
42Create the Data Encryption Config and key
43 generate an encryption key and create EncryptionConfig
44 distribute to each controller
45
46Bootstrapping etcd cluster
47 k8s stores cluster state in etcd
48
49 each controller
50 install etcd binary files
51 cp ca.pem, kubernetes-key.pem
52 get instance private ip
53 create etcd.service with parameters, keys, private ip
54 start etcd service
55
56 once etcd in all controller installed, verify by listing etcd cluster member
57
58Bootstrapping control plane
59 install these binaries, kube-api, kube-controller-manager, kube-scheduler, kubectl
60 cp ca.pem, kubernetes-key.pem, kubernetes.pem, service-account.pem, service-account.pem, encryption-config.yaml
61 get private ip
62 create services for those conponents
63 start service and verify kubectl get componentstatuses
64
65RBAC for Kubelet Authorization
66 configure RBAC permissions to allow the Kubernetes API Server to access the Kubelet API on each worker node
67
68Bootstrapping control plane
69 install worker binaries
70 configure CNI network
71 configure Containerd
72 configure Kubelet
73 configure Kube-proxy
74 start worker services
75 verify node `kubectl get nodes --kubeconfig admin.kubeconfig`
76
77Configure kubectl for remote access
78 set kubeconfig and verify `kubectl get componentstatuses` and `kubectl get nodes`
79
80Provisioning Pod Network Routes
81 create Route Table and Route in aws
82 validate Routes
83
84Deploy DNS cluster add on
85 install kube DNS
86 run dns lookup from Pod
87
88Smoke Test
89 Data encryption
90 Deployment
91 Portforward
92 Logs
93 Exec
94 Services
95 Untrusted Workloads
96 Check images/pods/containers on worker nodes using crictl
97
98Cleanup