diff --git a/README.md b/README.md index 501b895e6..73ad4c62d 100644 --- a/README.md +++ b/README.md @@ -22,11 +22,11 @@ The target audience for this tutorial is someone planning to support a productio Kubernetes The Hard Way guides you through bootstrapping a highly available Kubernetes cluster with end-to-end encryption between components and RBAC authentication. -* [kubernetes](https://github.com/kubernetes/kubernetes) v1.21.5 -* [containerd](https://github.com/containerd/containerd) v1.4.4 -* [coredns](https://github.com/coredns/coredns) v1.8.3 -* [cni-plugins](https://github.com/containernetworking/plugins) v0.9.1 -* [etcd](https://github.com/etcd-io/etcd) v3.4.15 +* [kubernetes](https://github.com/kubernetes/kubernetes) v1.29.1 +* [containerd](https://github.com/containerd/containerd) v1.7.13 +* [coredns](https://github.com/coredns/coredns) v1.11.1 +* [cni-plugins](https://github.com/containernetworking/plugins) v1.4.0 +* [etcd](https://github.com/etcd-io/etcd) v3.5.12 ## Labs diff --git a/deployments/coredns.yaml b/deployments/coredns.yaml index bf1a258a0..0ee21235c 100644 --- a/deployments/coredns.yaml +++ b/deployments/coredns.yaml @@ -27,6 +27,13 @@ rules: - nodes verbs: - get +- apiGroups: + - discovery.k8s.io + resources: + - endpointslices + verbs: + - list + - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -96,10 +103,10 @@ spec: - key: "CriticalAddonsOnly" operator: "Exists" nodeSelector: - beta.kubernetes.io/os: linux + kubernetes.io/os: linux containers: - name: coredns - image: coredns/coredns:1.6.2 + image: coredns/coredns:1.11.1 imagePullPolicy: IfNotPresent resources: limits: diff --git a/docs/01-prerequisites.md b/docs/01-prerequisites.md index c28cd6c21..0c89b1dd2 100644 --- a/docs/01-prerequisites.md +++ b/docs/01-prerequisites.md @@ -201,7 +201,7 @@ The basic VM configuration process is the same for the 6 VM (you can also config You have to: -* Install the [Ubuntu 18.04.4 LTS (Bionic Beaver) Server install image](https://releases.ubuntu.com/18.04/) on this VM. +* Install the [Ubuntu 22.04.3 LTS Server install image](https://releases.ubuntu.com/22.04/) on this VM. * Configure the network interface (see the network architecture). Example of `/etc/netplan/00-installer-config.yaml` file if ens18 is the name of your private network interface (you need to change the IP address depending on the installed server): diff --git a/docs/02-client-tools.md b/docs/02-client-tools.md index 6df4f4536..47d84a732 100644 --- a/docs/02-client-tools.md +++ b/docs/02-client-tools.md @@ -9,9 +9,8 @@ The `cfssl` and `cfssljson` command line utilities will be used to provision a [ On the **gateway-01** VM, download and install `cfssl` and `cfssljson`: ```bash -wget -q --show-progress --https-only --timestamping \ - https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssl \ - https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssljson + wget -q --show-progress --https-only --timestamping https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl_1.6.4_linux_amd64 -O cfssl + wget -q --show-progress --https-only --timestamping https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssljson_1.6.4_linux_amd64 -O cfssljson ``` ```bash @@ -55,7 +54,7 @@ Runtime: go1.13 The `kubectl` command line utility is used to interact with the Kubernetes API Server. On the **gateway-01** VM, download and install `kubectl` from the official release binaries: ```bash -wget https://storage.googleapis.com/kubernetes-release/release/v1.21.5/bin/linux/amd64/kubectl +wget https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl ``` ```bash @@ -77,7 +76,9 @@ kubectl version --client > Output: ```bash -Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.21.5", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"} +Client Version: v1.29.1 +Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 + ``` Next: [Provisioning Compute Resources](03-compute-resources.md) diff --git a/docs/07-bootstrapping-etcd.md b/docs/07-bootstrapping-etcd.md index 3cfd342ce..3418b5c89 100644 --- a/docs/07-bootstrapping-etcd.md +++ b/docs/07-bootstrapping-etcd.md @@ -22,14 +22,14 @@ Download the official etcd release binaries from the [etcd](https://github.com/e ```bash wget -q --show-progress --https-only --timestamping \ - "https://github.com/etcd-io/etcd/releases/download/v3.4.15/etcd-v3.4.15-linux-amd64.tar.gz" + "https://github.com/etcd-io/etcd/releases/download/v3.5.12/etcd-v3.5.12-linux-amd64.tar.gz" ``` Extract and install the `etcd` server and the `etcdctl` command line utility: ```bash -tar -xvf etcd-v3.4.15-linux-amd64.tar.gz -sudo mv etcd-v3.4.15-linux-amd64/etcd* /usr/local/bin/ +tar -xvf etcd-v3.5.12-linux-amd64.tar.gz +sudo mv etcd-v3.5.12-linux-amd64/etcd* /usr/local/bin/ ``` ### Configure the etcd Server diff --git a/docs/08-bootstrapping-kubernetes-controllers.md b/docs/08-bootstrapping-kubernetes-controllers.md index d3a4218e4..4c8f9f69d 100644 --- a/docs/08-bootstrapping-kubernetes-controllers.md +++ b/docs/08-bootstrapping-kubernetes-controllers.md @@ -28,10 +28,10 @@ Download the official Kubernetes release binaries: ```bash wget -q --show-progress --https-only --timestamping \ - "https://storage.googleapis.com/kubernetes-release/release/v1.21.5/bin/linux/amd64/kube-apiserver" \ - "https://storage.googleapis.com/kubernetes-release/release/v1.21.5/bin/linux/amd64/kube-controller-manager" \ - "https://storage.googleapis.com/kubernetes-release/release/v1.21.5/bin/linux/amd64/kube-scheduler" \ - "https://storage.googleapis.com/kubernetes-release/release/v1.21.5/bin/linux/amd64/kubectl" + "https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kube-apiserver" \ + "https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kube-controller-manager" \ + "https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kube-scheduler" \ + "https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl" ``` Install the Kubernetes binaries: @@ -126,7 +126,7 @@ Documentation=https://github.com/kubernetes/kubernetes [Service] ExecStart=/usr/local/bin/kube-controller-manager \\ - --address=0.0.0.0 \\ + --bind-address=0.0.0.0 \\ --cluster-cidr=10.200.0.0/16 \\ --cluster-name=kubernetes \\ --cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\ @@ -158,7 +158,7 @@ Create the `kube-scheduler.yaml` configuration file: ```bash cat < 15s v1.21.5 -worker-1 Ready 15s v1.21.5 -worker-2 Ready 15s v1.21.5 +worker-0 Ready 15s v1.29.1 +worker-1 Ready 15s v1.29.1 +worker-2 Ready 15s v1.29.1 +``` + +> [!NOTE] +> By default kube-proxy uses iptables to set up Service IP handling and load balancing. Unfortunately, it breaks our deployment and there's a hack to force Linux to run iptables even for bridge-only traffic: +> +> Run this on all control and worker nodes. + +```bash +sudo modprobe br_netfilter +echo "br-netfilter" >> /etc/modules-load.d/modules.conf +sysctl -w net.bridge.bridge-nf-call-iptables=1 ``` + Next: [Configuring kubectl for Remote Access](10-configuring-kubectl.md) diff --git a/docs/10-configuring-kubectl.md b/docs/10-configuring-kubectl.md index 31af2a0ff..5d816392b 100644 --- a/docs/10-configuring-kubectl.md +++ b/docs/10-configuring-kubectl.md @@ -48,6 +48,49 @@ etcd-2 Healthy {"health":"true"} etcd-0 Healthy {"health":"true"} ``` +However component statuses are deprecated in Kubernetes 1.19 and later, so the recommended way to check cluster health is: +```bash +kubectl get --raw='/readyz?verbose' +``` + +> Output: + +```bash +[+]ping ok +[+]log ok +[+]etcd ok +[+]etcd-readiness ok +[+]informer-sync ok +[+]poststarthook/start-kube-apiserver-admission-initializer ok +[+]poststarthook/generic-apiserver-start-informers ok +[+]poststarthook/priority-and-fairness-config-consumer ok +[+]poststarthook/priority-and-fairness-filter ok +[+]poststarthook/storage-object-count-tracker-hook ok +[+]poststarthook/start-apiextensions-informers ok +[+]poststarthook/start-apiextensions-controllers ok +[+]poststarthook/crd-informer-synced ok +[+]poststarthook/start-service-ip-repair-controllers ok +[+]poststarthook/rbac/bootstrap-roles ok +[+]poststarthook/scheduling/bootstrap-system-priority-classes ok +[+]poststarthook/priority-and-fairness-config-producer ok +[+]poststarthook/start-system-namespaces-controller ok +[+]poststarthook/bootstrap-controller ok +[+]poststarthook/start-cluster-authentication-info-controller ok +[+]poststarthook/start-kube-apiserver-identity-lease-controller ok +[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok +[+]poststarthook/start-legacy-token-tracking-controller ok +[+]poststarthook/start-kube-aggregator-informers ok +[+]poststarthook/apiservice-registration-controller ok +[+]poststarthook/apiservice-status-available-controller ok +[+]poststarthook/kube-apiserver-autoregistration ok +[+]autoregister-completion ok +[+]poststarthook/apiservice-openapi-controller ok +[+]poststarthook/apiservice-openapiv3-controller ok +[+]poststarthook/apiservice-discovery-controller ok +[+]shutdown ok +readyz check passed +``` + List the nodes in the remote Kubernetes cluster: ```bash @@ -58,9 +101,9 @@ kubectl get nodes ```bash NAME STATUS ROLES AGE VERSION -worker-0 Ready 90s v1.21.5 -worker-1 Ready 91s v1.21.5 -worker-2 Ready 90s v1.21.5 +worker-0 Ready 90s v1.29.1 +worker-1 Ready 91s v1.29.1 +worker-2 Ready 90s v1.29.1 ``` Next: [Provisioning Pod Network Routes](11-pod-network-routes.md) diff --git a/docs/12-dns-addon.md b/docs/12-dns-addon.md index 6f186b8c0..2a5591cd3 100644 --- a/docs/12-dns-addon.md +++ b/docs/12-dns-addon.md @@ -7,20 +7,9 @@ In this lab you will deploy the [DNS add-on](https://kubernetes.io/docs/concepts Get the CoreDNS yaml: ```bash -wget https://storage.googleapis.com/kubernetes-the-hard-way/coredns-1.8.yaml +kubectl apply -f https://raw.githubusercontent.com/DushanthaS/kubernetes-the-hard-way-on-proxmox/master/deployments/coredns.yaml ``` -Edit the `coredns.yaml` file to change CoreDNS configuration to enable DNS resolution for external name: - -```bash -sed '/.*prometheus :9153/a \ \ \ \ \ \ \ \ forward . /etc/resolv.conf' coredns.yaml -``` - -Deploy the `coredns` cluster add-on: - -```bash -kubectl apply -f coredns.yaml -``` > Output: @@ -52,7 +41,7 @@ coredns-699f8ddd77-gtcgb 1/1 Running 0 20s Create a `busybox` deployment: ```bash -kubectl run --generator=run-pod/v1 busybox --image=busybox:1.28 --command -- sleep 3600 +kubectl run busybox --image=busybox:1.28 --command -- sleep 3600 ``` List the pod created by the `busybox` deployment: