Skip to content

Commit ce92b3e

Browse files
sergelogvinovsergelogvinov
committed
feat(chart): add daemonset mode
Deploy CCM in daemonset mode. It's helpful when CNI doesn't have the node.cloudprovider.kubernetes.io/uninitialized taint. Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
1 parent 4771769 commit ce92b3e

File tree

8 files changed

+244
-16
lines changed

8 files changed

+244
-16
lines changed

‎Makefile

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,10 @@ docs:
112112
-f charts/proxmox-cloud-controller-manager/values.talos.yaml \
113113
--set-string image.tag=$(TAG) \
114114
charts/proxmox-cloud-controller-manager > docs/deploy/cloud-controller-manager-talos.yml
115+
helm template -n kube-system proxmox-cloud-controller-manager \
116+
--set-string image.tag=$(TAG) \
117+
--set useDaemonSet=true \
118+
charts/proxmox-cloud-controller-manager > docs/deploy/cloud-controller-manager-daemonset.yml
115119
helm-docs --sort-values-order=file charts/proxmox-cloud-controller-manager
116120

117121
release-update:

‎charts/proxmox-cloud-controller-manager/Chart.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ maintainers:
1414
# This is the chart version. This version number should be incremented each time you make changes
1515
# to the chart and its templates, including the app version.
1616
# Versions are expected to follow Semantic Versioning (https://semver.org/)
17-
version: 0.2.0
17+
version: 0.2.1
1818
# This is the version number of the application being deployed. This version number should be
1919
# incremented each time you make changes to the application. Versions are not expected to
2020
# follow Semantic Versioning. They should reflect the version the application is using.

‎charts/proxmox-cloud-controller-manager/README.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# proxmox-cloud-controller-manager
22

3-
![Version: 0.2.0](https://img.shields.io/badge/Version-0.2.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.4.0](https://img.shields.io/badge/AppVersion-v0.4.0-informational?style=flat-square)
3+
![Version: 0.2.1](https://img.shields.io/badge/Version-0.2.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.4.0](https://img.shields.io/badge/AppVersion-v0.4.0-informational?style=flat-square)
44

55
A Helm chart for Kubernetes
66

@@ -79,12 +79,11 @@ helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
7979
| podSecurityContext | object | `{"fsGroup":10258,"fsGroupChangePolicy":"OnRootMismatch","runAsGroup":10258,"runAsNonRoot":true,"runAsUser":10258}` | Pods Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
8080
| securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"seccompProfile":{"type":"RuntimeDefault"}}` | Container Security Context. ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod |
8181
| resources | object | `{"requests":{"cpu":"10m","memory":"32Mi"}}` | Resource requests and limits. ref: https://kubernetes.io/docs/user-guide/compute-resources/ |
82+
| useDaemonSet | bool | `false` | Deploy CCM in Daemonset mode. CCM will use hostNetwork. It allows to use CCM without CNI plugins. |
8283
| updateStrategy | object | `{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}` | Deployment update stategy type. ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment |
8384
| nodeSelector | object | `{}` | Node labels for data pods assignment. ref: https://kubernetes.io/docs/user-guide/node-selection/ |
8485
| tolerations | list | `[{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane","operator":"Exists"},{"effect":"NoSchedule","key":"node.cloudprovider.kubernetes.io/uninitialized","operator":"Exists"}]` | Tolerations for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ |
8586
| affinity | object | `{}` | Affinity for data pods assignment. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity |
8687
| extraVolumes | list | `[]` | Additional volumes for Pods |
8788
| extraVolumeMounts | list | `[]` | Additional volume mounts for Pods |
8889

89-
----------------------------------------------
90-
Autogenerated from chart metadata using [helm-docs v1.12.0](https://github.com/norwoodj/helm-docs/releases/v1.12.0)

‎charts/proxmox-cloud-controller-manager/templates/deployment.yaml

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,23 @@
11
apiVersion: apps/v1
2+
{{- if .Values.useDaemonSet }}
3+
kind: DaemonSet
4+
{{- else }}
25
kind: Deployment
6+
{{- end }}
37
metadata:
48
name: {{ include "proxmox-cloud-controller-manager.fullname" . }}
59
labels:
610
{{- include "proxmox-cloud-controller-manager.labels" . | nindent 4 }}
711
namespace: {{ .Release.Namespace }}
812
spec:
13+
{{- if not .Values.useDaemonSet }}
914
replicas: {{ .Values.replicaCount }}
1015
strategy:
1116
type: {{ .Values.updateStrategy.type }}
17+
{{- else }}
18+
updateStrategy:
19+
type: {{ .Values.updateStrategy.type }}
20+
{{- end }}
1221
selector:
1322
matchLabels:
1423
{{- include "proxmox-cloud-controller-manager.selectorLabels" . | nindent 6 }}
@@ -35,8 +44,14 @@ spec:
3544
serviceAccountName: {{ include "proxmox-cloud-controller-manager.serviceAccountName" . }}
3645
securityContext:
3746
{{- toYaml .Values.podSecurityContext | nindent 8 }}
47+
{{- if .Values.useDaemonSet }}
48+
dnsPolicy: ClusterFirstWithHostNet
49+
hostNetwork: true
50+
{{- end }}
51+
{{- with .Values.hostAliases }}
3852
hostAliases:
39-
{{- toYaml .Values.hostAliases | nindent 8 }}
53+
{{- toYaml . | nindent 8 }}
54+
{{- end }}
4055
initContainers: {{- toYaml .Values.initContainers | nindent 8 }}
4156
containers:
4257
- name: {{ .Chart.Name }}
@@ -89,17 +104,24 @@ spec:
89104
topologyKey: topology.kubernetes.io/zone
90105
weight: 1
91106
{{- end }}
92-
{{- with .Values.tolerations }}
93107
tolerations:
108+
{{- with .Values.tolerations }}
94109
{{- toYaml . | nindent 8 }}
95110
{{- end }}
111+
{{- if .Values.useDaemonSet }}
112+
- effect: NoSchedule
113+
key: node.kubernetes.io/not-ready
114+
operator: Exists
115+
{{- end }}
116+
{{- if not .Values.useDaemonSet }}
96117
topologySpreadConstraints:
97118
- maxSkew: 1
98119
topologyKey: kubernetes.io/hostname
99120
whenUnsatisfiable: DoNotSchedule
100121
labelSelector:
101122
matchLabels:
102123
{{- include "proxmox-cloud-controller-manager.selectorLabels" . | nindent 14 }}
124+
{{- end }}
103125
volumes:
104126
{{- if .Values.existingConfigSecret }}
105127
- name: cloud-config

‎charts/proxmox-cloud-controller-manager/values.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,11 @@ resources:
128128
cpu: 10m
129129
memory: 32Mi
130130

131+
# -- Deploy CCM in Daemonset mode.
132+
# CCM will use hostNetwork.
133+
# It allows to use CCM without CNI plugins.
134+
useDaemonSet: false
135+
131136
# -- Deployment update stategy type.
132137
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#updating-a-deployment
133138
updateStrategy:

‎docs/deploy/cloud-controller-manager-daemonset.yml

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
---
2+
# Source: proxmox-cloud-controller-manager/templates/serviceaccount.yaml
3+
apiVersion: v1
4+
kind: ServiceAccount
5+
metadata:
6+
name: proxmox-cloud-controller-manager
7+
labels:
8+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
9+
app.kubernetes.io/name: proxmox-cloud-controller-manager
10+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
11+
app.kubernetes.io/version: "v0.4.0"
12+
app.kubernetes.io/managed-by: Helm
13+
namespace: kube-system
14+
---
15+
# Source: proxmox-cloud-controller-manager/templates/role.yaml
16+
apiVersion: rbac.authorization.k8s.io/v1
17+
kind: ClusterRole
18+
metadata:
19+
name: system:proxmox-cloud-controller-manager
20+
labels:
21+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
22+
app.kubernetes.io/name: proxmox-cloud-controller-manager
23+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
24+
app.kubernetes.io/version: "v0.4.0"
25+
app.kubernetes.io/managed-by: Helm
26+
rules:
27+
- apiGroups:
28+
- coordination.k8s.io
29+
resources:
30+
- leases
31+
verbs:
32+
- get
33+
- create
34+
- update
35+
- apiGroups:
36+
- ""
37+
resources:
38+
- events
39+
verbs:
40+
- create
41+
- patch
42+
- update
43+
- apiGroups:
44+
- ""
45+
resources:
46+
- nodes
47+
verbs:
48+
- get
49+
- list
50+
- watch
51+
- update
52+
- patch
53+
- delete
54+
- apiGroups:
55+
- ""
56+
resources:
57+
- nodes/status
58+
verbs:
59+
- patch
60+
- apiGroups:
61+
- ""
62+
resources:
63+
- serviceaccounts
64+
verbs:
65+
- create
66+
- get
67+
- apiGroups:
68+
- ""
69+
resources:
70+
- serviceaccounts/token
71+
verbs:
72+
- create
73+
---
74+
# Source: proxmox-cloud-controller-manager/templates/rolebinding.yaml
75+
kind: ClusterRoleBinding
76+
apiVersion: rbac.authorization.k8s.io/v1
77+
metadata:
78+
name: system:proxmox-cloud-controller-manager
79+
roleRef:
80+
apiGroup: rbac.authorization.k8s.io
81+
kind: ClusterRole
82+
name: system:proxmox-cloud-controller-manager
83+
subjects:
84+
- kind: ServiceAccount
85+
name: proxmox-cloud-controller-manager
86+
namespace: kube-system
87+
---
88+
# Source: proxmox-cloud-controller-manager/templates/rolebinding.yaml
89+
apiVersion: rbac.authorization.k8s.io/v1
90+
kind: RoleBinding
91+
metadata:
92+
name: system:proxmox-cloud-controller-manager:extension-apiserver-authentication-reader
93+
namespace: kube-system
94+
roleRef:
95+
apiGroup: rbac.authorization.k8s.io
96+
kind: Role
97+
name: extension-apiserver-authentication-reader
98+
subjects:
99+
- kind: ServiceAccount
100+
name: proxmox-cloud-controller-manager
101+
namespace: kube-system
102+
---
103+
# Source: proxmox-cloud-controller-manager/templates/deployment.yaml
104+
apiVersion: apps/v1
105+
kind: DaemonSet
106+
metadata:
107+
name: proxmox-cloud-controller-manager
108+
labels:
109+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
110+
app.kubernetes.io/name: proxmox-cloud-controller-manager
111+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
112+
app.kubernetes.io/version: "v0.4.0"
113+
app.kubernetes.io/managed-by: Helm
114+
namespace: kube-system
115+
spec:
116+
updateStrategy:
117+
type: RollingUpdate
118+
selector:
119+
matchLabels:
120+
app.kubernetes.io/name: proxmox-cloud-controller-manager
121+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
122+
template:
123+
metadata:
124+
annotations:
125+
checksum/config: c69436cb1e16c36ff708b1003d3ca4c6ee6484d2524e2ba7d9b68f473acaa1ca
126+
labels:
127+
app.kubernetes.io/name: proxmox-cloud-controller-manager
128+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
129+
spec:
130+
enableServiceLinks: false
131+
priorityClassName: system-cluster-critical
132+
serviceAccountName: proxmox-cloud-controller-manager
133+
securityContext:
134+
fsGroup: 10258
135+
fsGroupChangePolicy: OnRootMismatch
136+
runAsGroup: 10258
137+
runAsNonRoot: true
138+
runAsUser: 10258
139+
dnsPolicy: ClusterFirstWithHostNet
140+
hostNetwork: true
141+
initContainers:
142+
[]
143+
containers:
144+
- name: proxmox-cloud-controller-manager
145+
securityContext:
146+
allowPrivilegeEscalation: false
147+
capabilities:
148+
drop:
149+
- ALL
150+
seccompProfile:
151+
type: RuntimeDefault
152+
image: "ghcr.io/sergelogvinov/proxmox-cloud-controller-manager:v0.4.0"
153+
imagePullPolicy: IfNotPresent
154+
args:
155+
- --v=2
156+
- --cloud-provider=proxmox
157+
- --cloud-config=/etc/proxmox/config.yaml
158+
- --controllers=cloud-node,cloud-node-lifecycle
159+
- --leader-elect-resource-name=cloud-controller-manager-proxmox
160+
- --use-service-account-credentials
161+
- --secure-port=10258
162+
livenessProbe:
163+
httpGet:
164+
path: /healthz
165+
port: 10258
166+
scheme: HTTPS
167+
initialDelaySeconds: 20
168+
periodSeconds: 30
169+
timeoutSeconds: 5
170+
resources:
171+
requests:
172+
cpu: 10m
173+
memory: 32Mi
174+
volumeMounts:
175+
- name: cloud-config
176+
mountPath: /etc/proxmox
177+
readOnly: true
178+
affinity:
179+
podAntiAffinity:
180+
preferredDuringSchedulingIgnoredDuringExecution:
181+
- podAffinityTerm:
182+
labelSelector:
183+
matchLabels:
184+
app.kubernetes.io/name: proxmox-cloud-controller-manager
185+
app.kubernetes.io/instance: proxmox-cloud-controller-manager
186+
topologyKey: topology.kubernetes.io/zone
187+
weight: 1
188+
tolerations:
189+
- effect: NoSchedule
190+
key: node-role.kubernetes.io/control-plane
191+
operator: Exists
192+
- effect: NoSchedule
193+
key: node.cloudprovider.kubernetes.io/uninitialized
194+
operator: Exists
195+
- effect: NoSchedule
196+
key: node.kubernetes.io/not-ready
197+
operator: Exists
198+
volumes:
199+
- name: cloud-config
200+
secret:
201+
secretName: proxmox-cloud-controller-manager
202+
defaultMode: 416

‎docs/deploy/cloud-controller-manager-talos.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ kind: ServiceAccount
55
metadata:
66
name: proxmox-cloud-controller-manager
77
labels:
8-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
8+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
99
app.kubernetes.io/name: proxmox-cloud-controller-manager
1010
app.kubernetes.io/instance: proxmox-cloud-controller-manager
1111
app.kubernetes.io/version: "v0.4.0"
@@ -18,7 +18,7 @@ kind: ClusterRole
1818
metadata:
1919
name: system:proxmox-cloud-controller-manager
2020
labels:
21-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
21+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
2222
app.kubernetes.io/name: proxmox-cloud-controller-manager
2323
app.kubernetes.io/instance: proxmox-cloud-controller-manager
2424
app.kubernetes.io/version: "v0.4.0"
@@ -106,7 +106,7 @@ kind: Deployment
106106
metadata:
107107
name: proxmox-cloud-controller-manager
108108
labels:
109-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
109+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
110110
app.kubernetes.io/name: proxmox-cloud-controller-manager
111111
app.kubernetes.io/instance: proxmox-cloud-controller-manager
112112
app.kubernetes.io/version: "v0.4.0"
@@ -137,8 +137,6 @@ spec:
137137
runAsGroup: 10258
138138
runAsNonRoot: true
139139
runAsUser: 10258
140-
hostAliases:
141-
[]
142140
initContainers:
143141
[]
144142
containers:

‎docs/deploy/cloud-controller-manager.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ kind: ServiceAccount
55
metadata:
66
name: proxmox-cloud-controller-manager
77
labels:
8-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
8+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
99
app.kubernetes.io/name: proxmox-cloud-controller-manager
1010
app.kubernetes.io/instance: proxmox-cloud-controller-manager
1111
app.kubernetes.io/version: "v0.4.0"
@@ -18,7 +18,7 @@ kind: ClusterRole
1818
metadata:
1919
name: system:proxmox-cloud-controller-manager
2020
labels:
21-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
21+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
2222
app.kubernetes.io/name: proxmox-cloud-controller-manager
2323
app.kubernetes.io/instance: proxmox-cloud-controller-manager
2424
app.kubernetes.io/version: "v0.4.0"
@@ -106,7 +106,7 @@ kind: Deployment
106106
metadata:
107107
name: proxmox-cloud-controller-manager
108108
labels:
109-
helm.sh/chart: proxmox-cloud-controller-manager-0.2.0
109+
helm.sh/chart: proxmox-cloud-controller-manager-0.2.1
110110
app.kubernetes.io/name: proxmox-cloud-controller-manager
111111
app.kubernetes.io/instance: proxmox-cloud-controller-manager
112112
app.kubernetes.io/version: "v0.4.0"
@@ -137,8 +137,6 @@ spec:
137137
runAsGroup: 10258
138138
runAsNonRoot: true
139139
runAsUser: 10258
140-
hostAliases:
141-
[]
142140
initContainers:
143141
[]
144142
containers:

0 commit comments

Comments
 (0)