Skip to content

Commit 47fd436

Browse files
badenerbdtapiacl
badenerb
authored and
dtapiacl
committed
(fleet/k8up) add k8up deployment
1 parent 83ae2e0 commit 47fd436

File tree

14 files changed

+24433
-0
lines changed

14 files changed

+24433
-0
lines changed

‎fleet/lib/k8up-crds/fleet.yaml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
defaultNamespace: k8up
3+
namespaceLabels:
4+
lsst.io/discover: "true"
5+
k8up-enabled: "true"
6+
labels:
7+
bundle: &name k8up-crds
8+
helm:
9+
releaseName: *name
10+
takeOwnership: true
11+
force: true
12+
timeoutSeconds: 60
13+
waitForJobs: true

‎fleet/lib/k8up-crds/k8up-crds.yaml

Lines changed: 24156 additions & 0 deletions
Large diffs are not rendered by default.

‎fleet/lib/k8up-post/fleet.yaml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
labels:
2+
bundle: &name k8up-post
3+
dependsOn:
4+
- selector:
5+
matchLabels:
6+
bundle: k8up
7+
targetCustomizations:
8+
- name: ruka
9+
clusterSelector:
10+
matchExpressions:
11+
- key: management.cattle.io/cluster-display-name
12+
operator: In
13+
values:
14+
- ruka
15+
kustomize:
16+
dir: overlays/ruka

‎fleet/lib/k8up-post/overlays/ruka/backup-daily.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
apiVersion: k8up.io/v1
3+
kind: Schedule
4+
metadata:
5+
name: schedule-test
6+
namespace: nexus
7+
spec:
8+
backend:
9+
s3:
10+
endpoint: s3.gaw.ls.lsst.org
11+
bucket: ruka-k8up
12+
accessKeyIDSecretRef:
13+
name: k8up-s3-credentials
14+
key: access_key
15+
secretAccessKeySecretRef:
16+
name: k8up-s3-credentials
17+
key: secret_key
18+
repoPasswordSecretRef:
19+
name: k8up-s3-credentials
20+
key: restic_password
21+
backup:
22+
schedule: 0 2 * * *
23+
failedJobsHistoryLimit: 2
24+
successfulJobsHistoryLimit: 2
25+
prune:
26+
schedule: 0 1 * * 0
27+
retention:
28+
keepLast: 10

‎fleet/lib/k8up-post/overlays/ruka/kustomization.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
resources:
2+
- backup-daily.yaml

‎fleet/lib/k8up-pre/clusterexternal-secret-k8up.yaml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
apiVersion: external-secrets.io/v1beta1
2+
kind: ClusterExternalSecret
3+
metadata:
4+
name: k8up-credentials
5+
spec:
6+
externalSecretName: k8up-s3-credentials
7+
refreshTime: 1m
8+
namespaceSelector:
9+
matchLabels:
10+
k8up-enabled: "true"
11+
externalSecretSpec:
12+
secretStoreRef:
13+
kind: ClusterSecretStore
14+
name: onepassword
15+
target:
16+
name: k8up-s3-credentials
17+
creationPolicy: Owner
18+
data:
19+
- secretKey: access_key
20+
remoteRef:
21+
key: k8up-credentials
22+
property: access_key
23+
- secretKey: secret_key
24+
remoteRef:
25+
key: k8up-credentials
26+
property: secret_key
27+
- secretKey: restic_password
28+
remoteRef:
29+
key: k8up-credentials
30+
property: restic_password

‎fleet/lib/k8up-pre/fleet.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
defaultNamespace: k8up
3+
labels:
4+
bundle: &name k8up-pre
5+
namespaceLabels:
6+
lsst.io/discover: "true"
7+
k8up-enabled: "true"
8+
helm:
9+
releaseName: *name
10+
timeoutSeconds: 60
11+
waitForJobs: true

‎fleet/lib/k8up/README.MD

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# 🔄 K8up Backup with Rancher Fleet
2+
3+
This repository configures [K8up](https://k8up.io/) via Rancher Fleet to automate persistent volume backups in Kubernetes using restic and an external S3-compatible object store.
4+
5+
---
6+
7+
## 📦 Repository Structure
8+
9+
| Path | Purpose |
10+
|------|---------|
11+
| `k8up-crds/` | Deploys K8up CustomResourceDefinitions (CRDs) |
12+
| `k8up-pre/` | Creates `ClusterExternalSecret` to sync S3 credentials into backup-enabled namespaces |
13+
| `k8up/` | Installs the K8up Helm chart and configures the backup environment (S3 endpoint, restic password, etc.) |
14+
| `k8up-post/` | Applies `Schedule` resources to trigger automated backups in selected namespaces |
15+
16+
---
17+
18+
## 🚀 How It Works
19+
20+
1. Any namespace labeled `k8up-enabled=true` will:
21+
- Receive the `k8up-s3-credentials` secret via `ClusterExternalSecret`.
22+
- Be eligible for backup if its PVCs are annotated.
23+
2. Backup schedules (defined in `k8up-post/`) trigger periodic restic jobs that back up PVC contents to the configured S3 bucket.
24+
25+
---
26+
27+
## 🧪 Sample Output: Querying Backups and Snapshots
28+
29+
### 🔍 View current backups (Example)
30+
31+
```bash
32+
kubectl get backup
33+
34+
NAME SCHEDULE REF COMPLETION PREBACKUP AGE
35+
demo-backup Succeeded NoPreBackupPodsFound 3m20s
36+
schedule-test-backup-b6bxz schedule-test NoPreBackupPodsFound 20s
37+
schedule-test-backup-sj5rt schedule-test NoPreBackupPodsFound 80s
38+
schedule-test-backup-whnkl schedule-test Failed NoPreBackupPodsFound 2m20s
39+
40+
```
41+
42+
### 🔍 View available snapshots (Example)
43+
44+
```bash
45+
kubectl get snapshots.k8up.io -n nexus
46+
47+
NAME DATE TAKEN PATHS REPOSITORY
48+
00dcfddc 2025-07-09T16:41:08Z /data/data-nexus-nexus3-0 s3:s3.gaw.ls.lsst.org/ruka-k8up
49+
0292e48c 2025-07-09T16:37:30Z /data/data-nexus-nexus3-0 s3:s3.gaw.ls.lsst.org/ruka-k8up
50+
0369a833 2025-07-09T16:41:14Z /data/data-nexus-nexus3-0 s3:s3.gaw.ls.lsst.org/ruka-k8up
51+
05869032 2025-07-09T16:32:54Z /data/data-nexus-nexus3-0 s3:s3.gaw.ls.lsst.org/ruka-k8up
52+
0644f76f 2025-07-09T16:31:58Z /data/data-nexus-nexus3-0 s3:s3.gaw.ls.lsst.org/ruka-k8up
53+
54+
```
55+
56+
## 🧪 Sample Output: Restore a backup.
57+
58+
### 🔍 Restore a PVC in a Namespace (Example)
59+
60+
```yaml
61+
62+
Create new pvc first
63+
64+
---
65+
kind: PersistentVolumeClaim
66+
apiVersion: v1
67+
metadata:
68+
name: restore-test-pvc
69+
namespace: k8up
70+
annotations:
71+
k8up.io/backup: "false"
72+
spec:
73+
accessModes:
74+
- ReadWriteOnce
75+
resources:
76+
requests:
77+
storage: 250Mi
78+
---
79+
Create restore job, to dump the lastest backup to that pvc
80+
81+
apiVersion: k8up.io/v1
82+
kind: Restore
83+
metadata:
84+
name: restore-testv2
85+
namespace: k8up
86+
spec:
87+
restoreMethod:
88+
folder:
89+
claimName: restore-test-pvc
90+
backend:
91+
repoPasswordSecretRef:
92+
name: k8up-s3-credentials
93+
key: restic_password
94+
s3:
95+
endpoint: s3.ruka.dev.lsst.org
96+
bucket: rubinobs-k8up-dev
97+
accessKeyIDSecretRef:
98+
name: k8up-s3-credentials
99+
key: access-key-id
100+
secretAccessKeySecretRef:
101+
name: k8up-s3-credentials
102+
key: secret-access-key
103+
104+
Then bind the pvc to a pod to view its contents
105+
```
106+
107+
> Backups created by K8up are fully compatible with the [Restic CLI documentation](https://docs.k8up.io/k8up/2.12/how-tos/restore.html). You can use the Restic client directly to inspect, restore, or manage snapshots outside of K8up workflows.

‎fleet/lib/k8up/fleet.yaml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
defaultNamespace: &name k8up
3+
namespaceLabels:
4+
lsst.io/discover: "true"
5+
k8up-enabled: "true"
6+
labels:
7+
bundle: *name
8+
dependsOn:
9+
- selector:
10+
matchLabels:
11+
bundle: k8up-pre
12+
- selector:
13+
matchLabels:
14+
bundle: k8up-crds
15+
helm:
16+
chart: *name
17+
releaseName: *name
18+
repo: https://k8up-io.github.io/k8up
19+
version: 4.8.4
20+
timeoutSeconds: 60
21+
waitForJobs: true
22+
targetCustomizations:
23+
- name: ruka
24+
clusterSelector:
25+
matchExpressions:
26+
- key: management.cattle.io/cluster-display-name
27+
operator: In
28+
values:
29+
- ruka
30+
helm:
31+
valuesFiles:
32+
- overlays/ruka/values.yaml

‎fleet/lib/k8up/overlays/ruka/values.yaml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
k8up:
2+
envVars:
3+
- name: BACKUP_S3_ENDPOINT
4+
value: s3.gaw.ls.lsst.org
5+
- name: BACKUP_GLOBALACCESSKEYID
6+
valueFrom:
7+
secretKeyRef:
8+
name: k8up-s3-credentials
9+
key: access_key
10+
- name: BACKUP_GLOBALSECRETACCESSKEY
11+
valueFrom:
12+
secretKeyRef:
13+
name: k8up-s3-credentials
14+
key: secret_key
15+
- name: RESTIC_PASSWORD
16+
valueFrom:
17+
secretKeyRef:
18+
name: k8up-s3-credentials
19+
key: restic_password
20+
21+
timezone: America/Santiago
22+
23+
resources:
24+
limits:
25+
memory: 2Gi
26+
requests:
27+
cpu: 1
28+
memory: 1Gi
29+
30+
metrics:
31+
serviceMonitor:
32+
enabled: true
33+
additionalLabels:
34+
lsst.io/monitor: "true"

0 commit comments

Comments
 (0)