Skip to content

Commit e099163

Browse files
authored
Merge pull request #70 from gianlucam76/sync
Sync
2 parents d9049e1 + 57c43ba commit e099163

12 files changed

+259
-23
lines changed

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ ARCH ?= amd64
2525
OS ?= $(shell uname -s | tr A-Z a-z)
2626
K8S_LATEST_VER ?= $(shell curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)
2727
export CONTROLLER_IMG ?= $(REGISTRY)/$(IMAGE_NAME)
28-
TAG ?= master
28+
TAG ?= main
2929

3030
# Get cluster-api version and build ldflags
3131
clusterapi := $(shell go list -m sigs.k8s.io/cluster-api)

README.md

Lines changed: 59 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
# Sveltos
77

8-
<img src="https://raw.githubusercontent.com/projectsveltos/sveltos-manager/master/logos/logo.png" width="100">
8+
<img src="https://raw.githubusercontent.com/projectsveltos/sveltos-manager/master/logos/logo.png" width="200">
99

1010
## What it is
1111
Sveltos is tool for policy driven management of kubernetes resources and helm charts in [ClusterAPI](https://github.com/kubernetes-sigs/cluster-api) powered Kubernetes clusters. Sveltos provides declarative APIs to provision features like Helm charts, ingress controllers, CNIs, storage classes and other resources in a given set of Kubernetes clusters. Sveltos is a freely available and open source. Sveltos is very lightweight and can be installed onto any Kubernetes clusters in minutes.
@@ -70,26 +70,63 @@ To see the full demo, have a look at this [youtube video](https://youtu.be/Ai5Mr
7070
3. Snapshotting (see [video](https://youtu.be/ALcp1_Nj9r4))
7171
4. Rollback (see [video](https://youtu.be/sTo6RcWP1BQ))
7272
5. Conflict detection
73-
6. Declaritive API and CLI
73+
6. Declaritive API and CLI [sveltosctl](https://github.com/projectsveltos/sveltosctl)
74+
75+
## Install Sveltos on your laptop using the make cmd
76+
Just execute `make create-cluster`
77+
The command will:
78+
- create a [KIND](https://sigs.k8s.io/kind) cluster on your laptop;
79+
- install ClusterAPI;
80+
- create a CAPI Cluster with Docker as infrastructure provider;
81+
- install CRD and the Deployment from this project;
82+
- create a ClusterProfile instance;
83+
- modify CAPI Cluster labels so to match ClusterProfile selector.
84+
85+
## Install Sveltos on Docker Desktop
86+
First, enable Kubernetes in Docker Desktop
87+
```
88+
1. from the Docker Dashboard, select the Setting icon, or Preferences icon if you use a macOS.
89+
2. select Kubernetes from the left sidebar.
90+
3. next to Enable Kubernetes, select the checkbox.
91+
4. select Apply & Restart to save the settings and then click Install to confirm.
92+
```
7493

75-
## Test Sveltos on your laptop
76-
If you want to test it out, just execute, `make create-cluster` and it will:
77-
1. create a [KIND](https://sigs.k8s.io/kind) cluster;
78-
2. install ClusterAPI;
79-
3. create a CAPI Cluster with Docker as infrastructure provider;
80-
4. install CRD and the Deployment from this project;
81-
5. create a ClusterProfile instance;
82-
6. modify CAPI Cluster labels so to match ClusterProfile selector.
94+
Second, install ClusterAPI
95+
```
96+
https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl
97+
```
8398

84-
## Install Sveltos
85-
If you want to install it on local or remote Kubernetes cluster.
99+
Third, initialize the management cluster
100+
```
101+
clusterctl init --infrastructure docker
102+
```
86103

87-
1. install ClusterAPI. [ClusterAPI instruction](https://cluster-api.sigs.k8s.io/user/quick-start.html) can be followed.
104+
Fourth, install Sveltos applying this manifest YAML
105+
```
106+
kubectl create -f https://raw.githubusercontent.com/projectsveltos/cluster-api-feature-manager/master/manifest/manifest.yaml
107+
```
88108

89-
2. install the CRD and Deployment in the management cluster applying this YAML file:
109+
Finally, if you have prometheus operator installed in your management cluster, you can apply following ServiceMonitor
110+
```
111+
kubectl create -f https://raw.githubusercontent.com/projectsveltos/cluster-api-feature-manager/master/manifest/service_monitor.yaml
112+
```
90113

114+
115+
## Install Sveltos on any local or remote Kubernetes cluster.
116+
117+
First, install ClusterAPI
91118
```
92-
kubectl create -f https://raw.githubusercontent.com/projectsveltos/cluster-api-feature-manager/master/manifest/manifest.yaml
119+
https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl
120+
```
121+
122+
Second, initialize the management cluster
123+
```
124+
https://cluster-api.sigs.k8s.io/user/quick-start.html#initialize-the-management-cluster
125+
```
126+
127+
Third, install Sveltos applying this manifest YAML
128+
```
129+
kubectl create -f https://raw.githubusercontent.com/projectsveltos/cluster-api-feature-manager/main/manifest/manifest.yaml
93130
```
94131

95132
# Understanding how to configure and use Sveltos
@@ -388,6 +425,13 @@ Another example of misconfiguration is when two different ClusterProfiles match
388425
In such a case, only one ClusterProfile will be elected and given permission to manage a specific helm release in a given CAPI cluster. Other ClusterProfiles will report such misconfiguration.
389426

390427

428+
## Metrics
429+
Sveltos exposes following metrics:
430+
1. projectsveltos_program_resources_time_seconds: time to deploy resources in a CAPI clusters (all ClusterProfiles and all CAPI clusters considered);
431+
2. projectsveltos_program_charts_time_seconds: time to deploy Helm charts in a CAPI clusters (all ClusterProfiles and all CAPI clusters considered);
432+
3. *clusterNamespace*_*clusterName*_program_resources_time_seconds: time to deploy resources in a specific CAPI clusters (all ClusterProfiles considered);
433+
4. *clusterNamespace*_*clusterName*_program_charts_time_seconds: time to deploy Helm charts in a specific CAPI clusters (all ClusterProfiles considered);
434+
391435
### Uninstall CRDs
392436
To delete the CRDs from the cluster:
393437

api/v1alpha1/clusterprofile_types.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,12 @@ func (r PolicyRef) String() string {
6464
return r.Kind + "-" + r.Namespace + "-" + r.Name
6565
}
6666

67+
type DryRunReconciliationError struct{}
68+
69+
func (m *DryRunReconciliationError) Error() string {
70+
return "mode is DryRun. Nothing is reconciled"
71+
}
72+
6773
// SyncMode specifies how features are synced in a workload cluster.
6874
// +kubebuilder:validation:Enum:=OneTime;Continuous;DryRun
6975
type SyncMode string

api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 15 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

config/default/manager_image_patch.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,5 @@ spec:
88
spec:
99
containers:
1010
# Change the value of image field below to your controller image URL
11-
- image: gianlucam76/sveltos-manager-amd64:master
11+
- image: gianlucam76/sveltos-manager-amd64:dev
1212
name: manager

controllers/handlers_helm.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ func undeployHelmCharts(ctx context.Context, c client.Client,
198198
return nil
199199
}
200200

201-
return fmt.Errorf("mode is DryRun. Nothing is reconciled")
201+
return &configv1alpha1.DryRunReconciliationError{}
202202
}
203203

204204
func helmHash(ctx context.Context, c client.Client, clusterSummaryScope *scope.ClusterSummaryScope,
@@ -321,7 +321,7 @@ func handleCharts(ctx context.Context, clusterSummary *configv1alpha1.ClusterSum
321321

322322
// In DryRun mode always return an error.
323323
if clusterSummary.Spec.ClusterProfileSpec.SyncMode == configv1alpha1.SyncModeDryRun {
324-
return fmt.Errorf("mode is DryRun. Nothing is reconciled")
324+
return &configv1alpha1.DryRunReconciliationError{}
325325
}
326326
return nil
327327
}

controllers/handlers_helm_test.go

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package controllers_test
1919
import (
2020
"context"
2121
"crypto/sha256"
22+
"errors"
2223
"reflect"
2324

2425
. "github.com/onsi/ginkgo/v2"
@@ -501,7 +502,10 @@ var _ = Describe("HandlersHelm", func() {
501502
// such release has never been installed.
502503
err = controllers.HandleCharts(context.TODO(), clusterSummary, testEnv.Client, nil, kubeconfig, klogr.New())
503504
Expect(err).ToNot(BeNil())
504-
Expect(err.Error()).To(Equal("mode is DryRun. Nothing is reconciled"))
505+
506+
var druRunError *configv1alpha1.DryRunReconciliationError
507+
ok := errors.As(err, &druRunError)
508+
Expect(ok).To(BeTrue())
505509

506510
currentClusterReport := &configv1alpha1.ClusterReport{}
507511
Eventually(func() bool {

controllers/handlers_resources.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ func deployResources(ctx context.Context, c client.Client,
101101
}
102102

103103
if clusterSummary.Spec.ClusterProfileSpec.SyncMode == configv1alpha1.SyncModeDryRun {
104-
return fmt.Errorf("mode is DryRun. Nothing is reconciled")
104+
return &configv1alpha1.DryRunReconciliationError{}
105105
}
106106
return nil
107107
}
@@ -156,7 +156,7 @@ func undeployResources(ctx context.Context, c client.Client,
156156
}
157157

158158
if clusterSummary.Spec.ClusterProfileSpec.SyncMode == configv1alpha1.SyncModeDryRun {
159-
return fmt.Errorf("mode is DryRun. Nothing is reconciled")
159+
return &configv1alpha1.DryRunReconciliationError{}
160160
}
161161
return nil
162162
}

manifest/manifest.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1274,7 +1274,7 @@ spec:
12741274
- --v=5
12751275
command:
12761276
- /manager
1277-
image: gianlucam76/sveltos-manager-amd64:master
1277+
image: gianlucam76/sveltos-manager-amd64:main
12781278
livenessProbe:
12791279
httpGet:
12801280
path: /healthz

manifest/service_monitor.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
apiVersion: monitoring.coreos.com/v1
2+
kind: ServiceMonitor
3+
metadata:
4+
labels:
5+
control-plane: controller-manager
6+
name: fm-controller-manager-metrics-monitor
7+
namespace: projectsveltos
8+
spec:
9+
endpoints:
10+
- bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
11+
path: /metrics
12+
port: https
13+
scheme: https
14+
tlsConfig:
15+
insecureSkipVerify: true
16+
selector:
17+
matchLabels:
18+
control-plane: controller-manager

pkg/deployer/metrics.go

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
/*
2+
Copyright 2022. projectsveltos.io. All rights reserved.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package deployer
18+
19+
import (
20+
"errors"
21+
"fmt"
22+
"strings"
23+
24+
"github.com/go-logr/logr"
25+
"github.com/prometheus/client_golang/prometheus"
26+
"sigs.k8s.io/controller-runtime/pkg/metrics"
27+
28+
"github.com/projectsveltos/cluster-api-feature-manager/pkg/logs"
29+
)
30+
31+
var (
32+
programResourceDurationHistogram = prometheus.NewHistogram(
33+
prometheus.HistogramOpts{
34+
Namespace: "projectsveltos",
35+
Name: "program_resources_time_seconds",
36+
Help: "Program Resources on a workload cluster duration distribution",
37+
Buckets: []float64{1, 10, 30, 60, 120, 180, 240},
38+
},
39+
)
40+
41+
programChartDurationHistogram = prometheus.NewHistogram(
42+
prometheus.HistogramOpts{
43+
Namespace: "projectsveltos",
44+
Name: "program_charts_time_seconds",
45+
Help: "Program Helm charts on a workload cluster duration distribution",
46+
Buckets: []float64{1, 10, 30, 60, 120, 180, 240},
47+
},
48+
)
49+
)
50+
51+
//nolint:gochecknoinits // forced pattern, can't workaround
52+
func init() {
53+
// Register custom metrics with the global prometheus registry
54+
metrics.Registry.MustRegister(programResourceDurationHistogram, programChartDurationHistogram)
55+
}
56+
57+
func newResourceHistogram(clusterNamespace, clusterName string,
58+
logger logr.Logger) prometheus.Histogram {
59+
60+
clusterInfo := strings.ReplaceAll(fmt.Sprintf("%s_%s", clusterNamespace, clusterName), "-", "_")
61+
histogram := prometheus.NewHistogram(
62+
prometheus.HistogramOpts{
63+
Namespace: clusterInfo,
64+
Name: "program_resources_time_seconds",
65+
Help: "Program Resources on a workload cluster duration distribution",
66+
Buckets: []float64{1, 10, 30, 60, 120, 180, 240},
67+
},
68+
)
69+
70+
err := metrics.Registry.Register(histogram)
71+
if err != nil {
72+
var registrationError *prometheus.AlreadyRegisteredError
73+
ok := errors.As(err, &registrationError)
74+
if ok {
75+
_, ok = registrationError.ExistingCollector.(prometheus.Histogram)
76+
if ok {
77+
return registrationError.ExistingCollector.(prometheus.Histogram)
78+
}
79+
logCollectorError(err, logger)
80+
return nil
81+
}
82+
logCollectorError(err, logger)
83+
return nil
84+
}
85+
86+
return histogram
87+
}
88+
89+
func newChartHistogram(clusterNamespace, clusterName string,
90+
logger logr.Logger) prometheus.Histogram {
91+
92+
clusterInfo := strings.ReplaceAll(fmt.Sprintf("%s_%s", clusterNamespace, clusterName), "-", "_")
93+
histogram := prometheus.NewHistogram(
94+
prometheus.HistogramOpts{
95+
Namespace: clusterInfo,
96+
Name: "program_resources_time_seconds",
97+
Help: "Program Helm Charts on a workload cluster duration distribution",
98+
Buckets: []float64{1, 10, 30, 60, 120, 180, 240},
99+
},
100+
)
101+
102+
err := metrics.Registry.Register(histogram)
103+
if err != nil {
104+
var registrationError *prometheus.AlreadyRegisteredError
105+
ok := errors.As(err, &registrationError)
106+
if ok {
107+
_, ok = registrationError.ExistingCollector.(prometheus.Histogram)
108+
if ok {
109+
return registrationError.ExistingCollector.(prometheus.Histogram)
110+
}
111+
logCollectorError(err, logger)
112+
return nil
113+
}
114+
logCollectorError(err, logger)
115+
return nil
116+
}
117+
118+
return histogram
119+
}
120+
121+
func logCollectorError(err error, logger logr.Logger) {
122+
logger.V(logs.LogInfo).Info(fmt.Sprint("failed to register collector: %w", err))
123+
}

0 commit comments

Comments
 (0)