Skip to content

Commit eab4b8b

Browse files
author
Maciek Gołaszewski
authored
Merge pull request #113 from canonical/KU-1226/integration-test-for-disableDefaultCNI-flag
Integration test for disable default cni flag
2 parents 7f6bc64 + f313dcc commit eab4b8b

File tree

6 files changed

+602
-308
lines changed

6 files changed

+602
-308
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ make run
6868

6969
As soon as the bootstrap and control-plane controllers are up and running you can apply the cluster manifests describing the desired specs of the cluster you want to provision. Each machine is associated with a MicroK8sConfig through which you can set the cluster's properties. Please review the available options in [the respective definitions file](./apis/v1beta1/microk8sconfig_types.go). You may also find useful the example manifests found under the [examples](./examples/) directory. Note that the configuration structure followed is similar to the the one of kubeadm, in the MicroK8sConfig you will find a CLusterConfiguration and an InitConfiguration sections. When targeting a specific infrastructure you should be aware of which ports are used by MicroK8s and allow them in the network security groups on your deployment.
7070

71-
Two workload cluster templates are available under the [templates](./templates/) folder, which are actively used to validate releases:
71+
Two workload cluster templates are available under the [templates](./templates) folder, which are actively used to validate releases:
7272
- [AWS](./templates/cluster-template-aws.yaml), using the [AWS Infrastructure Provider](https://github.com/kubernetes-sigs/cluster-api-provider-aws)
7373
- [OpenStack](./templates/cluster-template-openstack.yaml), using the [OpenStack Infrastructure Provider](https://github.com/kubernetes-sigs/cluster-api-provider-openstack)
7474

‎integration/README.md

Lines changed: 9 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -4,40 +4,10 @@
44

55
The integration/e2e tests have the following prerequisites:
66

7-
* an environment variable `CLUSTER_MANIFEST_FILE` pointing to the cluster manifest. Cluster manifests can be produced with the help of the templates found under `templates`. For example:
8-
```
9-
export AWS_REGION=us-east-1
10-
export AWS_SSH_KEY_NAME=capi
11-
export CONTROL_PLANE_MACHINE_COUNT=3
12-
export WORKER_MACHINE_COUNT=3
13-
export AWS_CREATE_BASTION=false
14-
export AWS_PUBLIC_IP=false
15-
export AWS_CONTROL_PLANE_MACHINE_FLAVOR=t3.large
16-
export AWS_NODE_MACHINE_FLAVOR=t3.large
17-
export CLUSTER_NAME=test-ci-cluster
18-
clusterctl generate cluster ${CLUSTER_NAME} --from "templates/cluster-template-aws.yaml" --kubernetes-version 1.25.0 > cluster.yaml
19-
export CLUSTER_MANIFEST_FILE=$PWD/cluster.yaml
20-
```
21-
22-
* Additional environment variables when testing cluster upgrades:
23-
```
24-
export CAPI_UPGRADE_VERSION=v1.26.0
25-
export CAPI_UPGRADE_MD_NAME=${CLUSTER_NAME}-md-0
26-
export CAPI_UPGRADE_MD_TYPE=machinedeployments.cluster.x-k8s.io
27-
export CAPI_UPGRADE_CP_NAME=${CLUSTER_NAME}-control-plane
28-
export CAPI_UPGRADE_CP_TYPE=microk8scontrolplanes.controlplane.cluster.x-k8s.io
29-
30-
# Change the control plane and worker machine count to desired values for in-place upgrades tests and create a new cluster manifest.
31-
CONTROL_PLANE_MACHINE_COUNT=1
32-
WORKER_MACHINE_COUNT=1
33-
clusterctl generate cluster ${CLUSTER_NAME} --from "templates/cluster-template-aws.yaml" --kubernetes-version 1.25.0 > cluster-inplace.yaml
34-
export CLUSTER_INPLACE_MANIFEST_FILE=$PWD/cluster-inplace.yaml
35-
36-
```
37-
38-
* `clusterctl` available in the PATH
39-
40-
* `kubectl` available in the PATH
7+
* make sure to have ssh key in aws `capi`in `us-east-1 region` if you do not have key refer
8+
to CAPI on [AWS prerequisites documentation](https://cluster-api-aws.sigs.k8s.io/topics/using-clusterawsadm-to-fulfill-prerequisites#ssh-key-pair)
9+
10+
* local testing requires the following to be available in the PATH: `clusterctl`, `kubectl`, `helm`
4111

4212
* a management cluster initialised via `clusterctl` with the infrastructure targeted as well as the version of the MicroK8s providers we want to be tested
4313

@@ -67,7 +37,7 @@ microk8s config > ~/.kube/config
6737

6838
#### Initialize infrastructure provider
6939

70-
Visit [here](https://cluster-api.sigs.k8s.io/user/quick-start.html#initialization-for-common-providers) for a list of common infrasturture providers.
40+
Visit [here](https://cluster-api.sigs.k8s.io/user/quick-start.html#initialization-for-common-providers) for a list of common infrastructure providers.
7141

7242
```bash
7343
clusterctl init --infrastructure <infra> --bootstrap - --control-plane -
@@ -83,7 +53,7 @@ docker push <username>/capi-bootstrap-provider-microk8s:<tag>
8353
sed "s,docker.io/cdkbot/capi-bootstrap-provider-microk8s:latest,docker.io/<username>/capi-bootstrap-provider-microk8s:<tag>," -i bootstrap-components.yaml
8454
```
8555

86-
Similarly for control-plane provider
56+
Similarly, for control-plane provider
8757
```bash
8858
cd control-plane
8959
docker build -t <username>/capi-control-plane-provider-microk8s:<tag> .
@@ -96,6 +66,9 @@ sed "s,docker.io/cdkbot/capi-control-plane-provider-microk8s:latest,docker.io/<u
9666
```bash
9767
kubectl apply -f bootstrap/bootstrap-components.yaml -f control-plane/control-plane-components.yaml
9868
```
69+
### Cluster definitions for e2e
70+
71+
Cluster definitions are stored in the [`manifests`](./cluster-manifests) directory.
9972

10073
#### Trigger the e2e tests
10174

‎integration/cluster-manifests/cluster-disable-default-cni.yaml

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
apiVersion: cluster.x-k8s.io/v1beta1
2+
kind: Cluster
3+
metadata:
4+
name: test-ci-cluster
5+
namespace: default
6+
spec:
7+
controlPlaneRef:
8+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
9+
kind: MicroK8sControlPlane
10+
name: test-ci-cluster-control-plane
11+
infrastructureRef:
12+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
13+
kind: AWSCluster
14+
name: test-ci-cluster
15+
---
16+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
17+
kind: AWSCluster
18+
metadata:
19+
name: test-ci-cluster
20+
namespace: default
21+
spec:
22+
bastion:
23+
enabled: false
24+
region: us-east-1
25+
sshKeyName: capi
26+
---
27+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
28+
kind: MicroK8sControlPlane
29+
metadata:
30+
name: test-ci-cluster-control-plane
31+
namespace: default
32+
spec:
33+
controlPlaneConfig:
34+
clusterConfiguration:
35+
portCompatibilityRemap: true
36+
initConfiguration:
37+
IPinIP: true
38+
addons:
39+
- dns
40+
- ingress
41+
confinement: classic
42+
disableDefaultCNI: true
43+
joinTokenTTLInSecs: 900000
44+
riskLevel: stable
45+
machineTemplate:
46+
infrastructureTemplate:
47+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
48+
kind: AWSMachineTemplate
49+
name: test-ci-cluster-control-plane
50+
replicas: 1
51+
upgradeStrategy: SmartUpgrade
52+
version: v1.27.0
53+
---
54+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
55+
kind: AWSMachineTemplate
56+
metadata:
57+
name: test-ci-cluster-control-plane
58+
namespace: default
59+
spec:
60+
template:
61+
spec:
62+
iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
63+
instanceType: t3.large
64+
publicIP: false
65+
sshKeyName: capi
66+
---
67+
apiVersion: cluster.x-k8s.io/v1beta1
68+
kind: MachineDeployment
69+
metadata:
70+
name: test-ci-cluster-md-0
71+
namespace: default
72+
spec:
73+
clusterName: test-ci-cluster
74+
replicas: 1
75+
selector:
76+
matchLabels: null
77+
template:
78+
spec:
79+
bootstrap:
80+
configRef:
81+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
82+
kind: MicroK8sConfigTemplate
83+
name: test-ci-cluster-md-0
84+
clusterName: test-ci-cluster
85+
infrastructureRef:
86+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
87+
kind: AWSMachineTemplate
88+
name: test-ci-cluster-md-0
89+
version: 1.27.0
90+
---
91+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
92+
kind: AWSMachineTemplate
93+
metadata:
94+
name: test-ci-cluster-md-0
95+
namespace: default
96+
spec:
97+
template:
98+
spec:
99+
iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
100+
instanceType: t3.large
101+
publicIP: false
102+
sshKeyName: capi
103+
---
104+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
105+
kind: MicroK8sConfigTemplate
106+
metadata:
107+
name: test-ci-cluster-md-0
108+
namespace: default
109+
spec:
110+
template:
111+
spec:
112+
clusterConfiguration:
113+
portCompatibilityRemap: true
114+
initConfiguration:
115+
confinement: classic
116+
disableDefaultCNI: true
117+
riskLevel: stable

‎integration/cluster-manifests/cluster-inplace.yaml

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
apiVersion: cluster.x-k8s.io/v1beta1
2+
kind: Cluster
3+
metadata:
4+
name: test-ci-cluster
5+
namespace: default
6+
spec:
7+
controlPlaneRef:
8+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
9+
kind: MicroK8sControlPlane
10+
name: test-ci-cluster-control-plane
11+
infrastructureRef:
12+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
13+
kind: AWSCluster
14+
name: test-ci-cluster
15+
---
16+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
17+
kind: AWSCluster
18+
metadata:
19+
name: test-ci-cluster
20+
namespace: default
21+
spec:
22+
bastion:
23+
enabled: false
24+
region: us-east-1
25+
sshKeyName: capi
26+
---
27+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
28+
kind: MicroK8sControlPlane
29+
metadata:
30+
name: test-ci-cluster-control-plane
31+
namespace: default
32+
spec:
33+
controlPlaneConfig:
34+
clusterConfiguration:
35+
portCompatibilityRemap: true
36+
initConfiguration:
37+
IPinIP: true
38+
addons:
39+
- dns
40+
- ingress
41+
confinement: classic
42+
joinTokenTTLInSecs: 900000
43+
riskLevel: stable
44+
machineTemplate:
45+
infrastructureTemplate:
46+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
47+
kind: AWSMachineTemplate
48+
name: test-ci-cluster-control-plane
49+
replicas: 1
50+
upgradeStrategy: SmartUpgrade
51+
version: v1.27.0
52+
---
53+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
54+
kind: AWSMachineTemplate
55+
metadata:
56+
name: test-ci-cluster-control-plane
57+
namespace: default
58+
spec:
59+
template:
60+
spec:
61+
iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
62+
instanceType: t3.large
63+
publicIP: false
64+
sshKeyName: capi
65+
---
66+
apiVersion: cluster.x-k8s.io/v1beta1
67+
kind: MachineDeployment
68+
metadata:
69+
name: test-ci-cluster-md-0
70+
namespace: default
71+
spec:
72+
clusterName: test-ci-cluster
73+
replicas: 1
74+
selector:
75+
matchLabels: null
76+
template:
77+
spec:
78+
bootstrap:
79+
configRef:
80+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
81+
kind: MicroK8sConfigTemplate
82+
name: test-ci-cluster-md-0
83+
clusterName: test-ci-cluster
84+
infrastructureRef:
85+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
86+
kind: AWSMachineTemplate
87+
name: test-ci-cluster-md-0
88+
version: 1.27.0
89+
---
90+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
91+
kind: AWSMachineTemplate
92+
metadata:
93+
name: test-ci-cluster-md-0
94+
namespace: default
95+
spec:
96+
template:
97+
spec:
98+
iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
99+
instanceType: t3.large
100+
publicIP: false
101+
sshKeyName: capi
102+
---
103+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
104+
kind: MicroK8sConfigTemplate
105+
metadata:
106+
name: test-ci-cluster-md-0
107+
namespace: default
108+
spec:
109+
template:
110+
spec:
111+
clusterConfiguration:
112+
portCompatibilityRemap: true
113+
initConfiguration:
114+
confinement: classic
115+
riskLevel: stable

0 commit comments

Comments
 (0)