remove environment variables from integration tests

KU-1226

Author: Maciej Golaszewski

README.md

@@ -9,12 +9,12 @@ This project offers a cluster API bootstrap provider controller that manages the
 ### Prerequisites
 
   * Install clusterctl following the [upstream instructions](https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl)
-```bash
+```
 curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.1.3/clusterctl-linux-amd64 -o clusterctl
 ```
 
   * Install a bootstrap Kubernetes cluster. To use MicroK8s as a bootstrap cluster:
-```bash
+```
 sudo snap install microk8s --classic
 sudo microk8s.config  > ~/.kube/config
 sudo microk8s enable dns
@@ -24,7 +24,7 @@ sudo microk8s enable dns
 
 To to configure clusterctl with the two MicroK8s providers edit `~/.cluster-api/clusterctl.yaml`
 and add the following:
-```yaml
+```
 providers:
   - name: "microk8s"
     url: "https://github.com/canonical/cluster-api-bootstrap-provider-microk8s/releases/latest/bootstrap-components.yaml"
@@ -47,18 +47,18 @@ Alternatively, you can build the providers manually as described in the followin
 ### Building from source
 
   * Install the cluster provider of your choice. Have a look at the [cluster API book](https://cluster-api.sigs.k8s.io/user/quick-start.html#initialization-for-common-providers) for your options at this step. You should deploy only the infrastructure controller leaving the bootstrap and control plane ones empty. For example assuming we want to provision a MicroK8s cluster on AWS:
-```bash
+```
 clusterctl init --infrastructure aws --bootstrap "-" --control-plane "-"
 ```
 
   * Clone the two cluster API MicroK8s specific repositories and start the controllers on two separate terminals:
-```bash
+```
 cd $GOPATH/src/github.com/canonical/cluster-api-bootstrap-provider-microk8s/
 make install
 make run
 ```
 And:
-```bash
+```
 cd $GOPATH/src/github.com/canonical/cluster-api-control-plane-provider-microk8s/
 make install
 make run
@@ -68,7 +68,7 @@ make run
 
 As soon as the bootstrap and control-plane controllers are up and running you can apply the cluster manifests describing the desired specs of the cluster you want to provision. Each machine is associated with a MicroK8sConfig through which you can set the cluster's properties. Please review  the available options in [the respective definitions file](./apis/v1beta1/microk8sconfig_types.go). You may also find useful the example manifests found under the [examples](./examples/) directory. Note that the configuration structure followed is similar to the the one of kubeadm, in the MicroK8sConfig you will find a CLusterConfiguration and an InitConfiguration sections. When targeting a specific infrastructure you should be aware of which ports are used by MicroK8s and allow them in the network security groups on your deployment.
 
-Two workload cluster templates are available under the [templates](./templates/) folder, which are actively used to validate releases:
+Two workload cluster templates are available under the [templates](./templates) folder, which are actively used to validate releases:
 - [AWS](./templates/cluster-template-aws.yaml), using the [AWS Infrastructure Provider](https://github.com/kubernetes-sigs/cluster-api-provider-aws)
 - [OpenStack](./templates/cluster-template-openstack.yaml), using the [OpenStack Infrastructure Provider](https://github.com/kubernetes-sigs/cluster-api-provider-openstack)
 

integration/README.md

@@ -4,47 +4,10 @@
 
 The integration/e2e tests have the following prerequisites:
 
-  * an environment variable `CLUSTER_MANIFEST_FILE` pointing to the cluster manifest. Cluster manifests can be produced with the help of the templates found under [`templates`](../templates). For example:
-  ```bash 
-export AWS_REGION=us-east-1
-export AWS_SSH_KEY_NAME=capi
-export CONTROL_PLANE_MACHINE_COUNT=3
-export WORKER_MACHINE_COUNT=3
-export AWS_CREATE_BASTION=false
-export AWS_PUBLIC_IP=false
-export AWS_CONTROL_PLANE_MACHINE_FLAVOR=t3.large
-export AWS_NODE_MACHINE_FLAVOR=t3.large
-export CLUSTER_NAME=test-ci-cluster
-clusterctl generate cluster ${CLUSTER_NAME} --from "templates/cluster-template-aws.yaml" --kubernetes-version 1.27.0 > cluster.yaml
-export CLUSTER_MANIFEST_FILE=$PWD/cluster.yaml
-  ```
-> NOTE: AWS_SSH_KEY_NAME is name of ssh key in AWS that you are plan to use, if you don't have one yet refer
-> to capi on [aws prerequisites documentation](https://cluster-api-aws.sigs.k8s.io/topics/using-clusterawsadm-to-fulfill-prerequisites#ssh-key-pair)
-  *  Additional environment variables when testing cluster upgrades:
-  ```bash
-export CAPI_UPGRADE_VERSION=v1.28.0
-export CAPI_UPGRADE_MD_NAME=${CLUSTER_NAME}-md-0
-export CAPI_UPGRADE_MD_TYPE=machinedeployments.cluster.x-k8s.io
-export CAPI_UPGRADE_CP_NAME=${CLUSTER_NAME}-control-plane
-export CAPI_UPGRADE_CP_TYPE=microk8scontrolplanes.controlplane.cluster.x-k8s.io
-# Change the control plane and worker machine count to desired values for in-place upgrades tests and create a new cluster manifest.
-CONTROL_PLANE_MACHINE_COUNT=1
-WORKER_MACHINE_COUNT=1
-clusterctl generate cluster ${CLUSTER_NAME} --from "templates/cluster-template-aws.yaml" --kubernetes-version 1.27.0 > cluster-inplace.yaml
-export CLUSTER_INPLACE_MANIFEST_FILE=$PWD/cluster-inplace.yaml
-```
-
-  *  Additional environment variables when testing disable default CNI flag:
-  ```bash
-export DISABLE_DEFAULT_CNI=true
-export POST_RUN_COMMANDS='["helm install cilium cilium/cilium  --namespace kube-system --set cni.confPath=/var/snap/microk8s/current/args/cni-network --set cni.binPath=/var/snap/microk8s/current/opt/cni/bin --set daemon.runPath=/var/snap/microk8s/current/var/run/cilium --set operator.replicas=1 --set ipam.operator.clusterPoolIPv4PodCIDRList=\"10.1.0.0/16\" --set nodePort.enabled=true"]' # install Calico in place default CNI
-clusterctl generate cluster ${CLUSTER_NAME} --from "templates/cluster-template-aws.yaml" --kubernetes-version 1.27.0 > cluster_disable_default_cni.yaml
-export CLUSTER_DISABLE_DEFAULT_CNI_MANIFEST_FILE=$PWD/cluster_disable_default_cni.yaml
-```
+  * make sure to have ssh key in aws `capi`in `us-east-1 region` if you do not have key refer
+    to CAPI on [AWS prerequisites documentation](https://cluster-api-aws.sigs.k8s.io/topics/using-clusterawsadm-to-fulfill-prerequisites#ssh-key-pair)
 
-  * `clusterctl` available in the PATH
-
-  * `kubectl` available in the PATH
+  * to run tests locally you will need to have on path fallowing programs `clusterctl`, `kubectl`, `helm`
 
   * a management cluster initialised via `clusterctl` with the infrastructure targeted as well as the version of the MicroK8s providers we want to be tested
 
@@ -77,7 +40,7 @@ microk8s config > ~/.kube/config
 Visit [here](https://cluster-api.sigs.k8s.io/user/quick-start.html#initialization-for-common-providers) for a list of common infrastructure providers.
 
 ```bash
-clusterctl init --infrastructure <infra> --bootstrap - --control-plane -
+  clusterctl init --infrastructure <infra> --bootstrap - --control-plane -
 ```
 
 #### Build Docker images and release manifests from the checked out source code
@@ -103,6 +66,9 @@ sed "s,docker.io/cdkbot/capi-control-plane-provider-microk8s:latest,docker.io/<u
 ```bash
 kubectl apply -f bootstrap/bootstrap-components.yaml -f control-plane/control-plane-components.yaml
 ```
+### Cluster definitions for e2e
+
+Cluster definition are stored in [`manifests`](./cluster-manifests) directory.
 
 #### Trigger the e2e tests
 

integration/cluster-manifests/cluster-disable-default-cni.yaml

@@ -0,0 +1,117 @@
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: Cluster
+metadata:
+  name: test-ci-cluster
+  namespace: default
+spec:
+  controlPlaneRef:
+    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+    kind: MicroK8sControlPlane
+    name: test-ci-cluster-control-plane
+  infrastructureRef:
+    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+    kind: AWSCluster
+    name: test-ci-cluster
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSCluster
+metadata:
+  name: test-ci-cluster
+  namespace: default
+spec:
+  bastion:
+    enabled: false
+  region: us-east-1
+  sshKeyName: capi
+---
+apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+kind: MicroK8sControlPlane
+metadata:
+  name: test-ci-cluster-control-plane
+  namespace: default
+spec:
+  controlPlaneConfig:
+    clusterConfiguration:
+      portCompatibilityRemap: true
+    initConfiguration:
+      IPinIP: true
+      addons:
+      - dns
+      - ingress
+      confinement: classic
+      disableDefaultCNI: true
+      joinTokenTTLInSecs: 900000
+      riskLevel: stable
+  machineTemplate:
+    infrastructureTemplate:
+      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+      kind: AWSMachineTemplate
+      name: test-ci-cluster-control-plane
+  replicas: 1
+  upgradeStrategy: SmartUpgrade
+  version: v1.27.0
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSMachineTemplate
+metadata:
+  name: test-ci-cluster-control-plane
+  namespace: default
+spec:
+  template:
+    spec:
+      iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
+      instanceType: t3.large
+      publicIP: false
+      sshKeyName: capi
+---
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: MachineDeployment
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  clusterName: test-ci-cluster
+  replicas: 1
+  selector:
+    matchLabels: null
+  template:
+    spec:
+      bootstrap:
+        configRef:
+          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+          kind: MicroK8sConfigTemplate
+          name: test-ci-cluster-md-0
+      clusterName: test-ci-cluster
+      infrastructureRef:
+        apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+        kind: AWSMachineTemplate
+        name: test-ci-cluster-md-0
+      version: 1.27.0
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSMachineTemplate
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  template:
+    spec:
+      iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
+      instanceType: t3.large
+      publicIP: false
+      sshKeyName: capi
+---
+apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+kind: MicroK8sConfigTemplate
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  template:
+    spec:
+      clusterConfiguration:
+        portCompatibilityRemap: true
+      initConfiguration:
+        confinement: classic
+        disableDefaultCNI: true
+        riskLevel: stable

integration/cluster-manifests/cluster-inplace.yaml

@@ -0,0 +1,117 @@
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: Cluster
+metadata:
+  name: test-ci-cluster
+  namespace: default
+spec:
+  controlPlaneRef:
+    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+    kind: MicroK8sControlPlane
+    name: test-ci-cluster-control-plane
+  infrastructureRef:
+    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+    kind: AWSCluster
+    name: test-ci-cluster
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSCluster
+metadata:
+  name: test-ci-cluster
+  namespace: default
+spec:
+  bastion:
+    enabled: false
+  region: us-east-1
+  sshKeyName: capi
+---
+apiVersion: controlplane.cluster.x-k8s.io/v1beta1
+kind: MicroK8sControlPlane
+metadata:
+  name: test-ci-cluster-control-plane
+  namespace: default
+spec:
+  controlPlaneConfig:
+    clusterConfiguration:
+      portCompatibilityRemap: true
+    initConfiguration:
+      IPinIP: true
+      addons:
+      - dns
+      - ingress
+      confinement: classic
+      disableDefaultCNI: false
+      joinTokenTTLInSecs: 900000
+      riskLevel: stable
+  machineTemplate:
+    infrastructureTemplate:
+      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+      kind: AWSMachineTemplate
+      name: test-ci-cluster-control-plane
+  replicas: 1
+  upgradeStrategy: SmartUpgrade
+  version: v1.27.0
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSMachineTemplate
+metadata:
+  name: test-ci-cluster-control-plane
+  namespace: default
+spec:
+  template:
+    spec:
+      iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
+      instanceType: t3.large
+      publicIP: false
+      sshKeyName: capi
+---
+apiVersion: cluster.x-k8s.io/v1beta1
+kind: MachineDeployment
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  clusterName: test-ci-cluster
+  replicas: 1
+  selector:
+    matchLabels: null
+  template:
+    spec:
+      bootstrap:
+        configRef:
+          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+          kind: MicroK8sConfigTemplate
+          name: test-ci-cluster-md-0
+      clusterName: test-ci-cluster
+      infrastructureRef:
+        apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+        kind: AWSMachineTemplate
+        name: test-ci-cluster-md-0
+      version: 1.27.0
+---
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
+kind: AWSMachineTemplate
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  template:
+    spec:
+      iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
+      instanceType: t3.large
+      publicIP: false
+      sshKeyName: capi
+---
+apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
+kind: MicroK8sConfigTemplate
+metadata:
+  name: test-ci-cluster-md-0
+  namespace: default
+spec:
+  template:
+    spec:
+      clusterConfiguration:
+        portCompatibilityRemap: true
+      initConfiguration:
+        confinement: classic
+        disableDefaultCNI: false
+        riskLevel: stable