Cilium Multi-Cluster with KIND

This repository provides a set of tools and examples for running a multi-cluster Cilium environment using KIND (Kubernetes in Docker).

It uses the Kubernetes Cloud Provider for KIND to provide LoadBalancer services for the KIND clusters, enabling the use of Cilium's multi-cluster features in a local development environment.

Getting Started

Prerequisites

• Docker
• kubectl
• kind
• Task
• Cilium CLI

Setup

1. Clone the repository:

   git clone https://github.com/your-username/cilium-kind-multicluster.git
   cd cilium-kind-multicluster

2. Create the KIND clusters:

   This will create two KIND clusters, cluster-1 and cluster-2, and initialize the cloud-provider-kind.

   task k8s:init

3. Install Cilium:

   This will install Cilium on both clusters.

   task cilium:install

4. Enable Clustermesh:

   This will enable the clustermesh on both clusters.

   task cilium:clustermesh:enable

5. Connect the clusters:

   This will connect the two clusters.

   task cilium:clustermesh:connect

6. Verify the status:

   task cilium:clustermesh:status

Examples

The k8s directory contains examples of how to use Cilium's multi-cluster features.

Global Service

This example demonstrates how to create a global service that is accessible across both clusters. The shared-service is a ClusterSetIP service that will be available on both clusters and will resolve to the same IP address.

graph TD
    subgraph "KIND Cluster 1"
        debug_pod(debug-pod)
    end

    subgraph "KIND Cluster 2"
        shared_pod(shared-pod)
    end

    debug_pod -- "http://shared-service" --> shared_pod

Loading

1. Deploy the global service:

   task k8s:global-service

2. Test the global service:

   First, create a debug pod:

   task test:global-service

   Then, from another terminal, exec into the debug pod and curl the shared-service:

   kubectl --context kind-cluster-1 exec -it debug1 -- curl http://shared-service

Inter-cluster Load Balancing

This example demonstrates inter-cluster load balancing. A service rebel-base is created on both clusters, and Cilium will load balance requests to this service across pods in both clusters.

graph TD
    subgraph "KIND Cluster 1"
        rebel_base_1(rebel-base-1)
    end

    subgraph "KIND Cluster 2"
        rebel_base_2(rebel-base-2)
    end

    client((client)) -- "http://rebel-base" --> rebel_base_1
    client -- "http://rebel-base" --> rebel_base_2

Loading

1. Deploy the inter-cluster load balancing example:

   task k8s:deploy-intercluster-loadbalancing

2. Test the inter-cluster load balancing:

   Run the following command to send 10 requests to the rebel-base service. You should see responses from pods in both cluster-1 and cluster-2.

   kubectl run --rm -it --image=alpine/curl --restart Never debug2 --context kind-cluster-1 -- sh -c 'for i in $(seq 1 10); do curl -sS http://rebel-base; done'

Teardown

To delete the clusters and clean up all resources, run:

task k8s:destroy