Skip to content

Adding a propsal for changing the KRO instance and resource labels #639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Adding a propsal for changing the KRO instance and resource labels #639

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e08ef60
Adding a propsal for changing the KRO instance and resource labels
barney-s Aug 18, 2025
c6397ab
Addressing review comments
barney-s Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions docs/design/proposals/labels.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Proposal: Standardize KRO Labels

## Problem statement

The current labels in KRO are used to track ownership and metadata of resources managed by the orchestrator. While these labels provide essential information, they lack a clear and standardized way to represent the relationships between different KRO components, such as ResourceGraphDefinitions (RGDs), instances, and the resources they generate. This ambiguity can make it difficult to query for related objects and understand the provenance of resources within a cluster.

Specifically, we need a clearer way to identify:
1. Which RGD is responsible for reconciling a given instance.
2. Which RGD a created resource originates from.
3. Which instance a created resource belongs to.

This proposal aims to introduce new, more descriptive labels to clarify these relationships, aligning with Kubernetes best practices for resource management and observability.

## Proposal

We propose introducing a set of standardized labels to better represent the relationships between RGDs, instances, and the resources they manage. This will improve the ability to discover, manage, and debug resources created and managed by KRO.

#### Overview

The proposal is to add the following labels:
- A label on an instance to indicate which RGD is reconciling it.
- A label on resources created by an instance reconciliation to indicate which RGD they come from.
- Labels on resources created by an instance reconciliation to indicate which instance created them.

This aligns with the common Kubernetes practice of using labels for discovery and organization of related resources, as seen in tools like Helm and Kustomize.

#### Design details
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we have a section addressing deeply nested RGDs (3+ levels)? thinking what happens when RGD1 creates an RGD2 instance, which creates an RGD3 instance, which creates an RGD4 instance ? With the two label scheme, we lose the intermediate relationships e.g an RGD4 instance would only show it was defined by RGD3 and controlled by RGD4, with no trace of RGD1 or RGD2. this means queries like "show me all resources that originated from RGD1, including those created indirectly through nested RGDs" won't be possible - which is fine, but worth noting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could get around this by having a root RGD, although this starts to get reaaaally complicated by then :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true. We can recursively enumerate and get the nested resources. But not with a single list query.

Question would be what do you do with the listed result ?

And this is a current pattern in k8s as well. Deployment -> Replicaset -> Pods.
There are ways to allow traceability by injecting labels downstream.


We will introduce the following new labels, which will be defined in `@pkg/metadata/labels.go`:

1. **Label for an instance being reconciled:**
- **Existing Labels:**
- kro.run/resource-graph-definition-id: 761e8fb7-14d5-4da1-b4f9-0a16fc334d6c
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to add some context about why I originally introduced the RGD ID label. The goal was to help the controller detect when RGDs are deleted and recreated, which creates completely new RGDs and can trigger unwanted cleanup. I was thinking we could have an RGD annotation like kro.run/allow-recreation (or maybe kro.run/allow-rgd-replacement?) that users would need to explicitly set. Without this annotation, kro would reject the recreation as a conflict to prevent accidental data loss. cc @matthchr @jakobmoellerdev @n3wscott

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might influence how we think about the labeling scheme - we still need some way to track RGD identity beyond just the name. but i fully agree the current implementation could be clearer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Kubernetes UID field is sufficient for this purpose because it is effectively an ID that gets recreated when the object is recreated. Any specific reason why you would want to roll your own?

- kro.run/resource-graph-definition-name: check-instance-creation
- Conflicts with label used in creation path for nested RGD (https://github.com/kro-run/kro/pull/631)
- **Proposed Label:**
- `kro.run/reconciled-by-rgd`
- **Value:** The name of the ResourceGraphDefinition (e.g., `my-rgd-name`).
- **Purpose:** To indicate that an instance is being actively reconciled against a specific RGD. This label will be applied to the instance itself.
- **Alternative Names:**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these alternative names set too or are these just suggestions? I would prefer to limit us to one.

- `kro.run/reconciling-rgd`
- `kro.run/template`

2. **Label for resources created during instance reconciliation:**
- **Existing Labels:**
- kro.run/resource-graph-definition-id: 761e8fb7-14d5-4da1-b4f9-0a16fc334d6c
- kro.run/resource-graph-definition-name: check-instance-creation
- **Proposed Label:**
- `kro.run/defined-by-rgd`
- **Value:** The name of the ResourceGraphDefinition (e.g., `my-rgd-name`).
- **Purpose:** To identify which RGD was used as a template to create a resource during the reconciliation of an instance. This label will be applied to all resources created by the instance controller.
- **Alternative Names:**
- kro.run/created-by-rgd
- kro.run/managed-by-rgd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think managed-by probably fits best since we continuously work on it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we have kro.run/managed-by-rgd it could be reused for the instance too. Is there a specific need to have two labels here?

- kro.run/owner-rgd
- kro.run/template-rgd

3. **Labels for resources to link back to the instance:**
- **Existing Label:**
- kro.run/instance-id: 6e6a1d95-4855-4062-b51b-8f288cb96365
- kro.run/instance-name: test-instance
- kro.run/instance-namespace: default
- **Proposed Labels:**
- `kro.run/managed-by-instance-group`: The API group of the instance (e.g., `mygroup.example.com`).
- `kro.run/managed-by-instance-kind`: The kind of the instance (e.g., `MyKind`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you define a group+kind, don't you also need to define a version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

kro.run/instance: production/my-web-app  # namespace/name format
kro.run/instance-gvk: myapp.io/v1/WebService  # optional for additional context ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another approach would be to just include an RGD ref?

- `kro.run/managed-by-instance-namespace`: The namespace of the instance (e.g., `default`).
- `kro.run/managed-by-instance-name`: The name of the instance (e.g., `my-instance`).
- The `managed-by` label is a common pattern in the Kubernetes ecosystem.
- **Purpose:** To provide a direct and queryable link from a created resource back to the instance that caused its creation. This set of labels uniquely identifies the instance.
- **Alternative Name prefix:**
- `kro.run/owner-`
- `kro.run/created-by-`

## Other solutions considered
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im wondering if it doesn't make sense to reuse the kubernetes labels or at least co-apply them. Im sure you thought about this too so it would be good to have a thought about this in the proposal

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 here i would love for us to adopt k8s standard labels. at least the app.kuberentes.io/managed-by and app.kubernets.io/part-of


1. We could continue using the existing labels, but their purpose is not as explicit for relationship tracking, which can lead to confusion for users and client tools.
2. We could also use annotations, but labels are better suited for this purpose as they are queryable via the Kubernetes API, which is a key requirement for observability and tooling.

## Scoping

#### What is in scope for this proposal?

- Defining and adding the new labels to the KRO API.
- Updating the instance controller to apply these labels to instances and created resources during reconciliation.
- Updating the official KRO documentation to reflect the new labels and their usage.
- Adding docs reflecting the new labels. Old labels would not be documented.
- Deprecate old labels in next minor release or the one after that.

#### What is not in scope?

- Changing how KRO uses annotations.
- Changing these existing labels:
- kro.run/kro-version: v0.4.0
- kro.run/owned: "true"

## Testing strategy

#### Requirements

- A running Kubernetes cluster to deploy KRO and test the label application.

#### Test plan

- **Unit tests:** Add unit tests for any new labeler functions in `pkg/metadata/labels.go` to ensure they construct the correct labels.
- **Integration tests:**
- Create an RGD and an instance of that RGD.
- Verify that the instance object has the `kro.run/reconciled-by` label pointing to the RGD.
- Verify that all resources created by the instance reconciliation have the `kro.run/created-by` label, and the `kro.run/instance-group`, `kro.run/instance-kind`, `kro.run/instance-namespace`, and `kro.run/instance-name` labels with the correct values.
- Update an instance to be reconciled by a different RGD and verify that the `kro.run/reconciled-by` label is updated accordingly.