SLO version labels #1547
Description
A few times now we have run into cases where the underlying SLO query needs to be updated for an existing SLO. In our current deployment at https://slo.wikimeida.org making this kind of change cause a sudden change in the dashboard graph history, without a good way to delineate why or filter between iterations of the SLO query.
My first thought was to add an slo version label, e.g. pyrra.dev/version: '1'. The workflow I had in mind was to bump the value each time the underlying SLO query is changed, and backfill metrics for that version. However, I'm noticing that the pyrra.dev/ labels aren't included in queries in the Pyrra UI (e.g. when exploring the prometheus query links) it seems they are only present as filters for the search UI and are included in the recording rule metric output.
Would there be a negative side-effect of including the pyrra.dev/ labels in the UI queries by default? Or perhaps via a toggle for each label?
Or is there a better way to accomplish what I'm trying to do? Namely keep clean SLO metrics history across changes without SLO name sprawl, and trying to avoid label overlaps during backfill.