Feature:4015 Log metadata changes

[Json-Andriopoulos]

• Allow to specify multiple entities in same log_metadata() call
• Batch submit RunMetadataResource objects
• Allow to attach metadata for multiple artifacts with infer
• Split RunMetadataResource resolution logic for testability

Describe changes

I implemented/fixed _ to achieve _.

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• I have added tests to cover my changes.
• I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
• IMPORTANT: I made sure that my changes are reflected properly in the following resources:
  • ZenML Docs
  • Dashboard: Needs to be communicated to the frontend team.
  • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
  • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[bcdurak]

Leaving a few small notes but overall the changes look good. We might need some documentation to reflect the new changes though.

[bcdurak]

I am wondering whether these could be a part of the respective VersionedIdentifiers. In the current version, it is impossible to log metadata for an output artifact of the step and an external artifact at the same time.

[Json-Andriopoulos]

I think the distinction we do leads to better usage patterns though. Use infer within a function or explicit values out of it makes sense. Why would you log metadata for another step's artifact? Sounds a bit bug-prone.

IMO we should try to guide users to safer usage and if they are restricted by the signatures or the functionality they can always use the client directly (as per the updated docs).

[schustmi]

Why would you log metadata for another step's artifact

No strong opinion on whether we allow it in the bulk operations, but I've seen some usage of this before on Slack and thought I'll mention it here:
Imagine the following very simple pipeline:

@pipeline
def training_pipeline():
  train_data, test_data = load_data()
  model = train(train_data)
  eval(test_data, model)

In the eval step, you might want to attach some metadata to the artifact of the model, e.g. some metrics how it performed on the test data.

[bcdurak]

+1 to @schustmi. That was the point that I was trying to make.

[bcdurak]

Depending on whether we change this or not, the docs might need some small updates as well.

[Json-Andriopoulos]

Ok, got the point. Cool, will change the implementation to enable usage of both modes. This should enable users to log metadata both by infering entities from step context and using provided entities from the arguments. Just keep into account that since these 2 options are separate blocks of execution and manipulate data differently I can not easily and cleanly merge the options by moving infer into VersionedIdentifier definition.

[github-actions]

Documentation Link Check Results

❌ Absolute links check failed
There are broken absolute links in the documentation. See workflow logs for details
✅ Relative links check passed
_{Last checked: 2025-10-23 10:18:01 UTC}

[bcdurak]

+1 to @schustmi. That was the point that I was trying to make.

[bcdurak]

Depending on whether we change this or not, the docs might need some small updates as well.

[bcdurak]

Changes look good, I love the attention to the docs and the tests.