Skip to content

Conversation

ecomodeller
Copy link
Member

@ecomodeller ecomodeller commented Sep 23, 2025

import modelskill as ms
import pandas as pd

time = pd.date_range("2000", periods=2)
cmp1 = ms.match(
    obs=ms.PointObservation(pd.Series([0, 0], index=time), name="foo"),
    mod=ms.PointModelResult(pd.Series([0, 0], index=time), name="bar"),
)
cmp2 = ms.match(
    obs=ms.PointObservation(pd.Series([0, 0], index=time), name="baz"),
    mod=ms.PointModelResult(pd.Series([0, 0], index=time), name="qux"),
)
# TODO should it be allowed to combine two comparers with different models, maybe not?
cc = ms.ComparerCollection([cmp1, cmp2])
cc.skill(metrics="rmse")
image

So far so good, but if I call

cc.score() 😟

it fails with a cryptical message since the weights vector is of the wrong length, if each comparers doesn't contain each observation.

        if len(data) != len(index):
>           raise ValueError(
                "Length of values "
                f"({len(data)}) "
                "does not match length of index "
                f"({len(index)})"
            )
E           ValueError: Length of values (4) does not match length of index (2)

This PR fixes this problem, and does some more refactoring related to weighted scoring.

@ecomodeller ecomodeller marked this pull request as ready for review September 23, 2025 20:55
@ecomodeller ecomodeller changed the title Fix score of multiple models Bug: Score of multiple models fails if observations are not in each comparer Sep 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant