metricbeat: factory: Introduce metricSetWithPath

[orestisfl]

Proposed commit message

Introduce metricSetWithPath interface in factory to allow metricsets to receive beat-specific paths. Update openai/usage module to use SetPath() instead of paths.Resolve() for state manager initialization.

Fixes #46987

Diagram

The following diagram illustrates how the beat.Paths object is passed from the Metricbeat beater down to a metricset component.

graph TD
    %% metricbeat/beater
    subgraph "metricbeat/beater"
        Run(["Metricbeat.Run()"])
        Paths["beat.Paths"]
    end

    %% metricbeat/mb/module
    subgraph "metricbeat/mb/module"
        NewFactory(["NewFactory()"])
        Create(["factory.Create()"])
    end

    %% openai/usage
    subgraph "openai/usage"
        New(["New()"])
        MetricSet["MetricSet (instance)"]
        SetPath(["MetricSet.SetPath()"])
        Fetch(["MetricSet.Fetch()"])
        State[stateManager]
    end

    %% Relationships
    Run -- "Holds instance of" --> Paths
    Run -- "Calls" --> NewFactory
    Paths -- "passed as argument" --> NewFactory
    NewFactory -- "Returns factory instance" --> Create
    Run -- "Calls" --> Create
    Create -- "Eventually invokes metricset factory" --> New
    New -- "Returns" --> MetricSet
    MetricSet --- SetPath
    MetricSet --- Fetch
    Create -- "Calls" --> SetPath
    SetPath -- "Accesses paths and initializes" --> State
    Fetch -- "Reads state from" --> State
    Run -- "Calls" --> Fetch

Loading

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Should not disrupt previous behavior.

How to test this PR locally

Related issues

• Related chore: metricbeat: openai: Implement ReportingMetricSetV2WithContext #47387
• Closes [beatreceiver] replace global path in metricbeat openai module #46987

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[leehinman]

I really don't think we should do this. Callers should be responsible for giving a valid *paths.Path. Also we really want to get rid of the paths.Paths global variable everywhere in the Beats codebase.

[orestisfl]

Sure, I can change that. It's a bit of a red flag that paths is a pointer always, especially in this case where we only create the store once. Should I just change it from *paths.Path to paths.Path or is it intentionally a pointer?

[leehinman]

Sure, I can change that. It's a bit of a red flag that paths is a pointer always, especially in this case where we only create the store once. Should I just change it from *paths.Path to paths.Path or is it intentionally a pointer?

I agree the pointer is "bad", but I haven't really looked at replacing it with a struct. I know some of the functions are built before InitPaths is run, which is what resolves default values and what is in the config. So in at least some places we would still need a pointer.

[leehinman]

Code changes LGTM.

For the Makefile changes, we might want that in a separate PR

[orestisfl]

Code changes LGTM.

For the Makefile changes, we might want that in a separate PR

Sorry, playing with some new tooling, didn't realize I staged those.

[mauri870]

LGTM. There is one small linter issue.

[pierrehilbert]

@lalit-satapathy could we please have someone to take a look here?
cc @ishleenk17

[shmsr]

Hi @orestisfl!

We do not support this OpenAI module; in fact, we never actually utilized it. Our initial plan was to use this module for integration, but immediately after we merged it, OpenAI announced their official Usage API. Consequently, we developed the solution in CEL instead. The CEL approach allows us to iterate faster and makes updates easier to deploy.

This module relies on an undocumented API from OpenAI. While that was originally the only way to collect data, it is now obsolete. Since we have the official API, we neither use nor support this module. Furthermore, because OpenAI no longer maintains the undocumented endpoint, users should avoid this module entirely.

Therefore, we are no longer accepting changes to this module. I will add a deprecated comment, I will see how to do it and will do the needful. We will only make an exception for changes that are critical blockers for other components. We will not accept any non-blocking changes as this module is no longer maintained.

In case you want to merge as this could be a blocker — like removing some function or use of some package or something of that nature that could be a blocker, please let us know. I will approve those PRs.

[rdner]

@orestisfl what are our next steps with this PR? Do we still need it?

[orestisfl]

Closing given that the module is deprecated.