Skip to content

Update checkpointing in alpine and maven pipelines #740

@JonoYang

Description

@JonoYang

When running the alpine and maven pipelines to collect package urls, both actions time out after 6 hours. We should have a way to keep track of where we are when we are processing an index file so when we are interrupted, we can start off where we were.

For maven, the order of packages in the index is deterministic for a given version of the index, when the index is updated, the order of packages is not always the same as the order it was in the previous index.

We would need to keep track of the index file checksum to see if we can resume from the index file or if we have to start over because the index is different. We would also need to track which package we last processed so we know where to pick up from.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions