Discussion on Pipeline breakpoints feature

[davidsbatista]

This is the discussion board for the experimental feature of adding breakpoints to a Pipeline

This experiment allows set breakpoints at any component in any pipeline, forcing the pipeline run to stop before that component runs.

Usage Examples

# Setting breakpoints
pipeline.run(
    data={"input": "value"},
    breakpoints={("component_name", 0)},  # Break at the first visit
    debug_path="debug_states/"
)

This will generate a JSON with the complete pipeline state before the next component is run, i.e.: the one receiving the output of the component set in the breakpoint

# Resuming from a saved state
state = Pipeline.load_state("debug_states/component_state.json")
pipeline.run(
    data={"input": "value"},
    resume_state=state
)

🧑‍🍳 See an example notebook here
💻 See the PR that adds the experiment here

Note: Experimental features live in this repository for a fixed period of time. We don't guarantee that we will continue maintaining experimental features. But if they are successful, stable and if you like it, we will move the feature to the core Haystack package.

[aymbot]

Great feature! As a developer working with haystack, I would like to know if this feature will be expanded to async pipelines as well?
I am in the process of developing a multi-agent architecture where some of the agents are executed within async pipelines, hence the interest.

[davidsbatista]

Yes, we are also planning to have it in the async version of the Pipeline. Can you test it as well on the sync version, or you can only test it in the async?