🎄 Advent of Haystack - try out Haystack 2.0-Beta

[TuanaCelik]

During the month of December, we will be releasing a challenge every few days starting December 4th (so not the traditional day to start 🙂), with the release of Haystack 2.0-Beta. We are not releasing Haystack 2.0 just yet, but we would like to share what is our first commitment to the next major release of Haystack with a set of fun challenges that you can use to have a taste of what Haystack 2.0 will look like.

You will see challenges go live on the Advent of Haystack page. And, we will use this Discussion page to share with you the solutions of each challenge later on. You can also use this page to ask questions or discuss any specific challenge with the rest of the community as well.

This is our first time hosting an Advent of Haystack, we hope you enjoy it, and we're looking forward to hear what your thoughts on the design of Haystack 2.0 are. During the period of time in which 2.0 is still in Beta, we will have the opportunity to incorporate your feedback!

You will be able to submit your challenges, alongside any feedback you would like to give us on how it went, what you found easy or difficult 🫶

[antonioalmeida]

Sounds fun! 😄 Any resources that you recommend catching up on to prepare? Haven't used Haystack in a while

[TuanaCelik]

Glad you're looking forward to it. To have a sneak peak already and to explore 2.0-Beta you can already start having a look at the unpublished 2.0 docs: https://docs.haystack.deepset.ai/v2.0/docs

[TuanaCelik]

First Challenge is live: https://haystack.deepset.ai/advent-of-haystack/day-1#challenge

If there are any questions, we can use this thread 👇

[lfunderburk]

Hi team

Chris Alexiuk and I will be tackling the challenges. Chris is wondering if we can instantiate the pipeline using a list, where each previous element connects to the next, and so on?

i.e. instead of 4 .connect() calls, we could do something like .connect(["component_1", "component_2", ...])?

[chris-alexiuk]

It's true, I was wondering!

[masci]

This is something we want to add indeed!

It won't work for any possible combination of components (think of two components with more than one input/output of the same type), but it would still save time on most of the use cases. I'll open an issue to track the feature request.

Thanks for your feedback!

[AndyTheFactory]

Maybe an easy "fix" would be to return self from connect in order to chain the connects?

pipeline.connect("fetcher", "converter").connect("converter", "splitter")

[kynthesis]

What if I connect the Splitter with the Prompt Builder without explicitly linking the Splitter's output to the Prompt Builder's input?

CASE A: pipeline.connect("splitter", "prompt_builder.documents")
CASE B: pipeline.connect("splitter.documents", "prompt_builder.documents")

They both seem to be working. But does it affect the overall performance?

[TuanaCelik]

Second challenge is live: https://haystack.deepset.ai/advent-of-haystack/day-2#challenge

If there are any questions, we can use this thread 👇

[julian-risch]

Hi @lfunderburk and @chris-alexiuk great to see you experimenting so much with Haystack 2.0 beta! What you describe here sounds like two separate issues that we so far weren't aware of. One is about tgi_llm raised 'AttributeError: 'str' object has no attribute 'value'' and the other is about pipeline.connect("tgi_ranker", "tgi_prompt_builder") leading to a wrong connection between tgi_ranker.documents and tgi_prompt_builder.question. Could you please open two issues so that we can assign team members to work on them in our next sprint planning? Thank you so much!

[lfunderburk]

#6513 TGI Generator unexpected behaviour

#6512 Changing names of components yields errors

[lfunderburk]

@chris-alexiuk can you create an issue for the ranker issue you experienced?

[AndyTheFactory]

what i noticed: if you set up the splitter with a too large of a chunk , for instance:

splitter = DocumentSplitter(split_length=300, split_overlap=5)

the ranker throws an exception (i suppose the tokenized text exceeds the max length of the sentence transformer):

The error is confusing, and i think it should just truncate the text. Should i post this as an issue?

[FHardow]

To the comment above - the same happens to me on a plain initialization for the DocumentSplitter.
You will need to set the split_length to something smaller.
-> DocumentSplitter(split_length=100)

[TuanaCelik]

Third challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-3#challenge

If there are any questions, we can use this thread 👇

[lagom0327]

When I tried to fulfill the challenge, I'm curious about how to see intermediate components output in pipelines.

According to this document,
Modify code to

from pathlib import Path
pipeline = Pipeline(debug_path=Path(".haystack_debug/"))
pipeline.add_component(instance=query_embedder, name="query_embedder")
pipeline.add_component(instance=retriever, name="retriever")
pipeline.add_component(instance=prompt_builder, name="prompt_builder")
pipeline.add_component(instance=llm, name="llm")

pipeline.connect("query_embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "llm")

result = pipeline.run(data={"query_embedder": {"text": query }, "prompt_builder": {"query": query}}, debug=True)

It throws an exception as below:

Output exceeds the [size limit](command:workbench.action.openSettings?[). Open the full output data [in a text editor](command:workbench.action.openLargeOutput?a7b0feb5-5c2f-4140-9aa4-de6aba8108ed)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[9], line 2
      1 query = "How do I use the openai embedder?"
----> 2 result = pipeline.run(data={"query_embedder": {"text": query }, "prompt_builder": {"query": query}}, debug=True)
      3 # result = pipeline.run(data={"query_embedder": {"text": query }, "prompt_builder": {"query": query}})
      4 print(result['llm']['replies'][0])

File ~/haystack2/venv/lib/python3.10/site-packages/haystack/pipeline.py:85, in Pipeline.run(self, data, debug)
     83 is_nested_component_input = all(isinstance(value, dict) for value in data.values())
     84 if is_nested_component_input:
---> 85     return self._run_internal(data=data, debug=debug)
     86 else:
     87     # flat input, a dict where keys are input names and values are the corresponding values
     88     # we need to convert it to a nested dictionary of component inputs and then run the pipeline
     89     # just like in the previous case
     90     pipeline_inputs, unresolved_inputs = self._prepare_component_input_data(data)

File ~/haystack2/venv/lib/python3.10/site-packages/haystack/pipeline.py:111, in Pipeline._run_internal(self, data, debug)
    100 """
    101 Runs the pipeline by invoking the underlying run to initiate the pipeline execution.
    102 
   (...)
    108 :raises PipelineRuntimeError: if any of the components fail or return unexpected output.
    109 """
...
--> 280     y.__dict__.update(state)
    281 if slotstate is not None:
    282     for key, value in slotstate.items():

TypeError: descriptor '__dict__' for 'SentenceTransformersTextEmbedder' objects doesn't apply to a 'SentenceTransformersTextEmbedder' object

Am I using this parameter incorrectly? Or debug mode is still being under construction ?

[TuanaCelik]

Hey @lagom0327 - I've also experienced this issue in some pipelines that use specific components. For example I had this issue with LinkContentFetcher which was closed by @masci yesterday: #6265

I suspect this is a pretty similar issue! But thanks fore reporting. @masci do you think it makes sense to have a separate issue for this?

[bilgeyucel]

Fourth challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-4#challenge

If there are any questions, we can use this thread 👇

[TuanaCelik]

Fifth challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-5#challenge

If there are any questions, we can use this thread 👇

[kynthesis]

Hi, I tried to run the solution but I've got IndexError: list index out of range.

from haystack import Pipeline, Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever

documents = [Document(content="Use pip to install a basic version of Haystack's latest release: pip install farm-haystack. All the core Haystack components live in the haystack repo. But there's also the haystack-extras repo which contains components that are not as widely used, and you need to install them separately.",
                      meta={"version": "1.15"}),
             Document(content="Use pip to install a basic version of Haystack's latest release: pip install farm-haystack[inference]. All the core Haystack components live in the haystack repo. But there's also the haystack-extras repo which contains components that are not as widely used, and you need to install them separately.",
                      meta={"version": "1.22"}),
             Document(content="Use pip to install only the Haystack 2.0 code: pip install haystack-ai. The haystack-ai package is built on the main branch which is an unstable beta version, but it's useful if you want to try the new features as soon as they are merged.",
                      meta={"version": "2.0"}),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(documents=documents)

pipeline = Pipeline()
pipeline.add_component(instance=InMemoryBM25Retriever(document_store=document_store), name="retriever")

query = "Haystack installation"
result = pipeline.run(data={"retriever": {"query": query, "filters": { "field": "meta.version", "operator": "in", "value": ["2.0"]}}})
print(result['retriever']['documents'][0].content)

[TuanaCelik]

Hey @kynthesis - seems you've caught a new bug, the retriever is returning an empty list right now. We're on it. Will update you here when we've got a fix 🙏

[TuanaCelik]

@kynthesis we found out what's going on and the solution now works:
We recently merged a PR that sets scale_score to False by default for InMemoryBM25Retriever, which means that while scores were being scaled between 0-1 before, now it isn't. The BM25 algorithm we use for InMemory is okapi, which you can change while setting up the document store. This algorithm returns negative scores too. So it means when we merged the change, it started returning an empty list because we filter out negative scores.

2 options to fix the issue in the solution:

• Change the algo: InMemoryDocumentStore(bm25_algorithm="BM25Plus")
• Or, keep that the same and set scale_score=True in the retriever: InMemoryBM25Retriever(document_store=document_store, scale_score=True)

Now the discussion is whether we shouldn't filter out the negative scores at all though since for our default algo, Okapi, that's totally valid...

Thanks for flagging this!! 💙

Edit: Issue tracked here: #6801

[TuanaCelik]

Sixth challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-6#challenge

If there are any questions, we can use this thread 👇

[nickprock]

Hi, I try to add component to the pipeline

preprocessing_pipeline = Pipeline()
preprocessing_pipeline.add_component(instance=file_type_router, name="file_type_router")
preprocessing_pipeline.add_component(instance=text_file_converter, name="text_file_converter")
preprocessing_pipeline.add_component(instance=markdown_converter, name="markdown_converter")
preprocessing_pipeline.add_component(instance=pdf_converter, name="pypdf_converter")
preprocessing_pipeline.add_component(joiner, name="joiner")
preprocessing_pipeline.add_component(cleaner, name="cleaner")
preprocessing_pipeline.add_component(splitter, name="splitter")
preprocessing_pipeline.add_component(document_embedder, name="document_embedder")
preprocessing_pipeline.add_component(document_writer, name="document_writer")

but I have an error

TypeError                                 Traceback (most recent call last)

[<ipython-input-7-eb8d0bcb5aa8>](https://localhost:8080/#) in <cell line: 6>()
      4 preprocessing_pipeline.add_component(instance=markdown_converter, name="markdown_converter")
      5 preprocessing_pipeline.add_component(instance=pdf_converter, name="pypdf_converter")
----> 6 preprocessing_pipeline.add_component(joiner, name="joiner")
      7 preprocessing_pipeline.add_component(cleaner, name="cleaner")
      8 preprocessing_pipeline.add_component(splitter, name="splitter")

TypeError: Pipeline.add_component() got multiple values for argument 'name'

it's probably a trivial thing but I can't solve it because I'm getting too hung up on it.

[lagom0327]

Hi @nickprock.
It seems like the issue is caused by not specifying the parameter names when calling the add_component method. According to the method definition:

def add_component(self, name: str, instance: Component) -> None:

The name parameter should be provided as the first argument when calling the method. Make sure that when you call add_component, you pass the values in the correct order.

You can avoid this error by explicitly specifying the parameter names.

preprocessing_pipeline.add_component(instance=joiner, name="joiner")

[nickprock]

Thanks @lagom0327 , I knew I had it in front of my eyes but couldn't see it

[TuanaCelik]

Seventh challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-7#challenge

If there are any questions, we can use this thread 👇

[TuanaCelik]

Eighth challenge is live 🤗: https://haystack.deepset.ai/advent-of-haystack/day-8#challenge

If there are any questions, we can use this thread 👇

[bilgeyucel]

The ninth challenge is out 🎅: https://haystack.deepset.ai/advent-of-haystack/day-9#challenge

If there are any questions, we can use this thread 👇

[anakin87]

Nice link 😄

I would try logging

import logging
logging.basicConfig()
logging.getLogger("haystack.core.pipeline.pipeline").setLevel(logging.DEBUG)

[bilgeyucel]

@filippo82 You can also try running the components on their own if you're curious to see the individual outputs. E.g.

from haystack.components.audio import LocalWhisperTranscriber

whisper = LocalWhisperTranscriber(model_name_or_path="small")
transcription = whisper.run(audio_files=["path/to/audio/file"])

[filippo82]

Thank you 🙏🏻 both your suggestions did the trick. I just wanted to make sure that the prompt template was getting correctly populated.

As a general comment, I think it would be useful to have a switch to allow a pipeline to output intermediate outputs. I think this could be useful for debugging purposes. Something like pipeline.run(..., output_intermediate_steps=True). Would you know if a similar request exists already in the shape of either a GitHub issue or discussion?

[lagom0327]

I'm having some problems when using TGI.

AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 llm.run(prompt="xxxxxxxx", generation_kwargs={"max_new_tokens": 150})
....

File ~/haystack2/venv/lib/python3.10/site-packages/haystack/components/generators/hugging_face_tgi.py:228, in HuggingFaceTGIGenerator._run_non_streaming(self, prompt, prompt_token_count, num_responses, generation_kwargs)
    222 for _i in range(num_responses):
    223     tgr: TextGenerationResponse = self.client.text_generation(prompt, details=True, **generation_kwargs)
    224     all_metadata.append(
    225         {
    226             "model": self.client.model,
    227             "index": _i,
--> 228             "finish_reason": tgr.details.finish_reason.value,
    229             "usage": {
    230                 "completion_tokens": len(tgr.details.tokens),
    231                 "prompt_tokens": prompt_token_count,
    232                 "total_tokens": prompt_token_count + len(tgr.details.tokens),
    233             },
    234         }
    235     )
    236     responses.append(tgr.generated_text)
    237 return {"replies": responses, "metadata": all_metadata}

AttributeError: 'str' object has no attribute 'value'

tgr.details.finish_reason is a string

[bilgeyucel]

Hi @lagom0327, thank you for reporting this, it seems like the same issue with this one: #6513
Would you mind adding more details to that issue? Knowing about your Haystack version, your OS, etc. would help us a lot 🙌

[masci]

The last challenge is out! https://haystack.deepset.ai/advent-of-haystack/day-10#challenge
Discussion thread 👇

[TuanaCelik]

Hi everyone 👋 We've come to the end of the first week of 2024 and with that, I'm happy to share with you the solutions to all of the Advent of Haystack challenges. Thank you to everyone who participated in the very first Advent of Haystack and a special thanks to those of you who submitted your results and solutions 🩵

Here are the solutions 👇

• Day 1 🎨: Completing Haystack 2.0 pipeline connections. - Solution
• Day 2 🐹: Building a RAG pipeline to find our favorite animal. - Solution
• Day 3 📚: Creating a prompt template to do referenced QA. - Solution
• Day 4 🌳: Creating a pipeline with branches. - Solution
• Day 5 ⏳: Using metadata to filter documents. - Solution
• Day 6 🚀: Preprocessing different file types and writing them into document stores. - Solution
• Day 7 🌍: Conditional RAG on the web QA. - Solution
• Day 8 ⏰: Creating a custom component for ranking based in date. - Solution
• Day 9 🎅: Transcribing audio and transforming style. - Solution
• Day 10 📜: Serializing pipelines with a custom Marshaller. - Solution

[filippo82]

That was fun! Please do it again next year 🙂

[annthurium]

aw thank you!

[Ashish-Soni08]

Thank you guys :) It was fun :)

Day 7 Fail: Bad Jinja :p

I had question about day 10, Why do we need to serialize pipelines? What is the purpose of doing it?

[masci]

@Ashish-Soni08 Any situation where having Python around isn't practical is a good example of how serialization can be helpful:

• store a pipeline in a database
• sending pipelines over the network
• interoperability (say you want to create a UI editor for pipelines, that could use Yaml to represent them)

There are probably more examples, but I hope this gives you an idea!

[Ashish-Soni08]

Thank you @masci :)