Discussion on Chat Memory components

[TuanaCelik]

This is the discussion board for a group of experimental components introduced as a solution to implement chat memory for chat with RAG-like use-cases.

• ChatMessageWriter
• ChatMessageRetriever
• InMemoryChatMessageStore

🧑‍🍳 Try the Cookbook here

Let us know how you get along with these components, and please do let us know if you have any feedback.

Note: Experimental features live in this repository for a fixed period of time. We don't guarantee that we will continue maintaining experimental features. But if they are successful, stable and if you like it, we will move the feature to the core Haystack package.

PS: The contributor for the first version of these experimental components is @vblagoje 🚀

[shadeMe]

From @anakin87

While preparing a notebook on Tools, I needed some form of short-term memory of the conversation.
I tried to use ChatMessageStore, ChatMessageWriter, andChatMessageRetriever.

Unfortunately, I have encountered some roadblocks with the ChatMessageRetriever.
You can find my experiments in this notebook.

---

Main problems I have encountered:
ChatMessageRetriever has no input sockets

• cannot receive any input from other components
• cannot be placed in a conditional branch
• must be placed at the beginning of a Pipeline
• since it does not need inputs, it always runs

I am aware of the ongoing work on the ChatMessageRetriever (see deepset-ai/haystack#8258), but my impression is that most of these issues will remain.

Given that my use case is not very exotic, I think it would be beneficial to reflect on how to improve and make this experimental component more flexible.

[kkmarv]

I want to leave my feedback as well and say that I greatly appreciate the Chat Memory components and think they are very useful. In my case I've used them for bootstrapping small examples I can show to colleagues.

However, I just want to point out a small inconsistency in the cookbook. In Build The Pipeline it says that a ListJoiner component will come to Haystack in version 2.8.0 which has just released. This is confusing since version 2.8.0 did not implement such a component. I would be glad to use it though!

[lambda-science]

About the InMemoryChatMessageStore

• To solve multiple call leading to confusion of conversation: the self.messages should probably be a dictionnary with a ThreadID/ChatID as key and list of ChatMessage as value. It could be auto-generated if the list of ChatMessage was empty or given by the user.

We should also have a way when running the pipeline to inject ChatMessages in the InMemoryChatMessageStore.
For example I build an API based on Haystack, my /chat endpoint can take a query and a chat_history as parameter that is a list of ChatMessages. This is done to let the developper adjust himself how many messages from history he wants to use instead of letting the pipeline define a top_k. This also allow to modify the history or start again a conversation even after the InMemoryChatMessageStore has been cleared.
To be more clear: not only the pipeline should be able to create and manage history. We should have input surface to put our own history of messages.

[lambda-science]

Also I believe that in CookBooks we shouldn't showcase the usage of Memory/History by feeding the history in a single big prompt. The proper way is to feed a list of ChatMessage in chronological order directly to the ChatGenerator.

[mariosconsta]

I am attaching my feature request here so the devs can keep track of it: Session Based chat message store.

I would like to make an addition as well other than the session idea:

-- Being able to load a conversation through a dictionary. Open WebUI's pipelines provide the conversation in the form of a dictionary. So if you want to resume a conversation that is not saved on memory, you can do so by using the dictionary.