Semantic Search Engine - Multi Language

[sureshvytla]

Dear Team,

We are working to fulfill the search use case using Haystack. Please review and guide us on the correct path.

Use Case
We have conversations and would like to search for conversations based on natural language queries.

Approach

• Document Store: Elastic Search (As per Haystack recommendation for prod)
• Retriever: EmbeddingRetriever using model sentence-transformers/multi-qa-mpnet-base-dot-v1 (also looked into DensePassageRetriever)

For scalability, we have split the pipelines

• Index Pipeline
• Search Pipeline

Index Pipeline

Consume messages from messaging bus and add documents, update embeddings

Search Pipeline

API takes a query and returns the results

Please review the pipeline and suggest the right set of nodes and models to improve the search results. Also, looking to support other than the English language.

Thanks,
Suresh

[anakin87]

Hello @sureshvytla,
for a use case like the one you described, your choices make sense to me.

Are you encountering any problems?

To deal with documents of different languages, you can take a look at multilingual models, which you can use in the EmbeddingRetriever.
Alternatively, you can also use the DocumentLanguageClassifier node and then use the detected language to filter the results.

[sureshvytla]

Hi @anakin87,

Thank you so much for your input, I will have a look at the multilingual models.

I'm concerned about the results from the pipeline, for example:
I'm searching from the conversations, is there a callback promised? The results include documents that don't have any context related to callback.

I thought of reviewing the approach for the semantic search pipeline.

• Does this approach best fit for the Semantic Search use case?
• Does the pipeline include all the required nodes?
• Is there a need to leverage Ranker to improve the results? I see an example using Ranker to improve search results. If needed what should be the model for the Ranker?
• Does the document store elastic search initialized properly with relevant settings? Similarity cosine and embedding_dim 768 etc.
• Does the document content is appropriate? The document content is in the form of conversation "participant-1: xxx \n participant-2: zzz \n ..."

Thanks,
Suresh

[anakin87]

What do you mean by this?

I'm searching from the conversations, is there a callback promised? The results include documents that don't have any context related to callback.

Ranker
The Ranker better orders the documents using a cross-encoder at the cost of some additional computation time.
It can make sense for your use case.
Here you can find some cross-encoders models. (It seems that there are no multilingual ones.)

Document Store configurations
You should choose the most appropriate similarity metric for your model. Usually, you can find it in Sentence Transformers docs.
For example, sentence-transformers/multi-qa-mpnet-base-dot-v1 supports dot_product.

[sureshvytla]

Hi @anakin87,

I'm searching from the conversations, is there a callback promised? The results include documents that don't have any context related to callback

The document's content is like a conversation between two persons like "participant-1: xxx \n participant-2: zzz \n ...", looking to pull the documents based on a particular query.

Again, thanks for the pointers. I've used the retriever model sentence-transformers/all-MiniLM-L6-v2 with cosine similarity and the results are better.

I will check sentence-transformers/multi-qa-mpnet-base-dot-v1 with dot_product and verify the results.