Skip to content

Commit 7d758fe

Browse files
authored
Add Needle integration (#262)
* Add Needle integration * Fix broken link
1 parent 7823bc3 commit 7d758fe

File tree

2 files changed

+101
-0
lines changed

2 files changed

+101
-0
lines changed

integrations/needle.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
layout: integration
3+
name: Needle
4+
description: Use Needle document store and retriever in Haystack.
5+
authors:
6+
- name: Needle Team
7+
socials:
8+
twitter: needlexAI
9+
linkedin: needlexai
10+
pypi: https://pypi.org/project/needle-haystack-ai
11+
repo: https://github.com/JANHMS/needle-haystack
12+
type: Document Store
13+
report_issue: https://github.com/JANHMS/needle-haystack/issues
14+
logo: /logos/needle.png
15+
version: Haystack 2.x
16+
---
17+
18+
# Needle RAG tools for Haystack
19+
20+
[![PyPI - Version](https://img.shields.io/pypi/v/needle-haystack-ai.svg)](https://pypi.org/project/needle-haystack-ai)
21+
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/needle-haystack-ai.svg)](https://pypi.org/project/needle-haystack-ai)
22+
23+
This package provides `NeedleDocumentStore` and `NeedleEmbeddingRetriever` component for use in Haystack projects.
24+
25+
## Usage ⚡️
26+
27+
Get started by installing the package via `pip`.
28+
29+
```bash
30+
pip install needle-haystack-ai
31+
```
32+
33+
### API Keys
34+
35+
We will show you building a common RAG pipeline using Needle tools and OpenAI generator.
36+
For using these tools you must set your environment variables, `NEEDLE_API_KEY` and `OPENAI_API_KEY` respectively.
37+
38+
You can get your Needle API key from from [Developer settings](https://needle-ai.com/dashboard/settings).
39+
40+
### Example Pipeline 🧱
41+
42+
In Needle document stores are called collections. For detailed information, see our [docs](https://docs.needle-ai.com).
43+
You can create a reference to your Needle collection using `NeedleDocumentStore` and use `NeedleEmbeddingRetriever` to retrieve documents from it.
44+
45+
```python
46+
from needle_haystack import NeedleDocumentStore, NeedleEmbeddingRetriever
47+
48+
document_store = NeedleDocumentStore(collection_id="<your-collection-id>")
49+
retriever = NeedleEmbeddingRetriever(document_store=document_store)
50+
```
51+
52+
Use the retriever in a Haystack pipeline. Example:
53+
54+
```python
55+
from haystack import Pipeline
56+
from haystack.components.generators import OpenAIGenerator
57+
from haystack.components.builders import PromptBuilder
58+
59+
prompt_template = """
60+
Given the following retrieved documents, generate a concise and informative answer to the query:
61+
62+
Query: {{query}}
63+
Documents:
64+
{% for doc in documents %}
65+
{{ doc.content }}
66+
{% endfor %}
67+
68+
Answer:
69+
"""
70+
71+
prompt_builder = PromptBuilder(template=prompt_template)
72+
llm = OpenAIGenerator()
73+
74+
# Add components to pipeline
75+
pipeline = Pipeline()
76+
pipeline.add_component("retriever", retriever)
77+
pipeline.add_component("prompt_builder", prompt_builder)
78+
pipeline.add_component("llm", llm)
79+
80+
# Connect the components
81+
pipeline.connect("retriever", "prompt_builder.documents")
82+
pipeline.connect("prompt_builder", "llm")
83+
```
84+
85+
Run your RAG pipeline:
86+
87+
```python
88+
prompt = "What is the topic of the news?"
89+
90+
result = basic_rag_pipeline.run({
91+
"retriever": {"text": prompt},
92+
"prompt_builder": {"query": prompt}
93+
})
94+
95+
# Print final answer
96+
print(result['llm']['replies'][0])
97+
```
98+
99+
# Support 📞
100+
101+
For detailed guides, take a look at our [docs](https://docs.needle-ai.com). If you have questions or requests you can contact us in our [Discord channel](https://discord.gg/JzJcHgTyZx).

logos/needle.png

50.7 KB
Loading

0 commit comments

Comments
 (0)