Skip to content

Commit a57ab79

Browse files
airibarneairibarne
authored
Add integration with DeepL translation services (#246)
* 📝 deepl-haystack integration * 📝 deepl-haystack Fix ToC * 📝 deepl-haystack Fix license sentence * 📝 Update company name * 📝 Improve documentation: recommend use of env vars * 📝 Pipeline example with simpler plain text converter
1 parent 727f9b8 commit a57ab79

File tree

2 files changed

+138
-0
lines changed

2 files changed

+138
-0
lines changed

‎integrations/deepl.md

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
layout: integration
3+
name: DeepL
4+
description: Use DeepL translation services with Haystack
5+
authors:
6+
- name: Dribia Data Research
7+
socials:
8+
github: dribia
9+
linkedin: https://www.linkedin.com/company/dribia
10+
pypi: https://pypi.org/project/deepl-haystack/
11+
repo: https://github.com/dribia/deepl-haystack
12+
type: Custom Component
13+
report_issue: https://github.com/dribia/deepl-haystack/issues
14+
logo: /logos/deepl.png
15+
version: Haystack 2.0
16+
toc: true
17+
---
18+
### **Table of Contents**
19+
- [Overview](#overview)
20+
- [Installation](#installation)
21+
- [Usage](#usage)
22+
- [Components](#components)
23+
- [Examples](#examples)
24+
- [Standalone](#standalone)
25+
- [Pipeline](#pipeline)
26+
- [License](#license)
27+
28+
## Overview
29+
30+
[DeepL](https://www.deepl.com/) is a powerful translation services provider, offering high-quality translations
31+
in multiple languages. This integration allows you to use DeepL's translation services with Haystack.
32+
33+
## Installation
34+
35+
```console
36+
pip install deepl-haystack
37+
```
38+
39+
## Usage
40+
41+
### Components
42+
43+
The DeepL Haystack integration introduces two components that can be used to
44+
obtain translations using the [DeepL API](https://www.deepl.com/en/pro-api).
45+
46+
- The `DeepLTextTranslator` to translate plain text (Python strings).
47+
- The `DeepLDocumentTranslator` to translate Haystack `Document` objects.
48+
49+
### API Key
50+
51+
To use the DeepL Haystack integration, you'll need to provide a DeepL API key.
52+
You can get one by signing up at the [DeepL API website](https://www.deepl.com/en/pro#developer).
53+
54+
Once obtained, **make sure to export it as an environment variable named `DEEPL_API_KEY`**
55+
in you working environment before running the examples below. Both the `DeepLTextTranslator`
56+
and the `DeepLDocumentTranslator` component constructors will expect this variable to be set.
57+
58+
An alternative way to provide the API key, although not recommended, would be to pass it through the
59+
`api_key` parameter of the components' constructor, using the Haystack
60+
[Secret](https://docs.haystack.deepset.ai/reference/utils-api#secret) utility.
61+
62+
## Examples
63+
64+
### Standalone
65+
66+
The following example shows how to translate a simple text:
67+
68+
```python
69+
from deepl_haystack import DeepLTextTranslator
70+
71+
translator = DeepLTextTranslator(source_lang="EN", target_lang="ES")
72+
73+
translated_text = translator.run("Hello, world!")
74+
print(translated_text)
75+
# {'translation': '¡Hola, mundo!', 'meta': {'source_lang': 'EN', 'target_lang': 'ES'}}
76+
```
77+
78+
Here, instead, we show how to translate a list of `Document` objects:
79+
80+
```python
81+
from haystack.dataclasses import Document
82+
83+
from deepl_haystack import DeepLDocumentTranslator
84+
85+
translator = DeepLDocumentTranslator(source_lang="EN", target_lang="ES")
86+
87+
documents_to_translate = [
88+
Document(content="Hello, world!"),
89+
Document(content="Goodbye, Joe!", meta={"name": "Joe"}),
90+
]
91+
92+
translated_documents = translator.run(documents_to_translate)
93+
print(
94+
"\n".join(
95+
[f"{doc.content}, {doc.meta}" for doc in translated_documents["documents"]]
96+
)
97+
)
98+
# ¡Hola, mundo!, {'source_lang': 'EN', 'target_lang': 'ES'}
99+
# ¡Adiós, Joe!, {'name': 'Joe', 'source_lang': 'EN', 'target_lang': 'ES'}
100+
```
101+
102+
### Pipeline
103+
104+
To use the DeepL components in a Haystack pipeline,
105+
you can use them as any other Haystack component.
106+
107+
```python
108+
from haystack import Pipeline
109+
from haystack.components.converters import TextFileToDocument
110+
from haystack.components.writers import DocumentWriter
111+
from haystack.dataclasses.byte_stream import ByteStream
112+
from haystack.document_stores.in_memory import InMemoryDocumentStore
113+
114+
from deepl_haystack import DeepLDocumentTranslator
115+
116+
document_store = InMemoryDocumentStore()
117+
118+
pipeline = Pipeline()
119+
pipeline.add_component(instance=TextFileToDocument(), name="converter")
120+
pipeline.add_component(
121+
instance=DeepLDocumentTranslator(target_lang="ES"),
122+
name="translator",
123+
)
124+
pipeline.add_component(
125+
instance=DocumentWriter(document_store=document_store), name="document_store"
126+
)
127+
pipeline.connect("converter", "translator")
128+
pipeline.connect("translator", "document_store")
129+
pipeline.run({"converter": {"sources": [ByteStream.from_string("Hello world!")]}})
130+
print(document_store.filter_documents())
131+
# [Document(id=..., content: '¡Hola, mundo!', meta: {'source_lang': 'EN', 'language': 'ES'})]
132+
```
133+
134+
### License
135+
136+
`deepl-haystack` is distributed under the terms of the
137+
[MIT](https://opensource.org/license/mit) license.

‎logos/deepl.svg

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)