Skip to content

Commit c0428e9

Browse files
authored
integrated monsterapi to haystack (#244)
* integrated monsterapi to haystack * Added draft.md * Updated MonsterAPI Integration * Added draft-intefration.md * Removed draft-integration.md * Added MonsterAPI logo * Updated monsterapi.md * Updated monsterapi.md * Updated monsterapi.md * Update monsterapi.md
1 parent df58b0d commit c0428e9

File tree

2 files changed

+107
-0
lines changed

2 files changed

+107
-0
lines changed

integrations/monsterapi.md

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
---
2+
layout: integration
3+
name: MonsterAPI
4+
description: Use open Language Models served by MonsterAPI
5+
authors:
6+
- name: monsterapi
7+
socials:
8+
github: qblocks
9+
twitter: monsterapi
10+
linkedin: https://www.linkedin.com/company/monster-api/
11+
pypi: https://pypi.org/project/haystack-ai
12+
repo: https://github.com/deepset-ai/haystack
13+
type: Model Provider
14+
report_issue: https://github.com/deepset-ai/haystack/issues
15+
logo: /logos/monsterapi.png
16+
version: Haystack 2.0
17+
toc: true
18+
---
19+
20+
### **Table of Contents**
21+
22+
- [Overview](#overview)
23+
- [Usage](#usage)
24+
25+
## Overview
26+
27+
MonsterAPI provides access to powerful language models designed for various text generation tasks. With the MonsterAPI integration, you can leverage these models within the Haystack framework for enhanced natural language processing capabilities.
28+
29+
To start using MonsterAPI, sign up for an API key [here](https://monsterapi.ai/). This key provides access to the MonsterAPI, which supports rapid inference and customization through various parameters.
30+
31+
## Usage
32+
33+
MonsterAPI's API is OpenAI compatible, making it easy to use within Haystack via OpenAI Generators.
34+
35+
### Using `Generator`
36+
37+
Here's an example of using a model served via MonsterAPI to perform question answering on a web page. You need to set the environment variable `MONSTER_API_KEY` and choose a [compatible model](https://developer.monsterapi.ai/).
38+
39+
```python
40+
from haystack import Pipeline
41+
from haystack.utils import Secret
42+
from haystack.components.fetchers import LinkContentFetcher
43+
from haystack.components.converters import HTMLToDocument
44+
from haystack.components.builders import PromptBuilder
45+
from haystack.components.generators import OpenAIGenerator
46+
47+
fetcher = LinkContentFetcher()
48+
converter = HTMLToDocument()
49+
prompt_template = """
50+
According to the contents of this website:
51+
{% for document in documents %}
52+
{{document.content}}
53+
{% endfor %}
54+
Answer the given question: {{query}}
55+
Answer:
56+
"""
57+
prompt_builder = PromptBuilder(template=prompt_template)
58+
llm = OpenAIGenerator(
59+
api_key=Secret.from_env_var("MONSTER_API_KEY"),
60+
api_base_url="https://llm.monsterapi.ai/v1/",
61+
model="microsoft/Phi-3-mini-4k-instruct",
62+
generation_kwargs = {"max_tokens": 256}
63+
)
64+
pipeline = Pipeline()
65+
pipeline.add_component("fetcher", fetcher)
66+
pipeline.add_component("converter", converter)
67+
pipeline.add_component("prompt", prompt_builder)
68+
pipeline.add_component("llm", llm)
69+
70+
pipeline.connect("fetcher.streams", "converter.sources")
71+
pipeline.connect("converter.documents", "prompt.documents")
72+
pipeline.connect("prompt.prompt", "llm.prompt")
73+
74+
result = pipeline.run({"fetcher": {"urls": ["https://developer.monsterapi.ai/docs/"]},
75+
"prompt": {"query": "What are the features of MonsterAPI?"}})
76+
77+
print(result["llm"]["replies"][0])
78+
```
79+
80+
### Using `ChatGenerator`
81+
82+
Here's an example of engaging in a multi-turn conversation with a MonsterAPI model. You need to set the environment variable `MONSTER_API_KEY` and choose a [compatible model](https://developer.monsterapi.ai/).
83+
84+
```python
85+
from haystack.components.generators.chat import OpenAIChatGenerator
86+
from haystack.dataclasses import ChatMessage
87+
from haystack.utils import Secret
88+
89+
generator = OpenAIChatGenerator(
90+
api_key=Secret.from_env_var("MONSTER_API_KEY"),
91+
api_base_url="https://llm.monsterapi.ai/v1/",
92+
model="microsoft/Phi-3-mini-4k-instruct",
93+
generation_kwargs = {"max_tokens": 256}
94+
)
95+
96+
messages = []
97+
98+
while True:
99+
msg = input("Enter your message or Q to exit\n ")
100+
if msg=="Q":
101+
break
102+
messages.append(ChatMessage.from_user(msg))
103+
response = generator.run(messages=messages)
104+
assistant_resp = response['replies'][0]
105+
print(assistant_resp.content)
106+
messages.append(assistant_resp)
107+
```

logos/monsterapi.png

4.7 KB
Loading

0 commit comments

Comments
 (0)