|
| 1 | +--- |
| 2 | +layout: integration |
| 3 | +name: MonsterAPI |
| 4 | +description: Use open Language Models served by MonsterAPI |
| 5 | +authors: |
| 6 | + - name: monsterapi |
| 7 | + socials: |
| 8 | + github: qblocks |
| 9 | + twitter: monsterapi |
| 10 | + linkedin: https://www.linkedin.com/company/monster-api/ |
| 11 | +pypi: https://pypi.org/project/haystack-ai |
| 12 | +repo: https://github.com/deepset-ai/haystack |
| 13 | +type: Model Provider |
| 14 | +report_issue: https://github.com/deepset-ai/haystack/issues |
| 15 | +logo: /logos/monsterapi.png |
| 16 | +version: Haystack 2.0 |
| 17 | +toc: true |
| 18 | +--- |
| 19 | + |
| 20 | +### **Table of Contents** |
| 21 | + |
| 22 | +- [Overview](#overview) |
| 23 | +- [Usage](#usage) |
| 24 | + |
| 25 | +## Overview |
| 26 | + |
| 27 | +MonsterAPI provides access to powerful language models designed for various text generation tasks. With the MonsterAPI integration, you can leverage these models within the Haystack framework for enhanced natural language processing capabilities. |
| 28 | + |
| 29 | +To start using MonsterAPI, sign up for an API key [here](https://monsterapi.ai/). This key provides access to the MonsterAPI, which supports rapid inference and customization through various parameters. |
| 30 | + |
| 31 | +## Usage |
| 32 | + |
| 33 | +MonsterAPI's API is OpenAI compatible, making it easy to use within Haystack via OpenAI Generators. |
| 34 | + |
| 35 | +### Using `Generator` |
| 36 | + |
| 37 | +Here's an example of using a model served via MonsterAPI to perform question answering on a web page. You need to set the environment variable `MONSTER_API_KEY` and choose a [compatible model](https://developer.monsterapi.ai/). |
| 38 | + |
| 39 | +```python |
| 40 | +from haystack import Pipeline |
| 41 | +from haystack.utils import Secret |
| 42 | +from haystack.components.fetchers import LinkContentFetcher |
| 43 | +from haystack.components.converters import HTMLToDocument |
| 44 | +from haystack.components.builders import PromptBuilder |
| 45 | +from haystack.components.generators import OpenAIGenerator |
| 46 | + |
| 47 | +fetcher = LinkContentFetcher() |
| 48 | +converter = HTMLToDocument() |
| 49 | +prompt_template = """ |
| 50 | +According to the contents of this website: |
| 51 | +{% for document in documents %} |
| 52 | + {{document.content}} |
| 53 | +{% endfor %} |
| 54 | +Answer the given question: {{query}} |
| 55 | +Answer: |
| 56 | +""" |
| 57 | +prompt_builder = PromptBuilder(template=prompt_template) |
| 58 | +llm = OpenAIGenerator( |
| 59 | + api_key=Secret.from_env_var("MONSTER_API_KEY"), |
| 60 | + api_base_url="https://llm.monsterapi.ai/v1/", |
| 61 | + model="microsoft/Phi-3-mini-4k-instruct", |
| 62 | + generation_kwargs = {"max_tokens": 256} |
| 63 | +) |
| 64 | +pipeline = Pipeline() |
| 65 | +pipeline.add_component("fetcher", fetcher) |
| 66 | +pipeline.add_component("converter", converter) |
| 67 | +pipeline.add_component("prompt", prompt_builder) |
| 68 | +pipeline.add_component("llm", llm) |
| 69 | + |
| 70 | +pipeline.connect("fetcher.streams", "converter.sources") |
| 71 | +pipeline.connect("converter.documents", "prompt.documents") |
| 72 | +pipeline.connect("prompt.prompt", "llm.prompt") |
| 73 | + |
| 74 | +result = pipeline.run({"fetcher": {"urls": ["https://developer.monsterapi.ai/docs/"]}, |
| 75 | + "prompt": {"query": "What are the features of MonsterAPI?"}}) |
| 76 | + |
| 77 | +print(result["llm"]["replies"][0]) |
| 78 | +``` |
| 79 | + |
| 80 | +### Using `ChatGenerator` |
| 81 | + |
| 82 | +Here's an example of engaging in a multi-turn conversation with a MonsterAPI model. You need to set the environment variable `MONSTER_API_KEY` and choose a [compatible model](https://developer.monsterapi.ai/). |
| 83 | + |
| 84 | +```python |
| 85 | +from haystack.components.generators.chat import OpenAIChatGenerator |
| 86 | +from haystack.dataclasses import ChatMessage |
| 87 | +from haystack.utils import Secret |
| 88 | + |
| 89 | +generator = OpenAIChatGenerator( |
| 90 | + api_key=Secret.from_env_var("MONSTER_API_KEY"), |
| 91 | + api_base_url="https://llm.monsterapi.ai/v1/", |
| 92 | + model="microsoft/Phi-3-mini-4k-instruct", |
| 93 | + generation_kwargs = {"max_tokens": 256} |
| 94 | +) |
| 95 | + |
| 96 | +messages = [] |
| 97 | + |
| 98 | +while True: |
| 99 | + msg = input("Enter your message or Q to exit\n ") |
| 100 | + if msg=="Q": |
| 101 | + break |
| 102 | + messages.append(ChatMessage.from_user(msg)) |
| 103 | + response = generator.run(messages=messages) |
| 104 | + assistant_resp = response['replies'][0] |
| 105 | + print(assistant_resp.content) |
| 106 | + messages.append(assistant_resp) |
| 107 | +``` |
0 commit comments