Skip to content

Agent\Functions Overview

Jump to bottom
Emir Sahin edited this page Aug 26, 2024 · 1 revision

Agents Module Overview

Agents.py defines various types of agent classes that leverage different functionalities for interacting with language models, handling different types of tasks, from text and image processing to executing code and handling internet-based tasks.

Class: Agent

Description:
A basic agent capable of using predefined or custom system prompts. Custom system prompts, if provided, will override any predefined prompts.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • agent_type (AgentType, optional): Specifies the type of agent. Defaults to None.
  • additional_system_instructions (str, optional): Additional system instructions to be included in the premade system prompt. Defaults to an empty string.
  • custom_system_prompt (str, optional): Custom system prompt to override the predefined one. Defaults to None.
  • format (str, optional): Response format, e.g., "json". Defaults to an empty string.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • get_prompt: Generates the appropriate prompt for a given question.
  • ask: Submits a prompt to the LLM and returns the response, optionally incorporating images and conversation history.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
agent = Agent(llm, AgentType.SUMMARIZER)
response = agent.ask("Summarize the key points of the latest financial report.")
print(response)

Class: ObjectDetectorAgent

Description:
An agent specialized in detecting objects within images, utilizing a multimodal LLM for vision tasks and another LLM for textual responses.

Constructor Parameters:

  • vision_llm (object): A multimodal LLM object capable of processing images.
  • text_llm (object): An LLM used for generating textual output.
  • vision_temperature (float, optional): Temperature for vision LLM responses. Defaults to 0.3.
  • text_temperature (float, optional): Temperature for text LLM responses. Defaults to 0.3.

Methods:

  • detect: Detects objects in provided images according to specific criteria or a predefined list of objects.

Example Usage:

vision_llm = load_vision_llm()  # Replace with your LLM loading method
text_llm = load_text_llm()  # Replace with your LLM loading method
detector = ObjectDetectorAgent(vision_llm, text_llm)
detected_items = detector.detect(["path/to/image.jpg"], objects=["cat", "dog"])
print(detected_items)

Class: PythonAgent

Description:
This agent is designed to handle code generation tasks and should always operate within a safe, controlled environment due to the risks associated with executing arbitrary code.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • ask: Generates Python code based on the provided prompt, returning the results.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
python_agent = PythonAgent(llm)
code_output = python_agent.ask("Write a Python function to calculate Fibonacci numbers.")
print(code_output)

Class: DataExtractor

Description:
Extracts specific data points from provided content. This agent is useful for structured data extraction tasks from text.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • reply_as_json (bool, optional): Whether to format the response as JSON. Defaults to False.
  • additional_system_instructions (str, optional): Additional instructions to include in the system prompt. Defaults to an empty string.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • ask: Extracts specified data points from the given content and returns them.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
extractor = DataExtractor(llm)
data = extractor.ask("Extract names and dates from the following meeting notes.", ["name", "date"])
print(data)

Class: PdfReader

Description:
Answers questions based on the content extracted from PDF documents. This agent is designed to interact with PDF files, making it ideal for document review and information retrieval tasks.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • ask: Reads content from PDF files and answers questions based on this content.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
pdf_reader = PdfReader(llm)
answer = pdf_reader.ask("What are the main topics discussed in this document?", ["path/to/document.pdf"])
print(answer)

Class: FunctionCaller

Description:
Calls functions dynamically based on user prompts. This agent uses an LLM to determine which function to call and with what parameters, based on the user's input.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • functions (list): A list of functions that the agent can call.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • get_function: Determines the appropriate function to call based on the prompt.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
function_caller = FunctionCaller(llm, [your_function_list])
result = function_caller.get_function("Calculate the area of a circle with radius 7.")
print(result)

Class: WebsiteReaderAgent

Description:
Reads content from specified URLs and answers questions based on this content. This agent is designed to fetch and interpret web content dynamically.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • ask: Retrieves content from a URL and uses it to answer a provided question.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
website_reader = WebsiteReaderAgent(llm)
response = website_reader.ask("Summarize the latest article on the provided URL.", "https://example.com/latest-article")
print(response)

Class: OnlineAgent

Description:
Utilizes internet access to search for information and answer prompts based on the results. This agent leverages external search capabilities to provide informed responses.

Constructor Parameters:

  • llm (object): An LLM object with an ask function.
  • temperature (float, optional): Controls the randomness of the LLM's responses. Defaults to 0.8.

Methods:

  • search: Conducts an internet search based on a prompt and answers using the gathered information.

Example Usage:

llm = load_your_llm()  # Replace with your LLM loading method
online_agent = OnlineAgent(llm)
search_response = online_agent.search("What is the current stock price of Apple?")
print(search_response)
Clone this wiki locally