Skip to content

feat: support pdf files in Prompt #452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
konrad-czarnota-ds opened this issue Mar 25, 2025 · 0 comments
Open

feat: support pdf files in Prompt #452

konrad-czarnota-ds opened this issue Mar 25, 2025 · 0 comments
Labels
feature New feature or request
Milestone

Comments

@konrad-czarnota-ds
Copy link
Collaborator

Feature description

I would like to be able to attach .pdf files to prompts in ragbits and use them when chatting with LLM. Here's a quick and dirty solution that I did to make it work that needs to be polished:

import base64
from pydantic import BaseModel
from ragbits.core.prompt import Prompt


class InstructionExtractionPromptInput(BaseModel):
    pdf_path: str


class InstructionExtractionPromptOutput(BaseModel):
    instructions: str


class InstructionExtractionPrompt(Prompt[InstructionExtractionPromptInput, InstructionExtractionPromptOutput]):
    user_prompt = """
    """

    image_input_fields = ["pdf_path"]

    @staticmethod
    def _create_message_with_image(image: str | bytes) -> dict:
        if type(image) == str and image.endswith(".pdf"):
            with open(image, "rb") as f:
                data = f.read()

            base64_string = base64.b64encode(data).decode("utf-8")

            return{
                        "type": "file",
                        "file": {
                            "filename": "instruction.pdf",
                            "file_data": f"data:application/pdf;base64,{base64_string}",
                        }
                    }
        else:
            return super()._create_message_with_image(image)

Motivation

LiteLLM supports files passing through API, I think ragbits should too as this might be necessary to handle different documents

Additional context

No response

@konrad-czarnota-ds konrad-czarnota-ds added the feature New feature or request label Mar 25, 2025
@konrad-czarnota-ds konrad-czarnota-ds moved this to Backlog in ragbits Mar 25, 2025
@mhordynski mhordynski added this to the Ragbits 1.1.0 milestone May 7, 2025
@mhordynski mhordynski moved this from Backlog to Ready in ragbits May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: Ready
Development

No branches or pull requests

2 participants