How do I process the attachments correctly?

[the-s-anton]

Love this gem

I am currently working on a project that utilizes this gem as its core. Here is what I want to get:

The user creates a new message record and attaches multiple files. I did review the documentation, and it does describe the setup of the attachments. However, what I am confused about is the controller itself.

As per the documentation on scalability, it suggests setting up the ActiveJob in a certain way:

# ex. Here is what I have right now
class LLMJob < ApplicationJob
  # To understand why we're using async_job, see:
  #
  # https://rubyllm.com/guides/async
  # https://paolino.me/async-ruby-is-the-future/
  self.queue_adapter = :async_job if Rails.env.production?
end

gem "async-job-adapter-active_job", "~> 0.14.1"

Ok, now let's move to the controllers. I would like to initiate the chat using a form and attach one or more files. However, we encounter a problem here because we cannot pass the files from the form into the job due to a garbage collection issue.

Ok, the workaround (which is perfect to me) is to create another model, let's call it Asset. Asset has one Attachment (ActiveStorage). Now I can do something like this

# 1. From the controller
assets = params[:files].each { |file| create_asset }
Chat::ProcessJob.perform_later(params[:ask], with: assets)

# 2. From the job
chat.ask "summarize the docs", with: assets

The issue here is that the messages will not get the reference to the assets. However, I can't display the file within a chat, but the solution still works.

Here is another way of doing it

# From the controller
chat.ask params[:ask], with: params[:files]

In this case, we are bypassing the Asset model, but we attach everything to the message, allowing us to display the attachment in the chat.

What will be the proper setup here? How can I let accounts control the assets in the settings while also displaying them nicely in the chat? What am I missing? It seems like I'm over-engineering at this stage.

[richet]

I worked around this by using ActiveStorage direct uploads in the browser and storing the signed_ids for the uploaded files in a hidden field that is submitted with the users message. I then pass the user message and the signed_ids to the job.

If signed_ids exist then I fetch the attachments blobs and call ask with attachments.

chat.ask(user_content, with: attachments) do |chunk|
...
end

Seems to work well. Next up is to do more validation and error handling for when files are too big for the context window.

[marckohlbrugge]

The trick is to create the user message yourself, so you can add attachments, etc. You will also want to replace chat.ask with chat.complete which manually triggers the assistant response.

In your MessagesController:

def create
  # Create and persist the user message immediately
  user_message = @chat.create_user_message(message_content)

  # Attach files if present
  if attachments.present?
    user_message.attachments.attach(attachments)
  end

  ChatStreamJob.perform_later(@chat.id)
end

And in ChatStreamJob:

class ChatStreamJob < ApplicationJob
  def perform(chat_id)
    chat = Chat.find(chat_id)

    # Process the latest user message
    chat.complete do |chunk|
      # Get the assistant message record (created before streaming starts)
      assistant_message = chat.messages.last
      if chunk.content && assistant_message
        assistant_message.update!(content: assistant_message.content + chunk.content)
      end
    end
  end
end