Skip to content

small improvement to checking for image attachments #4543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 17, 2025
Merged

small improvement to checking for image attachments #4543

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c306f86
small improvement to checking for image attachments
Apr 16, 2025
4b71bf1
better comments
Apr 16, 2025
038baa6
check centralized list of types instead of hardcoding them in the con…
Apr 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions backend/onyx/connectors/confluence/connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -373,11 +373,23 @@ def _fetch_page_attachments(
cql=attachment_query,
expand=",".join(_ATTACHMENT_EXPANSION_FIELDS),
):
attachment["metadata"].get("mediaType", "")
media_type: str = attachment.get("metadata", {}).get("mediaType", "")

# TODO(rkuo): this check is partially redundant with validate_attachment_filetype
# but doing the check here avoids an unnecessary download. Due for refactoring.
if not self.allow_images:
if media_type.startswith("image/"):
logger.info(
f"Skipping attachment because allow images is False: {attachment['title']}"
)
continue

if not validate_attachment_filetype(
attachment,
):
logger.info(f"Skipping attachment: {attachment['title']}")
logger.info(
f"Skipping attachment because it is not an accepted file type: {attachment['title']}"
)
continue

logger.info(f"Processing attachment: {attachment['title']}")
Expand Down
5 changes: 3 additions & 2 deletions backend/onyx/connectors/confluence/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,16 @@ def validate_attachment_filetype(
"""
Validates if the attachment is a supported file type.
"""
attachment.get("metadata", {})
media_type = attachment.get("metadata", {}).get("mediaType", "")

if media_type.startswith("image/"):
return is_valid_image_type(media_type)

# For non-image files, check if we support the extension
title = attachment.get("title", "")
extension = Path(title).suffix.lstrip(".").lower() if "." in title else ""

# NOTE(rkuo): Why doesn't this check
# ACCEPTED_PLAIN_TEXT_FILE_EXTENSIONS or ACCEPTED_DOCUMENT_FILE_EXTENSIONS?
return extension in ["pdf", "doc", "docx", "txt", "md", "rtf"]


Expand Down
Loading