Bugfix/chat images 2 #4630
Bugfix/chat images 2 #4630
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
This PR focuses on improving binary data handling in file sources, with several key changes:
- Changed error handling in
create_search_doc_from_user_filefrom 'replace' to 'strict' mode for better binary content detection inchat.py - Renamed and refactored file loading functions in
utils.pyfor clarity:load_all_user_files→load_in_memory_chat_filesandload_all_user_file_files→get_user_files - Added optimizations in
process_message.pyfor handling large user files with a new 'fast path' for small files - Moved
RECENT_DOCS_FOLDER_IDconstant fromuser_documents/api.pytochat_backend.pyfor better code organization
The changes improve error handling for binary files while making the codebase more maintainable through better function naming and documentation.
6 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile
| # 1. Load files specified by individual IDs | ||
| [(load_user_file, (file_id, db_session)) for file_id in user_file_ids] |
There was a problem hiding this comment.
logic: run_functions_tuples_in_parallel is called with a list comprehension but the + operator is outside the cast, which could cause type issues if the second part returns an unexpected type
| user_files.extend( | ||
| db_session.query(UserFile) | ||
| .filter(UserFile.folder_id == user_folder_id) | ||
| .all() | ||
| ) |
There was a problem hiding this comment.
style: consider using a single query with an IN clause instead of multiple queries in a loop for better performance
| user_files.extend( | |
| db_session.query(UserFile) | |
| .filter(UserFile.folder_id == user_folder_id) | |
| .all() | |
| ) | |
| user_files.extend( | |
| db_session.query(UserFile) | |
| .filter(UserFile.folder_id.in_(user_folder_ids)) | |
| .all() | |
| ) |
* don't hardcode -1 * extra spaces * fix binary data in blurb * add note to binary handling --------- Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
3 participants
Description
Fixes https://linear.app/danswer/issue/DAN-1923/fix-binary-data-in-filesourcecard
How Has This Been Tested?
[Describe the tests you ran to verify your changes]
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.