feat: error handling of search quality test #4722
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
Enhanced error handling and file management in search quality evaluation scripts, focusing on robustness and maintainability.
- Added file existence checks with clear error messages in
/backend/tests/regression/search_quality/generate_search_queries.pyandrun_search_eval.py - Switched to
pathlib.Pathfor more robust file operations across both scripts - Added multi-tenant support validation to prevent running in unsupported environments
- Merged
_compute_jaccard_similarityand_compute_average_missing_chunk_ratiointo single function for better efficiency - Added search quality test patterns (
eval-*,search_eval_config.yaml, JSON files) to.gitignore
💡 (3/5) Reply to the bot's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!
3 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile
evan-onyx
approved these changes
May 19, 2025
ferdinandl007
pushed a commit
to ferdinandl007/onyx
that referenced
this pull request
May 27, 2025
aronszanto
pushed a commit
to aronszanto/onyx
that referenced
this pull request
May 27, 2025
ZhipengHe
pushed a commit
to ZhipengHe/onyx
that referenced
this pull request
Jun 6, 2025
AnkitTukatek
pushed a commit
to TukaTek/onyx
that referenced
this pull request
Sep 23, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Added extra error handling to the search quality test script.
Added files to gitignore
Merged two functions which share some steps
How Has This Been Tested?
It runs
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.