Skip to content

[#262] Add med rule extraction tests and model evaluations #290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 28 commits into
base: listOfMed
Choose a base branch
from
Open

[#262] Add med rule extraction tests and model evaluations #290

wants to merge 28 commits into from

Conversation

sahilds1
Copy link
Collaborator

@sahilds1 sahilds1 commented May 23, 2025 •
edited
Loading

This PR reopens med rule extraction tests and model evaluations as part of #262 after #277

sahilds1 added 28 commits May 23, 2025 16:42
@sahilds1
Add med rule extraction tests and model evaluations
358b07f
@sahilds1
Add CLI support for eval script with input and output file arguments
5ce6eac
@sahilds1
Include token usage and cost estimation in Claude Haiku evals
6ac7c57
@sahilds1
Add logging support
c0e164e
@sahilds1
Replaced single input CSV with separate config and reference CSVs
0b41e09
@sahilds1
Updated main execution logic to branch evaluation based on model
86966d0
@sahilds1
Renamed evaluation/evals.py
6f46119
@sahilds1
Refactor evaluation to support modular model-specific handlers
3a3ebc5
@sahilds1
Add TODOs to the evaluation script
87226a5
@sahilds1
Add support for GPT-4.1 Nano and Claude Haiku 3 models in evaluation
2135771
@sahilds1
Fix supported_models is not defined
f2b321b
@sahilds1
Refactor model invocation logic with ModelFactory and handler classes
60c9179
@sahilds1
Replace ROUGE and BERTScore with Extractiveness metric for response e…
29f9835 
…valuation
@sahilds1
Add TODOs to evals.py and services.py
6a3624b
@sahilds1
Add docstrings and type hints for clarity
ad7e313
@sahilds1
Reformat using ruff
78a2fd6
@sahilds1
Reformat using ruff
467eed6
@sahilds1
Fix mypy errors
adf7788
@sahilds1
Refactor model handlers for type safety and readability
1980105
@sahilds1
Refactor evals.py for clarity, formatting, and maintainability
9fd525a
@sahilds1
Add LLM Output Evaluator evals.py README
a2a9b1a
@sahilds1
Add server/api/services/test_evals.py
4734298
@sahilds1
Merge branch 'listOfMed' into 262-extract-meds-rules
e00ade8
@sahilds1
Update the README
228fad2
@sahilds1
Check if the required columns are present
e41e2d2
@sahilds1
Updated the README
05f1f47
@sahilds1
Move and rename evaluation-related files
f9cb604
@sahilds1
Resolve merge conflicts
6a6eb7a
@sahilds1 sahilds1 requested a review from taichan03 June 17, 2025 22:38
@sahilds1 sahilds1 changed the title [WIP] [#262] Add med rule extraction tests and model evaluations [#262] Add med rule extraction tests and model evaluations Jun 17, 2025
@sahilds1 sahilds1 self-assigned this Jun 17, 2025
@sahilds1 sahilds1 marked this pull request as ready for review June 17, 2025 22:38
@sahilds1 sahilds1 requested a review from ricanontherun June 17, 2025 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@taichan03 taichan03 Awaiting requested review from taichan03

@ricanontherun ricanontherun Awaiting requested review from ricanontherun

At least 1 approving review is required to merge this pull request.

Assignees

@sahilds1 sahilds1

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sahilds1