Skip to content

dataset: Add hebrew v3 #3607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

dataset: Add hebrew v3 #3607

wants to merge 1 commit into from
+100 −0

Conversation

@Samoed
Copy link
Member

@Samoed Samoed commented Nov 23, 2025

Close #3578

@Samoed Samoed added the new dataset Issues related to adding a new task or dataset label Nov 24, 2025
KennethEnevoldsen
KennethEnevoldsen reviewed Nov 25, 2025
View reviewed changes
Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just run a model on both datasets to compare

I am kinda unsure how about the differences in the data

mteb/tasks/classification/heb/hebrew_sentiment_analysis.py
Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add superseeded_by

Copy link
Member Author

@Samoed Samoed Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V1 is already superseeded_by v2

Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v2 needs a superseeded_by v3

Copy link
Member Author

@Samoed Samoed Nov 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we should clenup it as in #2900, but I can't find script for this

Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it this one:
https://gist.github.com/AlexeyVatolin/ea3adc21aa7a767603ff393b22085adc

Copy link
Member Author

@Samoed Samoed Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. Try to run this too

@Samoed
Copy link
Member Author

Samoed commented Nov 25, 2025

Results for sentence-transformers/LaBSE

Results
HebrewSentimentAnalysis 0.616357
HebrewSentimentAnalysis.v2 0.68636
HebrewSentimentAnalysis.v3 0.629297

@KennethEnevoldsen
Copy link
Contributor

KennethEnevoldsen commented Nov 25, 2025

hmm would have expected it would have gotten better

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KennethEnevoldsen KennethEnevoldsen KennethEnevoldsen left review comments

Assignees

No one assigned

Labels

new dataset Issues related to adding a new task or dataset

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add v2 of HebrewSentimentAnalysis

3 participants

@Samoed @KennethEnevoldsen