Skip to content

Commit fd84082

Browse files
committed
fix: return scores 1 or 0 for llm answer match if it passes/fails, not only 0 always
1 parent 9eaee7c commit fd84082

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

evaluators/langevals/langevals_langevals/llm_answer_match.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,8 +103,11 @@ def evaluate(self, entry: LLMAnswerMatchEntry) -> SingleEvaluationResult:
103103
else:
104104
raise e
105105

106+
passed = "true" in str(result.is_correct).lower()
107+
106108
return LLMAnswerMatchResult(
107-
passed="true" in str(result.is_correct).lower(),
109+
passed=passed,
110+
score=1 if passed else 0,
108111
details=result.reasoning,
109112
cost=Money(amount=cost, currency="USD") if cost is not None else None,
110113
)

0 commit comments

Comments
 (0)