Skip to content

Commit f730d28

Browse files
bug fix for ModelSpecificMetrics referenced by str name
1 parent 1e5ed92 commit f730d28

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

src/vllm_judge/judge.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,9 @@ async def evaluate(
103103
MetricNotFoundError: If metric name not found
104104
ParseError: If unable to parse model response
105105
"""
106+
if metric and isinstance(metric, str):
107+
metric: Metric = self.get_metric(metric)
108+
106109
# Handle model-specific metrics
107110
if isinstance(metric, ModelSpecificMetric):
108111
if isinstance(content, dict):
@@ -139,8 +142,6 @@ async def evaluate(
139142
metric_template_vars = {}
140143

141144
if metric:
142-
if isinstance(metric, str):
143-
metric = self.get_metric(metric)
144145
# Use metric defaults but allow overrides
145146
criteria = criteria or metric.criteria
146147
rubric = rubric or metric.rubric

0 commit comments

Comments
 (0)