Skip to content

Commit 304c2e9

Browse files
[Benchmark] Add VLRMBench Support (#1259)
* add vlrmbench dataset * Translate comments to English * Fix code formatting issues * Reconstructed the dataset and the evaluation code of vlrmbench * Refactor and translate comments in VLRMBench dataset code to English * Consolidate dataset imports in __init__.py * Add multi_solution evaluation support in VLRMBench dataset * Add foresight evaluation support in VLRMBench dataset Updated initialization to warn about supported task types. Fix and update the md5 --------- Co-authored-by: Xinyu Fang <fangxinyutju202009@126.com>
1 parent 55fba74 commit 304c2e9

File tree

2 files changed

+405
-1
lines changed

2 files changed

+405
-1
lines changed

vlmeval/dataset/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@
8080
from .mmifeval import MMIFEval
8181
from .chartmimic import ChartMimic
8282
from .m4bench import M4Bench
83+
from .vlrmbench import VLRMBench
8384
from .mmhelix import MMHELIX
8485
from .medqbench_mcq import MedqbenchMCQDataset
8586
from .medqbench_caption import MedqbenchCaptionDataset
@@ -218,7 +219,7 @@ def evaluate(self, eval_file, **judge_kwargs):
218219
OmniEarthMCQBench, VisFactor, OSTDataset, OCRBench_v2, TreeBench, CVQA, M4Bench,
219220
AyaVisionBench, TopViewRS, VLMBias, MMHELIX, MedqbenchMCQDataset,
220221
MedqbenchPairedDescriptionDataset, MedqbenchCaptionDataset, ChartMuseum, ChartQAPro, ReasonMap_Plus,
221-
olmOCRBench, OceanOCRBench, MATBench
222+
olmOCRBench, OceanOCRBench, MATBench, VLRMBench
222223
]
223224

224225
VIDEO_DATASET = [

0 commit comments

Comments
 (0)