v0.3.2

What's Changed

• Merging Ola by @Devininthelab in #558
• [Evaluation] Improve string processing order for better whitespace handling by @Ryoo72 in #554
• modify utils.py by @shuyansy in #555
• Contribute EgoLife model and evaluation pipeline for EgoPlan & Egothink by @choiszt in #560
• Fix Ola path for GPUs by @Devininthelab in #562
• add OCRBench v2 by @99Franklin in #570
• [WIP][Model] Whisper + vLLM by @kylesayrs in #545
• Add VideoChat-Flash and InternVideo2.5 by @leexinhao in #568
• Add LiveXiv benchmark [ICLR 2025] by @NimrodShabtay in #572
• [Add Dataset] K-MMBench, K-SEED, K-MMStar, K-DTCBench, K-LLaVA-W by @jujeongho0 in #575
• [Dataset] Support VMCBench (CVPR 25) by @yuhui-zh15 in #573
• update merge mlvu_dev and mlvu_test by @shuyansy in #582
• add mmau task by @pbcong in #585
• [Feat] Support VideoLLaMA3 by @CircleRadon in #588
• Add WorldSense by @Devininthelab in #589
• [Add] Task "Multimodal RewardBench" by @seungyeonlj in #591
• [Task] adding MME-COT by @Luodian in #593
• Add README.md for MME-CoT by @CaraJ7 in #601
• [Tasks] New tasks for Visual Reasoning Collection by @Luodian in #600
• [Enhancement] Add LLM evaluation metric and integrate GPT-4o reasoning by @Luodian in #604
• [Feat] fix MME COT, add llm as judge eval by @Luodian in #605
• Fix hard-coded max_new_tokens for qwen2_5_vl model by @robinhad in #609
• Add Omni Bench by @ngquangtrung57 in #613
• [Feat] Fix MEGA-Bench evaluator, update doc by @woodfrog in #606
• [Feat] Adding libri long by @kcz358 in #618
• Update a new model Qwen-2.5-Omni by @Devininthelab in #615
• Modify the openai api to support o1 and o3 by @wenhuchen in #614
• [Model] support VoRA model by @sty-yyj in #616

New Contributors

• @Devininthelab made their first contribution in #558
• @Ryoo72 made their first contribution in #554
• @99Franklin made their first contribution in #570
• @kylesayrs made their first contribution in #545
• @leexinhao made their first contribution in #568
• @NimrodShabtay made their first contribution in #572
• @jujeongho0 made their first contribution in #575
• @yuhui-zh15 made their first contribution in #573
• @CircleRadon made their first contribution in #588
• @seungyeonlj made their first contribution in #591
• @robinhad made their first contribution in #609
• @wenhuchen made their first contribution in #614
• @sty-yyj made their first contribution in #616

Full Changelog: v0.3.1...v0.3.2