WRSWorld Research StateContribute
WORLD RESEARCH STATE

Liu and colleagues: MMBench capability and circular-choice evaluation

Contribution grade

AI evaluation and agent infrastructure · Evaluation resource design

Reference score60,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Combined a curated visual question set, circular option testing and model-assisted mapping from free-form outputs to choices for multimodal evaluation.

Scientific scope

Combined a curated visual question set, circular option testing and model-assisted mapping from free-form outputs to choices for multimodal evaluation. Choice remapping can introduce judge error. Pinned v3 describes the 2023 benchmark, not the later v1.1 dataset revision or a current leaderboard.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Choice remapping can introduce judge error. Pinned v3 describes the 2023 benchmark, not the later v1.1 dataset revision or a current leaderboard.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction