Representative examples across indoor, outdoor, and object-centric scenes, showing 3D scene graphs, visual prompts, and QA pairs.
Current vision-language models (VLMs) demonstrate strong capabilities on 2D perception and captioning, but remain brittle when asked to perform 3D-aware reasoning from a single image, such as estimating distances, sizes, occlusions, and navigation feasibility.
We introduce MonoSR, a large-scale benchmark for open-vocabulary spatial reasoning from monocular RGB images. MonoSR spans diverse domains including indoor scenes, outdoor driving and navigation, and object-centric views. For each image, we construct structured 3D scene graphs with aligned 2D/3D geometry and generate hierarchical question–answer pairs that probe (1) low-level geometric perception, (2) mid-level perspective-aware reasoning, and (3) high-level situational understanding.
We evaluate a range of state-of-the-art VLMs and 3D-aware models, and show that even the strongest systems fall far below human performance on challenging monocular spatial reasoning tasks, revealing substantial headroom for 3D-grounded multimodal reasoning from a single view.
MonoSR organizes monocular spatial reasoning into a hierarchy of skills, ranging from metric geometry to perspective-aware and situational reasoning.
Example tasks include estimating 3D distances between objects, judging which object is closer to the camera, reasoning about visibility and occlusion, and deciding whether a proposed navigation path is feasible without collisions.
MonoSR spans multiple domains to stress-test model robustness: indoor environments (homes, offices, corridors), outdoor driving and navigation scenes, and object-centric views with strong perspective distortion.
For each image, we align monocular 2D cues with 3D scene information, enabling fine-grained evaluation of how well models recover underlying 3D structure and spatial relations from a single view.
We decompose spatial reasoning into three levels and report per-level performance across domains. This reveals where models fail: low-level metric errors, perspective confusions, or high-level situational mistakes.
We evaluate a range of closed-source and open-source VLMs, as well as 3D-aware models. MonoSR reveals consistent failure modes on perspective-aware and situational tasks, highlighting substantial room for improving 3D-grounded reasoning from monocular inputs.
MonoSR is part of a growing line of work on 3D-aware vision-language understanding and spatial reasoning from images.
Benchmarks such as ScanQA, ReferIt3D, and various 3D VQA datasets focus on 3D scenes with point clouds or meshes. MonoSR complements these efforts by emphasizing monocular inputs and open-vocabulary spatial queries.
Recent 3D-aware VLMs and 3D-LLMs that integrate explicit 3D features (e.g., from depth or point clouds) provide promising directions but still struggle on challenging monocular setups where 3D cues must be inferred implicitly.
We hope MonoSR can serve as a standardized benchmark for future spatially grounded multimodal models and training strategies.
@article{wang2025monosr,
title={MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images},
author={Wang, Qirui and He, Jingyi and Pan, Yining and Yeo, Si Yong and Yang, Xulei and Li, Shijie},
journal={arXiv preprint arXiv:2511.19119},
year={2025}
}