ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models

Dataset Description

We introduce ViewSpatial-Bench, a comprehensive benchmark with over 5,700 question-answer pairs across 1,000+ 3D scenes from ScanNet and MS-COCO validation sets. This benchmark evaluates VLMs' spatial localization capabilities from multiple perspectives, specifically testing both egocentric (camera) and allocentric (human subject) viewpoints across five distinct task types.

ViewSpatial-Bench addresses a critical gap: while VLMs excel at spatial reasoning from their own perspective, they struggle with perspective-taking—adopting another entity's spatial frame of reference—which is essential for embodied interaction and multi-agent collaboration. The figure below shows the construction pipeline and example demonstrations of our benchmark.

The dataset contains the following fields:

Field Name	Description
question_type	Type of spatial reasoning task, includes 5 distinct categories for evaluating different spatial capabilities
image_path	Path to the source image, includes data from two sources: scannetv2_val (ScanNet validation set) and val2017 (MS-COCO validation set)
question	The spatial reasoning question posed to the model
answer	The correct answer to the question
choices	Multiple choice options available for the question

• Language(s) (NLP): en
• License: apache-2.0

Uses

With HuggingFace datasets library.

from datasets import load_dataset
ds = load_dataset("lidingm/ViewSpatial-Bench")

Benchmark

We provide benchmark results for various models on our benchmark. More model evaluations will be added.

Model	Camera-based Tasks			Person-based Tasks				Overall
	Rel. Dir.	Obj. Ori.	Avg.	Obj. Ori.	Rel. Dir.	Sce. Sim.	Avg.
Proprietary Models
GPT-4o	41.46	19.58	33.57	42.97	40.86	26.79	36.29	34.98
Gemini-2.0-Flash	45.29	12.95	33.66	41.16	32.78	21.90	31.53	32.56
GPT-5-mini	56.97	27.41	46.34	43.98	49.29	26.06	38.77	42.44
Gemini-2.5-Flash	52.62	23.09	42.00	42.97	42.16	20.27	34.22	37.99
Gemini-2.5-Pro	58.71	32.73	49.37	48.59	45.84	25.79	39.24	44.15
Gemini-3.0-Flash	62.94	35.54	53.08	44.88	60.69	26.24	42.40	47.58
GLM-4.6v	56.35	36.35	49.16	48.90	47.39	23.44	38.91	43.87
Doubao-Seed-1.8	62.10	45.28	56.05	44.98	62.47	33.67	45.74	50.74
Doubao-Seed-2.0	65.60	44.78	58.11	47.19	72.09	33.57	49.20	53.52
Open-Source General Models
InternVL2.5 (2B)	38.52	22.59	32.79	47.09	40.02	25.70	37.04	34.98
Qwen3-VL (4B)	46.98	28.01	40.16	45.68	29.22	17.74	30.48	35.17
Qwen2.5-VL (7B)	46.64	29.72	40.56	37.05	35.04	28.78	33.37	36.85
LLaVA-NeXT-Video (7B)	26.34	19.28	23.80	44.68	38.60	29.05	37.07	30.64
LLaVA-OneVision (7B)	29.84	26.10	28.49	22.39	31.00	26.88	26.54	27.49
InternVL2.5 (8B)	49.41	41.27	46.48	46.79	42.04	32.85	40.20	43.24
Qwen3-VL (8B)	54.60	30.32	45.87	45.28	35.75	26.79	35.61	40.58
Llama-3.2-Vision (11B)	25.27	20.98	23.73	51.20	32.19	18.82	33.61	28.82
InternVL3 (14B)	54.65	33.63	47.09	33.43	37.05	31.86	33.88	40.28
Kimi-VL-Instruct (16B)	26.85	22.09	25.14	63.05	43.94	20.27	41.52	33.58
Qwen2.5-VL (32B)	39.03	29.92	35.75	36.45	34.68	21.09	30.18	32.88
Qwen2.5-VL (72B)	50.65	26.71	42.04	42.17	42.76	24.80	35.82	38.83
Qwen3-VL-Thinking (235B)	59.73	36.95	51.54	43.67	48.93	31.67	40.67	45.94
Qwen3.5-Plus (397B)	62.21	38.65	53.74	50.20	68.17	38.37	50.90	52.28
Multi-View Spatial Fine-Tuning
Qwen2.5-VL (3B)	43.43	33.33	39.80	39.16	28.62	28.51	32.14	35.85
+SFT	83.59	87.65	85.05	90.16	71.14	75.75	79.31	82.09
Improvement over backbone	+40.16	+54.32	+45.25	+51.00	+42.52	+47.24	+47.17	+46.24
Random Baseline	25.16	26.10	25.50	24.60	31.12	26.33	27.12	26.33