hh721/vdr_temp

VDR: Video Deep Research Benchmark

VDR (Video Deep Research) is an evaluation benchmark for video-grounded deep research. Each item pairs a video with a question that is deliberately constructed so that neither the video alone nor the open web alone is sufficient to answer it: a system must extract concrete observations from the video, use them as search anchors to retrieve external information, and combine both to reach the answer.

• Videos: 49
• Questions: 100
• Split: test (evaluation only — the released items are sanitized and contain no ground-truth answers)
• Language: English

Dataset structure

The questions table is published as Parquet for easy browsing and loading:

data/vdr_questions.parquet   # 100 rows, one per question
videos/<video_id>.mp4        # 49 source videos (referenced by the table, not duplicated into Parquet)
questions/<video_id>/<qid>.json
video_question_map.json      # index of questions grouped by video

Videos are not embedded in the Parquet. Each row references its video by a repo-relative path (video) and a directly resolvable link (video_url), following the common pattern for larger video datasets where media is stored as files rather than inlined bytes.

Data fields

field	type	description
sample_id	string	Unique item identifier to report in predictions (e.g. 2PXEEdsJbnQ_q1).
video_id	string	Source video identifier (e.g. 2PXEEdsJbnQ).
question	string	The question to answer (source field: task_question).
video	string	Repo-relative path to the MP4, e.g. videos/<video_id>.mp4.
video_url	string	Directly resolvable URL to the MP4 on the Hub.
audio_required	bool	Whether answering requires the audio track.
image_search_required	bool	Whether answering requires image/visual web search.

Statistics

• 49 videos, 100 questions (1–3 questions per video: 14 videos with 1, 19 with 2, 16 with 3).
• Total video size ≈ 4.77 GB (per-video min / median / max ≈ 2.8 / 25.1 / 1534.6 MB).
• audio_required: 7 questions. image_search_required: 6 questions.

Usage

from datasets import load_dataset

ds = load_dataset("hh721/vdr_temp", split="test")
print(ds[0]["sample_id"], ds[0]["question"])

Fetching a video for an item:

import urllib.request

item = ds[0]
urllib.request.urlretrieve(item["video_url"], f'{item["video_id"]}.mp4')
# or use the repo-relative path with huggingface_hub.hf_hub_download(..., item["video"])

Evaluation protocol

Each item is answered by: (1) watching the video and locating the moments relevant to the question; (2) extracting observable search anchors (a landmark, on-screen text, logo, spoken name, distinctive object, …); (3) searching the open web from those anchors and keeping the URL + a supporting verbatim quote for every external fact; (4) combining video observations with the retrieved documents. Answers should be the shortest response that fully satisfies the question; for multi-part questions, answer each part semicolon-separated in the order asked, and write UNKNOWN for any part that cannot be established rather than guessing. audio_required / image_search_required flag items that additionally need the audio track or image-based search.

Limitations and notes

• No ground-truth is released. The public items are sanitized for open evaluation; do not source answers from any other file in this package.
• Questions require live open-web access; retrievable facts may change over time.
• Videos are sourced externally. Licensing/usage of the underlying video content is the responsibility of the dataset owner and downstream users — please confirm the terms before redistribution or commercial use. (License field intentionally left unset pending the owner's confirmation.)

Citation

@misc{vdr_benchmark,
  title  = {VDR: Video Deep Research Benchmark},
  author = {hh721},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/datasets/hh721/vdr_temp}}
}