Datasets & Benchmarks

The first egocentric VideoQA dataset collected from blind and visually impaired individuals. Questions reflect genuine in-situation needs for visual assistance. State-of-the-art MLLMs achieve ~60% vs. 87.4% human performance.

The first dataset for egocentric visual intention grounding. Models must localise objects in first-person views based on implicit intention queries — e.g. "somewhere to sit" rather than "chair". Challenges models on affordance understanding and contextual reasoning.

A benchmark for egocentric scene-text aware video question answering. Models must read and reason about text appearing naturally in first-person videos — signs, labels, displays — to answer questions about the wearer's environment and activities.

A benchmark for question-answering on dense video events, requiring models to answer and ground questions about multiple events in long videos. Challenges MLLMs to faithfully comprehend and reason across extended time periods with multiple overlapping events.

A benchmark for scene-text grounding in text-based video question answering. Models must read scene text in videos and localise the spatio-temporal evidence for their answers. Current MLLMs achieve only 28% grounding accuracy vs. 77% for humans.

Extends NExT-QA by requiring models to localise the temporal evidence for their answers — not just answer correctly, but show where in the video. Exposes hallucination and shortcut learning in video-language models.

A new annotation layer on Assembly101 providing coarse-level action labels based on part positioning, enabling the study of ordering mistake detection in assembly procedures.

A large-scale multi-view dataset of people assembling and disassembling 101 take-apart toy vehicles without fixed instructions. The first dataset with simultaneous static (8) and egocentric (4) recordings. Awarded the EgoVis 2022/2023 Distinguished Paper Award. Now available on Hugging Face.

A video question-answering benchmark pushing models to reason about why and how events happen, rather than just describing them. Widely adopted as a standard benchmark for video-language models.

A collection of cooking recipe videos for zero-shot activity anticipation. Each video is paired with an ingredient list and step-wise instructions, enabling anticipation from structured procedural knowledge without seen training examples.

Real-world indoor and outdoor scenes captured with a custom multi-modal SLAM rig (64-channel LiDAR, 4 wide-angle RGB cameras, IMU) for benchmarking 3D Gaussian Splatting from coarsely-posed images and noisy LiDAR point clouds, without SfM initialisation.

A synthetic hand image dataset and synthesis pipeline studying the synthetic-to-real domain gap in 3D hand pose estimation. Identifies key gap components (forearm, image frequency, hand pose, object occlusions) and demonstrates synthetic data can match real data performance when these are addressed.