Paper at the HCV Workshop at ECCV 2026

Workshop · ECCV 2026 Workshops

“The Ego-Allocentric Dialogue: World-Centred Grounded 3D Reasoning via Persistent Primitives”, accepted at the 3rd Workshop on Human-inspired Computer Vision (HCV 2026), held with ECCV 2026.

To answer situated questions and act within 3D environments, an agent must reason jointly about its own position, the layout of the surrounding scene, and the identity of objects viewed from changing perspectives. Human cognition addresses this challenge through three complementary principles: an egocentric representation of the observer, an allocentric map of the environment, and viewpoint-invariant semantics anchored to world content.

Existing Vision–Language models (VLMs) satisfy these biological requirements only partially, presenting a clear dichotomy: they either ground language in a persistent 3D representation while neglecting the observer’s ego-allocentric geometry, or integrate localisation cues over viewpoint-dependent 2D features that lack a stable 3D representation.

We introduce GroundSplat, a human-inspired VLM that fuses relative geometry with a persistent 3D Gaussian Language Field, recovering the agent’s pose through dedicated situation tokens. By jointly grounding semantics and geometry in a common bird’s-eye-view representation, GroundSplat enables embodied 3D reasoning. Preliminary results show significant gains on SQA3D while maintaining comparable performance on ScanQA, indicating that the tight coupling of persistent allocentric semantic and egocentric geometry is the key lever for embodied 3D reasoning.

GroundSplat architecture: a multi-view video clip is encoded by a frozen 3D Gaussian Language Field and the CUT3R visual encoder; visual tokens, the situation text, position and orientation tokens and the question are fed to LLaVA-OneVision 7B with LoRA, trained with allocentric representation, situational and answer losses.
The GroundSplat architecture: visual tokens from a persistent 3D Gaussian Language Field and from the CUT3R encoder, together with the situation description, dedicated position and orientation tokens and the question, are processed by LLaVA-OneVision 7B fine-tuned with LoRA.
Publication

The Ego-Allocentric Dialogue: World-Centred Grounded 3D Reasoning via Persistent Primitives

Related news

← All news