Prior-SG Segments Open-Plan Spaces With LLM Scene Priors
Most scene-graph pipelines still pretend houses are boxes of rooms with walls. Open-plan labs, warehouses, and lofts break that assumption.
Prior-SG (arXiv:2608.06170, Aug 6, 2026) from Giorgio Tonetti, Laurent Kneip, Abel Gawel, and Marco Hutter reframes hierarchical 3D scene graph generation as a probabilistic alignment problem between what the robot sees and what an LLM expects for the task.
Pipeline
- Instance Graph: as the robot explores, RGB-D streams aggregate into a physically grounded instance map using multi-scale, open-vocabulary feature fusion.
- Prior Graph: an LLM synthesizes a logical expectation of environment structure and a task-relevant vocabulary for the current high-level goal.
- MAP alignment: a Markov Random Field fuses visual, geometric, and discrete-object “experts” with those topological priors to resolve local ambiguity.
The claimed payoff: zero-shot ontological flexibility — the same map can be re-partitioned when the task prompt changes, not only when walls appear.
Evaluation (paper claims)
- Validated on diverse simulated residential datasets and large open-plan real-world environments
- State-of-the-art semantic region segmentation accuracy vs recent baselines (as reported)
- Robust delineation of distant functional boundaries without physical walls
No wall-separated-room heuristic is required for the core method.
A Human’s Take
LLM priors on room labels scare me when the model invents a “pantry” that is not there — but open-plan segmentation that can re-slice for “prepare dinner” vs “clean floors” is exactly what mobile manipulators need. I want failure cases when the prior graph lies; SOTA bars alone do not ship.