QUORUM: Multi-View Feature Consensus for Open-Vocabulary SLAM in Dynamic Scenes

Nasser, Zaid; Iumanov, Mikhail; Li, Tianhao; Popov, Maxim; Mahmoud, Jaafar; Kolyubin, Sergey

Abstract

We present QUORUM, an online semantic SLAM system that enables geometry-aware open-vocabulary fusion, where multi-view high-level features vote for a dense pixel-level visual-language embedding field. These embeddings are consumed at four stages of the SLAM stack: the optical-flow prior, factor graph topology, a cross-view residual inside dense bundle adjustment, and the per-pixel shape of the robust kernel. This fusion in BA is wrapped by a temporal stability field that aggregates cross-view embedding agreement, separating genuinely static surfaces from actively moving objects, which improves robustness in dynamic environments.

Unlike existing approaches that require calibrated, posed RGB-D input, QUORUM operates directly on raw monocular RGB video streams, requiring no prior camera intrinsics, depth sensors, or pose initialization. Experiments demonstrate that QUORUM achieves state-of-the-art results on the dynamic TUM-RGBD benchmark while maintaining competitive performance against offline open-vocabulary methods that rely on calibrated data and static scene assumptions. QUORUM bridges a critical gap in real-world deployment, enabling robust open-vocabulary semantic grounding for autonomous robotics and unconstrained in-the-wild video streams. The code of QUORUM is open-source.

Contributions

What QUORUM Introduces

No existing system jointly achieves real-time operation, open-vocabulary grounding, robust odometry, accurate mapping and dynamic-scene robustness. QUORUM closes that gap with three ideas.

01

One Embedding Field, Four Entry Points

A single dense field from RADSeg — an agglomerative foundation model built on RADIO — is extracted once per keyframe and read by four stages: the flow prior, the factor-graph topology, a bundle-adjustment residual, and the shape of the robust kernel.

02

Temporally Consistent Adaptive Robust Kernel

A dynamic-aware optimization scheme that aggregates cross-view agreement over a keyframe's whole neighbourhood before mapping it to the Barron shape parameter — so a displaced object that looks locally stationary is still caught.

03

Calibration-Free Online Open-Vocabulary SLAM

A ready-to-deploy system supporting free-form language grounding in 3D, directly from uncalibrated monocular RGB video — with no depth, no pose priors, and no category-specific supervision.

Pipeline

System Overview

QUORUM ingests a raw, uncalibrated monocular RGB stream and jointly optimizes camera intrinsics, poses and dense metric depth inside one tightly coupled, end-to-end differentiable factor graph.

QUORUM pipeline: uncalibrated RGB stream through GeoCalib, RADSeg, DROID-SLAM flow and depth models into dense tightly coupled bundle adjustment, producing a language-groundable semantic map — **Fig. 1 — The QUORUM pipeline.** Intrinsics are bootstrapped with GeoCalib and then co-optimized inside bundle adjustment. RADSeg features, dense flow and monocular metric depth all feed one vision-language-geometric energy, wrapped by an adaptive robust kernel. Text queries are matched in SigLIP space against the fused 3D map.

01

Camera Init

GeoCalib bootstraps intrinsics from sampled frames — then they are co-optimized in BA rather than held fixed. No calibration target.

Backbone

02

Keyframe Selection

Only frames whose weighted dense optical flow relative to the last keyframe exceeds a motion threshold enter the factor graph.

Backbone

03

Depth Prior

Metric depth per keyframe from a monocular foundation model, converted to disparity and downsampled ×8 to match the flow resolution.

Backbone

04

Feature Extraction

Dense RADSeg embeddings, run sliding-window over overlapping regions and PCA-compressed to D = 256 — one shared basis for every term.

QUORUM

05

Bundle Adjustment

Poses, disparities and intrinsics jointly refined by minimizing a vision-language-geometric energy, with graph topology augmented by the same field.

QUORUM

06

OV Grounding

Compressed RADSeg features of 3D points are decoded and projected into SigLIP latent space, then matched against the text query.

QUORUM

Method

One Field, Four Consumers

RADSeg features are extracted once per keyframe. Because all four consumers read the same field, the marginal cost of each is small relative to what it contributes.

(A)

Semantic Flow Prior

Blended into the geometric flow prior, weighted so semantics dominate exactly where photometric confidence is low.

(B)

Graph Topology

Penalizes edges that are geometrically close but semantically dissimilar, and retrieves loop closures.

(C)

BA Residual

Cross-view feature alignment enters bundle adjustment directly, constraining pose and depth.

(D)

Robust Kernel Shape

Multi-view agreement sets a per-pixel Barron shape parameter, so each pixel carries its own loss.

The consensus rule

Agreement measured across a single pair of frames is too noisy to steer a loss shape. QUORUM instead votes across a keyframe's whole neighbourhood, combining the mean cross-view similarity with its variance:

𝒮 = mean(cs) · ( 1 − var(cs) )

A high score means a genuinely static surface. A low score flags either active motion (low mean) or an object displaced between visits (high variance) — and the variance term is what catches the second case, which looks perfectly stationary from any single viewpoint. The score then sets the shape parameter α of the Barron loss:

Static surfacesα = 2 · quadratic ℓ₂

Movable objectsα = 1 · pseudo-Huber

Moving agentsα = −2 · Geman–McClure

Adaptive robust kernels based on Barron's general loss: cosine similarity between two views produces a per-pixel alpha map, alongside the family of Barron loss curves — **Fig. 2 — Adaptive robust kernels.** RADSeg features from two views give a per-pixel similarity map, which the stability field turns into a per-pixel α — visibly isolating the seated people from the static room.

Full derivations, the complete energy function and every hyperparameter are in the paper.

Experiments

Quantitative Results

The two capabilities are evaluated separately, on the benchmark that isolates each: pose accuracy under scene dynamics on the eight fr3 sequences of TUM RGB-D, and open-vocabulary 3D semantic segmentation on eight Replica scenes. All experiments run on an Intel Xeon Gold 5320 with a single NVIDIA RTX 4090, at 8–10 FPS end-to-end.

QUORUM attains the best average ATE (1.68 cm) of all methods listed, from the most constrained input: it is the only entry that is simultaneously calibration-free, depth-free and pose-free. The five RGB-D baselines are handed metric depth directly, and QUORUM still improves on the best of them by 16%. The most informative comparison is against ViPE, which shares our backbone but handles dynamics by running Grounding DINO and SAM over a manually specified list of dynamic classes: QUORUM cuts average ATE from 2.17 cm to 1.68 cm while removing that dependency entirely.

Method	fr3 / walking				fr3 / sitting
Method	xyz	rpy	half	static	xyz	rpy	half	static	Avg ↓
RGB-D input · metric depth given
Dyna-SLAM	1.64	3.54	2.96	0.68	1.27	—	1.86	—	2.00*
DLD-SLAM	1.85	4.24	2.19	0.56	—	—	—	—	2.21*
V3D-SLAM	1.53	7.81	2.29	0.65	0.87	1.69	1.47	0.58	2.11
DGS-SLAM	4.10	—	5.50	0.60	—	—	4.40	—	3.65*
RoDyn-SLAM	8.30	—	5.60	1.70	—	—	2.70	—	4.58*
RGB input · monocular depth · scale aligned to GT before ATE
DynaMoN	1.40	3.90	2.00	1.40	0.90	2.10	1.90	0.50	1.76
ViPE (SAM)	2.40	3.46	2.52	0.54	1.43	3.80	2.57	0.60	2.17
QUORUM (ours)	1.84	3.13	2.07	0.56	1.10	2.74	1.51	0.56	1.68

Table 1 — SLAM performance on TUM-RGBD. RMSE of the Absolute Trajectory Error in cm, lower is better. QUORUM consumes monocular RGB only and recovers depth from a foundation model, so absolute scale is not observable from the input; following the other RGB-only entries, trajectory scale is aligned to ground truth before ATE is computed, whereas RGB-D entries need no such alignment.

1st Best 2nd Second 3rd Third — not reported · * average over reported sequences only

Our semantic map is built during real-time operation rather than as an offline post-process. QUORUM ranks in the top three on every without-background metric and is again the only system requiring neither calibration, depth, nor pose; all competitors except RayFronts are offline. The pair QUORUM_GT / QUORUM isolates exactly what ground-truth depth, pose and calibration are worth, since nothing else differs between the two rows.

Method	Without Background			With Background			Properties
Method	mIoU ↑	f-mIoU ↑	Acc ↑	mIoU ↑	f-mIoU ↑	Acc ↑	Online	w/o Calib	w/o Depth	w/o Pose
ConceptFusion	21.07	31.51	35.65	20.38	35.75	41.58	✕	✕	✕	✕
ConceptGraphs	11.63	16.61	19.80	11.72	21.35	28.28	✕	✕	✕	✕
HOV-SG	16.93	31.45	34.74	19.29	30.64	35.17	✕	✕	✕	✕
NACLIP-3D	20.37	35.08	47.47	15.30	16.98	26.23	✕	✕	✕	✕
Trident-3D	21.30	43.34	54.79	20.63	38.53	50.31	✕	✕	✕	✕
RayFronts	39.37	62.03	68.80	27.73	43.37	54.45	✓	✕	✕	✕
QUORUM_GT	29.51	52.24	59.80	28.19	54.44	65.21	✓	✕	✕	✕
QUORUM (ours)	24.25	50.63	59.25	19.00	37.13	48.38	✓	✓	✓	✓

Table 2 — Open-vocabulary 3D semantic segmentation on Replica (8 scenes: office0–4, room0–2), following the protocol of RayFronts. QUORUM_GT is the identical system given ground-truth depth, pose and calibration. Without background the uncalibrated setting costs little — f-mIoU falls 1.6 points. With background it costs 17.3 points: structural classes are large, weakly textured and observed under low parallax, exactly where a monocular depth prior is least reliable. That QUORUM_GT is the best method in the table on all three with-background metrics localizes the problem to the geometry, not the embedding field.

1st Best 2nd Second 3rd Third ✓ supported ✕ not supported

All ablations are reported on the four TUM walking sequences. Every variant runs under one common configuration, so rows within a table differ only by the ablated component — absolute ATE therefore differs slightly from Table 1, which reports the final system.

Kernel variant	fr3 / walking
Kernel variant	xyz	rpy	half	static	Avg ↓
No robust kernel (ℓ₂)	1.91	3.41	2.41	0.56	2.07
Fixed pseudo-Huber (α = 1)	1.84	3.26	2.06	0.56	1.93
Per-edge semantic (α from cs_ij)	1.88	3.27	2.78	0.56	2.12
Multi-view, mean & variance (full)	1.84	3.13	2.07	0.56	1.90

Table 3 — Multi-view aggregation is what makes a semantic kernel usable. The third row makes the shape adaptive but computes it from a single edge, with no aggregation — and is the worst entry in the table. It is the multi-view aggregation, not the use of semantics as such, that makes the mechanism work.

Consumer removed	fr3 / walking
Consumer removed	xyz	rpy	half	static	Avg ↓
None of (a)–(d) — geometric pipeline	1.82	3.36	2.82	0.55	2.14
Full w/o (a) semantic flow prior	1.86	3.15	2.16	0.56	1.93
Full w/o (b) graph topology	1.81	3.22	2.20	0.56	1.95
Full w/o (c) BA residual	1.79	3.37	2.14	0.57	1.97
Full w/o (d) adaptive kernel	1.84	3.26	2.06	0.56	1.93
Full system (ours)	1.84	3.13	2.07	0.56	1.90

Table 4 — Each consumer of the embedding field earns its place. Leave-one-out rather than add-one-in, because the question is whether each stage earns its place in the system as shipped. The first row disables all four at once and recovers the underlying geometric pipeline.

Radar plots of mIoU and accuracy per Replica scene for PCA dimensions 32, 64, 128, 256 and the full-dimensional baseline — **Fig. 3 — RADSeg PCA feature dimensionality on Replica.** D = 256 (red) tracks the full-dimensional baseline to within 1% mIoU (0.242 vs 0.249) at a quarter of the storage — which is what makes a dense per-voxel embedding map affordable online.

Prior work reviewed along five axes: localization, calibration, semantics, real-time operation and dynamic-scene handling. Predefined in the Dynamic column marks a system that rejects motion only for a manually specified list of classes.

Method	Online	Semantics	Grounding	Localization	Dynamic	Calib-free
Geometric SLAM
ORB-SLAM3	✓	✕	✕	✓	✕	✕
RGBDS-SLAM	✓	Closed	✕	✓	✕	✕
Dynamic SLAM
Dyna-SLAM	✓	Closed	✕	✓	✓	✕
DynaMoN	✕	Closed	✕	✓	✓	✕
Offline open-vocabulary scene understanding
ConceptFusion	✕	Open	✓	✕	✕	✕
ConceptGraphs	✕	Open	✓	✕	✕	✕
HOV-SG	✕	Open	✓	✕	✕	✕
Trident-3D	✕	Open	✓	✕	✕	✕
Real-time open-vocabulary SLAM
OVO-SLAM	✓	Open	✓	✓	✕	✕
RayFronts	✓	Open	✓	✕	✕	✕
Feed-forward SLAM
VGGT-SLAM	✓	✕	✕	✓	✕	✓
ViPE	✓	Predefined	✕	✓	Predefined	✓
QUORUM (ours)	✓	Open	✓	✓	✓	✓

Table 5 — Comparison of state-of-the-art SLAM methods. Columns indicate online operation, semantic understanding type (Closed = predefined classes, Open = open-vocabulary), geometric grounding (the ability to textually access and interact with the 3D map), localization, dynamic-scene handling, and the calibration requirement. QUORUM is the only system that satisfies all six.

✓ Supported ✕ Not supported Open Open-vocabulary Predefined Manually specified class list

Qualitative Results

Grounding & Embedding Maps

Free-form text queries resolved against the 3D map, and RGB-PCA colorization of the dense embedding field on both benchmark and real-world captures.

Six 3D point cloud views of Replica scenes with query responses highlighted: table, pillow, RADSeg PCA colorization, shelf, cabinet and table — **Fig. 4 — Open-vocabulary grounding on Replica.** Heat responses (red) and retrieved object boxes (green) for the queries *table*, *pillow*, *shelf* and *cabinet*, alongside the RGB-PCA colorized RADSeg point cloud. Queries are matched in SigLIP space — none of these categories is supervised.

Real-world office captured with a phone camera: RGB frame, reconstructed point cloud, and RGB-PCA colorized embedding field — **Fig. 5 — In-the-wild office, phone camera.** RGB frame, reconstructed geometry, and RGB-PCA colorization of the embedding field. Uncalibrated handheld capture, no depth sensor.

RGB-PCA colorization of the embedding field for Replica office 3 — **Fig. 6 — RGB-PCA colorization, Replica office 3.** Distinct surfaces receive distinct embeddings — the same field that drives grounding also drives the stability field used by the robust kernel.

Four synchronized panels from a live video stream: RGB input, embedding field, estimated depth, and fused geometry — **Fig. 7 — Live stream, four views.** RGB input, embedding field, estimated depth and fused geometry from a single uncalibrated monocular video, all produced online in the same pass.

Citation

BibTeX

quorum.bib

@inproceedings{nasser2026quorum,
  title     = {QUORUM: Multi-View Feature Consensus for
               Open-Vocabulary SLAM in Dynamic Scenes},
  author    = {Nasser, Zaid and Iumanov, Mikhail and Li, Tianhao and
               Popov, Maxim and Mahmoud, Jaafar and Kolyubin, Sergey},
  booktitle = {Proceedings of the 3rd Workshop on Neural SLAM (NeuSLAM),
               European Conference on Computer Vision (ECCV) Workshops},
  year      = {2026},
  url       = {https://github.com/be2rlab/QUORUM}
}