Visual–inertial dense mapping

Accord Universal Online Metric Localization and Dense Mapping from one Camera and an IMU

The paper

Abstract

A filter gives the poses, a depth model gives the surfaces, and the agreement between views makes their scales one.

Against the dense pipelines

Results at a glance

The mean F-score at 5 cm of each pipeline's dense map, per test set. Every map is placed by its own trajectory, so a wrong scale or a drifting trajectory costs it points.

How it works

Three processes, none waiting for another

A filter for the poses, a depth model for the map, and a correction that makes the scale of every image agree.

Accord's architecture: a VIO filter tracks the camera at sensor rate; a GPU worker turns windows of keyframes into metric submaps placed in a graph that loop closures correct; a third process corrects the scale of each image.
TrackingAn OpenVINS filter tracks the camera from the images and the inertial samples, on the CPU, every frame. Its poses are metric and never wait for the map.
Metric anchorWindows of keyframes go to Depth Anything 3 on the GPU with the filter's poses. Keypoints triangulated under those poses anchor each window's depth to metric scale.
Submap graphEach window becomes a submap in a factor graph that loop closures correct. The map never writes back to the filter.
Agreement between viewsImages of different windows that see one surface agree on one scale per image, by robust least squares, in a process the mapper never waits for.

In your browser

Interactive maps

Every cloud is the one its score report scored, placed in the laser scan's frame by its own trajectory, nothing fitted to the scan. Colour it by its distance to the scan to see where it is right; turn the scan on to see what was not mapped; compare puts two pipelines side by side under one camera.

drag · scroll · right-drag
loading…

Watch it build

Videos

The maps as they are built, and Accord beside FoundationVIO outdoors.

Replays in Rerun

A run's recording in Rerun's own viewer: the filter's trajectory, the map's, every submap as the graph placed it, keyframes and loop closures, frame by frame. The viewer (about 30 MB) and the recording load only when you open one.

Open source

Run it

Linux, Docker with the NVIDIA container toolkit and an NVIDIA GPU: 6 GB for DA3-Base, 24 GB for DA3-Giant, the model of the maps above. Set up once, then any dataset.

# the code, then the image, OpenVINS, DA3-Base, XFeat
git clone --recursive REPOSITORY
cd ACCORD
make setup
make weights MODEL=giant

# data
make euroc SEQS='V1_01_easy V2_01_easy'
make ori SEQS='r01 r02'
make hilti SEQS='exp18 exp16 exp21'
make spires SEQS='cc02 cc03 cc05'
make tumvi SEQS='room1 room4'
# live, with the viewer at localhost:9090
./accord run V1_01_easy --viz
./accord run room4 --dataset tumvi --viz
./accord run r01 --dataset ori --rate 0.3 --viz
./accord run exp18 --dataset hilti --viz
./accord run cc03 --dataset spires --viz

# your own camera and IMU, then the scores
./accord run NAME --dataset custom --viz
./accord score runs/V1_01_easy
scripts/score.sh runs/V1_01_easy