derivadas.dev

Pablo Díaz Viñambres

MSc Informatics @ TUM

Matching Disconnected Shapes: 3D Correspondence under Partiality with Disconnection

Shape MatchingPyTorch3D Generative ModelsResearch


Shape matching is a classical problem in Geometry Processing that aims to find point-to-point correspondences between two 3D surfaces. Partial shape matching does it when one of them is incomplete. Most methods, including the one we build on (ULRSSM), lean on functional maps: a shared basis built from the eigenfunctions of each shape’s Laplace-Beltrami operator (LBO), which implicitly assumes a single connected surface. If one cuts a shape into several spatially disconnected fragments, the functional map falls apart. Existing benchmarks (e.g. SHREC16, BeCoS) only handle the single connected component. Together with Mehdi Chehaider, we explored what happens under real disconnection, where existing correspondence methods fail almost immediately.

We first built a dataset for this, adapting the BeCoS codebase that uses raycasting for fragmenting shapes over different base benchmarks (e.g. SCAPE, FAUST, DeformingThings4D, TOSCA). Beyond a random-viewpoint baseline, we designed a non-convexity-driven viewpoint selection strategy: it renders depth maps from candidate directions, detects the depth discontinuities that split a silhouette, and samples the views that yield well-balanced disconnected components. This way, we are able to generate hard examples quickly — drag the shapes below to see one test mesh per dataset, with each of its largest disconnected components rendered in its own color and stray sliver fragments greyed out:

FAUST — 5 largest components
SCAPE — 5 largest components
TOSCA — 5 largest components
DT4D Animals — 5 largest components

Reconstruction as a matching prior

Our core idea for actually solving the matching problem is using a 3D generative model as a shape prior to reconstruct the full object from its fragments, match against that reconstruction, and then project the correspondence back onto the disconnected mesh:

Mapping through reconstructions: a disconnected mesh M_A is completed into a reconstruction M_recA via nearest-neighbor correspondence, matched against the template M_0 through ULRSSM, and the composed map is compared against the ground-truth correspondence.

We compared four reconstruction approaches, first a baseline geometric bridging method, then point-cloud completion via PoinTr and Poisson Surface Reconstruction, and finally two image-to-3D generative models, Hunyuan3D and Pixal3D. We trained ULRSSM on each and measured geodesic error and reconstruction quality. Hunyuan3D came out on top, producing smooth, well-aligned reconstructions that preserved most of the correspondence quality, while bridging turned out to be a surprisingly strong non-learned baseline on human meshes. Our main finding is that matching disconnected shapes is now a tractable problem thanks to recent 3D generative models, and we aim to keep working on this line of research and publish a paper describing the detailed approach soon.

Results

Every mesh below is colored by transferring the Template’s own coordinate-based coloring through its predicted point-to-point correspondence, so a smooth, coherent color means accurate matching and a scrambled color means it broke down. You can choose to show the mapping (via Nearest-Neighbors) to the disconnected mesh, which is what’s evaluated, or the full reconstruction, which is what’s matched.

Template
GT
Bridging
PoinTr
Hunyuan3D
Pixal3D
FAUST
TOSCA
n/a
n/a

The following two tables confirm the visual results. While Hunyuan3D and Pixal3D are mostly tied in terms of reconstruction quality, Hunyuan3D leads in matching quality, which we attribute to a more regular geometry.

FAUST - reconstruction method metrics

MethodCD-L1 (cm)↓CD-L2 (cm²)↓F1@2cm↑F1@5cm↑
PoinTr3.09±1.2827.95±23.310.58±0.160.81±0.11
Hunyuan3D2.41±1.0215.57±19.150.63±0.150.90±0.08
Pixal3D2.56±1.3219.33±30.750.59±0.140.90±0.10

FAUST - matching quality metrics

MethodGeodesic Error↓AOC↑
Bridging0.230±0.1280.276±0.189
PoinTr0.311±0.0860.140±0.136
Hunyuan3D0.078±0.0970.561±0.194
Pixal3D0.327±0.2280.246±0.289