derivadas.dev

Pablo Díaz Viñambres

MSc Informatics @ TUM

Matching Disconnected Shapes: 3D Correspondence under Partiality with Disconnection

Shape MatchingPyTorch3D Generative ModelsResearch


Shape matching is a classical problem in Geometry Processing that aims to find point-to-point correspondences between two 3D surfaces. Partial shape matching does it when one of them is incomplete. Most methods, including the one we build on (ULRSSM), lean on functional maps: a shared basis built from the eigenfunctions of each shape’s Laplace-Beltrami operator (LBO), which implicitly assumes a single connected surface. If one cuts a shape into several spatially disconnected fragments, the functional map falls apart. Existing benchmarks (e.g. SHREC16, BeCoS) only handle the single connected component. Together with Mehdi Chehaider, I explored what happens under real disconnection, where existing correspondence methods fail almost immediately.

We first built a dataset for this, adapting the BeCoS codebase that uses raycasting for fragmenting shapes over different base benchmarks (e.g. SCAPE, FAUST, DeformingThings4D, TOSCA). Beyond a random-viewpoint baseline, we designed a non-convexity-driven viewpoint selection strategy: it renders depth maps from candidate directions, detects the depth discontinuities that split a silhouette, and samples the views that yield well-balanced disconnected components. This way, we are able to generate hard examples quickly — drag the shapes below to see one test mesh per dataset, with each of its largest disconnected components rendered in its own color and stray sliver fragments greyed out:

FAUST — 5 largest components
SCAPE — 5 largest components
TOSCA — 5 largest components
DT4D Animals — 5 largest components

Reconstruction as a matching prior

Our core idea for actually solving the matching problem is using a 3D generative model as a shape prior to reconstruct the full object from its fragments, match against that reconstruction, and then project the correspondence back onto the disconnected mesh:

Mapping through reconstructions: a disconnected mesh M_A is completed into a reconstruction M_recA via nearest-neighbor correspondence, matched against the template M_0 through ULRSSM, and the composed map is compared against the ground-truth correspondence.

We compared four reconstruction approaches, first a baseline geometric bridging method, then point-cloud completion via PoinTr and Poisson Surface Reconstruction, and finally two image-to-3D generative models, Hunyuan3D and Pixal3D. We trained ULRSSM on each and measured geodesic error and reconstruction quality. Hunyuan3D came out on top, producing smooth, well-aligned reconstructions that preserved most of the correspondence quality, while bridging turned out to be a surprisingly strong non-learned baseline on human meshes. Our main finding is that matching disconnected shapes is now a tractable problem thanks to recent 3D generative models, and we aim to keep working on this line of research and publish a paper describing the detailed approach soon.

Results

Every mesh below is colored by transferring the Template’s own coordinate-based coloring through its predicted point-to-point correspondence, so a smooth, coherent color means accurate matching and a scrambled color means it broke down. You can choose to show the mapping (via Nearest-Neighbors) to the disconnected mesh, which is what’s evaluated, or the full reconstruction, which is what’s matched.

Template
GT
Bridging
PoinTr
Hunyuan3D
Pixal3D
FAUST
TOSCA
n/a
n/a

The following two tables confirm the visual results. While Hunyuan3D and Pixal3D are mostly tied in terms of reconstruction quality, Hunyuan3D leads in matching quality, which we attribute to a more regular geometry.

FAUST - reconstruction method metrics

MethodCD-L1 (cm)↓CD-L2 (cm²)↓F1@2cm↑F1@5cm↑
PoinTr3.09±1.2827.95±23.310.58±0.160.81±0.11
Hunyuan3D2.41±1.0215.57±19.150.63±0.150.90±0.08
Pixal3D2.56±1.3219.33±30.750.59±0.140.90±0.10

FAUST - matching quality metrics

MethodGeodesic Error↓AOC↑
Bridging0.230±0.1280.276±0.189
PoinTr0.311±0.0860.140±0.136
Hunyuan3D0.078±0.0970.561±0.194
Pixal3D0.327±0.2280.246±0.289