derivadas.dev

Pablo Díaz Viñambres

MSc Informatics @ TUM

Primimix: Semantic Planar Splatting for 3D Structured Reconstruction

PyTorchCUDAGaussian SplattingResearch


Primimix is a hybrid scene-decomposition and rendering pipeline developed by Binyu Fang, Katy Grossmann, Billy Herrmann, Danit Niwattananan and myself for our TUM Data Innovation Lab project, originally titled Semantic Planar Splatting for 3D Structured Reconstruction and supervised by Prof. Dr. Olaf Wysocki from the CV4DT Chair at the University of Cambridge. The research problem we tackle is that 3D Gaussian Splatting renders indoor scenes beautifully, but only as an unstructured cloud of millions of kernels. While there are approaches in the literature that regularize those Gaussians, or represent them with alternative primitives (e.g. 2DGS, MeshSplatting, GaMeS…) there is still not a dominant representation that is photorealistic, geometrically accurate and structured. Our approach combines a dual primitive representation, including planes and superquadrics, that are optimized jointly with 2D Gaussians placed on top of them. This way, the primitives provide a scaffold for both foreground objects (superquadrics) and background planar regions (planes), and the Gaussians equipped with Spherical Harmonics coefficients add texture and geometric detail to the representation.

Primimix pipeline: MoGe cloud densification, 3D instance segmentation, primitive initialization and Gaussian skinning, two-phase optimization

Given posed images alone, we densify a point cloud, segment it into instances, and split them into two primitive sets: bounded planes P\mathcal{P} for structural surfaces, and superquadrics Q\mathcal{Q} for furniture and clutter, together forming S=PQ\mathcal{S} = \mathcal{P} \cup \mathcal{Q}. Superquadrics are an eleven-parameter family spanning cuboids, cylinders and ellipsoids, defined by radii ri=(rix,riy,riz)r_i=(r_i^x, r_i^y, r_i^z) and shape exponents εi=(εi1,εi2)\varepsilon_i=(\varepsilon_i^1, \varepsilon_i^2).

Geometric parametrization of the planar primitive P_i and the superquadric primitive Q_i

We then bind 2D Gaussian “skins” to every primitive in S\mathcal{S}, first optimizing geometry against depth and normals, then freezing it to refine appearance. On a representative ScanNet++ scene, this lets Primimix explain the room with around 620 explicit blocks and 1.6M surface Gaussians, against the 1.4-2.2M fully unstructured Gaussians other splatting baselines need to render the same room with no explicit structure at all:

Representation structure: Primimix's primitive decomposition and render side by side with the unstructured 3D Gaussian Splatting point cloud for the same scene

On the geometric side, Primimix leads both baselines on Chamfer distance, completeness and F-score on the ScanNet++ dataset:

MethodChamfer↓Accuracy↓Completeness↓F@5↑
PlanarSplatting10.999.5812.4144.70
3D Gaussian Flats18.604.6032.5950.24
Primimix (Ours)6.637.006.2662.80

The project was awarded a 1.0 at TUM DI-Lab, and we’re now refining the method and working towards a publication.