SILSASliding-Window Slice Latents forTopology-Preserving High-Resolution 3D Generation

Tianjiao Yu, Xinzhuo Li, Yifan Shen, Ying Shen, Onkar Susladkar, Ismini Lourentzou
PLAN Lab · University of Illinois Urbana-Champaign

NeurIPS 2026

PaperarXivResults

TL;DR

We introduce SILSA, a framework for generating high-resolution 3D shapes from a single image using compact slice-based representations. SILSA coordinates overlapping slices across three axes and preserves thin structures, holes, and connectivity, producing detailed shapes with substantially fewer tokens.

SILSA-generated 3D shapes with thin structures and detailed geometry

Topology-preserving high-resolution 3D generation

SILSA generates detailed 3D geometry from an image using 384 slice tokens.

Abstract

High-resolution 3D generation increasingly relies on voxel latents and multi-stage pipelines that first predict active structure and then synthesize local geometry. While effective, this design fragments continuous surfaces into many local tokens, inflates generation cost, and often weakens topological consistency for thin or highly connected shapes.

We introduce SILSA, a topology-aware 3D generation framework that represents shapes with compact sliding-window slice latents. Instead of generating expensive voxel tokens, SILSA uses a fixed set of overlapping slices along the three canonical axes, where each token summarizes a local depth window to preserve cross-sectional continuity and support single-stage rectified-flow generation. A Slice VAE encodes oriented surface samples into multi-axis slice latents and reconstructs them with a sparse volumetric decoder, while a Volumetric Anchor Lattice coordinates directional slice streams through a shared 3D workspace. To preserve structural correctness, we introduce slice-level topology supervision that matches persistence diagrams and aligns Betti transitions across neighboring slices.

Experiments show that SILSA improves structural fidelity while substantially reducing generation cost. SILSA improves PSNR by 8.7%, coverage by 5.96 absolute points, and Betti error by 9.2% over the strongest baseline, while using 70.0% fewer tokens than the next-most compact baseline and over 98% fewer tokens than sparse or hierarchical tokenizers, effectively reducing training memory by 40.4% and inference time by 58.5%. Qualitative results further show improved preservation of thin structures, repeated components, and long-range connectivity.

Contributions

  • Sliding-window slice latents provide a compact representation of 3D shapes. We introduce SILSA, a topology-aware generation framework that replaces expensive local voxel tokens with a fixed set of overlapping depth-window slices along the three canonical axes. The representation preserves cross-sectional continuity and supports single-stage rectified-flow generation.
  • SliceVAE and a Volumetric Anchor Lattice coordinate reconstruction. A Slice VAE encodes oriented surface samples into multi-axis slice latents and reconstructs geometry with a sparse volumetric decoder. A shared 3D anchor workspace coordinates the directional slice streams.
  • Slice-level topology supervision preserves structural correctness. Persistence-diagram matching and Betti-transition alignment across neighboring slices encourage correct cross-sectional topology, preserving thin structures, repeated components, and long-range connectivity.
  • Experiments measure structural fidelity and generation cost. SILSA improves PSNR by 8.7%, coverage by 5.96 absolute points, and Betti error by 9.2% over the strongest baseline. It uses 70.0% fewer tokens than the next-most compact baseline and over 98% fewer than sparse or hierarchical tokenizers, reducing training memory by 40.4% and inference time by 58.5%.

SILSA Method

A topology-aware SliceVAE learns the representation. An image-conditioned rectified-flow transformer generates its 384 slice latents, and the frozen decoder reconstructs geometry through sparse volumetric upsampling.

SILSA architecture: a multi-axis sliding-window encoder creates slice latents; image-conditioned rectified flow coordinates them through a Volumetric Anchor Lattice; a sparse decoder reconstructs a mesh and receives slice-wise topology supervision.
From slices to surfaces. Overlapping windows encode local geometry; volumetric anchors connect the axes.

Slice-wise topology supervision and inter-slice consistency

Thin supports can disconnect, holes can close, and nearby parts can merge even when a reconstruction looks locally similar. SILSA supervises both the topology of each cross-section and the transitions between neighboring slices.

Components + holes + where they change

Cross-sections along three axes, with connected-component and hole persistence bars showing how topology changes through a shape.

Sliding-window slice representation

Voxel tokenization divides a surface into many local pieces. SILSA instead summarizes overlapping depth windows along three axes. This compact, fixed layout preserves cross-sectional continuity and supports a single-stage generator.

128 x + 128 y + 128 z = 384 tokens
01 / REPRESENT

Sliding-window SliceVAE

Each token aggregates oriented surface samples across a local depth window. A sparse volumetric decoder reconstructs the high-resolution mesh.

02 / COORDINATE

Volumetric Anchor Lattice

A shared 3D workspace lets the x-, y-, and z-axis streams exchange spatial evidence as the rectified-flow transformer generates the slice latents.

03 / PRESERVE

Slice-wise topology loss

Persistent-homology supervision preserves components and holes. Betti-transition matching aligns where they appear, disappear, split, or merge.

Quantitative Results

Trained on Trellis-500K. Evaluated on 200 randomly sampled Toys4K assets and 50 in-the-wild images, with no training-set overlap.

Efficiency comparison

Training memory and time: batch size 4 on one A100; training time is seconds per iteration. Inference is end-to-end seconds per shape. Variable-length token counts are mean ± standard deviation. ↓ Lower is better. Best and second best highlighted.

Efficiency comparison from the paper. Lower is better for all columns.
Method Tokens ↓ Stages ↓ Parameters ↓ Train memory ↓ Train time ↓ Inference ↓ CD ↓
Dora 1,280 1 124 M 14.6 GB 0.38 s 0.82 s 9.76
XCube 64,821 2 87 M 68.4 GB 1.18 s 2.74 s 3.21
Trellis 19,847 ± 4,312 2 347 M 42.7 GB 0.71 s 1.93 s 1.12
SparseFlex 87,453 ± 18,264 2 213 M 55.4 GB 1.47 s 1.15 s 0.61
SILSA 384 1 96 M 8.7 GB 0.21 s 0.34 s 0.59

VAE reconstruction quality

Betti error measures the average mismatch in connected components and holes. F-score thresholds are 0.01 and 0.005. ↓ Lower is better; ↑ higher is better. Best and second best highlighted.

VAE reconstruction quality from the paper.
Method CD ↓ F-score @ 0.01 ↑ F-score @ 0.005 ↑ IoU ↑ Betti error ↓
Dora 9.76 67.92 38.71 64.85 4.916
XCube 3.21 81.74 52.06 78.13 4.382
Trellis (SLAT) 1.12 92.83 71.45 86.29 2.871
SparseFlex 0.61 96.18 83.62 92.54 1.743
SILSA 0.59 96.79 84.03 93.01 1.582

Image-to-3D generation

CLIP measures image-output alignment; FD and KD measure generative quality; PSNR and LPIPS measure visual fidelity; COV and MMD measure distributional fidelity. ↓ Lower is better; ↑ higher is better. Best and second best highlighted.

+8.7% PSNR

32.74 vs. 30.12 for SparseFlex in image-to-3D generation.

+5.96 points coverage

79.08% vs. 73.12% for SparseFlex, an absolute gain.

40.4% less training memory

8.7 GB vs. 14.6 GB for Dora under the reported A100 setup.

Image-to-3D generation performance from the paper.
Method CLIP ↑ FD ↓ KD ↓ PSNR ↑ LPIPS ↓ COV (%) ↑ MMD (‰) ↓
Shap-E 80.16 34.64 0.87 16.84 0.21 61.41 19.19
LN3Diff 82.79 26.98 0.76 18.73 0.19 55.21 19.84
Direct3D 74.12 24.97 0.33 22.36 0.17 58.72 18.46
3DTopia-XL 76.46 24.21 0.29 22.06 0.18 58.93 17.62
InstantMesh 84.41 20.13 0.29 25.72 0.11 66.84 16.72
GaussianAnything 80.91 22.46 0.44 23.84 0.15 60.01 15.47
XCube 84.91 10.32 0.09 23.99 0.13 73.01 14.92
Dora 80.35 22.84 0.23 24.68 0.13 67.42 15.63
SAR3D 84.67 22.12 0.18 26.31 0.10 70.30 15.12
Trellis 85.03 10.31 0.08 24.01 0.14 72.10 14.36
SparseFlex 88.22 11.16 0.08 30.12 0.05 73.12 14.52
SILSA 87.94 10.16 0.08 32.74 0.05 79.08 14.02

BibTeX

@inproceedings{yu2026silsa,
  title={SILSA: Sliding-Window Slice Latents for Topology-Preserving High-Resolution 3D Generation},
  author={Yu, Tianjiao and Li, Xinzhuo and Shen, Yifan and Shen, Ying and Nguyen, Kiet A. and Juvekar, Adheesh Sunil and Lourentzou, Ismini},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2026}
}