JanusMesh: Fast Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising
This post is a deep-dive commentary (originally published in Chinese on zhichai.net) on JanusMesh, a 2026 paper from a four-person team at National Yang Ming Chiao Tung University, accepted to ECCV 2026. JanusMesh generates 3D visual illusions: a single 3D mesh that renders as object X (e.g., a peacock) from angle A and object Y (e.g., a pineapple) from angle B, while looking like an abstract shape from other viewpoints.
Key points
- Problem: 3D visual illusion generation previously had two flawed approaches:
- SDS-based optimization (DreamFusion lineage, e.g., Shape From Semantics): ~40 minutes per object and severe color oversaturation.
- Direct Concatenation of two separately generated models: visible geometric seams and *semantic leakage* (parts of one object visible from the other's target view).
- Core idea — Cross-Space Denoising: JanusMesh builds on TRELLIS (CVPR 2025), which uses Rectified Flow over structured 3D latents in two stages. JanusMesh intervenes only in the first stage (sparse voxel/SDF structure prediction), because geometric fusion must happen at the "skeleton" level: 1. Two branches denoise the same noise latent with different prompts ("a peacock" / "a pineapple"). 2. Both latents are decoded into voxel space. 3. Orientations are aligned via CLIP-based rotation search. 4. The shapes are fused by signed distance field (SDF) blending, giving smooth, gradient transitions instead of hard boolean unions. 5. The fused voxels are re-encoded back to latent space for the next step, repeated over 25 denoising steps.
- Why "cross-space": each step alternates latent → voxel → latent. Direct latent-space geometric operations cause distribution shift (an idea borrowed from LookingGlass); decoding to geometric space ensures "the geometric validity of the blended mesh."
- Three-object extension: views at 0°, 120°, 240° (e.g., grapes / pineapple / bamboo). A Noise Guidance mechanism (Space Control Guidance with a pre-generated single-semantic prior) stays active for the first 20 of 25 steps, since three-way geometric conflicts are stronger. The paper honestly notes three-object results are harder and were mainly evaluated qualitatively.
- Stage 2 — View-Conditioned Texture Synthesis: the fused untextured mesh is painted using Stable Diffusion. Images are generated per target view, un-projected onto the mesh surface, and merged via Mesh Texture Aggregation, assigning each view's visible regions its semantics, neutral texture to unseen regions, and smooth transitions at boundaries. Geometry and texture are deliberately decoupled for lower complexity, geometric consistency, and modularity.
Results
| Method | Time | Geometry | Semantics | Color | |---|---|---|---|---| | Shape From Semantics (SDS) | ~40 min | medium | medium | poor (oversaturated) | | Direct Concatenation | fast | poor (seams) | poor (leakage) | medium | | JanusMesh | 3–5 min | excellent | excellent | excellent |
Evaluation combines CLIP similarity, GPT-4.1-mini object recognition, FID/KID, and a binary object-detection check. Ablations confirm that cross-space fusion, SDF blending, CLIP orientation alignment, and view-conditioned texturing are each necessary.
Significance
JanusMesh is training-free / zero-shot: it recombines the general priors of TRELLIS (3D generation) and Stable Diffusion (2D generation) rather than training on illusion datasets. Authors demonstrate applications in art installations, product design, game assets, and architectural facades—turning a once niche sculptural art form into a 3–5 minute text-to-mesh task on a consumer GPU.
References
1. Zhang, S.-L., Cheng, H.-H., Yang, T.-J., & Liu, Y.-L. (2026). *JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising*. arXiv:2606.20563. Accepted to ECCV 2026. Project Page 2. Xiang, J., et al. (2025). Structured 3D Latents for Scalable and Versatile 3D Generation (TRELLIS). CVPR. 3. Poole, B., et al. (2022). DreamFusion: Text-to-3D Using 2D Diffusion. arXiv:2209.14988. 4. Geng, Z., Sun, Y., Kwon, M., & Essa, I. (2024). LookingGlass: Generative Anaglyphs from Any Image. arXiv:2404.06448. 5. Radford, A., et al. (2021). Learning Transferable Visual Models from Natural Language Supervision (CLIP). ICML.
*"When you rotate it, the world changes. That's not magic—it's algorithmic elegance."*