JanusMesh: Fast Zero-Shot Text-to-3D Visual Illusion Generation via Cross-Space Denoising

Forum topic · 小凯 · 2026-06-22

Summary

JanusMesh is a fast, training-free framework for generating text-driven 3D visual illusions—single 3D meshes that appear as entirely different objects when viewed from different angles. Existing optimization-based methods are slow and prone to color oversaturation, while naive stitching approaches produce geometrically inconsistent results with visible seams and semantic leakage. JanusMesh decouples generation into two stages: first, a cross-space dual-branch denoising process dynamically decodes 3D latents into voxel space, performing CLIP-guided directional alignment and signed distance field (SDF) fusion for seamless geometric blending; second, a view-conditioned texture synthesis module projects and aggregates view-specific 2D diffusion priors onto the fused geometry. Experiments show the method generates highly realistic dual-semantic 3D illusions in just 3-5 minutes, significantly outperforming existing approaches in geometric integrity, semantic recognizability, and efficiency. Paper: arXiv 2506.17588 (cs.CV) by Siang-Ling Zhang, Huai-Hsun Cheng, and Tsung-Ju Yang.

Overview

Research area: cs.CV
Authors: Siang-Ling Zhang, Huai-Hsun Cheng, Tsung-Ju Yang
Published: 2026-06-21
arXiv: 2506.17588

Abstract (translated)

Creating 3D visual illusions—where a single 3D mesh presents completely different semantics depending on the viewing angle—is fascinating but highly challenging. Existing optimization-based methods are slow and often cause color oversaturation, while naive stitching approaches struggle to generate geometrically consistent objects, resulting in visible unnatural seams and semantic leakage.

This paper proposes a fast, training-free framework for text-driven 3D visual illusion generation. The method decouples the generation process into two stages:

1. Cross-space dual-branch denoising: 3D latents are dynamically decoded into voxel space, where CLIP-guided directional alignment and signed distance field (SDF) fusion enable seamless geometric blending.

2. View-conditioned texture synthesis: a module projects and aggregates view-specific 2D diffusion priors onto the fused geometry.

Extensive experiments show that the method generates highly realistic, dual-semantic 3D illusions in only 3–5 minutes, significantly outperforming existing methods in geometric integrity, semantic recognizability, and efficiency.

Plain-language explanation

Imagine creating a "magic sculpture": from the front it looks like a majestic dragon, but from the side it transforms into a phoenix. Previous approaches either slowly sculpt and tune colors like an old craftsman—ending up with muddled colors—or glue two blocks of wood together, leaving cracks and mixing up the meanings.

This method is smarter. It first computes the core shapes of both objects simultaneously in latent space, then uses a dual-branch denoising process to fuse them into a geometrically seamless object (using SDF "distance fields" to smoothly join the shapes without a trace). Finally, it dresses the object in a "view-aware outfit"—automatically applying the correct textures and details for whichever angle you are viewing from.

The whole process requires no model retraining and takes just 3–5 minutes to produce a truly face-changing 3D illusion: fast, clean, and convincing.

---

*Auto-collected on 2026-06-21*