MUA: How Wavelet Magic Squeezes a 768-Layer Silk Dress Into a VR Headset
> *"You stand in a Meta Quest 3, facing a virtual person. Her dress sways in the wind, every fold clearly visible — and all of this is driven by a battery-powered chip strapped to your head. This isn't magic; this is wavelet decomposition."*
The Gap Between Two Worlds
Before MUA, digital human research faced a stark trade-off:
- Server-class methods (UMA, ASH, Ani Gaussians): ultra-detailed avatars using 768x768 dynamic texture maps and multi-layer 2D convolutions. UMA delivers stunning quality but needs 1804 GFLOPS — only 9.9 FPS even on a server GPU.
- Mobile methods (TaoAvatar, 3DGS-Avatar, SqueezeMe): run in real time on headsets, but lose dynamic detail — clothes look like folded plastic. TaoAvatar hits 87.8 FPS but only 28.66 PSNR (vs. MUA's 32.31).
- The bottom level is an approximation (low-frequency, global structure)
- Upper levels are detail sub-bands (horizontal/vertical/diagonal high frequencies)
- Each level halves resolution, so compute can be allocated precisely: cheap large-scale processing for coarse shapes, high-resolution work only where needed (faces, hands, folds)
- Approximation: \(a_{j}[n] = \frac{1}{\sqrt{2}}(a_{j+1}[2n] + a_{j+1}[2n+1])\)
- Detail: \(d_{j}[n] = \frac{1}{\sqrt{2}}(a_{j+1}[2n] - a_{j+1}[2n+1])\)
- vs. UMA: FLOPs are 1/3500th (0.52 vs 1804), parameters 1/7, frame rate 18x — while test-pose PSNR is nearly identical (27.31 vs 27.66)
- vs. TaoAvatar: PSNR 32.31 vs 28.66; LPIPS 53.88 vs 90.74 — visibly much better
- Meta Quest 3 (standalone): 24 FPS native real-time on its Snapdragon XR2 Gen 2 — past the immersion threshold for VR
- RTX 3090: 180+ FPS — enabling VTubers, game NPCs, and telepresence avatars
- Novel-pose generalization still lags training-pose quality (27.31 vs 32.31 PSNR)
- Only single-person scenarios validated; complex lighting and multi-person interaction unknown
- Training still requires a teacher model and large data — no from-scratch light model
- Generalization across diverse fabric materials (silk, denim, leather) remains to be verified
- [3] Mildenhall et al., "NeRF," ECCV 2020
- [4] Kerbl et al., "3D Gaussian Splatting," ACM TOG 42(4), 2023
- [6] Li et al., "Animatable Gaussians," CVPR 2024
- [10] Pang et al., "ASH," CVPR 2024
- [11] Zhu et al., "UMA," arXiv:2506.01802, 2025
- [13] "SqueezeMe," SIGGRAPH 2025
- [14] Chen et al., "TaoAvatar," CVPR 2025
MUA — Mobile Ultra-detailed Animatable Avatars (by Heming Zhu, Guoxing Sun, Marc Habermann) — refuses this trade-off.
Background: Why Compression Is Hard for Avatars
The field evolved from video-based replay to NeRF (implicit neural radiance fields, too slow for rendering), then to 3D Gaussian Splatting (3DGS) — splatting colored 3D Gaussian blobs and flattening them per view, which GPUs parallelize well.
The hard part: animatable avatars have almost no inter-frame redundancy. Each frame's clothing wrinkles are recomputed from cloth physics, skeletal motion, and deformation — unlike video compression, you can't exploit temporal similarity. The efficiency must come from the single-frame representation itself.
Key Ideas
1. Multi-level Wavelet Decomposition
Any signal can be split into scales. MUA decomposes the high-resolution texture map (e.g., 768x768) into multiple levels:
For the Haar wavelet:
2. Low-Rank Spatial Factorization
Using the intuition of SVD (\(A = U \Sigma V^T\)), textures in each wavelet sub-band are stored not as full matrices but as low-rank factors — exploiting the fact that texture appearance is governed by a few intrinsic dimensions (glossiness, roughness, translucency, elasticity). The paper reports a 10x model size reduction.
3. Wavelet-guided Multi-level Spatial Factorized Blendshapes
Classic blendshapes interpolate geometry between a base shape and target expressions. MUA extends blendshapes into texture space — materials, lighting, and wrinkle details are dynamically interpolated — but crucially operates on the low-rank factors of wavelet coefficients, not raw high-res textures.
4. Multi-Level Knowledge Distillation
Instead of training a compact model from scratch, MUA distills from the heavy UMA teacher:
1. The teacher generates 768x768 (or 512x512) dynamic textures from skeletal pose, decomposed into wavelet sub-bands 2. Each sub-band is distilled separately: PCA subspace + light MLP for low frequencies; factorized 1D convolutions for mid/high frequencies 3. Joint optimization with pixel L2 loss, perceptual loss, and per-level sub-band feature losses
This "layered teaching" gives the student a high-quality target manifold, simplifying optimization and preserving generalization.
Benchmark Results (Table I)
| Method | Train PSNR | Test PSNR | GFLOPS | Params (M) | FPS | |---|---|---|---|---|---| | ASH | 35.96 | 27.50 | 1804 | 181.5 | 10.0 | | UMA | 36.80 | 27.66 | 1804 | 184.0 | 9.9 | | 3DGS-Avatar | 25.55 | 24.87 | 6.84 | 3.23 | 35.2 | | TaoAvatar | 28.66 | 28.01 | 1.80 | 592 | 87.8 | | MUA | 32.31 | 27.31 | 0.52 | 26.67 | 182.2 |
Key takeaways:
Ablations (Table II) confirm every component matters: PCA-only explodes parameters to 974.5M; single-level factorization scores worse (29.10 vs 29.91 train PSNR); replacing factorized 1D convs with 2D convs raises cost from 0.52 to 13.7 GFLOPS.
Hardware Validation
Why It Works: The Feynman Test
The post argues MUA passes Feynman's smell tests: no cargo-cult use of trendy architectures. Each design choice has a clear functional purpose — wavelets to allocate compute across frequencies, low-rank to exploit intrinsic texture structure, distillation to transfer the teacher's generalization, 1D factorized convs to replace costly 2D ones. And "demonstration beats argument": the project page shows side-by-side Quest 3 footage where MUA's clothing folds move convincingly while TaoAvatar's look plastic.