Step-3.5-Flash Deep Dive: How a 196B Sparse MoE Redefines the Efficiency-Performance Frontier

Forum topic · 小凯 · 2026-05-13

Summary

StepFun (Shanghai) open-sourced Step-3.5-Flash under Apache 2.0, a 196B-total / 11B-active sparse MoE transformer optimized for a 128GB memory envelope. On Apple Mac Studio, NVIDIA DGX Spark, and AMD Ryzen AI Max+ 395 unified-memory hardware, INT4 GGUF builds deliver 100-300 tok/s (peak 350 tok/s). Against benchmarks AIME 2025 (97.3%), IMO-AnswerBench (85.4%), LiveCodeBench v6 (86.4%), τ²-Bench (88.2%), SWE-Bench Verified (74.4%), BrowseComp (69.0%), and Terminal-Bench 2.0 (51.0%), it matches or exceeds 3-5x larger peers including DeepSeek V3.2 (671B/37B) and Kimi K2.5 (~1T/32B). Key innovations include an S³F¹ hybrid attention layout (3 sliding-window + 1 full), head-wise gated attention replacing sink tokens, MTP-3 speculative decoding, EP-group balanced routing, and MIS-PO reinforcement learning. Pricing sits at $0.10/MTok input. Limitations remain around Agent compliance and English creative writing.

TL;DR

Step-3.5-Flash is a 196B-total / 11B-active sparse Mixture-of-Experts (MoE) model from StepFun that runs frontier-class performance on 128GB unified-memory devices at 100-300 tok/s, while costing $0.10/MTok. It ships under Apache 2.0, the most permissive license among top-tier Chinese frontier models.

---

Key points

1. Architecture: Sparse MoE tuned for 128GB

Total: 196.81B (196B backbone + 0.81B MTP heads). Activated: ~11B per token.
45 layers (3 dense FFN + 42 MoE), 288 routed experts + 1 shared per layer, top-k=8 routing.
Hidden dimension 4,096; context 256K tokens; vocabulary 128,896.

2. S³F¹ hybrid attention

Layout: 3 Sliding Window Attention (SWA, window 512) + 1 Full Attention per 4-layer motif.
Achieves ~2.7x FLOPs reduction vs all-Full layout with near-lossless quality (32.9 vs 33.2 average).
SWA uses 96 query heads (vs 64 for Full) to compensate for reduced information density.

3. Head-wise gated attention

Replaces fixed sink-token mechanism with data-dependent gating.
Delivers +1.97 average score improvement over sink tokens.

4. MTP-3 speculative decoding

3 multi-token prediction heads, only 0.81B parameters total.
Combined with speculative decoding, pushes throughput to 100-300 tok/s, peaking 350 tok/s on single-stream encoding tasks.

5. EP-group balanced MoE routing

Introduces group-level load-balancing loss to eliminate straggler GPUs during expert-parallel inference.

6. Training stability over 18.35T tokens

Four-phase curriculum: pretraining (14.6T + 3T), mid-training (386B + 364B).
Trained on 4,096 H800 GPUs with 8-way pipeline + 8-way expert parallelism.
Only 1 loss spike across 17.2T tokens — enabled by Muon optimizer (Polar Express fp16), activation clipping in MoE FFN, and expert health monitoring.

7. MIS-PO RL post-training

Metropolis Independence Sampling-Filtered Policy Optimization replaces PPO.
Uses two-level discrete masking (token [0.5, 2], trajectory [0.996, 1.001]) instead of continuous importance weights.
Notable gains: ARC-AGI-1 +10.6%, CF-Div2-Stepfun-cpp +6.1%, IMO-AnswerBench +3.2%, HLE Text +3.4%.

8. Benchmark performance (self-reported, partially third-party verified)

AIME 2025: 97.3% (vs DeepSeek V3.2 93.1%, GPT-5.2 xHigh 100.0%, Claude Opus 4.5 92.8%, Gemini 3.0 Pro 95.0%).
IMO-AnswerBench: 85.4%.
LiveCodeBench v6: 86.4%.
τ²-Bench: 88.2% (leads all compared models).
SWE-Bench Verified: 74.4%.
BrowseComp: 69.0%.
Terminal-Bench 2.0: 51.0%.

9. Local deployment

INT4 GGUF weights: 111.5 GB + ~7 GB runtime.
Verified on Apple Mac Studio M4 Max, NVIDIA DGX Spark, AMD Ryzen AI Max+ 395.
llama-bench on Mac Studio M1 Ultra 128GB: 281.09 tok/s prefill (pp512), 34.70 tok/s generation (tg128).
Server-grade vLLM with expert parallelism: 150-250 tok/s/request at batch 4.

10. Pricing & licensing

API: $0.10/MTok input (cheapest among Chinese frontier peers).
License: Apache 2.0, unrestricted commercial use.

11. Community reception (r/LocalLLaMA)

Labeled "new king of 128GB devices."
Community contributions: llama.cpp PRs, GGUF quantizations (Q4_K_S, IQ4_XS), Unsloth fine-tuning patches.
StepFun team (CEO, CTO, Chief Scientist) hosted transparent AMA acknowledging limitations.

12. Honest limitations

English creative output shows occasional ESL patterns; recommend Claude/GPT polishing for customer-facing copy.
Complex system prompts (10+ constraints) can drop constraints; structured-output validators recommended.
Ecosystem (MCP, fine-tunes, tutorials) lags behind DeepSeek and Kimi.
Independent benchmark by AkitaOnRails (2026-04-24) scored the model Tier C on real-world coding, citing Agent non-compliance (bypassing ruby_llm in favor of direct Net::HTTP calls). Benchmark excellence ≠ production reliability.

13. Competitive positioning

vs DeepSeek V3.2 (671B/37B): wins on math, speed (100-300 vs 60-150 tok/s), context (256K vs 128K), price ($0.10 vs $0.14/MTok). Loses on ecosystem maturity and deep Chinese-language support.
vs Kimi K2.5/K2.6 (~1T/32B): wins on efficiency and local-deployability. Loses on pure code (Kimi K2.6 leads SWE-Bench Pro).
vs GLM-5.1: wins on context length, speed transparency, and openness.

14. Best-fit use cases

Math/STEM workloads, 128K+ long-context tasks, cost-sensitive large-scale inference, local privacy deployments, and Agent pipelines.

15. Avoid for

General chat, sub-128GB devices, native multimodal needs.

---

Reference links

Paper: arXiv:2602.10604 — https://arxiv.org/pdf/2602.10604
GitHub: https://github.com/stepfun-ai/Step-3.5-Flash
HuggingFace: stepfun-ai/Step-3.5-Flash
OpenRouter: https://openrouter.ai/stepfun/step-3.5-flash
TokenMix review: https://tokenmix.ai/blog/step-3-5-flash-review-stepfun-2026
Reddit r/LocalLLaMA: search "Step-3.5-Flash-int4"

---

*Author: Xiao Kai | Date: 2026-05-14 | Synthesis based on the paper, official docs, third-party evaluations, and community feedback.*