Apple MLX Update: CUDA Backend, Ecosystem Expansion, and Performance Snapshot

Forum topic · ✨步子哥 · 2025-09-24

Summary

As of September 2025, Apple's MLX machine learning framework has entered an accelerated phase of feature completion and ecosystem growth. Versions 0.19 through 0.24 introduced an experimental CUDA backend (Apple's first official NVIDIA GPU support, with Llama-3.1-405B inference roughly 1.8x faster than llama.cpp on an RTX 4090), fused-attention and RoPE fusion that cut 128k-context peak memory by 28% and first-token latency by 35% on M3 Max with 4-bit quantization, and support for Qwen3-MoE, Gemma-3, and other recent architectures. The ecosystem expanded with Alibaba's 32 official Qwen3-MLX quantized models, LM Studio v0.3's native MLX backend offering 1.5-2x token throughput over GGUF, and mlx-vlm multimodal support for Gemma-3 vision. MLX retains a zero-copy unified memory advantage on Apple Silicon with 4-bit quality on par with llama.cpp Q4_K_M but 20-40% faster. The CUDA backend remains early-stage, lacking FP8, flash-attn, and tensor parallelism; a roadmap preview targets FP8 GEMM, CPU backend, distributed inference, and a PyTorch exporter in 0.25.

Apple MLX Status Update

As of September 2025, MLX has entered an accelerated phase of "feature completion + ecosystem expansion" over the past six months. The main developments can be summarized as three themes and seven key items:

1. Feature Completion

Versions 0.19 → 0.24 shipped 5 minor releases on the mainline. A new CUDA backend was added — Apple's first official NVIDIA GPU support — with Llama-3.1-405B inference on an RTX 4090 running at roughly 1.8× the speed of llama.cpp.
Fused-attention and RoPE-fusion were introduced: on M3 Max with 4-bit quantization, 128k-context peak memory dropped 28% and first-token latency improved by 35%.
mlx-lm now supports the latest architectures including Qwen3-MoE, StableLM-2-1.6B, and Gemma-3, with LoRA fine-tuning scripts. From version 0.24, runtime dynamic RoPE-base, sliding window, and logits-scale adjustments are available — experiments no longer require re-converting weights.

2. Ecosystem Expansion

Alibaba Qwen released 32 official Qwen3-MLX quantized models in one batch (0.6B–235B across 4/6/8-bit and BF16), runnable across the entire lineup from iPhone 15 to Mac Studio. Industry observers view this as groundwork for a China-market Apple Intelligence.
LM Studio v0.3 officially integrated the MLX backend, with a UI filter for "Apple Silicon optimized" models; compared against GGUF versions of the same models, token/s improves 1.5–2×.
The mlx-vlm subproject merged the Gemma-3-1B/7B vision towers, enabling 1344×896 multi-image conversational inference on a 16 GB M3 — marking MLX's coverage of speech-text-vision multimodality.

3. Performance & Boundaries

Strengths: MLX retains its "zero-copy unified memory" moat on Apple Silicon; 4-bit quantization quality matches llama.cpp Q4_K_M while being 20–40% faster.
Limitations: The CUDA backend is currently "usable but basic" — no FP8, flash-attn, or tensor parallelism; Windows/Linux users must compile it themselves, with only an official Dockerfile provided.
Roadmap (0.25 preview): FP8 GEMM, CPU backend, distributed inference, and a PyTorch exporter, expected to merge into the mainline around November 2025.

Conclusion

MLX has moved beyond its niche "M-series only" positioning, breaking out ecologically via CUDA support and first-day adaptation of major models from Alibaba and Google. On macOS it is currently the best local LLM experience available; on NVIDIA it remains at a "taste-it-and-try" stage. If your primary development environment is a Mac, MLX is a safe first-choice inference/fine-tuning backend; for production environments dominated by CUDA cards, it is advisable to wait for the 0.25 stable release or adopt a dual-backend ONNX-MLX strategy.