Apple MLX Status Update: CUDA Backend, Ecosystem Growth, and Roadmap (September 2025)

Forum topic · ✨步子哥 · 2025-09-24

Summary

As of September 2025, Apple's MLX framework has entered an accelerated phase of feature completion and ecosystem expansion. Five minor releases (0.19 to 0.24) introduced an official CUDA backend for NVIDIA GPUs, where Llama-3.1-405B achieves roughly 1.8x the inference speed of llama.cpp on an RTX 4090. Fused-attention and RoPE-fusion optimizations cut peak memory by 28% and time-to-first-token by 35% for 128k contexts on M3 Max with 4-bit quantization. mlx-lm now supports Qwen3-MoE, Gemma-3, and other new architectures with LoRA fine-tuning scripts. Alibaba released 32 official Qwen3-MLX quantized models, LM Studio v0.3 integrated the MLX backend with 1.5-2x speedups over GGUF, and mlx-vlm added Gemma-3 vision support. MLX remains strongest on Apple Silicon via its zero-copy unified memory, while CUDA support is still early-stage, lacking FP8, flash-attn, and tensor parallelism. Version 0.25 plans FP8 GEMM, CPU backend, distributed inference, and a PyTorch exporter.

As of September 2025, MLX has entered an accelerated phase of "feature completion + ecosystem expansion" over the past six months. The main progress can be summarized as three themes covering seven key developments.

1. Feature Completion

CUDA backend added (releases 0.19 → 0.24): Apple's first official support for NVIDIA GPUs. Llama-3.1-405B inference on an RTX 4090 reaches roughly 1.8× the speed of llama.cpp.
Fused-attention and RoPE-fusion: On M3 Max with 4-bit quantization, peak memory for 128k contexts dropped 28%, and time-to-first-token improved by 35%.
mlx-lm architecture support: Now supports Qwen3-MoE, StableLM-2-1.6B, Gemma-3, and other recent architectures, with LoRA fine-tuning scripts. From v0.24, runtime dynamic RoPE-base, sliding window, and logits-scale adjustments are available — experiments no longer require re-converting weights.

2. Ecosystem Expansion

Alibaba Qwen: Alibaba's Tongyi Qianwen team released 32 official Qwen3-MLX quantized models (0.6B–235B × 4/6/8-bit / BF16), runnable across the full hardware range from iPhone 15 to Mac Studio — viewed in industry circles as groundwork for a China-market "Apple Intelligence."
LM Studio v0.3: Officially integrated the MLX backend, with one-click UI filtering for "Apple Silicon optimized" models. Compared to GGUF versions of the same models, tokens/s improved 1.5–2×.
mlx-vlm: Merged Gemma-3-1B/7B vision towers, enabling multi-image dialogue inference at 1344×896 on a 16 GB M3 — marking MLX's coverage of speech-text-vision multimodality.

3. Performance and Boundaries

Strengths: MLX retains its "zero-copy unified memory" moat on Apple Silicon. 4-bit quantization quality matches llama.cpp Q4_K_M, with 20–40% better speed.
Limitations: The CUDA backend is "usable but early-stage" — no FP8, flash-attn, or tensor parallelism. Windows/Linux users must compile from source; the project only provides a Dockerfile.
Roadmap (0.25 preview): FP8 GEMM, CPU backend, distributed inference, and a PyTorch exporter, expected to merge into the mainline around November 2025.

Conclusion

MLX has moved beyond its niche "M-series exclusive" positioning, breaking into the broader ecosystem through CUDA support and first-day adaptation of major models from Alibaba and Google. On macOS it is currently the best framework for local LLM inference and fine-tuning; on NVIDIA hardware it remains at the "experimental but usable" stage. If your primary development environment is a Mac, MLX is a safe first choice as an inference/fine-tuning backend. If your production environment is primarily CUDA-based, consider waiting for the official 0.25 release or adopting a dual-backend "ONNX-MLX" strategy.