Running a 753B-Parameter GLM-5.2 Model Locally on Two M5 Max MacBooks at 16 Tokens/Second

Forum topic · 小凯 · 2026-07-06

Summary

A community experiment reportedly ran Zhipu's GLM-5.2, a 753-billion-parameter mixture model, fully locally across two M5 Max MacBook Pro laptops using the extreme IQ1_S quantization format, achieving around 16 tokens per second. The article explains how quantization compresses model weights from 16/32-bit floats down to roughly one bit per value, shrinking the model to roughly 200GB so it fits within the combined 256GB of unified memory offered by two M5 Max machines. It highlights Apple's unified memory architecture as the key hardware enabler, since CPU and GPU share one memory pool without data transfers. Notably, community tests showed the quantized 753B model beating a Qwen 27B at Q8 precision on some coding tasks, demonstrating how scale can offset precision loss. Beyond the technical feat, the post argues this signals a shift in AI accessibility: local inference, while rougher than cloud services, extends frontier-scale models to offline and resource-constrained scenarios.

An Apparently Impossible Feat

On June 30, 2026, a community member reportedly ran Zhipu's newly released GLM-5.2 — a model with 753 billion total parameters — entirely locally on two MacBook Pros with M5 Max chips. No cloud, no rented servers. The setup achieved roughly 16 tokens per second using the extremely aggressive IQ1_S quantization format.

At that speed, you can type a question, wait a second or two, and get a coherent streaming answer — usable for coding, editing, and analysis tasks.

Quantization: Compressing the Elephant

Quantization compresses not the text a model produces, but its weights — the numbers that determine how it thinks. Instead of storing each weight as a 16- or 32-bit float, quantization shortens the representation to 8-bit, 4-bit, or even ~1-bit.

IQ1_S is an extreme quantization scheme in the GGUF format, storing each value with a bit over one bit of information. The library of 750 billion books becomes a stack of summary cards that fit in a backpack.

The cost is precision loss. Yet community comparisons showed the IQ1_S-quantized GLM-5.2 753B outperforming a Qwen 27B at Q8 (high-precision) quantization on some coding tasks. Scale can compensate for precision: 750 billion slightly blurred parameters still carry more information capacity than 27 billion crisp ones — like a compressed high-resolution JPEG looking sharper than a lossy low-resolution PNG at certain distances.

Why M5 Max Can Handle It

Apple's M-series chips use a Unified Memory Architecture: CPU, GPU, and Neural Engine share one memory pool, avoiding the constant data shuttling of traditional PC designs.

IQ1_S-quantized GLM-5.2 753B: roughly ~200GB
Two M5 Max machines × 128GB unified memory = 256GB combined

It barely fits, requiring careful memory management in practice, but the core math works: unified memory lets the model live inside the chips rather than being split awkwardly across them.

What It Means: Power Moving Downstream

Running frontier-scale models was once the exclusive domain of hyperscalers with million-dollar GPU clusters. Two hobbyists with two laptops now replicate a slice of that capability at home.

This doesn't mean local inference replaces the cloud — cloud remains more efficient and convenient. But the boundary has moved: a journalist analyzing documents offline on a plane, a doctor searching medical literature in a remote area, a student running a personal assistant in a dorm — scenarios that once demanded connectivity now work locally.

Closing Thought

The IQ1_S local run of GLM-5.2 is rough: imperfect precision, slower than the cloud. But it's real, done in a bedroom on two MacBooks by community enthusiasts — reminiscent of early personal computers, crude but era-opening. A 753-billion-parameter model breathing on laptops isn't the endpoint; it's a beginning.