Model Quantization Explained: FP32, FP16, BF16, INT8 and INT4 Visual Tutorial

Forum topic · 小凯 · 2026-03-27

Summary

This tutorial from the Easy AI series on zhichai.net introduces model quantization, the process of converting high-precision floating-point numbers in neural networks into lower-precision representations to reduce model size and speed up inference while preserving accuracy. It presents an interactive visualization website built with components including a binary visualizer, format selector, comparison table, and a step-by-step quantization process visualizer. The tutorial compares FP32 (1 sign + 8 exponent + 23 mantissa bits), FP16 (5 exponent + 10 mantissa), BF16 (8 exponent + 7 mantissa), INT8, and INT4 formats using a concrete example such as 3.14. Key takeaways: floating-point precision depends on mantissa length; integer formats require a Scale factor for dequantization, with INT8 achieving acceptable error (ideally under 1%) while INT4 is only suitable for coarse representation; and INT8 is widely used in large-model quantization, offering a favorable balance between accuracy and speed at roughly half the size of FP16.

Model Quantization (Easy AI Tutorial)

> Model compression and quantization techniques

Overview

Model quantization is the process of converting high-precision floating-point numbers in neural networks into lower-precision representations. The goal is to reduce model size and improve inference speed while keeping accuracy loss as small as possible.

This post accompanies an interactive tutorial website built from several visualization components:

Header — site title and brief introduction
FormatSelector — switch between numerical formats (FP32, FP16, BF16, INT8, INT4)
BinaryVisualization — shows the bit layout of each format, bit by bit
ComparisonTable — side-by-side comparison of formats
QuantizationProcessVisualizer — step-by-step animation of quantization and dequantization (Scale computation, quantization, binary representation, restore)
QuantizationVisualizer — binary representation and precision-loss details for a chosen format

Number Formats Compared

| Format | Layout | Notes | |---|---|---| | FP32 | 1 sign + 8 exponent + 23 mantissa | Full float, extremely high precision | | FP16 | 1 sign + 5 exponent + 10 mantissa | Half precision, reduced mantissa | | BF16 | 1 sign + 8 exponent + 7 mantissa | Keeps FP32 exponent range, much lower mantissa precision | | INT8 | 8-bit unsigned integer | Requires quantization with Scale to restore; e.g. 3.14 with Scale = 0.0157; error ideally within 1% | | INT4 | 4-bit signed integer (-8 to 7) | Very low precision, large error, only for coarse representation; e.g. 3.426 with Scale = 1.142 |

INT8 covers the range 0–255 (unsigned; signed range -128 to 127), INT4 covers 0–15 (signed -8 to 7).

Key Conclusions

Floating-point precision depends on mantissa length. FP32's 23-bit mantissa represents values precisely; FP16's 10 bits lose noticeable precision; BF16's 7 bits lose substantial decimal detail.
Integer formats require quantization. With a well-chosen Scale factor, decimals can be restored within a limited range; INT4's tiny range makes accurate decimal representation nearly impossible.
INT8 is the mainstream choice for LLM quantization. It balances precision and speed, roughly halving size compared with FP16, and careful Scale design plus calibration algorithms keep error within acceptable bounds.

Quantization Process

The visualizer walks through five steps with animation and custom input:

1. Input a floating-point number (validated against a min/max range) 2. Determine the Scale factor — maps the original float range onto the target integer range:

Scale = (max − min) ÷ (integer max − integer min)

3. Perform quantization using the quantization formula 4. Inspect the binary representation 5. Dequantize (restore) and compare with the original

The Scale factor defines how much float value each integer unit represents. Depending on the input, quantization error may be tiny (good quantization) or noticeable (normal precision loss).

---

*Part of the Easy AI tutorial series.*