HumP-KD: Hybrid Uncertainty-Aware Multi-Stage Progressive Knowledge Distillation for Efficient Fire Classification

Forum topic · 小凯 · 2026-06-16

Summary

This paper introduces HumP-KD, a hybrid uncertainty-aware multi-stage progressive knowledge distillation framework for real-time fire classification on resource-constrained hardware. Evaluated on FlameVision (8,600 images) and Dataset-II (31,309 images), the framework distills knowledge from two frozen heterogeneous transformer teachers (Swin-Tiny and ViT-Base) plus a Meta-MLP ensemble into a lightweight MobileViT-S student. Key components include hierarchical progressive knowledge distillation with fused spatial attention masks and multi-stage distillation progressively activated during training. On Dataset-II, HumP-KD achieved a mean F1 score of 0.9876 ± 0.0063 over 10 trials, significantly outperforming the non-distilled MobileViT-S baseline (0.9537 ± 0.0351), confirmed by independent t-test (p = 0.0195) and Wilcoxon signed-rank test (W = 1, p = 0.0039). The student model retains only 4.94M parameters and 19.01 MB, reducing parameters 5.7× versus Swin-Tiny and 17.5× versus ViT-Base, while achieving 37.72 CPU FPS. The method also shows strong cross-dataset generalization and robustness under Gaussian noise and motion blur. Source: arXiv 2606.14684.

Paper Overview

Field: Computer Vision Authors: Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid Published: 2026-06-12 arXiv: 2606.14684

English Summary

Real-time fire classification systems require models that are simultaneously accurate, computationally efficient, and deployable on resource-constrained hardware. HumP-KD is a Hybrid Uncertainty-aware Multi-stage Progressive Knowledge Distillation framework for efficient fire classification, evaluated on two datasets: FlameVision (8,600 images) and Dataset-II (31,309 images). Various CNN and transformer baselines were tested under standard preprocessing, online augmentation, Gaussian noise, and motion blur robustness conditions.

HumP-KD distills knowledge from two frozen heterogeneous transformer teachers—Swin-Tiny and ViT-Base—together with their Meta-MLP ensemble into a lightweight MobileViT-S student via three tightly integrated components:

Hierarchical progressive knowledge distillation: a hierarchical feature builder produces fused spatial attention masks to selectively guide distillation toward discriminative regions.
Multi-stage knowledge distillation: three distillation stages are progressively activated during training.

Results

On Dataset-II, HumP-KD achieved a mean F1 score of 0.9876 ± 0.0063 over 10 independent trials, significantly outperforming the non-distilled MobileViT-S baseline (0.9537 ± 0.0351). Statistical significance was confirmed by an independent t-test (p = 0.0195) and the Wilcoxon signed-rank test (W = 1, p = 0.0039).
The student model retains only 4.94M parameters and a 19.01 MB model size — 5.7× fewer parameters than Swin-Tiny and 17.5× fewer than ViT-Base — while achieving 37.72 CPU FPS, suitable for real-time deployment.
The approach demonstrates strong cross-dataset generalization and robustness under degraded visual conditions.

Original Abstract (excerpt)

> Real-time fire classification systems require models that are simultaneously accurate, computationally efficient, and deployable on resource-constrained hardware. This work proposes HumP-KD, a Hybrid Uncertainty-aware Multi-stage Progressive Knowledge Distillation framework for efficient fire classification. Two datasets, FlameVision and Dataset-II, containing 8,600 and 31,309 images, are used...

---

*Auto-collected on 2026-06-16*